Home

Awesome

<img src="https://camo.githubusercontent.com/5b298bf6b0596795602bd771c5bddbb963e83e0f/68747470733a2f2f692e696d6775722e636f6d2f7031527a586a512e706e67" align="left" width="144px" height="144px"/>

My home operations repository šŸŽ›šŸ”Ø

... managed by Flux Renovate, and GitHub Actions šŸ¤–

<br /> <div align="center">

Discord talos kubernetes pre-commit GitHub Workflow Status Lines of code

</div>

šŸ“– Overview

This is a mono repository for my home infrastructure and Kubernetes cluster implementing Infrastructure as Code (IaC) and GitOps practices using tools like Kubernetes, Flux, Renovate and GitHub Actions.

Feel free to open a Github issue or join the k8s@home Discord if you have any questions.


ā›µ Kubernetes

This repo generally attempts to follow the structure and practices of the excellent k8s-at-home/template-cluster-k3, check it out if you're uncomfortable starting out with an immutable operating system.

Installation

The cluster is running on Talos Linux, an immutable and ephemeral Linux distribution built around Kubernetes, deployed on bare-metal. Rook Ceph running hyper-converged with workloads provides persistent block and object storage, while a seperate server provides bulk (NFS) file storage.

Core components

GitOps

Flux watches my cluster folder (see Directories below) and makes the changes to my cluster based on the YAML manifests.

Renovate watches my entire repository looking for dependency updates, when they are found a PR is automatically created. When PRs are merged, Flux applies the changes to my cluster.

Directories

This Git repository contains the following directories (kustomizatons) under cluster.

šŸ“ cluster      # k8s cluster defined as code
ā”œā”€šŸ“ bootstrap  # contains the initial kustomization used to install flux
ā”œā”€šŸ“ flux       # flux, gitops operator, loaded before everything
ā”œā”€šŸ“ crds       # custom resources, loaded before šŸ“ core and šŸ“ apps
ā”œā”€šŸ“ charts     # helm repos, loaded before šŸ“ core and šŸ“ apps
ā”œā”€šŸ“ config     # cluster config, loaded before šŸ“ core and šŸ“ apps
ā”œā”€šŸ“ core       # crucial apps, namespaced dir tree, loaded before šŸ“ apps
ā””ā”€šŸ“ apps       # regular apps, namespaced dir tree, loaded last

Networking

NameCIDR
Kubernetes Nodes10.75.40.0/24
Kubernetes external services (Cilium w/ BGP)10.75.45.0/24
Kubernetes pods172.22.0.0/16
Kubernetes services172.24.0.0/16

šŸŒ DNS

Ingress Controller

Over WAN, I have port forwarded ports 80 and 443 to the load balancer IP of my ingress controller that's running in my Kubernetes cluster.

Cloudflare works as a proxy to hide my homes WAN IP and also as a firewall. When not on my home network, all the traffic coming into my ingress controller on port 80 and 443 comes from Cloudflare. In VyOS I block all IPs not originating from Cloudflares list of IP ranges.

šŸ”ø Cloudflare is also configured to GeoIP block all countries except a few I have whitelisted

Internal DNS

k8s_gateway is deployed on my router running VyOS. With this setup, k8s_gateway has direct access to my clusters ingress records and serves DNS for them in my internal network.

Without much engineering of DNS @home, these options have made my VyOS router a single point of failure for DNS. I believe this is ok though because my router should have the most uptime of all my systems.

External DNS

external-dns is deployed in my cluster and configured to sync DNS records to Cloudflare. The only ingresses external-dns looks at to gather DNS records to put in Cloudflare are ones where I explicitly set an annotation of external-dns.home.arpa/enabled: "true"


šŸ”§ Hardware

DeviceCountOS Disk SizeData Disk SizeRamOperating SystemPurpose
Dell R2201120GB SSDN/A16GBVyOS 1.4Router
HP S01-pf10003120GB SSDN/A8GBTalos LinuxKubernetes Control Nodes
HP S01-pf10003120GB SSD1TB NVMe (rook-ceph)32GBTalos LinuxKubernetes Workers
SuperMicro SC8361120GB SSD16x8TB + 16x3TB ZFS RAIDZ2192GBUbuntu 20.04NFS
Brocade ICX 66101N/AN/AN/AN/ACore Switch
Raspberry Pi 4B132GB SD CardN/A4GBPiKVMNetwork KVM
TESmart 8 Port KVM Switch1N/AN/AN/AN/ANetwork KVM switch for PiKVM
APC SUA3000RMXL3U w/ NIC1N/AN/AN/AN/AUPS
APC AP79301N/AN/AN/AN/APDU

šŸ¤ Thanks

Thanks to all folks who donate their time to the Kubernetes @Home community. A lot of inspiration for my cluster came from those that have shared their clusters over at awesome-home-kubernetes.


šŸ“œ Changelog

See commit history


šŸ” License

See LICENSE