A Four Node Kubernetes Cluster for Your Home Lab

While I have this on my mind, I decided to build a microk8s cluster from a bunch of raspberry pis that I have laying around. Why? In practice more useful than minikube which is also great for local development.

What we will be using

1. Ubuntu 22.04.03 Linux distribution
   • 32 bit for contro pi 3b armhf
   • 64 bit for 4 arm64
2. Ansible
3. microk8s
4. prometheus, grafana, node_exporter

Using five raspberry pi’s. One for an Ansible control to deploy software and four for our microk8s cluster

Hardware Setup

IP Allocation

I added address reservations on my router to automatically assign an IP to a known MAC address on a Raspberry pi. This ensures that the pi will have the same IP address each time it boots. No messing with the network configuration in the operating system every time I re-install the OS. Since this is technically a test environment where I will burn-and-build at will. I will not go through this excercise as there ae many ways to do this on a DHCP server.

To obtain the MAC aaddress, I first installed an OS on the pi and noted down the MAC.

OS Installation

Use raspberry pi imager. You can download the softwaere here https://www.raspberrypi.com/software/

Ansible

for automation

Copy Ansible keys to other hosts from cloudletcmd

ssh-copy-id -i ansible_rsa.pub pi@192.168.1.200
ssh-copy-id -i ansible_rsa.pub pi@192.168.1.201
ssh-copy-id -i ansible_rsa.pub pi@192.168.1.202
ssh-copy-id -i ansible_rsa.pub pi@192.168.1.203

ansible-playbook update.yml –ask-become-pass
ansible all -m ping

For microk8s

sudo nano /boot/firmware/cmdline.txt

add the following to the file
cgroup_enable=memory cgroup_memory=1

then reboot
sudo reboot

MicroK8s

sudo groupadd --system prometheus
sudo useradd -s /sbin/nologin --system -g prometheus prometheus
sudo apt update
sudo apt -y install wget curl vim
sudo apt autoremove
curl -s https://api.github.com/repos/prometheus/node_exporter/releases/latest | grep browser_download_url | grep linux-arm64 | cut -d '"' -f 4 | wget -qi -
tar xvf node_exporter-1.5.0.linux-armv7.tar.gz
cd node_exporter-1.5.0.linux-armv7/
sudo mv node_exporter /usr/local/bin
node_exporter --version

A basic service config for node_exporter:
sudo vim /etc/systemd/system/node_exporter.service

sudo systemctl daemon-reload; sudo systemctl start node_exporter; sudo systemctl enable node_exporter; sudo systemctl status node_exporter

Remove microk8s

sudo microk8s reset

sudo snap remove microk8s