Running a Kubernetes Cluster Using Proxmox and Rancher

(Part 1)

Currently I have a Hyve Zeuss 1U server running in my home lab. It is a modest setup with 32GB of RAM and 32 CPU cores. In the future I will be looking to upgrade the RAM. In my previous post I mentioned that I was running pi-hole as “the spine” of my homelab. That pihole instance is running on my home server.

Since I am currently working for a company that uses kubernetes extensively, I decided to add kubernetes to my homelab as the runtime for all my services, internal and external.

After a lot of tinkering, testing, and botched kubernetes vms, I finally decided to take that knowledge and create a fast and reliable guide for running kubernetes on proxmox.

Here are the specs for the cluster we are going to create. You can scale each node up but I would not recommend scaling down on the RAM or CPU cores.

Step 1: Download Ubuntu Server 20.04

You will need the ISO for Ubuntu Server 20.04. You can grab it here. I tried using 21.10 and I ran into several problems.

Step 2: Create 3 (or more) virtual machines in Proxmox

You need to create at least 3 virtual machines. I have 4 since I wanted an extra worker node to handle my workloads. The GUI node will be hosting our Rancher GUI, the master node will be our etcd, scheduling, and control plane, and our worker nodes will run our workloads.

Step 3: Set-up, Update & Upgrade, Install Docker

Run through the Ubuntu set-up, log onto the server via SSH, update and upgrade, and finally install docker. For rancher, a specific version of docker seems to be needed. Use the following script to install docker:

curl https://releases.rancher.com/install-docker/20.10.sh | sh

Step 4: Set up Rancher K3s

Time to set up your Rancher K3s cluster. This “cluster” (it really isn’t) is the management cluster for your kubernetes cluster. The setup process is easy.

Run the following command on your GUI node:

sudo docker run --privileged -d --restart=unless-stopped -p 80:80 -p 443:443 rancher/rancher

This command will start rancher in a docker container on your GUI node.

After the docker container is up, you can hit the internal IP of the GUI node at https://<INTERNAL_IP>:443

You will be welcomed by a screen asking you to create a password. run the command show and create an admin password.

After you have signed in, you will be greeted by the Rancher Dashboard.

Step 5: Create your Kubernetes Cluster

Now it’s time to create your cluster.

Click the “Create button.

Click the “Custom” option.

Name your cluster

Keep these options to their defaults.

Change the “Nginx Ingress” to disabled. We are going to install traefik later.

Click “Next” and continue.

Step 6: Join your master and worker nodes to the cluster.

Now it’s time to add your master and worker nodes to your new cluster.

To register your master node, make sure these boxes are checked. Click on the command to copy to your clipboard and paste it into the terminal with your SSH session to the master node. (You can use your master node as a worker node as well. I would not recommend this.

To register your worker nodes, make sure these boxes are checked. Click on the command to copy to your clipboard and paste it into the terminal with your SSH session to the worker nodes.

Step 7: Wait

The registration process can take a few minutes. one of my clusters took 10 minutes to create. Be patient. Get some coffee and chill. You will see it hang at “Waiting for API” for the longest amount of time.

Step 8: You are done!

Well done! We are done creating our cluster. HOWEVER, we still have to add a few components before we can start self-hosting workloads. Make sure to read Part 2 of this series to learn how to use Metallb, Longhorn, and Traefik.