Kubernetes Setup

Build a Cluster with

• multipass + k3s or talos + proxmox
• argocd gitops continuous delivery for k8s
• onepassword-operator via their helm chart for managing secrets
• superset via their helm chart for building dashboards etc.
• minio-operator via their helm chart for providing object storage
• eck-operator via there helm chart for standing up elasticsearch
• kube-prometheus-stack via their helm chart for monitoring
• longhorn providing storage for the cluster
• cloudflare zero trust tunnels exposing services externally (in lieu of ingress etc)

Bootstrap Cluster

Set the kubectl context to the cluster we are setting up.

set k8s_context_name default
# or
set k8s_context_name admin@talos-proxmox-cluster
kubectl config use-context $k8s_context_name

./bin/create-secret-onepassword

ArgoCD

Install

kubectl create namespace argocd \
  && kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml \
  && sleep 5 \
  && kubectl wait --for=condition=Ready pods --all -n argocd --timeout=300s

App of Apps

set env xcel-on-prem
kubectl apply -f apps/$env.yaml \
  && kubectl rollout restart deployment argocd-server --namespace argocd \
  && kubectl rollout status deployment/argocd-server --namespace argocd

Default ArgoCD Password

set ARGOCD_PASSWORD $(kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d)
echo $ARGOCD_PASSWORD | pbcopy

Environments

name	locaction	cluster type	host	description
local	local machine	orbstack	localhost	a local testing cluster
xcel-on-prem	trashcans	talos	proxmox	Talos cluster running in Proxmox on the trashcan
amfaro-dev	trashcans	talos	proxmox	Talos cluster running in Proxmox on the trashcan

Tasks

set-context

Inputs: NAME Inputs: EXTERNAL_DOMAIN

pgrep cloudflared | xargs kill -9 || true

cloudflared access tcp \
  --hostname k8s.$EXTERNAL_DOMAIN \
  --url 127.0.0.1:1234 \
  > /dev/null 2>&1 &

kubectl config use-context admin@$NAME-cluster