Update local-rke2-state secret type and configure API URLs in client secret

[bk201]

Fixes:

• Can't bootstrap clusters with Rancher v2.7.5-rc2 #30
  • Fleet needs API server URL and CA certs to be set in the client secret:
    • apiServerURL: value of Rancher setting "internal-api-url"
    • apiServerCA: value of Rancher setting "internal-cacerts"
• rancherd is creating the local-rke-state secret as type Opaque instead of the correct type rke.cattle.io/cluster-state  #29
  • Update secret type.

[ibrokethecloud]

LGTM.

[guangbochen]

Testing this PR with the customized rancherd and encountering the following error:

rancherd info
    Rancher:    v2.7.5-rc4
    Kubernetes: v1.25.9+rke2r1
    Rancherd:   a24febb (a24febb)

===
INFO[0824] [stderr]: error: timed out waiting for the condition on clusters/local
INFO[0824] [stderr]: time="2023-06-07T14:52:51Z" level=error msg="will retry failed command [/var/lib/rancher/rke2/bin/kubectl -n fleet-local wait --for=condition=Provisioned=true clusters.provisioning.cattle.io local]: exit status 1"
INFO[0869] [stderr]: error: timed out waiting for the condition on clusters/local
INFO[0869] [stderr]: time="2023-06-07T14:53:37Z" level=error msg="will retry failed command [/var/lib/rancher/rke2/bin/kubectl -n fleet-local wait --for=condition=Provisioned=true clusters.provisioning.cattle.io local]: exit status 1"

opensuse:/etc/rancher/rancherd # ks get machines -A
NAMESPACE     NAME                  CLUSTER   NODENAME   PROVIDERID        PHASE          AGE     VERSION
fleet-local   custom-0865dbf259e7   local                                  Provisioning   5m31s
fleet-local   custom-3abc76723d12   local     opensuse   rke2://opensuse   Running        3m33s

opensuse:/etc/rancher/rancherd # ks get node -A
NAME         STATUS     ROLES                       AGE     VERSION
opensuse     Ready      control-plane,etcd,master   11m     v1.25.9+rke2r1
opensuse-2   NotReady   <none>                      3m10s   v1.25.9+rke2r1

config.yaml

role: cluster-init
token: "aa bb cc"
kubernetesVersion: v1.25.9+rke2r1
rancherVersion: v2.7.5-rc4
rancherInstallerImage: rancher/system-agent-installer-rancher:v2.7.5-rc4
labels:
 - harvesterhci.io/managed=true
extraConfig:
  disable:
  - rke2-snapshot-controller
  - rke2-snapshot-controller-crd
  - rke2-snapshot-validation-webhook

rancherValues:
  rancherImagePullPolicy: IfNotPresent
  rancherImage: rancher/rancher
  rancherImageTag: v2.7.5-rc4
  noDefaultAdmin: false
  features: multi-cluster-management=false,multi-cluster-management-agent=false
  useBundledSystemChart: true
  bootstrapPassword: admin
  global:
    cattle:
      psp:
        enabled: false

cluster.yaml.zip

[bk201]

It seems related to the calico provision:

  - lastTransitionTime: "2023-06-07T14:48:42Z"
    message: 'waiting for probes: calico'
    reason: Waiting
    status: Unknown
    type: Reconciled

maybe we can test by specifying canal like Harvester:

https://github.com/harvester/vagrant-rancherd/blob/8b8d7964c550561ea494f2c9cb21829676de1bdf/Vagrantfile#L76-L81

[guangbochen]

LGTM, this works well with the Canal network, the default Calico network will encounter rancher/rancher#41191 with Rancher v2.7+.

os1:/etc/rancher/rke2 # ks get node
NAME   STATUS   ROLES                       AGE    VERSION
os1    Ready    control-plane,etcd,master   7m4s   v1.25.9+rke2r1
os1:/etc/rancher/rke2 # ks get machines -A
NAMESPACE     NAME                  CLUSTER   NODENAME   PROVIDERID   PHASE     AGE   VERSION
fleet-local   custom-ba180b5832ac   local     os1        rke2://os1   Running   60s
os1:/etc/rancher/rke2 # ks get clusters -A
NAMESPACE     NAME    READY   KUBECONFIG
fleet-local   local   true    local-kubeconfig
os1:/etc/rancher/rke2 # ks get pod -n cattle-system
NAME                                                              READY   STATUS      RESTARTS   AGE
apply-system-agent-upgrader-on-os1-with-ec2ce2f8ad2d7b9ad-x45xt   0/1     Completed   0          76s
helm-operation-lrfcn                                              0/2     Completed   0          4m23s
helm-operation-pxwfg                                              0/2     Completed   0          5m2s
helm-operation-zsgfk                                              0/2     Completed   0          4m32s
rancher-64d44d6788-dctqw                                          1/1     Running     0          7m1s
rancher-webhook-5d9dd9cc94-kxmx9                                  1/1     Running     0          4m19s
system-upgrade-controller-74db4c9dd-725j8                         1/1     Running     0          113s