• Kubernetes Submit Queue's avatar
    Merge pull request #65594 from liggitt/node-csr-addresses-2 · 337dfe0a
    Kubernetes Submit Queue authored
    Automatic merge from submit-queue (batch tested with PRs 65052, 65594). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
    
    Derive kubelet serving certificate CSR template from node status addresses
    
    xref https://github.com/kubernetes/features/issues/267
    fixes #55633
    
    Builds on https://github.com/kubernetes/kubernetes/pull/65587
    
    * Makes the cloud provider authoritative when recording node status addresses
    * Makes the node status addresses authoritative for the kube-apiserver determining how to speak to a kubelet (stops paying attention to the hostname label when determining how to reach a kubelet, which was only done to support kubelets < 1.5)
    * Updates kubelet certificate rotation to be driven from node status
      * Avoids needing to compute node addresses a second time, and differently, in order to request serving certificates.
      * Allows the kubelet to react to changes in its status addresses by updating its serving certificate
      * Allows the kubelet to be driven by external cloud providers recording node addresses on the node status
    
    test procedure:
    ```sh
    # setup
    export FEATURE_GATES=RotateKubeletServerCertificate=true
    export KUBELET_FLAGS="--rotate-server-certificates=true --cloud-provider=external"
    
    # cleanup from previous runs
    sudo rm -fr /var/lib/kubelet/pki/
    
    # startup
    hack/local-up-cluster.sh
    
    # wait for a node to register, verify it didn't set addresses
    kubectl get nodes 
    kubectl get node/127.0.0.1 -o jsonpath={.status.addresses}
    
    # verify the kubelet server isn't available, and that it didn't populate a serving certificate
    curl --cacert _output/certs/server-ca.crt -v https://localhost:10250/pods
    ls -la /var/lib/kubelet/pki
    
    # set an address on the node
    curl -X PATCH http://localhost:8080/api/v1/nodes/127.0.0.1/status \
      -H "Content-Type: application/merge-patch+json" \
      --data '{"status":{"addresses":[{"type":"Hostname","address":"localhost"}]}}'
    
    # verify a csr was submitted with the right SAN, and approve it
    kubectl describe csr
    kubectl certificate approve csr-...
    
    # verify the kubelet connection uses a cert that is properly signed and valid for the specified hostname, but NOT the IP
    curl --cacert _output/certs/server-ca.crt -v https://localhost:10250/pods
    curl --cacert _output/certs/server-ca.crt -v https://127.0.0.1:10250/pods
    ls -la /var/lib/kubelet/pki
    
    # set an hostname and IP address on the node
    curl -X PATCH http://localhost:8080/api/v1/nodes/127.0.0.1/status \
      -H "Content-Type: application/merge-patch+json" \
      --data '{"status":{"addresses":[{"type":"Hostname","address":"localhost"},{"type":"InternalIP","address":"127.0.0.1"}]}}'
    
    # verify a csr was submitted with the right SAN, and approve it
    kubectl describe csr
    kubectl certificate approve csr-...
    
    # verify the kubelet connection uses a cert that is properly signed and valid for the specified hostname AND IP
    curl --cacert _output/certs/server-ca.crt -v https://localhost:10250/pods
    curl --cacert _output/certs/server-ca.crt -v https://127.0.0.1:10250/pods
    ls -la /var/lib/kubelet/pki
    ```
    
    ```release-note
    * kubelets that specify `--cloud-provider` now only report addresses in Node status as determined by the cloud provider
    * kubelet serving certificate rotation now reacts to changes in reported node addresses, and will request certificates for addresses set by an external cloud provider
    ```
    337dfe0a
Name
Last commit
Last update
..
src Loading commit data...
BUILD Loading commit data...
OWNERS Loading commit data...
README.md Loading commit data...