• Kubernetes Submit Queue's avatar
    Merge pull request #63887 from luxas/kubeadm_kubelet_integration · 773ced14
    Kubernetes Submit Queue authored
    Automatic merge from submit-queue (batch tested with PRs 63914, 63887, 64116, 64026, 62933). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
    
    kubeadm: Write kubelet config file to disk and persist in-cluster
    
    **What this PR does / why we need it**:
    In order to make configuration flow from the cluster level to node level, we need a way for kubeadm to tell the kubelet what config to use. As of v1.10 (I think) the kubelet can read `--config` using the kubelet Beta ComponentConfiguration API, so now we have an interface to talk to the kubelet properly.
    
    This PR:
     - Writes the kubelet ComponentConfig to `/var/lib/kubelet/config.yaml` on init and join
     - Writes an environment file to source in the kubelet systemd dropin `/var/lib/kubelet/kubeadm-flags.env`. This file contain runtime flags that should be passed to the kubelet.
     - Uploads a ConfigMap with the name `kubelet-config-1.X`
     - Patches the node object so that it starts using the ConfigMap with updates using Dynamic Kubelet Configuration, **only if the feature gate is set** (currently alpha and off by default, not intended to be switched on in v1.11)
     - Updates the phase commands to reflect this new flow
    
    The kubelet dropin file I used now looks like this:
    ```
    # v1.11.x dropin as-is at HEAD
    # /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
    ---
    [Service]
    Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
    Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
    EnvironmentFile-=/var/lib/kubelet/kubeadm-flags.env
    # Should default to 0 in v1.11: https://github.com/kubernetes/kubernetes/pull/63881, and hence not be here in the real v1.11 manifest
    Environment="KUBELET_CADVISOR_ARGS=--cadvisor-port=0"
    # Should be configurable via the config file: https://github.com/kubernetes/kubernetes/issues/63878, and hence be configured using the file in v1.11
    Environment="KUBELET_CERTIFICATE_ARGS=--rotate-certificates=true"
    ExecStart=
    ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_CADVISOR_ARGS $KUBELET_CERTIFICATE_ARGS $KUBELET_EXTRA_ARGS
    ---
    # v1.11.x dropin end goal
    # /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
    ---
    [Service]
    Environment="KUBELET_KUBECONFIG_ARGS=--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf"
    Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
    EnvironmentFile-=/var/lib/kubelet/kubeadm-flags.env
    ExecStart=
    ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_CONFIG_ARGS $KUBELET_KUBEADM_ARGS $KUBELET_EXTRA_ARGS
    ---
    # Environment file dynamically created at runtime by "kubeadm init"
    # /var/lib/kubelet/kubeadm-flags.env
    KUBELET_KUBEADM_ARGS=--cni-bin-dir=/opt/cni/bin --cni-conf-dir=/etc/cni/net.d --network-plugin=cni
    ```
    
    **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
    Fixes https://github.com/kubernetes/kubeadm/issues/822
    Fixes https://github.com/kubernetes/kubeadm/issues/571
    
    **Special notes for your reviewer**:
    
    **Release note**:
    
    ```release-note
    "kubeadm init" now writes a structured and versioned kubelet ComponentConfiguration file to `/var/lib/kubelet/config.yaml` and an environment file with runtime flags (you can source this file in the systemd kubelet dropin) to `/var/lib/kubelet/kubeadm-flags.env`.
    ```
    @kubernetes/sig-cluster-lifecycle-pr-reviews @mtaufen
    773ced14
Name
Last commit
Last update
..
clicheck Loading commit data...
cloud-controller-manager Loading commit data...
controller-manager Loading commit data...
gendocs Loading commit data...
genkubedocs Loading commit data...
genman Loading commit data...
genswaggertypedocs Loading commit data...
genutils Loading commit data...
genyaml Loading commit data...
hyperkube Loading commit data...
importverifier Loading commit data...
kube-apiserver Loading commit data...
kube-controller-manager Loading commit data...
kube-proxy Loading commit data...
kube-scheduler Loading commit data...
kubeadm Loading commit data...
kubectl Loading commit data...
kubelet Loading commit data...
kubemark Loading commit data...
linkcheck Loading commit data...
BUILD Loading commit data...
OWNERS Loading commit data...