[release-1.20] - Backport add etcd backup restore s3 1.20

* Update to Kubernetes v1.20.5
* vendor: bumps for some containerd deps
* go: bump to 1.16.2 for arm
Signed-off-by: Jacob Blain Christen <jacob@rancher.com>

Signed-off-by: Brian Downs <brian.downs@gmail.com>

Addresses k3s-io/k3s#3067 and CVE-2021-21334
Signed-off-by: Jacob Blain Christen <jacob@rancher.com>

Signed-off-by: Brian Downs <brian.downs@gmail.com>

* have state stored in etcd at completed start and remove unneeded code
Signed-off-by: Brian Downs <brian.downs@gmail.com>
(cherry picked from commit 7c99f864)

* Add functionality for etcd snapshot/restore to and from S3 compatible backends.
* Update etcd restore functionality to extract and write certificates and configs from snapshot.
Signed-off-by: Brian Downs <brian.downs@gmail.com>
(cherry picked from commit 4d1f9eda)

* add ability to perform an etcd on-demand snapshot via cli
Signed-off-by: Brian Downs <brian.downs@gmail.com>
(cherry picked from commit 13229019)

Signed-off-by: Chris Kim <oats87g@gmail.com>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
(cherry picked from commit e628fe9280e8cb577973e1a8951dc80f04db5d88)

Servers should always be upgraded before agents, but generally this
isn't required because things are compatible between versions. In this
case we're OK with failing closed if the user upgrades out of order, but
we should give a clearer message about what steps are required to fix
the issue.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
(cherry picked from commit 31bbae1eb1f0372752a0479f19a3ce49ac30a24d)

We have had a couple issues with newer agents not working with old
servers or vice versa. Add a CI test to test variations on
uplevel/downlevel server/agent against latest, stable, and the previous
branch.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
(cherry picked from commit 357c375d04737ed8300c9b5caaf25c7966a85013)

* Add disable flags for control components (#2900)

* Add disable flags to control components
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* golint
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* more fixes
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fixes to disable flags
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Add comments to functions
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Fix joining problem
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* more fixes
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* golint
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fix ticker
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* fix role labels
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* more fixes
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* remove etcd member if disable etcd is passed
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* update dynamiclistener
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* change error to warn when removing self from etcd members
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Mark disable components flags as experimental
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Update to v1.20.4
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

* Update to v1.20.4
Signed-off-by: galal-hussein <hussein.galal.ahmed.11@gmail.com>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
(cherry picked from commit 457daeadd63c2001e68488f4cb0a84ca7ef1c4db)

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

* Collect IPs from all pods before deciding to use internal or external addresses

@Taloth correctly noted that the code that iterates over ServiceLB pods
to collect IP addresses was failing to add additional internal IPs once
the map contained ANY entry from a previous node. This may date back to
when ServiceLB used a Deployment instead of a DaemonSet, so there was
only ever a single pod.

The new behavior is to collect all internal and external IPs, and then
construct the address list of a single type - external if there are any,
otherwise internal.

https://github.com/k3s-io/k3s/issues/1652#issuecomment-774497788Signed-off-by: Brad Davidson <brad.davidson@rancher.com>
Co-authored-by: Brian Downs <brian.downs@gmail.com>

* Improve handling of comounted cpu,cpuacct controllers
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

Add opensuse 15 vagrant provision

* Add tests to clientaccess/token
* Fix issues in clientaccess/token identified by tests
* Update tests to close coverage gaps
* Remove redundant check turned up by code coverage reports
* Add warnings if CA hash will not be validated
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

We also need to be more careful about setting the crictl.yaml path,
as it doesn't have kubectl's nice behavior of checking multiple
locations. It's not safe to assume that it's in the user's home data-dir
just because we're not running as root.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

* Update local-path-provisioner and helper busybox
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

Previously, k3s.service was failing when the EnvironmentFile does not exist:
```
Feb 02 17:17:30 suda-ws01 systemd[1]: k3s.service: Failed to load environment files: No such file or directory
Feb 02 17:17:30 suda-ws01 systemd[1]: k3s.service: Failed to run 'start' task: No such file or directory
Feb 02 17:17:30 suda-ws01 systemd[1]: k3s.service: Failed with result 'resources'.
Feb 02 17:17:30 suda-ws01 systemd[1]: Failed to start Lightweight Kubernetes.
```

ref: https://unix.stackexchange.com/questions/404199/documentation-of-equals-minus-in-systemd-unit-filesSigned-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

Fix issue 900

cgroup2 support was introduced in PR 2584, but got broken in f3de60ff

It was failing with "F1210 19:13:37.305388    4955 server.go:181] cannot set feature gate SupportPodPidsLimit to false, feature is locked to true"
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

Since a recent commit, rootless mode was failing with the following errors:

```
E0122 22:59:47.615567      21 kuberuntime_manager.go:755] createPodSandbox for pod "helm-install-traefik-wf8lc_kube-system(9de0a1b2-e2a2-4ea5-8fb6-22c9272a182f)" failed: rpc error: code = Unknown desc = failed to create network namespace for sandbox "285ab835609387f82d304bac1fefa5fb2a6c49a542a9921995d0c35d33c683d5": failed to setup netns: open /var/run/netns/cni-c628a228-651e-e03e-d27d-bb5e87281846: permission denied
...
E0122 23:31:34.027814      21 pod_workers.go:191] Error syncing pod 1a77d21f-ff3d-4475-9749-224229ddc31a ("coredns-854c77959c-w4d7g_kube-system(1a77d21f-ff3d-4475-9749-224229ddc31a)"), skipping: failed to "CreatePodSandbox" for "coredns-854c77959c-w4d7g_kube-system(1a77d21f-ff3d-4475-9749-224229ddc31a)" with CreatePodSandboxError: "CreatePodSandbox for pod \"coredns-854c77959c-w4d7g_kube-system(1a77d21f-ff3d-4475-9749-224229ddc31a)\" failed: rpc error: code = Unknown desc = failed to create containerd task: io.containerd.runc.v2: create new shim socket: listen unix /run/containerd/s/8f0e40e11a69738407f1ebaf31ced3f08c29bb62022058813314fb004f93c422: bind: permission denied\n: exit status 1: unknown"
```

Remove symlinks to /run/{netns,containerd} so that rootless mode can create their own /run/{netns,containerd}.
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

* Add functionality to bind custom IP address for Etcd metrics endpoint
Signed-off-by: yuriydzobak <yurii.dzobak@lotusflare.com>

* Replace k3s cloud provider wrangler controller with core node informer

Upstream k8s has exposed an interface for cloud providers to access the
cloud controller manager's node cache and shared informer since
Kubernetes 1.9. This is used by all the other in-tree cloud providers;
we should use it too instead of running a dedicated wrangler controller.

Doing so also appears to fix an intermittent issue with the uninitialized
taint not getting cleared on nodes in CI.
Signed-off-by: Brad Davidson <brad.davidson@rancher.com>

Signed-off-by: Brad Davidson <brad.davidson@rancher.com>