Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Default some unbound cluster/gce env vars

**What this PR does / why we need it**:

Sets defaults for two env vars used by cluster/gce/* scripts so as to
avoid the following warnings when bringing a cluster up for test
```
METADATA_CONCEALMENT_NO_FIREWALL: unbound variable
CUSTOM_KUBE_DASHBOARD_BANNER: unbound variable
```

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #60850

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

dd status=none does not exist on macOS

**What this PR does / why we need it**:

When running cluster/kubectl.sh on macOS 10.13.6, the use of the
`status=none` operand leads to `dd: unknown operand status` being
printed out as an error message. Redirecting to /dev/null does
the same thing, supressing transfer status.

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 66284, 66690). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Exit gce kube-up.sh early if openssl is LibreSSL

**What this PR does / why we need it**:

macOS has an openssl binary, but it's actually LibreSSL, which doesn't play well with the easyrsa script that cluster/gce/util.sh uses to generate certs

Instead of waiting until we generate certs to discover easyrsa doesn't work, consider openssl a prereq for gce, and include a check for the version string starting with OpenSSL

Also, mirror kube-up.sh's "... calling" output in kube-down.sh

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes kubernetes/community#1954

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Move the` k8s.io/kubernetes/pkg/util/pointer` package to` k8s.io/utils/pointer`

**What this PR does / why we need it**:
Move `k8s.io/kubernetes/pkg/util/pointer` to  `shared utils` directory, so that we can use it  easily.
Close #66010 accidentally, and can't reopen it, so the same as #66010 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix wrong format output in e2e log

**What this PR does / why we need it**:
When i run e2e, occuring  some  output log as below:
```
STEP: Creating pod1
STEP: Creating a pod
Jul 28 14:09:12.876: INFO: pod "security-context-bd6750a8-922c-11e8-abe9-286ed488d743" created on Node "127.0.0.1"
Jul 28 14:09:12.876: INFO: Running '/usr/local/bin/kubectl --server=http://127.0.0.1:8080 --kubeconfig=/var/run/kubernetes/admin.kubeconfig exec --namespace=e2e-tests-persistent-local-volumes-test-dlbf7 security-context-bd6750a8-922c-11e8-abe9-286ed488d743 -- /bin/sh -c cat /mnt/volume1/test-file'
Jul 28 14:09:13.119: INFO: stderr: ""
Jul 28 14:09:13.119: INFO: stdout: "test-file-content\n"
Jul 28 14:09:13.119: INFO: podRWCmdExec out: "test-file-content\n" err: %!q(<nil>)
STEP: Writing in pod1
Jul 28 14:09:13.119: INFO: Running '/usr/local/bin/kubectl --server=http://127.0.0.1:8080 --kubeconfig=/var/run/kubernetes/admin.kubeconfig exec --namespace=e2e-tests-persistent-local-volumes-test-dlbf7 security-context-bd6750a8-922c-11e8-abe9-286ed488d743 -- /bin/sh -c mkdir -p /mnt/volume1; echo /tmp/local-volume-test-b97fe819-922c-11e8-abe9-286ed488d743 > /mnt/volume1/test-file'
Jul 28 14:09:13.331: INFO: stderr: ""
Jul 28 14:09:13.331: INFO: stdout: ""
Jul 28 14:09:13.331: INFO: podRWCmdExec out: "" err: %!q(<nil>)
STEP: Deleting pod1
STEP: Deleting pod security-context-bd6750a8-922c-11e8-abe9-286ed488d743 in namespace e2e-tests-persistent-local-volumes-test-dlbf7
STEP: Creating pod2
STEP: Creating a pod
```
the `podRWCmdExec out: "" err: %!q(<nil>)` line are wrong
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 66489, 66728, 66739). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

remove incomplete check of ipvs modules in hack/local-up-cluster.sh

**What this PR does / why we need it**:
Currently `hack/local-up-cluster.sh` executes `sudo modprobe -a ip_vs ip_vs_rr ip_vs_wrr ip_vs_sh nf_conntrack_ipv4` to check whether the ipvs required modules exist, which leaves out the scenario https://github.com/kubernetes/kubernetes/issues/63801 mentioned.
Since `func CanUseIPVSProxier` in  `pkg/proxy/ipvs/proxier.go` covers all scenarios, maybe we should just remove this part instead of adding codes.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 66489, 66728, 66739). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Speed up volume modifications on AWS

Volume becomes reusable after it reached optimizing state.

/sig storage
/sig aws


```release-note
Make EBS volume expansion faster
```

cc @d-nishi @kokhang

Automatic merge from submit-queue (batch tested with PRs 66489, 66728, 66739). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Reuse iptablesContainerPortalArgs, remove function iptablesContainerNodePortArgs

**What this PR does / why we need it**:

reuse iptablesContainerPortalArgs, remove function iptablesContainerNodePortArgs

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Simplify device manager: make endpoint stateless

While reviewing devicemanager code, found the caching layer on endpoint is redundant.
Here are the 3 related objects in picture:

**devicemanager  <->  endpoint <-> plugin**

plugin is the source of truth for devices and device health status.
devicemanager maintain healthyDevices, unhealthyDevices, allocatedDevices based on updates
from plugin.

So there is no point for endpoint to cache devices, this patch is removing the cache layer,
endpoint becomes stateless, which i believe should be the case (but i do welcome review
if i missed something here).

also removing the Manager.Devices() since i didn't find any caller of this other than test.

if we need to get all devices from manager in future, it just need to return healthyDevices + unhealthyDevices, so don't have to call endpoint after all.

This patch makes code more readable, data model been simplified.



**What this PR does / why we need it**:
this patch simplify the device manager code, make it more maintainable.

**Which issue(s) this PR fixes** *:
this is a refactor of device manager code

**Special notes for your reviewer**:
will need to rebase the code if #58755 get checked-in first.

**Release note**:

```release-note
None
```

/sig node
/cc @jiayingz @RenaudWasTaken @vishh @saad-ali @vikaschoudhary16 @vladimirvivien @anfernee

Automatic merge from submit-queue (batch tested with PRs 66686, 66760). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Error in return value should be tested

**What this PR does / why we need it**:
Error in return value should be tested
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

test image for a release 1.7 based sample-apiserver

**What this PR does / why we need it**:
In our e2e test suite we have use an image "gcr.io/kubernetes-e2e-test-images:k8s-aggregator-sample-apiserver:1.7v2". We need a way to build a fresh image that can we can use instead of that one. Especially we need one that has a multi-arch fat manifest so e2e tests can be run across multiple architectures.

This is especially important since we are in the process of promoting the test in question to the conformance suite - https://github.com/kubernetes/kubernetes/pull/63947

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
/cc @mkumatag 
/cc @ixdy 
/cc @luxas 

**Release note**:

```release-note
NONE
```

the caching layer on endpoint is redundant.

Here are the 3 related objects in picture:
devicemanager <-> endpoint <-> plugin

Plugin is the source of truth for devices
and device health status.

devicemanager maintain healthyDevices,
unhealthyDevices, allocatedDevices based on updates
from plugin.

So there is no point for endpoint caching devices,
this patch is removing this caching layer on endpoint,

Also removing the Manager.Devices() since i didn't
find any caller of this other than test, i am adding a
notification channel to facilitate testing,

If we need to get all devices from manager in future,
it just need to return healthyDevices + unhealthyDevices,
we don't have to call endpoint after all.

This patch makes code more readable, data model been simplified.

Change-Id: I1429e4866cf20dd88781f37a124ccff65334f534

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Switch off leader election for scheduler and kube/cloud controller

**What this PR does / why we need it**:

We now have leader election on by default, for a single node
local-up-cluster, this is not needed. Let's switch it off

This will reduce the flakiness and timeouts we see in the local e2e CI jobs.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 66723, 66732). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix a duplicate case in TestUpdateNodeObjects

**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

enable etcd logging in local-e2e jobs

**What this PR does / why we need it**:

We are not saving the etcd logs and just redirecting the output
to /dev/null. In this change, we set ETCD_LOGFILE to the same
directory where we log other kube relates processes.


**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
See example run in http://gcsweb.k8s.io/gcs/kubernetes-jenkins/pr-logs/pull/66608/pull-kubernetes-local-e2e/254/artifacts/kubetest-local263115757/

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 62444, 66358, 66724, 66726). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Move kubelet serving cert rotation to beta

xref https://github.com/kubernetes/features/issues/267

This is exercised in the alpha gke e2es, and can be enabled in the non-alpha gke e2es once it no longer requires an alpha feature gate.

```release-note
Kubelet serving certificate bootstrapping and rotation has been promoted to beta status.
```

Automatic merge from submit-queue (batch tested with PRs 62444, 66358, 66724, 66726). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Grab docker log using a soft link in local-up-cluster

**What this PR does / why we need it**:
Would be useful to debug problems like timeouts and missing images etc
for the local e2e jobs.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 62444, 66358, 66724, 66726). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

GetMasterHost should not return port

**What this PR does / why we need it**:
masterUrl.Host may include port, e.g. 10.114.51.201:8443
GetMasterHost() should not return port

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
NONE

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

refactor server.go to simplify the invokes to kubeFlags and KubeConfiguration

**What this PR does / why we need it**:
Since kubeFlags and KubeletConfiguration have been fields of KubeletServer, we just need to pass the reference of KubeletServer to the following functions. This will simplify the migrations of flags such as BootstrapCheckpointPath and others, as we don't have to specify from which object the migrated field comes.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:
@mtaufen 

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 66593, 66727, 66558). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Mark --pod/-p flag of kubectl exec command as deprecated

**What this PR does / why we need it**:
Marks the `--pod` (`-p` shorthand) flag of kubectl `exec` command as deprecated.
Hides the flag from the help menu, but shows a message when command is executed with this flag.

**Which issue this PR fixes**:
Fixes:  https://github.com/kubernetes/kubectl/issues/104

This is a remake of PR https://github.com/kubernetes/kubernetes/pull/54629.

**Release note**:
```release-note
Flag --pod (-p shorthand) of kubectl exec command marked as deprecated
```

Automatic merge from submit-queue (batch tested with PRs 66593, 66727, 66558). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Bump version since we added dnsmasq to dnsutils image

Change-Id: Icf2335e95b0c5505be295b83f6d339269a8d418e



**What this PR does / why we need it**:
Forgot to bump version number in https://github.com/kubernetes/kubernetes/pull/66695

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 66593, 66727, 66558). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

remove the outdate comments in tryRegisterWithAPIServer

**What this PR does / why we need it**:
some judgement about ExternalID removed in #61877, so remove the outdate comments in tryRegisterWithAPIServer


**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 66225, 66648, 65799, 66630, 66619). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Implement InstanceShutdownByProviderID for vSphere CP

**What this PR does / why we need it**:
To detach volumes from shutdown nodes #66181

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 66225, 66648, 65799, 66630, 66619). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Bump to k8s.gcr.io/metadata-proxy:v0.1.10

This PR bumps the metadata-proxy to a version which is compatible with `gcloud`'s GCE Metadata access_token logic.

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 66225, 66648, 65799, 66630, 66619). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Percent-encode illegal characters in user.Info.Extra keys

This percent-encodes characters in `X-Remote-Extra-` and `Impersonate-Extra-` keys which aren't valid for header names per [RFC 7230](https://tools.ietf.org/html/rfc7230#section-3.2.6) (plus "%" to avoid breaking keys which contain them). The API server then blindly unescapes these keys.

Reviewer note:
Old clients sending keys which were `%`-escaped by the user will have their values unescaped by new API servers. New clients sending keys containing illegal characters (or "%") to old API servers will not have their values unescaped. This version skew incompatibility is a compromise discussed in #63682.

Fixes #63682

PTAL @mikedanese 

**Release note**:
```release-note
action required: the API server and client-go libraries have been fixed to support additional non-alpha-numeric characters in UserInfo "extra" data keys. Both should be updated in order to properly support extra data containing "/" characters or other characters disallowed in HTTP headers.
```

Automatic merge from submit-queue (batch tested with PRs 66225, 66648, 65799, 66630, 66619). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update Azure Go SDK to v19.0.0 and get availability zone for VirtualMachineScaleSetVM

**What this PR does / why we need it**:

Continue of #66242. This PR updates Azure Go SDK to v19.0.0 (with compute API 2018-04-01) and gets availability zones for VirtualMachineScaleSetVM.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
Azure Go SDK has been upgraded to v19.0.0 and VirtualMachineScaleSetVM now supports availability zones.
```

/sig azure
/assign @brendandburns @khenidak @andyzhangx

Automatic merge from submit-queue (batch tested with PRs 66225, 66648, 65799, 66630, 66619). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix error binding of edit output format

**What this PR does / why we need it**:
`kubectl edit xxx/xxx -o json` won't print result in json format
**Release note**:
```release-note
NONE
```

We now have leader election on by default, for a single node
local-up-cluster, this is not needed. Let's switch it off

Change-Id: I5cc4c3b9f74e318f378f239d1e8ee84603b3a6ef

Automatic merge from submit-queue (batch tested with PRs 58755, 66414). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Flexvolume e2e test for attachable driver: Introduced delay after pod…

… deletion to ensure detach happens



**What this PR does / why we need it**: Removes the possibility of verbose logs from controller manager caused by this issue.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #66408

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 58755, 66414). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use probe based plugin watcher mechanism in Device Manager

**What this PR does / why we need it**:
Uses this probe based utility in the device plugin manager.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56944 

**Notes For Reviewers**:
Changes are backward compatible and existing device plugins will continue to work. At the same time, any new plugins that has required support for probing model (Identity service implementation), will also work. 


**Release note**
```release-note
Add support kubelet plugin watcher in device manager.
```
/sig node
/area hw-accelerators
/cc /cc @jiayingz @RenaudWasTaken @vishh @ScorpioCPH @sjenning @derekwaynecarr @jeremyeder @lichuqiang @tengqm @saad-ali @chakri-nelluri @ConnorDoyle

Volume becomes reusable after it reached optimizing state