Added a semicolon on line 10 after "hosts" to improve sentence structure and prevent confusion.

Automatic merge from submit-queue (batch tested with PRs 58539, 58685). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Openstack: Fill size attribute for the V3 API volumes

The getVolume method in OpenStack provider is not filling the Size for the V3 API type volumes. This
breaks the PV resizing of Cinder volumes which compares the existing volume size with the new request. This leads to redundant volume resize calls to the cloud provider that end with errors.

cc: @gnufied

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 58539, 58685). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Force use of Makefile for update

**What this PR does / why we need it**:
This forces the use of `make update` instead of `hack/update-all.sh`. The main reason for this is ensuring that `make generated_files` runs first, as the files it generates are used in some of the other update scripts. I've seen this a couple times bite contributors where they just haven't regenerated the code yet, so the update-all scripts don't fully do their job.

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 58480, 58549). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Use backup location to load cloud config for OpenStack

**What this PR does / why we need it**:

Since we are transitioning to external cloud provider, we need a way
to use the existing cinder volume plugin (from kubelet). With external
cloud manager kubelet will be run with --cloud=provider=external and
no --cloud-config file will be in the command line. So we need a way
to load the openstack config file from somewhere.

Taking a cue from kubeadm, which currently is picking up "/etc/kubernetes/cloud-config"
https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/phases/controlplane/manifests.go#L44

let's support the scenario where we fall back to this static location if
there is no cloud provider specified in the command line.

This has been tested with local-up-cluster using the following params:
EXTERNAL_CLOUD_PROVIDER=true
CLOUD_PROVIDER=openstack
CLOUD_CONFIG=/etc/kubernetes/cloud-config


**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 58480, 58549). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove apiVersion from scheduler extender example configuration

Follow up to the discussion in #58440



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

The getVolume method in OpenStack provider is not
filling the Size for the V3 API type volumes. This
breaks the PV resizing of Cinder volumes.

Automatic merge from submit-queue (batch tested with PRs 58438, 58523, 58513, 57948). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix csi ut print

**What this PR does / why we need it**:
fix csi ut print

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 58438, 58523, 58513, 57948). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add Namespace to glusterfs custom volume names.

For better isolation of volumes, users/admins would like to include `namespace` in custom volume names. This patch adds that support.
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>



**What this PR does / why we need it**:

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note

```

Automatic merge from submit-queue (batch tested with PRs 58438, 58523, 58513, 57948). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

network spaces support for kubernetes-worker charm

**What this PR does / why we need it**:
Adds support for network spaces in the kubernetes-worker charm.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/406
**Special notes for your reviewer**:

**Release note**:

```release-note
Added support for network spaces in the kubernetes-worker charm
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix apiserver crash caused by nil pointer

**What this PR does / why we need it**:

1. $ref should be not supported
2. redefine CRClientGetter interface 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #58427

**Release note**:


```release-note
CustomResourceDefinitions: OpenAPI v3 validation schemas containing `$ref`references are no longer permitted (valid references could not be constructed previously because property ids were not permitted either). Before upgrading, ensure CRD definitions do not include those `$ref` fields.
```

Automatic merge from submit-queue (batch tested with PRs 58590, 58667). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix logs message formating

**What this PR does / why we need it**:

Fix logs message formating.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

fix the wrong err print of assumepod

**What this PR does / why we need it**:
I think the err print is wrong, just opposite the original meaning.
/cc @timothysc 

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 57867, 58490, 58502, 58134). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add additional unit tests for Azure cloud provider.

@feiskyer @andyzhangx @khenidak

Automatic merge from submit-queue (batch tested with PRs 57867, 58490, 58502, 58134). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Openstack: register metadata.hostname as node name

**What this PR does / why we need it**:
Currently Openstack can boot up instances with the name like `xyz/abc`, which is not a valid kubelet node name. While `hostname` retrieved from `meta_data.json` has already been sanitized 
 by Openstack to valid DNS-1123 format string. It's safe to register this `metadata.hostname` as valid kubelet node name.

/kind bug
/sig openstack

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #57765

**Special notes for your reviewer**:
/assign @dims @FengyunPan 

**Release note**:

```release-note
Openstack: register metadata.hostname as node name
```

Automatic merge from submit-queue (batch tested with PRs 57867, 58490, 58502, 58134). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

GCE: neg to use generated code

GCE: neg to use generated code

```release-note
NONE
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

trace patch operations

Just like `update`, `create`, `get` and `delete` operations.

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 58412, 56132, 58506, 58542, 58394). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

don't stop informer delivery on error

If an informer delivery fails today, we stop delivering to it entirely.  The pull updates the code to skip that particular notification, delay, and continue delivery with the next time.

/assign derekwaynecarr 
/assign ncdc
/assign ash2k

@derekwaynecarr This would change the "the controller isn't doing anything?!" to "the controller missed my (individual) resource!"


```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 58412, 56132, 58506, 58542, 58394). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Adding support for changing default backend and nginx container images

**What this PR does / why we need it**:
Adding support for changing the images used by kubernetes worker for nginx and the default backend.
**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #
https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/470
**Special notes for your reviewer**:

**Release note**:

```release-note
Added nginx and default backend images to kubernetes-worker config.
```

Automatic merge from submit-queue (batch tested with PRs 58412, 56132, 58506, 58542, 58394). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Update Instances to use generated code

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 58412, 56132, 58506, 58542, 58394). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

 Implement upgrade-aliases.sh to migrate a route-based k8s cluster to use IP aliases in GCE.

**What this PR does / why we need it**:
This PR implements upgrade-aliases.sh, which is used to migrate a route-based k8s cluster to use IP aliases in GCE.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
"NONE"
```

Automatic merge from submit-queue (batch tested with PRs 58412, 56132, 58506, 58542, 58394). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Improve the upgrade test for ingress.

**What this PR does / why we need it**:
This PR improves the existing upgrade e2e test for ingress-gce. Specifically, we add a test which upgrades ingress with a image built from HEAD of the ingress-gce repo. 

```release-note
None
```

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix autoscaler deployment bug

**What this PR does / why we need it**:
This PR add `selector` in yaml: dns-horizontal-autoscaler.yaml, and upgrade the `apiVersion` of `Deployment` to `apps/v1`

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #

**Special notes for your reviewer**:

```release-note
None
```

Automatic merge from submit-queue (batch tested with PRs 58547, 57228, 58528, 58499, 58618). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Don't run godep restore in jenkins verify

As far as I can tell, this is not needed any more, and just causes a huge slowdown on every verify run.  

/sig testing

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 58547, 57228, 58528, 58499, 58618). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove getOldSecurityGroupName() from OpenStack cloud provider

Related to #53764
The getOldSecurityGroupName() is used to get the old security
group name, we can remove it now.

**What this PR does / why we need it**:
#53764

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 58547, 57228, 58528, 58499, 58618). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add TLS min version flag

Adds a flag for controlling the minimum TLS level allowed.

/assign liggitt

@kubernetes/sig-node-pr-reviews @k8s-mirror-api-machinery-pr-reviews 

```release-note
--tls-min-version on kubelet and kube-apiserver allow for configuring minimum TLS versions
```

Automatic merge from submit-queue (batch tested with PRs 58547, 57228, 58528, 58499, 58618). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

pass down runtime-config through apiserver chain

**What this PR does / why we need it**:

>kube-apiserver is actually a delegation chain of aggregator+kube+apiextensions. Let's pass down runtime-config through the chain, each layer removing the groups it knows about.

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #57189

**Special notes for your reviewer**:

make a new pkg `k8s.io/apiserver/pkg/server/resourceconfig`, and 
1. move resourceconfig related code to it. 
2. abstract generic used function `MergeAPIResourceConfigs`, put it here.

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 58547, 57228, 58528, 58499, 58618). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Send correct resource version for delete events from watch cache

Fixes #58545 

the watch cache filtering is returning the previous object content intact, including resource version. this is the logic the watch cache uses:
```go
switch {
case curObjPasses && !oldObjPasses:
	watchEvent = watch.Event{Type: watch.Added, Object: event.Object.DeepCopyObject()}
case curObjPasses && oldObjPasses:
	watchEvent = watch.Event{Type: watch.Modified, Object: event.Object.DeepCopyObject()}
case !curObjPasses && oldObjPasses:
	watchEvent = watch.Event{Type: watch.Deleted, Object: event.PrevObject.DeepCopyObject()}
}
```

when processing a delete event, we should be sending the old object's content *but* with the event's resource version set in it. corresponding logic exists in the uncached stores:

https://github.com/kubernetes/kubernetes/blob/77ac663df427d1ae0cb45adb0a3eba263809c837/staging/src/k8s.io/apiserver/pkg/storage/etcd/etcd_watcher.go#L401-L403

https://github.com/kubernetes/kubernetes/blob/77ac663df427d1ae0cb45adb0a3eba263809c837/staging/src/k8s.io/apiserver/pkg/storage/etcd3/watcher.go#L373-L378


```release-note
Fixes an issue where the resourceVersion of an object in a DELETE watch event was not the resourceVersion of the delete itself, but of the last update to the object. This could cause clients receiving the watch event to revert to an old "last observed" resource version, and disrupt their ability to re-establish watches properly.
```

Automatic merge from submit-queue (batch tested with PRs 58144, 57149). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

make quota reusable

Quota is logically a re-useable component so that replenishment and admission can be done in constituent servers.  This makes it possible to embed.  More cleanup is probably required, but this is the minimum.

Automatic merge from submit-queue (batch tested with PRs 58144, 57149). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fixed crash in kubectl cp when path has multiple leading slashes

**What this PR does / why we need it**: Fixed crash in kubectl cp when path has multiple leading slashes
For example:
```
$ kubectl cp pod://path/to/somewhere localfile
tar: Removing leading `//' from member names
panic: runtime error: slice bounds out of range

goroutine 1 [running]:
k8s.io/kubernetes/pkg/kubectl/cmd.untarAll(0x363e840, 0xc420198120, 0x7fff5fbffc13, 0x1, 0x7fff5fbffbfa, 0x18, 0x0, 0x0)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/cp.go:329 +0x7b4
k8s.io/kubernetes/pkg/kubectl/cmd.copyFromPod(0x3671940, 0xc420261dd0, 0xc4202f9200, 0x3644480, 0xc42000c020, 0x0, 0x0, 0x7fff5fbffbed, 0xb, 0x7fff5fbffbf9, ...)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/cp.go:237 +0x375
k8s.io/kubernetes/pkg/kubectl/cmd.runCopy(0x3671940, 0xc420261dd0, 0xc4202f9200, 0x3644480, 0xc42000c018, 0x3644480, 0xc42000c020, 0xc4203490a0, 0x2, 0x2, ...)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/cp.go:134 +0x35b
k8s.io/kubernetes/pkg/kubectl/cmd.NewCmdCp.func1(0xc4202f9200, 0xc4203490a0, 0x2, 0x2)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/kubectl/cmd/cp.go:72 +0x89
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).execute(0xc4202f9200, 0xc420348ee0, 0x2, 0x2, 0xc4202f9200, 0xc420348ee0)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:603 +0x234
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc42038d440, 0x8000104, 0x0, 0xffffffffffffffff)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:689 +0x2fe
k8s.io/kubernetes/vendor/github.com/spf13/cobra.(*Command).Execute(0xc42038d440, 0xc420261dd0, 0x3644440)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/spf13/cobra/command.go:648 +0x2b
k8s.io/kubernetes/cmd/kubectl/app.Run(0x0, 0x0)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubectl/app/kubectl.go:41 +0xd5
main.main()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubectl/kubectl.go:27 +0x26
```

**Which issue(s) this PR fixes** None

**Special notes for your reviewer**: None

**Release note**:
```release-note
Fixed crash in kubectl cp when path has multiple leading slashes
```

Update LoadBalancer to use generated code

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Remove op field as it is no longer needed

```release-note
NONE
```

Since we are transitioning to external cloud provider, we need a way
to use the existing cinder volume plugin (from kubelet). With external
cloud manager kubelet will be run with --cloud=provider=external and
no --cloud-config file will be in the command line. So we need a way
to load the openstack config file from somewhere.

Taking a cue from kubeadm, which currently is picking up "/etc/kubernetes/cloud-config"
https://github.com/kubernetes/kubernetes/blob/master/cmd/kubeadm/app/phases/controlplane/manifests.go#L44

let's support the scenario where we fall back to this static location if
there is no cloud provider specified in the command line.

This has been tested with local-up-cluster using the following params:
EXTERNAL_CLOUD_PROVIDER=true
CLOUD_PROVIDER=openstack
CLOUD_CONFIG=/etc/kubernetes/cloud-config