Automatic merge from submit-queue (batch tested with PRs 49420, 49296, 49299, 49371, 46514)

Refactoring taint functions to reduce sprawl

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #45060

**Special notes for your reviewer**:
@gmarek @timothysc @k82cn @jayunit100 - I moved some fn's to helpers and some to utils. LMK, if you are ok with this change.

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 49420, 49296, 49299, 49371, 46514)

Fix: PV metric is not namespaced

**What this PR does / why we need it**: The PV metric of juju deployments is not namespaced. This PR fixes this bug. 

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/348

**Special notes for your reviewer**:

**Release note**:

```NONE
```

Automatic merge from submit-queue (batch tested with PRs 49420, 49296, 49299, 49371, 46514)

Avoid looking up instance id until we need it

**What this PR does / why we need it**:

currently kube-controller-manager cannot run outside of a vm started
by openstack (with --cloud-provider=openstack params). We try to read
the instance id from the metadata provider or the config drive or the
file location only when we really need it. In the normal scenario, the
controller-manager uses the node name to get the instance id.
https://github.com/kubernetes/kubernetes/blob/41541910e1699975a8f9202a89b6865e45921194/pkg/volume/cinder/attacher.go#L149

The localInstanceID is currently used only in the test case, so let
us not read it until it is really needed.

So let's try to find the instance-id only when we need it.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 49420, 49296, 49299, 49371, 46514)

Update status to show failing services.

**What this PR does / why we need it**: Report on charm status any services that are not running.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/341

**Special notes for your reviewer**:

**Release note**:

```Report failing services in Juju deployed clusters.
```

Automatic merge from submit-queue (batch tested with PRs 49420, 49296, 49299, 49371, 46514)

Fix controller crash because of nil volume spec

For volumes that don't support bulk volume verification, a nil
volume spec can cause crash of controller.

Fixes #49418

**Release note**:
```release-note
Fixes #49418 where kube-controller-manager can panic on volume.CanSupport methods and enter a crash loop.
```

Automatic merge from submit-queue (batch tested with PRs 46210, 48607, 46874, 46598, 49240)

Remove types.generated.go generated for internal API types

We don't need generated codecs for internal types.

Automatic merge from submit-queue (batch tested with PRs 46210, 48607, 46874, 46598, 49240)

Make "kubectl version" json format output more readable.

**What this PR does / why we need it**:
##39858 adds a flag --output to `kubectl version`, but the json format output is displayed in one line. It's not so readable. This PR fixes it.

and

- adds a shorthand for `output`
- ~~refactors that: if `--short` is specified, `--output` will be ignored~~

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #43750

**Special notes for your reviewer**:
/cc @php-coder @alejandroEsc 

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 46210, 48607, 46874, 46598, 49240)

Ipv6 support for ChooseHostInterface (part 3 of 3)

**What this PR does / why we need it**:
Provides IPv6 support for the ChooseHostInterface() and ChooseBindAddress() functions
in the apimachinery's util/net/interface.go file. It strives to maintain backward compatibility with existing use of IPv4, but allow IPv6 to be used when available.  This is part 3 of a set of PRs, and has been rebased on top of #46138, which is rebased on top of #46044.

This is part of an overall effort to provide IPv6 support in the code, by addressing places where IPv4 logic exists and making it IPv6 ready.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #44848

**Special notes for your reviewer**:
You can look at the other PRs for the commit description of what each of them does. I do see some unrelated test failures in those commits - appears to be flakes.

**Release note**:

```release-noteNONE
```

Automatic merge from submit-queue (batch tested with PRs 46210, 48607, 46874, 46598, 49240)

kubeadm: Make the hostPath volume mount code more secure

**What this PR does / why we need it**:

 - Refactors the hostpath volume mounting code for the Static Pods
 - Splits out the functionality that was in a big function to something testable
 - Unit test a lot
 - Adds support for mounting external etcd CA/cert/key files in an other path than `/etc/ssl/certs`. Before this you **had** to have your files in there or the apiserver would crashloop
 - Significantly improves comment coverage
 - Now only mounts the bare essentials instead of nearly everything. For example, don't mount full `/etc/kubernetes` when the only thing you need is `/etc/kubernetes/scheduler.conf`
 - Make everything but the etcd datadir read-only for components.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

Fixes: https://github.com/kubernetes/kubeadm/issues/341

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
cc @kubernetes/sig-cluster-lifecycle-pr-reviews

Automatic merge from submit-queue

Add AzureFile,FC,Flex,Flocker volume source to describe printer.

**What this PR does / why we need it**:
Display other volume sources in describe printer.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
Also need CephFS. It will be added in #46124 

/cc @rootfs 

_ADD: all the volume source printers are sorted out of order. I'd like to sort them alphabetically in order to be more readable in another PR after this PR merged. WDYT?_ 

**Release note**:

```release-note
NONE
```

For volumes that don't support bulk volume verifiction, a nil
volume spec can cause crash of controller.

Automatic merge from submit-queue

fix race in proxy unit test

fixes #49413, follow up to https://github.com/kubernetes/kubernetes/pull/49353 to fix a race in the unit test:


```
go test -v k8s.io/kubernetes/vendor/k8s.io/kube-aggregator/pkg/apiserver -run TestProxyUpgrade$
handler_proxy_test.go:328: invalid hostname + insecure: expected called=true, got false
testing.go:610: race detected during execution of test
```
https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/pr-logs/pull/48746/pull-kubernetes-unit/42230/

Automatic merge from submit-queue

Add a new API version apps/v1beta2

xref: #49135
This PR adds a new API version `apps/v1beta2` which contains a copy (of types, conversions, and defaults) of `apps/v1beta1` StatefulSet, Deployment, and their subresources. Note that `apps/v1beta2` is still WIP and we will make breaking changes to it before releasing 1.8.

Moving core controllers (StatefulSet, Deployment, ReplicaSet, DaemonSet) to `apps/v1beta2` is the first step of moving them to `apps/v1` (GA). 

This PR is a starting point for DaemonSet and ReplicaSet to move from `/extensions` to `/apps` and for Deployment and StatefulSet to make some breaking changes (e.g. new defaults and/or remove deprecated fields).

```release-note
Add a new API version apps/v1beta2
```

Automatic merge from submit-queue

Use specified ServerName in aggregator TLS validation

Fixes #49354

The aggregator sets a ServerName in the proxier tlsConfig, but the code path handling websocket upgrade requests did not honor it, and instead tried to verify TLS using the dialed host

* Honors ServerName if already set in tls.Config
* Adds unit tests for upgrade functionality via the aggregator
* Fixes mutation of shared tlsConfig.ServerName in spdy roundtripper

```release-note
Websocket requests to aggregated APIs now perform TLS verification using the service DNS name instead of the backend server's IP address, consistent with non-websocket requests.
```

Automatic merge from submit-queue

Reduce hollow proxy mem/node

As likely expected, kubemark-scale failed to even start with n1-standard-8 nodes. Because 1/3rd of our hollow nodes didn't even get scheduled due to their requests:

```
I0720 17:45:08.139] Found only 3325 ready hollow-nodes while waiting for 5000.
I0720 17:45:20.435] 3326 hollow-nodes are reported as 'Running'
I0720 17:45:20.442] 1675 hollow-nodes are reported as NOT 'Running'
```

If we want to experiment with smaller nodes anyway, then this change is needed. Though we most likely will end up OOM'ing.

Explanation for new value:
We have 62.5 hollow-node / real-node
=> mem available per hollow node = 30GB / 62.5 = 480MB
minus 100MB (kubelet)
minus 20MB (npd) 
=> 360MB for proxy should be = 100MB + 5000*(mem/node)
=> 50KB mem/node (with some slight slack)

cc @kubernetes/sig-scalability-misc

Automatic merge from submit-queue

Add an integration test library and some integration tests for scheduler

**What this PR does / why we need it**:

1. Add an integration test library (utils.go) for scheduler testing.
2. Cleaned up some of the tests in scheduler_test.go with the new integration test library.
3. Add priority_test.go with a couple of examples on how to test scheduler priority function in integration tests.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

ref/ #48176

@kubernetes/sig-scheduling-pr-reviews 
@davidopp @k82cn @vikaschoudhary16

Automatic merge from submit-queue

Auto-calculate master disk and root disk sizes in GCE

@gmarek PR https://github.com/kubernetes/kubernetes/pull/49282 didn't fix the issue because MASTER_DISK_SIZE was defaulting to 20GB in config-test.sh before being calculated inside get-master-disk-size() where you use pre-existing value if any.

It should be fixed by this now.

Automatic merge from submit-queue

word spell error

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```NONE

Automatic merge from submit-queue

Explicitly use Core() client when creating eventClint

@sttts

Automatic merge from submit-queue (batch tested with PRs 49328, 49285, 49307, 49127, 49163)

Cleanup storage e2e test names

**What this PR does / why we need it**:
Some test names had redundant [sig-storage] tags.  Also, some tests still had [Volume] tag.  This PR removes those tags.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Release note**:

```release-note
NONE
```
/release-note-none
/sig storage

Automatic merge from submit-queue (batch tested with PRs 49328, 49285, 49307, 49127, 49163)

Make definite mount timeout for glusterfs volume mount.

The `backup-volfile-servers` mount option allows to specify more than 
one server to be contacted in single mount command. With this option in place,
it is not required to iterate over all the servers in the addrlist.
A mount attempt with this option  will fetch all the servers mentioned in
the list, Reference # https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3/html/Administration_Guide/sect-Native_Client.htmlSigned-off-by: Humble Chirammal <hchiramm@redhat.com>

Automatic merge from submit-queue (batch tested with PRs 49328, 49285, 49307, 49127, 49163)

fuzzer: remove unreachable code

Remove unreachable code in fuzzers.

**Release note**:

```
NONE
```

/cc @sttts

Automatic merge from submit-queue (batch tested with PRs 49328, 49285, 49307, 49127, 49163)

fix mutation in statefulset sync

Original issue: https://github.com/openshift/origin/issues/15324

Seems like something inside `UpdateStatefulSet` mutates the StatefulSet. We might need to dig deeper to find the exact location where the non-copy mutation happens.

Automatic merge from submit-queue (batch tested with PRs 49328, 49285, 49307, 49127, 49163)

kubeadm: don't customize etcd selinux label

The original change that added the unconfined label included a comment
indicating it won't be needed in the future.
See: https://github.com/kubernetes/kubernetes/pull/33555#issuecomment-251126908

That time is now. https://github.com/kubernetes/kubernetes/pull/33663
has landed and means we no longer have to go out of our way to make that
work.

Removing the label also increases security since there wasn't really a
good reason for etcd to be run with such broad selinux privileges.

This also will allow kubeadm to avoid errors on distros without an spc_t
type, such as Gentoo and Container Linux (at the time of writing at
least).

Fixes https://github.com/kubernetes/kubeadm/issues/269

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 48565, 49172)

On GCE check whether NODE_LOCAL_SSDS=0 and handle this case appropriately

**What this PR does / why we need it**: Presently if you are using a mac and GCE and specify NODE_LOCAL_SSDS=0, or use the default, you end up with 2 local SSDs.

**Which issue this PR fixes** : fixes https://github.com/kubernetes/kubernetes/issues/49171

**Special notes for your reviewer**:

I've discovered that this issue is due to https://github.com/kubernetes/kubernetes/blob/b353792f9c349abb50a312bc1adaea1b3a8e115c/cluster/gce/util.sh#L579

If NODE_LOCAL_SSDS=0, this evaluates to $(seq 0)

```
$ for i in $(seq 0); do echo $i; done
1
0
```

From man seq on mac osx

```
The seq utility prints a sequence of numbers, one per line (default), from first (default 1), 
to near last as possible, in increments of incr (default 1).When first is larger than last the 
default incr is -1.
```

This was run on mac with the seq manpage indicating it comes from BSD Feb 19 2010.

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 48565, 49172)

Use $(location) to find generated output paths.

Along with the build-infra PR https://github.com/kubernetes/repo-infra/pull/19, this allows Kubernetes binaries (e.g. hyperkube) to be built as external dependencies in other Bazel projects.

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 49222, 49333, 48708, 49337)

Fix issue in installing containerized mounter

Fix PR #49335

PR #49157 causes failure when installing containerized mounter. This
PR is a fix for it

Automatic merge from submit-queue (batch tested with PRs 49222, 49333, 48708, 49337)

Allow to override build date

See https://reproducible-builds.org/ for why this is good
and https://reproducible-builds.org/specs/source-date-epoch/
for the definition of this variable.

Background: issue #48710
when trying to build the kubernetes package
for the openSUSE Linux distribution
```
/usr/bin/hyperkube
/usr/bin/kubeadm
/usr/src/kubernetes/platforms/linux/amd64/e2e.test
```
had a date+time string embedded in them in the format
2018-08-15T19:08:34Z

which meant that each build produced new binaries
even when nothing actually changed.
Those then had to be synced to mirrors, wasting their and our users' bandwidth.

Automatic merge from submit-queue (batch tested with PRs 49222, 49333, 48708, 49337)

GCE: Fix network/subnet url logic

Split if conditions: fixes #48521

/assign @bowei 

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 49222, 49333, 48708, 49337)

glbc: change the label of the l7-lb-controller pod

This ensures that the default http backend service doesn't include this
pod as its endpoint. This fixes #49159

Automatic merge from submit-queue (batch tested with PRs 49330, 49252, 49262, 49278, 49334)

Add project to pd delete node gcloud command

**What this PR does / why we need it**: Add `--project=` to `gcloud compute instances list` calls from `Pod Disks should be able to detach from a node which was deleted`.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: Fixes #https://github.com/kubernetes/kubernetes/issues/49185

**Special notes for your reviewer**:

CC @kubernetes/sig-storage-pr-reviews

Automatic merge from submit-queue (batch tested with PRs 49330, 49252, 49262, 49278, 49334)

Simplify master-worker relation missing message

**What this PR does / why we need it**: Simplify messaging of missing relation in Juju deployments

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes https://github.com/juju-solutions/bundle-canonical-kubernetes/issues/309

**Special notes for your reviewer**:

**Release note**:

```NONE
```

Automatic merge from submit-queue (batch tested with PRs 49330, 49252, 49262, 49278, 49334)

Enable garbage collector e2e tests

These tests are not running in pre-submit: see https://github.com/kubernetes/test-infra/blob/753266cb7d77456c2395521bece25eca51bfedcc/jobs/config.json#L9207

Automatic merge from submit-queue

Do not allow vendor code to import staging code

Otherwise we cannot update the imported staging code.