Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)

Adding legacy ABAC for 1.6

This is a fork of a previous [pull request](https://github.com/kubernetes/kubernetes/pull/42014) to include feedback as the original author is unavailable.

Adds a mechanism to optionally enable legacy abac for 1.6 to provide a migration path for existing users.

Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)

Revert conditional updates to configmap that breaks ingress

Don't prevent configmap updates to happen on existing ingress-uid configmap despite the lack of ingress objects. Otherwise, ingress objects get created with the wrong name.

Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)

Apiserver: wait for Etcd to become available on startup

fixes #37704

Automatic merge from submit-queue (batch tested with PRs 41931, 39821, 41841, 42197, 42195)

Admission Controller: Add Pod Preset

Based off the proposal in https://github.com/kubernetes/community/pull/254

cc @pmorie @pwittrock 

TODO:
- [ ] tests



**What this PR does / why we need it**: Implements the Pod Injection Policy admission controller

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
Added new Api `PodPreset` to enable defining cross-cutting injection of Volumes and Environment into Pods.
```

Automatic merge from submit-queue

Add RBAC roles for bootstrap controllers

Supercedes https://github.com/kubernetes/kubernetes/pull/42221

When locking down controllers to individual RBAC roles we need to make sure that the bootstrap controllers have the right permissions.

This adds the roles and bindings at the correct namespace scopes for the bootstrap-signer and token-cleaner controllers.

@liggitt ptal
@jbeda @luxas you got a good way to test this?  It must not be covered in normal e2e or we'd've seen the issue before.

Automatic merge from submit-queue

AWS: run k8s master in different account or different provider 

Currently the master and the nodes must run in the same account. With this change the master can run in a different AWS account, on a different cloud provider or on premise.

## Release Notes
```release-note
AWS cloud provider: allow to run the master with a different AWS account or even on a different cloud provider than the nodes.
```

Automatic merge from submit-queue (batch tested with PRs 41644, 42020, 41753, 42206, 42212)

Update defaultbackend image to 1.3

Update `gcr.io/google-containers/defaultbackend` to the latest version.

See https://github.com/kubernetes/contrib/pull/2386

/cc @ixdy

Automatic merge from submit-queue (batch tested with PRs 41644, 42020, 41753, 42206, 42212)

Make kubemark hollow node logging verbosity level configurable

cc @kubernetes/sig-scalability-misc @wojtek-t @gmarek

Automatic merge from submit-queue (batch tested with PRs 41644, 42020, 41753, 42206, 42212)

Burstable QoS cgroup has cpu shares assigned

**What this PR does / why we need it**:
This PR sets the Burstable QoS cgroup cpu shares value to the sum of the pods cpu requests in that tier.  We need it for proper evaluation of CPU shares in the new QoS hierarchy.

**Special notes for your reviewer**:
It builds against the framework proposed for https://github.com/kubernetes/kubernetes/pull/41833

Automatic merge from submit-queue (batch tested with PRs 41644, 42020, 41753, 42206, 42212)

Ingress-glbc upgrade tests

Basically #41676 but with some fixes and added comments.  @bprashanth has been away this week and it's desirable to have this in before code freeze.

Automatic merge from submit-queue (batch tested with PRs 41644, 42020, 41753, 42206, 42212)

Ensure pod cgroup is deleted prior to deletion of pod

**What this PR does / why we need it**:
This PR ensures that the kubelet removes the pod cgroup sandbox prior to deletion of a pod from the apiserver.   We need this to ensure that the default behavior in the kubelet is to not leak resources.

Automatic merge from submit-queue

Adding vishh to cluster/gce owners

Automatic merge from submit-queue

Re-writing of the resolv.conf file generated by docker

Fixes #17406 

Docker 1.12 will contain feature "The option --dns and --net=host should not be mutually exclusive" (docker/docker#22408)
This patch adds optional support for this ability in kubelet (for now in case of "hostNetwork: true" set all dns settings are ignored if any).
To enable feature use newly added kubelet flag: --allow-dns-for-hostnet=true

Signed-off-by: Jess Frazelle <acidburn@google.com>

Signed-off-by: Jess Frazelle <acidburn@google.com>

Signed-off-by: Jess Frazelle <acidburn@google.com>

export functions from pkg/api/validation

add settings API

add settings to pkg/registry

add settings api to pkg/master/master.go

add admission control plugin for pod preset

add new admission control plugin to kube-apiserver

add settings to import_known_versions.go

add settings to codegen

add validation tests

add settings to client generation

add protobufs generation for settings api

update linted packages

add settings to testapi

add settings install to clientset

add start of e2e

add pod preset plugin to config-test.sh
Signed-off-by: Jess Frazelle <acidburn@google.com>

Automatic merge from submit-queue

Adding OWNERS file for test/integration/federation

Ref https://github.com/kubernetes/kubernetes/issues/40705

File copied from federation/OWNERS.

Existing pending PRs for this diectory:
https://github.com/kubernetes/kubernetes/pull/42278 and https://github.com/kubernetes/kubernetes/pull/42225

cc @kubernetes/sig-federation-pr-reviews

Automatic merge from submit-queue

HPA Controller: Use Custom Metrics API

This commit switches over the HPA controller to use the custom metrics
API.  It also converts the HPA controller to use the generated client
in k8s.io/metrics for the resource metrics API.

In order to enable support, you must enable
`--horizontal-pod-autoscaler-use-rest-clients` on the
controller-manager, which will switch the HPA controller's MetricsClient
implementation over to use the standard rest clients for both custom
metrics and resource metrics.  This requires that at the least resource
metrics API is registered with kube-aggregator, and that the controller
manager is pointed at kube-aggregator.  For this to work, Heapster
must be serving the new-style API server (`--api-server=true`).

Before this merges, this will need kubernetes/metrics#2 to merge, and a godeps update to pull that in.
It's also semi-dependent on kubernetes/heapster#1537, but that is not required in order for this to merge.

**Release note**:
```release-note
Allow the Horizontal Pod Autoscaler controller to talk to the metrics API and custom metrics API as standard APIs.
```

Automatic merge from submit-queue

statefulset: wait for pvc cache sync

#42056 switched the statefulset controller to use the pvc shared informer/lister, but accidentally left out waiting for its cache to sync.

cc @kubernetes/sig-apps-pr-reviews @kargakis @foxish @kow3ns @smarterclayton @deads2k

Automatic merge from submit-queue

fix rsListerSynced and podListerSynced for DeploymentController

**What this PR does / why we need it**:

There is a mistake when initializing `DeploymentController`'s `rsListerSynced` and `podListerSynced` in `NewDeploymentController`, they are all initialized to `Deployment`'s `Informer`, so the `DeploymentController` maybe running before the `ReplicaSet` cache and `Pod` cache has been synced.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

Indeed according unit test is neccessary, but this bug fix is simple, and if the tests is neccessary I will submit another PR later.

**Release note**:

```release-note
```

Automatic merge from submit-queue

Implement support for mount options in PVs

**What this PR does / why we need it**:

This PR implements support for mount options in PersistentVolume via `volume.beta.kubernetes.io/mount-options` annotation.

**Which issue this PR fixes** 

Fixes https://github.com/kubernetes/features/issues/168

**Release note**:
```
Enable additional, custom mount options to be passed to PersistentVolume objects via volume.beta.kubernetes.io/mount-options annotation.
```

Automatic merge from submit-queue

show patch error as string not byte slice

Debugging for https://github.com/kubernetes/kubernetes/issues/39471 shows

```
client_test.go:309: Failed updating patchpod with patch type application/json-patch+json: Operation cannot be fulfilled on pods "patchpod": there is a meaningful conflict:
	 diff1=[123 34 109 101 116 97 100 97 116 97 34 58 123 34 108 97 98 101 108 115 34 58 123 34 102 111 111 34 58 110 117 108 108 125 44 34 114 101 115 111 117 114 99 101 86 101 114 115 105 111 110 34 58 34 49 52 50 50 34 125 125]
	, diff2=[123 34 109 101 116 97 100 97 116 97 34 58 123 34 108 97 98 101 108 115 34 58 110 117 108 108 125 125]
```

I don't speak byte slice, so this makes it a string for next time.

@sttts

Automatic merge from submit-queue

Extensible Userspace Proxy

This PR refactors the userspace proxy to allow for custom proxy socket implementations.
It changes the the ProxySocket interface to ensure that other packages can properly implement it (making sure all arguments are publicly exposed types, etc), and adds in a mechanism for an implementation to create an instance of the userspace proxy with a non-standard ProxySocket.
Custom ProxySockets are useful to inject additional logic into the actual proxying.  For example, our idling proxier uses a custom proxy socket to hold connections and notify the cluster that idled scalable resources need to be woken up.

Also-Authored-By: Ben Bennett bbennett@redhat.com

Introduced chages:
   1. Re-writing of the resolv.conf file generated by docker.
      Cluster dns settings aren't passed anymore to docker api in all cases, not only for pods with host network:
      the resolver conf will be overwritten after infra-container creation to override docker's behaviour.

   2. Added new one dnsPolicy - 'ClusterFirstWithHostNet', so now there are:
      - ClusterFirstWithHostNet - use dns settings in all cases, i.e. with hostNet=true as well
      - ClusterFirst - use dns settings unless hostNetwork is true
      - Default

Fixes #17406

Add support for mount options via annotations on PVs

Set the vpcID when dummy is created (+1 squashed commit)
Squashed commits:
[0b1ac6e83e] Use the VPC flag and KubernetesClusterTag as identifier (+1 squashed commit)
Squashed commits:
[962bc56e38] Remove again availabilityZone and fix naming (+1 squashed commit)
Squashed commits:
[e3d1b41807] Use the VCID flag as identifier (+1 squashed commit)
Squashed commits:
[5b99fe6243] Add flag for external master

Automatic merge from submit-queue

Add apps/v1beta1 deployments with new defaults

This pull introduces deployments under `apps/v1beta1` and fixes #23597 and #23304.

TODO:

* [x] - create new type `apps/v1beta1.Deployment`
* [x] - update kubectl (stop, scale)
* [ ] - ~~new `kubectl run` generator~~ - this will only duplicate half of generator code, I suggest replacing current to use new endpoint
* [ ] - ~~create extended tests~~ - I've added integration and cmd tests verifying new endpoints
* [ ] - ~~create `hack/test-update-storage-objects.sh`~~ - see above

This is currently blocked by https://github.com/kubernetes/kubernetes/pull/38071, due to conflicting name `v1beta1.Deployment`. 

```release-note
Introduce apps/v1beta1.Deployments resource with modified defaults compared to extensions/v1beta1.Deployments.
```

@kargakis @mfojtik @kubernetes/sig-apps-misc

Automatic merge from submit-queue (batch tested with PRs 42316, 41618, 42201, 42113, 42191)

Make 'docker-email' optional on dockercfg secrets

It is not required for most username/password registries.

Fixes #41727

Automatic merge from submit-queue (batch tested with PRs 42316, 41618, 42201, 42113, 42191)

Fixing a couple of things in the example, and bumping docker version

**What this PR does / why we need it**:

Updates the Cassandra version, and fixes a couple of issues in the example documentation.

fixes #40435

Automatic merge from submit-queue (batch tested with PRs 42316, 41618, 42201, 42113, 42191)

[Kubemark] Add init container in hollow node for setting inotify limit of node to 200

Fixes #41713 

Along with adding the init container, I also changed the manifest to a yaml as otherwise the entire init container annotation would have to be in a single line (with escaped characters), as json doesn't allow multi-line strings.

cc @kubernetes/sig-scalability-misc @wojtek-t @gmarek @Random-Liu

Automatic merge from submit-queue (batch tested with PRs 42316, 41618, 42201, 42113, 42191)

Support unqualified and partially qualified domain name in DNS query in Windows kube-proxy

**What this PR does / why we need it**:
In Windows container networking, --dns-search is not currently supported on Windows Docker. Besides, even with --dns-suffix, inside Windows container DNS suffix is not appended to DNS query names. That makes unqualified domain name or partially qualified domain name in DNS query not able to resolve.

This PR provides a solution to resolve unqualified domain name or partially qualified domain name in DNS query for Windows container in Windows kube-proxy. It uses well-known Kubernetes DNS suffix as well host DNS suffix search list to append to the name in DNS query. DNS packet in kube-proxy UDP stream is modified as appropriate.

This PR affects the Windows kube-proxy only.

**Special notes for your reviewer**:
This PR is based on top of Anthony Howe's commit 48647fb9, 0e37f0a8 and 7e2c71f6 which is already included in the PR 41487. Please only review commit b9dfb69d.

**Release note**:

```release-note
Add DNS suffix search list support in Windows kube-proxy.
```

This commit adds the staging repos to the GOPATH in
hack/verify-godeps.sh.  This allows vendored librarys
to depend on staging repos and not break verify-godeps.

This also adds `hack/godep-restore.sh`, which acts like
`godep restore`, but sets the GOPATH appropriate as well.

This commit introduces a dependency on k8s.io/metrics, for
use with the horizontal pod autoscaler.