Automatic merge from submit-queue

remove abac

Remove the abac authorizer as an authorizer for e2e.

Automatic merge from submit-queue

retry RBAC initialization for up to 30 seconds, kill server on failure

RBAC initialization needs to complete in order to bootstrap a cluster.  When the bootstrapping fails (etcd not ready has happened in e2e runs), things fail badly and we don't even kill the API server to force it to retry.  This retries for up to 30 seconds and kills the server if it never succeeds.

Fixes https://github.com/kubernetes/kubernetes/issues/39108

Automatic merge from submit-queue (batch tested with PRs 39075, 39350, 39353)

Move pkg/api.{Context,RequestContextMapper} into pkg/genericapiserver/api/request

**Based on #39350**

Automatic merge from submit-queue (batch tested with PRs 39075, 39350, 39353)

Move pkg/apiserver to pkg/genericapiserver/api

Automatic merge from submit-queue

no need to sleep for last retry

break for needless sleep

Automatic merge from submit-queue

Try parse golang logs by default

Glog by default logs to stderr, so Stackdriver Logging shows them all as errors. This PR makes fluentd try to parse messages using glog format and if succeeded, set timestamp and severity accordingly.

CC @piosz @fgrzadkowski

Automatic merge from submit-queue

Ensure that assumed pod won't be expired until the end of binding

In case if api server is overloaded and will reply with 429 too many requests error, binding may take longer than ttl of scheduler cache for assumed pods https://github.com/kubernetes/kubernetes/blob/1199d42210c8ff6b232a80c79490c6a3c38a5de6/pkg/client/restclient/request.go#L787-L850 

This problem was mitigated by using this fix https://github.com/kubernetes/kubernetes/commit/e4d215d5084078c561659ab18ed6397e2e2f0d84 and increased rate limit for api server. But it is possible that it will occur again.

Automatic merge from submit-queue

fix swapped new/old job obj for validating object meta update

@nikhiljindal

In such cases when api server is overloaded and returns a lot of
429 (too many requests) errors - binding may take a lot of time
to succeed due to retry policy implemented in rest client.
In such events cache ttl for assumed pods wasn't big enough.

In order to minimize probability of such errors ttl for assumed pods
will be counted from the time when binding for particular pod is finished
(either with error or success)

Change-Id: Ib0122f8a76dc57c82f2c7c52497aad1bdd8be411

Automatic merge from submit-queue

DaemonSet ObservedGeneration

Extracting ObserverdGeneration part from #31693. It also implements #7328 for DaemonSets.

cc @kargakis

Automatic merge from submit-queue (batch tested with PRs 39351, 39322)

Remove the deprecated --reconcile-cidr flag from kubelet

**What this PR does / why we need it**:

Removes the `--reconcile-cidr` flag that has been deprecated since v1.5

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
The `--reconcile-cidr` kubelet flag was removed since it had been deprecated since v1.5
```
@thockin

Automatic merge from submit-queue

Update Google API dependencies

In order to experiment with alpha or beta GCP APIs (that are new), we need to make the client compatible with v1 client. And there were some cosmetic changes that make them incompatible with the version that we were using before (e.g. passing pointers vs non-pointers in gensupport functions).

Automatic merge from submit-queue (batch tested with PRs 39150, 38615)

Add work queues to PV controller

PV controller should not use Controller.Requeue, as as it is not available in
shared informers. We need to implement our own work queues instead, where we
can enqueue volumes/claims as we want.

Automatic merge from submit-queue

kubectl: ignore deleted pods in the rolling updater

Fixes https://github.com/kubernetes/kubernetes/issues/37955

The Deployment controller already ignores deleted pods or more precisely the ReplicaSet controller ignores deleted pods ([pod classification](https://github.com/kubernetes/kubernetes/blob/a30b6e2d163ce5363daf1106350f46ea99982a64/pkg/controller/replicaset/replica_set.go#L600) ignores [deleted pods](https://github.com/kubernetes/kubernetes/blob/a30b6e2d163ce5363daf1106350f46ea99982a64/pkg/controller/controller_ref_manager.go#L63)) and the Deployment controller reuses what's in the ReplicaSet status.

@kubernetes/sig-apps-misc @kubernetes/sig-cli-misc

Automatic merge from submit-queue

Add Persistent Volume E2E in the context of a disrupted kubelet

This PR adds a test suite for persistent volumes affected by a disrupted kubelet.  Two cases are presented:

1.  A volume mounted via PVC remains accessible after a kubelet restart.
2.  When a pod is deleted while the kubelet is down, the mounted volume is unmounted successfully.

PV controller should not use Controller.Requeue, as as it is not available in
shared informers. We need to implement our own work queues instead where we
can enqueue volumes/claims as we want.

Automatic merge from submit-queue

Add default imagePullPolicy for initContainers.

fixes #38542

Merge pull request #36221 from pospispa/86-5-add-checks-and-documentation-about-template-pods-for-recycling

Automatic merge from submit-queue (batch tested with PRs 37959, 36221)

Recycle Pod Template Check

The kube-controller-manager has two command line arguments (--pv-recycler-pod-template-filepath-hostpath and --pv-recycler-pod-template-filepath-nfs) that specify a recycle pod template. The recycle pod template may not contain the volume that shall be recycled.

A check is added to make sure that the recycle pod template contains at least a volume.

cc: @jsafrane

Automatic merge from submit-queue

Add ready replicas in Deployments

Fixes https://github.com/kubernetes/kubernetes/issues/28381

@kubernetes/deployment PTAL

Automatic merge from submit-queue

Extend logging in restore-from-backup script

Automatic merge from submit-queue

Copy workqueue to client-go

The workqueue is very useful when building controllers, so this PR copied it to client-go's staging area.

Fix https://github.com/kubernetes/kubernetes/issues/33497

Update generated for 2017

Automatic merge from submit-queue

Kubelet: add image ref to ImageService interfaces

This PR adds image ref (digest or ID, depending on runtime) to PullImage result, and pass image ref in CreateContainer instead of image name. It also

* Adds image ref to CRI's PullImageResponse
* Updates related image puller
* Updates related testing utilities

~~One remaining issue is: it breaks some e2e tests because they [checks image repoTags](https://github.com/kubernetes/kubernetes/blob/master/test/e2e/framework/util.go#L1941) while docker always returns digest in this PR. Should we update e2e test or continue to return repoTags in `containerStatuses.image`?~~

Fixes #38833.

Automatic merge from submit-queue

cluster/gce: Rename coreos to container-linux.

Since coreos distro has been be renamed to container linux, we need to follow the naming convention here as well.

We also need to rename the libvirt-coreos as well, but to make review easier, that will happen in another PR.

cc @euank @philips

Run hack/verify-flags-underscore.py -e > hack/verify-flags/exceptions.txt