When we are mounting a lot of volumes, we frequently hit rate limits.

Reduce the frequency with which we poll the status; introduces a bit of
latency but probably matches common attach times pretty closely, and
avoids causing rate limit problems everywhere.

Also, we now poll for longer, as when we timeout, the volume is in an
indeterminate state: it may be about to complete.  The volume controller
can tolerate a slow attach/detach, but it is harder to tolerate the
indeterminism.

Finally, we ignore a sequence of errors in DescribeVolumes (up to 5 in a
row currently).  So we will eventually return an error, but a one
off-failure (e.g. due to rate limits) does not cause us to spuriously
fail.

Automatic merge from submit-queue

Enabled Rescheduler e2e for GKE

The Rescheduler is now enabled on GKE (cl/132886258).

Ref #32500

@pwittrock let's wait for cherry-pick until it passes on gke-serial.

Automatic merge from submit-queue

Allow webhook authenticator to use TokenReviewsInterface

Refactors the authentication webhook to be able to be fed a kubeconfig file or a TokenReviewsInterface 

Fixes a bug with webhooks not retrying on 429/500 errors

Related to https://github.com/kubernetes/kubernetes/pull/32518#discussion_r78450170

Automatic merge from submit-queue

Allow namespace to be loaded from in-cluster config

This is a follow on from the previous commit that fixed ClientConfig.
Namespace can also be defaulted from ICC, and this correctly handles
that logic. Also add two debugging lines to ensure that it is easier in
the future to uncover problems here.

Fixes #32352

@deads2k

Automatic merge from submit-queue

Move push-ci-build.sh to kubernetes/release repo

Automatic merge from submit-queue

Doc API group suffix, add test to catch new groups

Spawned from discussion in https://github.com/kubernetes/kubernetes/pull/31887

Doc and add tests to ensure new API groups are suffixed.

Also changed the doc to reference an API group containing the suffix as a starting point for new API groups.

Automatic merge from submit-queue

Improvements on OpenAPI spec generation

- Generating models using go2idl library (no reflection anymore)
- Remove dependencies on go-restful/swagger
- Generate one swagger.json file for each web-service
- Bugfix: fixed a bug in trie implementation

Reference: #13414


**Release note**:

```release-note
Generate separate OpenAPI spec for each API GroupVersion on /<Group>/<Version>/swagger.json
```

Automatic merge from submit-queue

modify error return

err is nil here,chage it to fmt.Errorf("cannot find SCSI controller in VM")

Automatic merge from submit-queue

delete the member before delete the pool

fixes https://github.com/kubernetes/kubernetes/issues/32285. it is necessary to delete members before delete the pool.

Automatic merge from submit-queue

Change all jsonmerge to strategicpatch

Implement all jsonmerge changed to strategicpatch in edit.go.

Automatic merge from submit-queue

add alias deploy for deployment

deploy is also alias of deployment. add alias deploy for deployment so the help can see deploy in the Aliases like below:
```sh
k8s@k8s-node1:~/go/workspace/src/k8s.io/kubernetes/cmd/kubectl$ ./kubectl create dep -h
Create a deployment with the specified name.

Aliases:
deployment, dep, deploy
```

Revert "dockershim: add unit tests for sandbox/container status"

Automatic merge from submit-queue

Ensure that we are closing files.

**What this PR does / why we need it**: In several places we are leaking file descriptors. This could be problematic on systems with low ulimits for them.

**Release note**:
```release-note
```

Automatic merge from submit-queue

Generate 1 5 clientset

Generate the 1.5 clientset. Stop updating 1.4 clientset. Remove 1.2 clientset.

@nikhiljindal @lavalamp 

I will rebase #31994 atop of this one.

Automatic merge from submit-queue

Revert "Merge pull request #30513 from tmrts/kubelet-rkt-cri/use-imag…

This reverts commit aff7dfca, reversing
changes made to 7a4d81ea.

See https://github.com/kubernetes/kubernetes/pull/30513#issuecomment-245949664, this breaks tests for the rkt runtime.

The original PR also changed the source of image truth from the api-service to cli, which could have further implications which @yifan-gu could speak better to, so I think it's safer to just revert for now and discuss further in the rktlet repo / pr. The reverted code effectively already exists (with the bug in question already fixed) here https://github.com/kubernetes-incubator/rktlet/pull/5 .. once that's merged, we can vendor and call it over here in place of #30513.

@yifan-gu / @tmrts  if you think continuing with the changes and rolling forwards (just adding the `--full` flag to list to fix the immediate bug) is better, feel free to close this and open a PR with that change, but I think this approach is overall better for the reasons in the previous paragraph.

cc @tmrts @yifan-gu @kubernetes/sig-rktnetes @pskrzyns

Automatic merge from submit-queue

remove the rest of the non-generated clients from the kubectl code

Die `Client` Die!

It's always bigger than you think.  Last bit @kargakis after this, it's gone.

Automatic merge from submit-queue

Revert "e2e: only tear down existing cluster if it exists"

Reverts kubernetes/kubernetes#32420

This broke more things. We're going to backport the `|| true` fix to 1.2 -- doing just enough maintenance on old branches to keep upgrades working with new infra changes.

Automatic merge from submit-queue

Add node e2e density test using 60 QPS for benchmark

This PR adds a new benchmark node e2e density test which sets Kubelet API QPS limit from default 5 to 60, through ConfigMap. 

The latency caused by API QPS limit is as large as ~30% when creating a large batch of pods (e.g. 105). It makes the pod startup latency, as well creation throughput underestimated. This test helps us to know the real performance of Kubelet core.

Automatic merge from submit-queue

e2e/log-dump: Collect kernel log with journald

Related to #31928

The kern.log file does not exist on journald distros typically.

cc @vishh @Random-Liu

Automatic merge from submit-queue

Namespace Controller handles items with finalizers gracefully

This PR does the following:

1. ensures the "orphan" finalizer is not added to items during DELETE COLLECTION calls
2. does not treat presence of a finalizer as an unexpected error condition.

The 15s wait should only happen when finalizers not added by GC are used.

I am aware of any finalizer like that at this time.

Fixes https://github.com/kubernetes/kubernetes/issues/32519

customize intstr, unversioned.time, and quantity OpenAPI definitions.

- Generating models using go2idl library (no reflection anymore)
- Remove dependencies on go-restful/swagger
- Generate one swagger.json file for each web-service
- Bugfix: fixed a bug in trie implementation

Automatic merge from submit-queue

Add some docs about the missing node e2e scheduling

Automatic merge from submit-queue

Fix memory eviction test parameters!

The parameters currently in master should NOT have come through in b9f0bd95. Must have happened when I squashed. Apologies.

Automatic merge from submit-queue

update error handling for daemoncontroller

Updates the DaemonSet controller to cleanly requeue with ratelimiting on errors, make use of the `utilruntime.HandleError` consistently, and wait for preconditions before doing work.

@ncdc @liggitt @sttts My plan is to use this one as an example of how to handle requeuing, preconditions, and processing error handling.
@foxish fyi

related to https://github.com/kubernetes/kubernetes/issues/30629

Automatic merge from submit-queue

Fix race condition in updating attached volume between master and node

This PR tries to fix issue #29324. The cause of this issue is that a race
condition happens when marking volumes as attached for node status. This
PR tries to clean up the logic of when and where to mark volumes as
attached/detached. Basically the workflow as follows,
1. When volume is attached sucessfully, the volume and node info is
added into nodesToUpdateStatusFor to mark the volume as attached to the
node.
2. When detach request comes in, it will check whether it is safe to
detach now. If the check passes, remove the volume from volumesToReportAsAttached
to indicate the volume is no longer considered as attached now.
Afterwards, reconciler tries to update node status and trigger detach
operation. If any of these operation fails, the volume is added back to
the volumesToReportAsAttached list showing that it is still attached.

These steps should make sure that kubelet get the right (might be
outdated) information about which volume is attached or not. It also
garantees that if detach operation is pending, kubelet should not
trigger any mount operations.

Automatic merge from submit-queue

vendor: update github.com/coreos/go-oidc client package

This change updates the github.com/coreos/go-oidc package to it's latest commit
(since we don't version that package).

Notable changes:

- Throw out JWTs with invalid claims early (coreos/go-oidc#97, brougt up in #30457)
- Remove the capnslog dependency (coreos/go-oidc#95)
- Support for Azure AD oddities (coreos/go-oidc#87)

cc @kubernetes/sig-auth

Automatic merge from submit-queue

dockershim: add unit tests for sandbox/container status

Part of #31459

Automatic merge from submit-queue

federation e2e: Initialising cluster var before using it

Fixing https://github.com/kubernetes/kubernetes/pull/31904#discussion_r78433657

cc @jianhuiz @kubernetes/sig-cluster-federation