Automatic merge from submit-queue

Disable kubernetes-discovery in local-up-cluster.sh

fix #38257

Fixes local-up-cluster until kubernetes-discovery flags are hooked up

Automatic merge from submit-queue

add default label to rbac bootstrap policy

allow people to retrieve information of bootstrap policy by label :

`kubectl get clusterroles -l key=value` 
`kubectl get clusterrolebindings -l key=value`

Automatic merge from submit-queue

update local-up-cluster to allow full authentication proxying

Adds group and header information in auth proxy authenticator options for `local-up-cluster.sh`.  Must have been missed in the rebase madness.

Automatic merge from submit-queue (batch tested with PRs 38282, 38281)

Fix skip logic in e2e framework

Fixes 37792

cc: @wojtek-t

Automatic merge from submit-queue

e2e: mark 3rd portforwarding test as flaky

Follow-up of https://github.com/kubernetes/kubernetes/pull/38194 and https://github.com/kubernetes/kubernetes/issues/27680#issuecomment-265392033.

cc @kubernetes/sig-testing

Automatic merge from submit-queue (batch tested with PRs 38276, 38193)

Collect controller-manager logs from kubemark

Automatic merge from submit-queue

Bump Cluster Autoscaler to 0.4.0

This is the  same binary as 0.4.0-beta1, that has been tested for the last couple weeks.

@saad-ali This should be cherry-picked to 1.5 release.

```release-note
Cluster Autoscaler in version 0.4.0
```

cc: @fgrzadkowski @piosz @jszczepkowski

Automatic merge from submit-queue (batch tested with PRs 38181, 38128, 36711)

etcd2: have prefix always prepended

The prefix issue is discussed in #36290.

This is fixing etcd2 behavior separately.

**release note**:
```
etcd2: have prefix always prepended
```

Automatic merge from submit-queue (batch tested with PRs 38181, 38128, 36711)

Adding correct secret type for Ceph RBD storageclass provisioner example

StorageClass now requires provider-specific secret types, adding them to the RBD provisioning docs.

Automatic merge from submit-queue (batch tested with PRs 38181, 38128, 36711)

Remove redundant conditional statement

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**:

1. remove redundant conditional statement
2. replace  errors.New(fmt.Sprintf with fmt.Errorf 

Thanks.

**Special notes for your reviewer**:

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
```

Automatic merge from submit-queue

Cache additional information in schedulercache.NodeInfo to speedup scheduler

Ref #35117

Automatic merge from submit-queue

Enable OpenAPI spec validation

Spec validation was failing on Jenkins. I am enabling it in this PR to figure out if we can get it pass. No review is necessary until all test passes.

Automatic merge from submit-queue (batch tested with PRs 37693, 38085)

fix typo in scale

fix typo.

Automatic merge from submit-queue (batch tested with PRs 37693, 38085)

Pipe get options to storage

Ref #37473

Automatic merge from submit-queue (batch tested with PRs 38173, 38151, 38197, 38221)

Revert "Disable "overlapping deployment" test on GKE temporarily. #32947"

Reverts kubernetes/kubernetes#38157

Automatic merge from submit-queue (batch tested with PRs 38173, 38151, 38197, 38221)

test: wait for ready replica set before adopting

Reworked version of https://github.com/kubernetes/kubernetes/pull/36439 which was reverted in https://github.com/kubernetes/kubernetes/pull/38049. This PR doesn't use any of the new status API added in replica sets so it should cause no trouble with upgrade tests.

@kubernetes/deployment @smarterclayton

Automatic merge from submit-queue (batch tested with PRs 38173, 38151, 38197, 38221)

Include resource type headers in diff report

**What this PR does / why we need it**: when checking for leaked resources, we report a failure only if there were additions to the diff - we don't want resources that were deleted during the run to count as a failure.

The logic for this removes the resource type headers from the diff report, however, so it's harder to understand what leaked.

For example, in https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-alpha-features-release-1.5/196/, the test failure shows
```
Error: 4 leaked resources
+NAME                              REGION       IP_ADDRESS      IP_PROTOCOL  TARGET
+a66878dbaad1f11e695b342010af0002  us-central1  104.198.182.83  TCP          us-central1/targetPools/a66878dbaad1f11e695b342010af0002
+NAME                              REGION       SESSION_AFFINITY  BACKUP  HEALTH_CHECKS
+a66878dbaad1f11e695b342010af0002  us-central1                            a66878dbaad1f11e695b342010af0002
```

but the [actual diff](https://storage.googleapis.com/kubernetes-jenkins/logs/ci-kubernetes-e2e-gke-alpha-features-release-1.5/196/artifacts/gcp-resources-diff.txt) contains more information:
```
--- /workspace/_artifacts/gcp-resources-before.txt	2016-12-01 14:39:14.260049037 -0800
+++ /workspace/_artifacts/gcp-resources-after.txt	2016-12-01 14:39:14.260049037 -0800
@@ -34,8 +34 @@ [ addresses ]
-NAME                              REGION       ADDRESS          STATUS
-a03d0e020a6bc11e6855f42010af0001  us-central1  104.197.135.88   RESERVED
-a23f20a47a6a011e6829a42010af0004  us-central1  104.197.147.212  RESERVED
-a32291b51a5e611e6b73342010af0002  us-central1  104.198.171.178  RESERVED
-a3c93cf79a65811e6b7c242010af0001  us-central1  104.198.58.57    RESERVED
-a9ac5bd1ba62811e68d1142010af0004  us-central1  104.154.134.135  RESERVED
-adae4c1a7a69511e6b2df42010af0000  us-central1  104.198.182.83   RESERVED
-adb4a80caa60311e689a942010af0001  us-central1  104.154.116.195  RESERVED
+
@@ -64 +57,2 @@ [ forwarding-rules ]
-
+NAME                              REGION       IP_ADDRESS      IP_PROTOCOL  TARGET
+a66878dbaad1f11e695b342010af0002  us-central1  104.198.182.83  TCP          us-central1/targetPools/a66878dbaad1f11e695b342010af0002
@@ -68 +62,2 @@ [ target-pools ]
-
+NAME                              REGION       SESSION_AFFINITY  BACKUP  HEALTH_CHECKS
+a66878dbaad1f11e695b342010af0002  us-central1                            a66878dbaad1f11e695b342010af0002
```

This PR will include the `[ type header ]`s in the report if resources are leaked.

cc @fejta

Automatic merge from submit-queue

fix typo in ingress

fix typo.

Automatic merge from submit-queue

Fix pods/log OpenAPI/Swagger spec to return string instead of v1.Pod

log operation is part of pod and had the assumption of returning a v1.Pod. First I added the support for an operation to be able to customize its return object then add the support to pods/log operation to return an string as an object sample.

Fixes #37881
Cc @kubernetes/sig-api-machinery

Automatic merge from submit-queue

add a configuration for kubelet to register as a node with taints

and deprecate --register-schedulable

ref #28687 #29178

cc @dchen1107 @davidopp @roberthbailey

Automatic merge from submit-queue (batch tested with PRs 37032, 38119, 38186, 38200, 38139)

New ns param for NewClusterVerification

**What this PR does / why we need it**: Allows the test to specify alternate namespaces to when waiting for pods to be in a specific state.

**Which issue this PR fixes**: fixes #38138

**Special notes for your reviewer**: Minor fix

**Release note**: None

Automatic merge from submit-queue (batch tested with PRs 37032, 38119, 38186, 38200, 38139)

etcd2: remove unnecessary PrevValue in SetOption

ref: https://github.com/kubernetes/kubernetes/issues/37994

Summary:
- PrevValue is set in HTTP header, and large value (>1MB) could exceed check limit
- We don't need PrevValue indeed since we already use PrevIndex in SetOptions and each PrevIndex corresponds to each PrevValue.

I don't really think we need extra tests for this. There is already test for GuaranteedUpdate covering its use cases.

Automatic merge from submit-queue (batch tested with PRs 37032, 38119, 38186, 38200, 38139)

Remove genericapiserver.Options.MasterServiceNamespace

Deprecated and not used anymore.

Automatic merge from submit-queue (batch tested with PRs 37032, 38119, 38186, 38200, 38139)

Detect long-running requests from parsed request info

Follow up to https://github.com/kubernetes/kubernetes/pull/36064

Uses parsed request info to more tightly match verbs and subresources

Removes regex-based long-running request path matching (which is easily fooled)

```release-note
The --long-running-request-regexp flag to kube-apiserver is deprecated and will be removed in a future release. Long-running requests are now detected based on specific verbs (watch, proxy) or subresources (proxy, portforward, log, exec, attach).
```

Automatic merge from submit-queue

Add integration tests for desire state of world populator

Add integration tests for desire state of world populator
    
This adds tests for code introduced here :
https://github.com/kubernetes/kubernetes/issues/26994
    
Via integration test we can now verify that if pod delete
event is somehow missed by AttachDetach controller - it still
get cleaned up by Desired State of World populator.

Automatic merge from submit-queue (batch tested with PRs 38194, 37594, 38123, 37831, 37084)

[kubeadm] Fail fast if there already is a node in the cluster with that name

kubeadm Fail fast if there is another node with the same name already in the cluster. 

Fixes #36255

cc @kubernetes/sig-cluster-federation

Automatic merge from submit-queue (batch tested with PRs 38194, 37594, 38123, 37831, 37084)

Improve kubeadm reset

Depends on: https://github.com/kubernetes/kubernetes/pull/36474
Broken out from: https://github.com/kubernetes/kubernetes/pull/37568
Carries: https://github.com/kubernetes/kubernetes/pull/35709, @camilocot

This makes the `kubeadm reset` command more robust and user-friendly.
I'll rebase after #36474 merges...

cc-ing reviewers: @mikedanese @errordeveloper @dgoodwin @jbeda

Automatic merge from submit-queue (batch tested with PRs 38194, 37594, 38123, 37831, 37084)

remove unnecessary fields from genericapiserver config

Cleans up some unnecessary fields in the genericapiserver config.

Automatic merge from submit-queue (batch tested with PRs 38194, 37594, 38123, 37831, 37084)

Better compat with very old iptables (e.g. CentOS 6)

Fixes reported issue with CentOS6 iptables 1.4.7 (ancient)

Older iptables expanded things like 0x4000 into 0x00004000, which defeats the
fallback "check" logic.

Fixes #37416

Automatic merge from submit-queue

e2e: mark portforward tests as flaky

As long as we don't get https://github.com/kubernetes/kubernetes/issues/27673 and https://github.com/kubernetes/kubernetes/issues/27680 under control, this PR marks them as `[Flaky]`.

There is https://github.com/kubernetes/kubernetes/pull/37072 in flight which might improve the situation, but is blocked on socat issues right now with half-open connections.

cc @kubernetes/sig-testing

Automatic merge from submit-queue

Fix GCI mounter issue

Automatic merge from submit-queue

Skip not registered nodes in labeling in CA e2e tests

This PR fixes problems with querying for not yet registered nodes. The underlying problem is related to the way the test is written. So we apply labels to the existing nodes, create pods that require N+1 nodes  with the labels and expect a new node to be added. But the new node is created without the labels. As soon as the node is spotted it is labeled. But sometimes it is too late. CA notices that the new node doesn't solve the problem and ask for another, hoping that this time it will get the node with the labels.  The node is added by MIG but it takes a minute or more for the node to start and register in kubernetes. At this moment the labeling is started. The list of nodes to be labeled is taken from MIG. The extra node is there. But it is not in kubernetes yet. So 404 error is returned on labeling attempt and test fails.

This PR filters the list of nodes to be labeled and applies the labels only on the fully registered nodes.

Fixes 404 in #33754

cc: @jszczepkowski @piosz @fgrzadkowski