Automatic merge from submit-queue

AWS: Support default value for ExternalHost

Automatic merge from submit-queue

Make a consistent name ( GlusterFS instead of Gluster) in variables a…

Signed-off-by: Humble Chirammal hchiramm@redhat.com

Automatic merge from submit-queue

Switch cluster autoscaler to 0.4.0-beta1

Switch Kubernetes to new 0.4.0-beta1 Cluster Autoscaler. The release contains mainly bugfixes:
* unschedulable nodes don't stop cluster autoscaler
* better logging
* events for deltions
* bulk delete for empty nodes

cc: @fgrzadkowski @piosz @jszczepkowski

Automatic merge from submit-queue

HPA: Consider unready pods separately

**Release note**:

``` release-note
The Horizontal Pod Autoscaler now takes the readiness of pods into account when calculating desired replicas.
```

Currently, the HPA considers unready pods the same as ready pods when
looking at their CPU and custom metric usage.  However, pods frequently
use extra CPU during initialization, so we want to consider them
separately.

This commit causes the HPA to consider unready pods as having 0 CPU
usage when scaling up, and ignores them when scaling down.  If, when
scaling up, factoring the unready pods as having 0 CPU would cause a
downscale instead, we simply choose not to scale.  Otherwise, we simply
scale up at the reduced amount calculated by factoring the pods in at
zero CPU usage.

Similarly, if we are missing metrics for any pods, those pods will be
considered as having 0% CPU when scaling up, and 100% CPU when
scaling down.  As with the unready pods calculation, this cannot change
the direction of the scale.

The effect is that unready pods cause the autoscaler to be a bit more
conservative -- large increases in CPU usage can still cause scales,
even with unready pods in the mix, but will not cause the scale factors
to be as large, in anticipation of the new pods later becoming ready and
handling load.

Automatic merge from submit-queue

options, kube-apiserver: clarify scheme on etcd endpoints

**What this PR does / why we need it**:

Fix typo in `kube-apiserver` flag.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

None

Automatic merge from submit-queue

'Max' and 'MIn' don't seem to used anywhere, so I would suggest removing them

Signed-off-by: Yanqiang Miao miao.yanqiang@zte.com.cn

Automatic merge from submit-queue

improve and modify log

1, the content of a unified writing, compared to the following line of failure (314th lines)
2, “instance” should be “node”

Automatic merge from submit-queue

Print osImage and kubeletVersion for nodes before and after GCE upgrade

This will print, e.g.:
```
== Pre-Upgrade Node OS and Kubelet Versions ==
name: "e2e-test-mtaufen-master", osImage: "Google Container-VM Image", kubeletVersion: "v1.4.5-beta.0.45+90d209221ec8dc-dirty"
name: "e2e-test-mtaufen-minion-group-jo79", osImage: "Debian GNU/Linux 7 (wheezy)", kubeletVersion: "v1.4.5-beta.0.45+90d209221ec8dc-dirty"
name: "e2e-test-mtaufen-minion-group-ox5l", osImage: "Debian GNU/Linux 7 (wheezy)", kubeletVersion: "v1.4.5-beta.0.45+90d209221ec8dc-dirty"
name: "e2e-test-mtaufen-minion-group-qvbq", osImage: "Debian GNU/Linux 7 (wheezy)", kubeletVersion: "v1.4.5-beta.0.45+90d209221ec8dc-dirty"
```

Let me know what output format you prefer and I'll see if I can make it work, I have the extent of flexibility allowed by jsonpath.

Automatic merge from submit-queue

Add timeout argument to ExecInContainer

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**: This is related to https://github.com/kubernetes/kubernetes/issues/26895. It brings a timeout to the signature of `ExecInContainer` so that we can take timeouts into account in the future. Unlike my first attempt in https://github.com/kubernetes/kubernetes/pull/27956, it doesn't immediately observe the timeout, because it is impossible to do it with the current state of the Docker Remote API (the default exec handler implementation).

**Special notes for your reviewer**: This shares commits with https://github.com/kubernetes/kubernetes/pull/27956, but without some of them that have more controversial implications (actually supporting the timeouts). The original PR shall be closed in the current state to preserve the history (instead of dropping commits in that PR).

Pinging the original people working on this change: @ncdc @sttts @vishh @dims 

**Release note**:

<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->

``` release-note
NONE
```

Automatic merge from submit-queue

Fix non-default etcd image handling in kubemark

Doesn't touch production code.

Automatic merge from submit-queue

Deleting dead code from federation service controller

Automatic merge from submit-queue

Node Conformance Test: Containerize the node e2e test

For #30122, #30174.
Based on #32427, #32454.

**Please only review the last 3 commits.**

This PR packages the node e2e test into a docker image:
- 1st commit: Add `NodeConformance` flag in the node e2e framework to avoid starting kubelet and collecting system logs. We do this because:
  - There are all kinds of ways to manage kubelet and system logs, for different situation we need to mount different things into the container, run different commands. It is hard and unnecessary to handle the complexity inside the test suite.
- 2nd commit: Remove all `sudo` in the test container. We do this because:
  - In most container, there is no `sudo` command, and there is no need to use `sudo` inside the container.
  - It introduces some complexity to use `sudo` inside the test. (https://github.com/kubernetes/kubernetes/issues/29211, https://github.com/kubernetes/kubernetes/issues/26748) In fact we just need to run the test suite with `sudo`.
- 3rd commit: Package the test into a docker container with corresponding `Makefile` and `Dockerfile`. We also added a `run_test.sh` script to start kubelet and run the test container. The script is only for demonstration purpose and we'll also use the script in our node e2e framework. In the future, we should update the script to start kubelet in production way (maybe with `systemd` or `supervisord`).

@dchen1107 @vishh 
/cc @kubernetes/sig-node @kubernetes/sig-testing



**Release note**:

<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->

``` release-note
Release alpha version node test container gcr.io/google_containers/node-test-ARCH:0.1 for users to verify their node setup.
```

Automatic merge from submit-queue

Adding cadcading deletion support for federated secrets

Ref https://github.com/kubernetes/kubernetes/issues/33612

Adding cascading deletion support for federated secrets.
The code is same as that for namespaces.  Just ensuring that DeletionHelper functions are called at right places in secret_controller.
Also added e2e tests.

cc @kubernetes/sig-cluster-federation @caesarxuchao

```release-note
federation: Adding support for DeleteOptions.OrphanDependents for federated secrets. Setting it to false while deleting a federated secret also deletes the corresponding secrets from all registered clusters.
```

Currently, the HPA considers unready pods the same as ready pods when
looking at their CPU and custom metric usage.  However, pods frequently
use extra CPU during initialization, so we want to consider them
separately.

This commit causes the HPA to consider unready pods as having 0 CPU
usage when scaling up, and ignores them when scaling down.  If, when
scaling up, factoring the unready pods as having 0 CPU would cause a
downscale instead, we simply choose not to scale.  Otherwise, we simply
scale up at the reduced amount caculated by factoring the pods in at
zero CPU usage.

The effect is that unready pods cause the autoscaler to be a bit more
conservative -- large increases in CPU usage can still cause scales,
even with unready pods in the mix, but will not cause the scale factors
to be as large, in anticipation of the new pods later becoming ready and
handling load.

Similarly, if there are pods for which no metrics have been retrieved,
these pods are treated as having 100% of the requested metric when
scaling down, and 0% when scaling up.  As above, this cannot change the
direction of the scale.

This commit also changes the HPA to ignore superfluous metrics -- as
long as metrics for all ready pods are present, the HPA we make scaling
decisions.  Currently, this only works for CPU.  For custom metrics, we
cannot identify which metrics go to which pods if we get superfluous
metrics, so we abort the scale.

Automatic merge from submit-queue

kuberuntime: remove unused GetNs code

Automatic merge from submit-queue

Deploy kube-dns with cluster-proportional-autoscaler

This PR integrates [cluster-proportional-autoscaler](https://github.com/kubernetes-incubator/cluster-proportional-autoscaler) with kube-dns for DNS horizontal autoscaling. 

Fixes #28648 and #27781.

Automatic merge from submit-queue

CRI: Add security context for sandbox/container

Part of #29478. This PR 
- adds security context for sandbox and fixes #33139
- encaps container security context to `SecurityContext` and adds missing features
- Note that capability is not fully accomplished in this PR because it is under discussion at  #33614.

cc/ @yujuhong @yifan-gu @Random-Liu @kubernetes/sig-node

Automatic merge from submit-queue

Emit event when scheduling daemon fails

Automatic merge from submit-queue

Fix kubectl drain for statefulset

Support deleting pets for `kubectl drain`. 
Use evict to delete pods.

Fixes: #33727

```release-note
Adds support for StatefulSets in kubectl drain.
Switches to use the eviction sub-resource instead of deletion in kubectl drain, if server supports.
```

@foxish @caesarxuchao

Automatic merge from submit-queue

Rename experimental-runtime-integration-type to experimental-cri

Also rename the field in the component config to `EnableCRI`

Automatic merge from submit-queue

copy PodInitialized condition to v1

Copied from pkg/api/types.go

I might batch this change with other similar changes, but want to gets this reviewed first.

cc @dchen1107 @yujuhong

Automatic merge from submit-queue

Add constants for node role labels

Automatic merge from submit-queue

Default kube-proxy to the old behavior for proxier sync.

Fix #36281.

This PR defaults `minSyncPeriod` to 0 and makes kube-proxy fall back to the old behavior to fix the immediate problem #36266.

@bprashanth

Automatic merge from submit-queue

Fix config file names inside fluentd-gcp image

Need this in order to merge https://github.com/kubernetes/kubernetes/pull/36358

Because on container-vm we need implicitly used configuration file

@piosz

Automatic merge from submit-queue

Fix a data race condition in federation namespace controller

Fixes https://github.com/kubernetes/kubernetes/issues/36364

Fixing federation namespace controller to create a copy of namespace fetched from the store before editing it. Without it there is a data race condition with other readers of the store.

cc @kubernetes/sig-cluster-federation @mwielgus @madhusudancs

Automatic merge from submit-queue

Node Conformance Test: Mark more conformance test

For https://github.com/kubernetes/kubernetes/issues/30122.

This PR:
1) Removes unused image test.
2) Marks more conformance tests based on https://docs.google.com/spreadsheets/d/1yib6ypfdWuq8Ikyo-rTcBGHe76Xur7tGqCKD9dkzx0Y/edit?usp=sharing.

Notice that 2 tests are not marked conformance for now:
1. **OOM score test:** The test is serial and is verifying host PID directly. The test should start a pod with PID=host and verify inside the pod. @vishh 
2. **Summary api test:** The assumption made in the test doesn't always make sense for arbitrary image, for example: The fs capacity bounds is only [(100mb, 100gb)](https://github.com/kubernetes/kubernetes/blob/master/test/e2e_node/summary_test.go#L62). @timstclair 
3. We should consider mark **[cgroup manager test](https://github.com/kubernetes/kubernetes/blob/master/test/e2e_node/cgroup_manager_test.go)** as conformance test. 

@dchen1107 @vishh @timstclair 
/cc @kubernetes/sig-node

Automatic merge from submit-queue

Removed EXPERIMENTAL from KUBE_REPLICATE_EXISTING_MASTER flag.

The e2e tests cover cases like cluster size changed, parameters
changed, ConfigMap got deleted, autoscaler pod got deleted, etc.
They are separated into a fast part(could be run parallelly) and
a slow part(put in [serial]). The fast part of the e2e tests cost
around 50 seconds to run.