Automatic merge from submit-queue

integration: cleanup unused API

Automatic merge from submit-queue

Make GKE detect-instance-groups work on Mac.

Make the fix from #27803 also work on mac.

The GNU `expr` command supports both the `expr match STRING REGEXP` and `expr STRING : REGEXP` command syntax.

The BSD `expr` command only has the `expr STRING : REGEXP` syntax.

@fabioy @a-robinson 

Automatic merge from submit-queue

Declare out of disk when there is no free inodes

#21546

Automatic merge from submit-queue

Add MinReadySeconds to rolling updater

Add MinReadySeconds support to RollingUpdater that allows to specify the number of seconds to wait on top of the pod is "ready" because its readiness probe passed.

Automatic merge from submit-queue

Includes the number of free inodes in stat summary

#21546 

Automatic merge from submit-queue

Adding inode availability in eviction policy

Updates the proposal to include inodes as an eviction parameter.

Automatic merge from submit-queue

Remove reference to documentation webpage from kubectl describe nodes output

Fixes #23778

Update CHANGELOG

Summarized what I thought were the most interesting changes for
all users, and for specific cloud providers.

Still need better list of breaking changes.

Automatic merge from submit-queue

Add a new container runtime interface

This PR includes a proposal and a Go file to re-define the container runtime interface.
This is based on the original doc: https://docs.google.com/document/d/1ietD5eavK0aTuMQTw6-21r67UU73_vqYSUIPFdA0J5Q/

The umbrella issues is #22964

/cc @kubernetes/sig-node 

Automatic merge from submit-queue

Use `CreatedByAnnotation` constant

A nit but didn't want the strings to get out of sync.
Signed-off-by: Doug Davis <dug@us.ibm.com>

Automatic merge from submit-queue

Add additional testing scenarios for compute resource requests=0

I was asked about the qos tier of a pod that specified 

`--requests=cpu=0,memory=0 --limits=cpu=100m,memory=1Gi`

and in just investigating current behavior, realized we should have an explicit test case to ensure that 0 values are preserved in defaulting passes, and that this is still a burstable pod (but the lowest for that tier as it related to eviction)

/cc @vishh 

This commit includes a proposal and a Go file to re-define the container
runtime interface.

Note that this is an experimental interface and is expected to go through
multiple revisions once developers start implementing against it. As stated in
the proposal, there are also individual issues to carry discussions of
specific features.

Automatic merge from submit-queue

Update scheduler_extender.md

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

The filter call should actually return a schedulerapi.ExtenderFilterResult with an api.NodeList in it, instead of a raw api.NodeList.

Automatic merge from submit-queue

Update "kubectl get all" to display resource type as part of name

fixes #23838
release-note-none

When running "kubectl get all", or printing any output with mixed resource kinds, an additional column is added to the output with each resource's kind:

`kubectl get all --all-namespaces`
```
NAMESPACE     NAME                         DESIRED          CURRENT       AGE
default       rc/docker-registry-1         1                1             23h
testproject   rc/node-1                    0                0             2d
NAMESPACE     NAME                         CLUSTER-IP       EXTERNAL-IP   PORT(S)                 AGE
default       svc/docker-registry          172.30.36.42     <none>        5000/TCP                23h
default       svc/kubernetes               172.30.0.1       <none>        443/TCP,53/UDP,53/TCP   7d
testproject   svc/ruby-ex                  172.30.187.128   <none>        8080/TCP                6d
NAMESPACE     NAME                         READY            STATUS        RESTARTS                AGE
default       po/docker-registry-1-cpf8o   1/1              Running       1                       23h
```

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

kubectl: don't display an empty list when trying to get a single resource that isn't found

Return immediately when attempting to get a singular resource that isn't found, so that we avoid
printing out a List if the output format is something like json or yaml.

Before:

```
$ kubectl get pod/foo -o yaml
apiVersion: v1
items: []
kind: List
metadata: {}
pods "foo" not found
```

After:

```
$ kubectl get pod/foo -o yaml
pods "foo" not found
```

Fixes #28243 

@kubernetes/kubectl @kubernetes/rh-ux @smarterclayton @liggitt @deads2k @metral 

Automatic merge from submit-queue

Graceful deletion bumps object's generation

Fix. #24935

cc @caesarxuchao 

Automatic merge from submit-queue

Add version and flag logging to kube-dns.

Helps with #28358

Stopgap until we have a general solution for #27744

Automatic merge from submit-queue

Print kube-dns pod logs on federation e2e test failure.

Automatic merge from submit-queue

Fix node confomance test

Fixes https://github.com/kubernetes/kubernetes/issues/28255, https://github.com/kubernetes/kubernetes/issues/28250, https://github.com/kubernetes/kubernetes/issues/28341.

The main reason of the flake is that in the failed test expects the `PodPhase` to keep `Pending`. It did `Eventually` check and `Consistently` check for 5 seconds. However, the default `PodPhase` is `Pending`, when the check passes, the `PodStatus` could still be in default state.

After that, the test expects the container status to be `Waiting`, which may not be the case, because the default `ContainerStatuses` is empty, and the pod could still be in the default state.

This PR changes the test to ensure `ContainerStatuses` first and then check the `PodPhase` after that.

Mark P1 because the test fails relatively frequently and does block some PRs.

@pwittrock 

/cc @liangchenye @ncdc 
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

E2e tests to check whether cluster autoscaling scale down works when one node pool is not autoscaled

E2e tests to check whether cluster autoscaling scale down works when one node pool is not autoscaled.

Automatic merge from submit-queue

Allow specifying secret data using strings

This PR allows specifying non-binary data values in `Secret` objects as `"stringData":{"key":"string value"}`, in addition to the existing base64 []byte serializations in the `data` field.

On write, the keys and values in the `stringData` field are merged to the `data` map, overwriting any values already present in the `data` map. The move is one-way, the `stringData` field is never output when reading from the API.

A Secret could be created like this:
```
{
  "kind":"Secret",
  "apiVersion":"v1",
  "metadata":{"name":"mysecret"},
  "data":{
    "image":"<base64-encoded-jpg>"
  },
  "stringData":{
    "username": "myuser",
    "password": "mypassword"
  }
}
```
and when read from the API would look like this:
```
{
  "kind":"Secret",
  "apiVersion":"v1",
  "metadata":{"name":"mysecret",...},
  "data":{
    "image":"<base64-encoded-jpg>"
    "username": "bXl1c2Vy",
    "password": "bXlwYXNzd29yZA=="
  }
}
```

Automatic merge from submit-queue

fixed the typo causing incorrect comparison with cVer and cVer; now checking against sVer

fixes #25252

Automatic merge from submit-queue

Kubelet: Add the docker semver back.

Fixes #28221.
This PR:
1) Add the semver back https://github.com/kubernetes/kubernetes/pull/20020
2) Remove the code in https://github.com/kubernetes/kubernetes/pull/20827, because docker 1.8 is not officially supported now, and we want to deprecate it. https://github.com/kubernetes/kubernetes/pull/27208
3) Add a test for docker version comparison.

XRef https://github.com/kubernetes/kubernetes/pull/28223

@yujuhong 
/cc @ingvagabund 

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Install pip in kubekins test image and install AWS cli when needed

Follow-on to https://github.com/kubernetes/test-infra/pull/66. This fixes AWS hopefully.

I haven't pushed the image yet.

cc @fejta 

Automatic merge from submit-queue

fix return value

Automatic merge from submit-queue

invalid arg during applying SELinux label

Tested with Ubuntu 16.04 with SELINUX support. Actually, after `chcon `with long label got: `chcon: failed to set type security context component to ‘system_u:object_r:svirt_sandbox_file_t:s0’: Invalid argument`

Automatic merge from submit-queue

Use slices of items to clean up after tests

Fixes #27582.

We used to maintain a pointer variable for each process to kill after the
tests finish. @lavalamp suggested using a slice instead, which is a much
cleaner solution. This implements @lavalamp's suggestion and also extends
the idea to tracking directories that need to be removed after the tests finish.

This also means that we should no longer check for nil `killCmd`s inside
`func (k *killCmd) Kill() error {...}` (see #27582 and #27589). If a nil
`killCmd` makes it in there, something is bad elsewhere and we want to see
the nil pointer exception immediately.

Mentioning @timstclair and @euank wrt the original issue/PR. 

Automatic merge from submit-queue

Should set default value for --service-node-port-range flag before verifying

For the flag `--service-node-port-range` of kube-apiserver, we know that it defaults to `30000-32767` if not specified. But if we only pass the flag `--kubernetes-service-node-port` with a valid value between `30000-32767` when starting kube-apiserver, a fatal error will occurs as the last below. It means that service port range is not 30000-32767 but empty. The log is from code [DefaultAndValidateRunOptions-->ValidateRunOptions-->verifyServiceNodePort](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/genericapiserver.go#L580) where the flags are verified. 

After tracing the apiserver related code, we can find the call stack:
```
func main() {
	......
	s := options.NewAPIServer()
	......
	app.Run(s)
	......
}
```
In the `app.Run`, it calls [genericapiserver.DefaultAndValidateRunOptions(s.ServerRunOptions)](https://github.com/xiangpengzhao/kubernetes/blob/master/cmd/kube-apiserver/app/server.go#L80). But the `--kubernetes-service-node-port` hasn't been defaulted before there, so it's empty. It's then defaulted in `app.Run`-->[master.New](https://github.com/xiangpengzhao/kubernetes/blob/master/cmd/kube-apiserver/app/server.go#L276)-->[genericapiserver.New](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/master/master.go#L179)-->[setDefaults](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/genericapiserver.go#L338)-->[defaultServiceNodePortRange](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/genericapiserver.go#L281).

So, we have to set default value for `--kubernetes-service-node-port` in [NewServerRunOptions](https://github.com/xiangpengzhao/kubernetes/blob/master/pkg/genericapiserver/options/server_run_options.go#L105), as is done for `--secure-port` and/or `--insecure-port`. The `NewServerRunOptions` will be called in options.[NewAPIServer](https://github.com/xiangpengzhao/kubernetes/blob/master/cmd/kube-apiserver/app/options/options.go#L50)().

Hope that I have described the issue clearly. Thanks!

```
root@vm:~# kube-apiserver --etcd-servers=http://172.16.1.11:4001 --service-cluster-ip-range=192.168.122.0/24 --insecure-bind-address=0.0.0.0 --logtostderr=false --log-dir=/home/paas/zxp/log/kube --v=10 --kubernetes-service-node-port=30001 &
[2] 24629
root@vm:~# F0627 23:46:37.308726   24629 genericapiserver.go:580] Kubernetes service port range  doesn't contain 30001
goroutine 1 [running]:
k8s.io/kubernetes/vendor/github.com/golang/glog.stacks(0x44f2500, 0x0, 0x0, 0x0)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:766 +0xb8
k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).output(0x44d2020, 0xc800000003, 0xc820238000, 0x438c73b, 0x13, 0x244, 0x0)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:717 +0x259
k8s.io/kubernetes/vendor/github.com/golang/glog.(*loggingT).printf(0x44d2020, 0xc800000003, 0x3223dc0, 0x33, 0xc8204c4cc8, 0x2, 0x2)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:655 +0x1d4
k8s.io/kubernetes/vendor/github.com/golang/glog.Fatalf(0x3223dc0, 0x33, 0xc8204c4cc8, 0x2, 0x2)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/vendor/github.com/golang/glog/glog.go:1145 +0x5d
k8s.io/kubernetes/pkg/genericapiserver.verifyServiceNodePort(0xc8202a8400)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:580 +0x1d5
k8s.io/kubernetes/pkg/genericapiserver.ValidateRunOptions(0xc8202a8400)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:605 +0x2f
k8s.io/kubernetes/pkg/genericapiserver.DefaultAndValidateRunOptions(0xc8202a8400)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/pkg/genericapiserver/genericapiserver.go:612 +0x4e
k8s.io/kubernetes/cmd/kube-apiserver/app.Run(0xc820224fc0, 0x0, 0x0)
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/app/server.go:80 +0x70
main.main()
	/home/paas/zxp/code/k8s/fork/kubernetes/_output/local/go/src/k8s.io/kubernetes/cmd/kube-apiserver/apiserver.go:50 +0x121

[2]+  Exit 255                kube-apiserver --etcd-servers=http://172.16.1.11:4001 --service-cluster-ip-range=192.168.122.0/24 --insecure-bind-address=0.0.0.0 --logtostderr=false --log-dir=/home/paas/zxp/log/kube --v=10 --kubernetes-service-node-port=30001
root@vm:~#

```

Automatic merge from submit-queue

Add printf formatting for rrset objects.

Without this you just get two pointers in the debug log.

Before:

```
I0627 21:48:44.136615       1 dns.go:215] Existing recordset {0xc820168830 0xc820691540} is not equal to needed recordset &{0xc820168848 0xc820686040}, removing existing and adding needed.
```

After:

```
I0627 22:26:46.221856       1 dns.go:215] Existing recordset                <(clouddns) "federated-service.e2e-tests-service-cuza5.federation.svc.us-central1-c.us-central1.kube.5yetis.net." type=CNAME rrdatas=["federated-service.e2e-tests-service-cuza5.federation.svc.us-central1.kube.5yetis.net."] ttl=180>
I0627 22:26:46.221885       1 dns.go:216] ... not equal to needed recordset <(clouddns) "federated-service.e2e-tests-service-cuza5.federation.svc.us-central1-c.us-central1.kube.5yetis.net." type=CNAME rrdatas=["federated-service.e2e-tests-service-cuza5.federation.svc.us-central1.kube.5yetis.net."] ttl=180>
I0627 22:26:46.221919       1 dns.go:217] ... removing existing and adding needed.
```