Automatic merge from submit-queue

Kubelet: Set PruneChildren when removing image.

This is a bug introduced during switching to engine-api. https://github.com/kubernetes/kubernetes/issues/23563.

When removing image, there is an option `noprune`:
```
If prune is true, ancestor images will each attempt to be deleted quietly.
```

In go-dockerclient, the default value of the option is ["noprune=false"](https://github.com/fsouza/go-dockerclient/blob/master/image.go#L171), which means that ancestor images should be also removed. This is the expected behaviour.

However in engine-api, the option is changed to `PruneChildren`, and the default value is `PruneChildren=false`, which means that ancestor images won't be removed.
This makes `ImageRemove` only remove the first layer of the image, which causes the image garbage collection not working as expected.

This should be fixed in 1.3.
And thanks to @ronnielai for finding the bug! :)

/cc @kubernetes/sig-node 

Automatic merge from submit-queue

docker_manager: Correct determineContainerIP args

This could result in the network plugin not retrieving the pod ip in a
call to SyncPod when using the `exec` network plugin.
The CNI and kubenet network plugins ignore the name/namespace arguments,
so they are not impacted by this bug.

I verified the second included test failed prior to correcting the
argument order.

Fixes #29161 

cc @yujuhong 

Automatic merge from submit-queue

pkg/probe/http: don't compare error strings in tests

TestHTTPProbeChecker fails on the Go1.7 release candidates. The
package's history show that this was the case for Go1.5 and Go1.6
as well.

The test depend on errors holding specific string values, behavior
not guarenteed in the standard library API, and causing new test
failures every minor Go release. Just look for an error rather than
trying to inspect it using string comparison. If we feel this
impacts coverage we can add more test cases.

Fixes #15952

Automatic merge from submit-queue

Add defer

Add defer？

Automatic merge from submit-queue

Fix RBAC authorizer of ServiceAccount

RBAC authorizer assigns a role to a wrong service account.

How to reproduce

1.Create role and rolebinding to allow default user in kube-system namespace to read secrets in kube-system namespace.

```
# kubectl create -f role.yaml
# kubectl create -f binding.yaml
```

```yaml
# role.yaml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
  name: secret-reader
  namespace: kube-system
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "watch", "list"]
    nonResourceURLs: []
```

```yaml
# binding.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
  name: read-secrets
  namespace: kube-system
subjects:
  - kind: ServiceAccount
    name: default
    namespace: kube-system
roleRef:
  kind: Role
  namespace: kube-system
  name: secret-reader
  apiVersion: rbac.authorization.k8s.io/v1alpha1
```

2.Set a credential of default user

```
$ kubectl config set-credentials default_user --token=<token_of_system:serviceaccount:kube-system:default>
$ kubectl config set-context default_user-context --cluster=test-cluster --user=default_user
$ kubectl config use-context default_user-context
```

3.Try to get secrets as default user in kube-system namespace

```
$ kubectl --namespace=kube-system get secrets
the server does not allow access to the requested resource (get secrets)
```

As shown above, default user could not access to secrets.
But if I have kube-system user in default namespace, it is allowed access to secrets.


4.Create a service account and try to get secrets as kube-system user in default namespace

```
# kubectl --namespace=default create serviceaccount kube-system
serviceaccount "kube-system" created
$ kubectl config set-credentials kube-system_user --token=<token_of_system:serviceaccount:default:kube-system>
$ kubectl config set-context kube-system_user-context --cluster=test-cluster --user=kube-system_user
$ kubectl config use-context kube-system_user-context
$ kubectl --namespace=kube-system get secrets
NAME                  TYPE                                  DATA      AGE
default-token-8pyb3   kubernetes.io/service-account-token   3         4d

```

Automatic merge from submit-queue

Information is opposite to real meaning to express

master is not equal to expectedMaster, the meaning should be the master is unexpected:
	master, err := mesosCloud.Master(clusterName)
	if master != expectedMaster {
		t.Fatalf("Master returns the expected value: (expected: %#v, actual: %#v", expectedMaster, master)

Automatic merge from submit-queue

Modify err output format from %s to %v

t.Errorf err output format should be %v

Automatic merge from submit-queue

format number not consistent with real variable number

glog.Infof format number not consistent with real variable number， should add %s for second var because loadBalancerName is string:
func (c *Cloud) ensureLoadBalancer(namespacedName types.NamespacedName, loadBalancerName string, ...

Automatic merge from submit-queue

"server.go" directory error

In file "docs\devel\profiling.md", line 55:
"In 'pkg/master/server/server.go' more servers are created“
Here server.go directory is wrong, should be :pkg/kubelet/server/server.go

Automatic merge from submit-queue

Fix wrong variable of error

Should not log `err` but `cerr`.

Automatic merge from submit-queue

Proposal for ControllerReference

Proposal for including the reference pointing to the owning "collection" (controller) for objects that can be grouped. The goal is to prevent a situation when two controllers are fighting over some resources.

cc @bgrant0607 @lavalamp @caesarxuchao @davidopp @fgrzadkowski @wojtek-t @kubernetes/sig-api-machinery

Automatic merge from submit-queue

Warn when missing cloud-provider on kube controller

Automatic merge from submit-queue

Allow handlers earlier in a request flow to inject a UID for an object

This lets admission controllers specify a stable UID for an object prior to its creation.  That lets the admission controller then record a reference to the object on another resource using that stable UID prior to the object being created.  This would be a prerequisite for supporting quota reservations.

/cc @smarterclayton @lavalamp @deads2k 

RBAC authorizer assigns a role to a wrong service account.

Automatic merge from submit-queue

Start namespace controller in node e2e

Fix https://github.com/kubernetes/kubernetes/issues/28320.
Based on https://github.com/kubernetes/kubernetes/pull/28807, only the last 2 commits are new.

Before this PR, there was no namespace controller running in node e2e test infrastructure. We can not enable the [`delete-namespace`](https://github.com/kubernetes/kubernetes/blob/f2ddd60eb9e7e9e29f7a105a9a8fa020042e8e52/test/e2e/framework/test_context.go#L109) flag in the test framework.
So after the test running, there will be running pod left on the test node. This seems to be acceptable in our test infrastructure because we create an new instance each time.

However, in 1.4 we may want to provide part of the test as node conformance test to the user, they definitely don't want the test to leave tons of pods on their node after test running.

Currently, there is no easy way to only start namespace controller in kube-controller-manager (confirmed with @mikedanese), so in this PR I started a "uncontainerized" one in the test infrastructure.

This PR:
* Started the namespace controller in the node e2e test infrastructure and enable the automatic namespace deletion.
* Change the privileged test to use framework (@yujuhong), so that all node e2e tests are using the framework and test pods will be cleaned up by namespace controller.

/cc @kubernetes/sig-node 

Automatic merge from submit-queue

Make a link in docs clickable

Automatic merge from submit-queue

Info about enabling the hostpath provisioner

The README for persistent volume provisioning had information about the
hostpath provisioner for testing purposes, but lacked instructions on
how to enable it.

Automatic merge from submit-queue

Revert "Follow on for 1.4 to default HTTP2 on by default"

This reverts commit efe25553  
in order to address: #29001 #28537

Automatic merge from submit-queue

Fix generic scheduler test

Automatic merge from submit-queue

Fix a typo

Automatic merge from submit-queue

Bump the default etcd version in the Makefile to 3.0.3

Fixes: #29132

I haven't had time to manually validate the arm and arm64 version yet, but I think it should be fine.

cc @xiang90 @hongchaodeng @timothysc @lavalamp @wojtek-t @thockin @kubernetes/sig-scalability @Pensu @laboger

Automatic merge from submit-queue

Add proposal for service externalName

This is a proposal to address: #13748.

@smarterclayton @ncdc @thockin.  Please check this out when you have time, hopefully this is okay :-D

I created the proposal because was unsure if the feature would be able to go in if there isn't a proposal already merged, because of this mail to kubernetes-pm: https://groups.google.com/forum/#!topic/kubernetes-pm/Ki63EztfZMo.

So, IIUC it would be nice to have the proposal merged ASAP (I think the interface looks ok for all, so hopefully this will be easy) so we can have this feature in 1.4 as you guys ( @smarterclayton @ncdc ) need.

This reverts commit efe25553.

- Github flavored markdown doesn't support links inside codeblocks

Revert "Add and delete load balancer in front of apiserver."

Add and delete load balancer in front of apiserver.

Automatic merge from submit-queue

Change the name of the secret that delivers federation kubeconfig.

```release-note
Federation API server kubeconfig secret consumed by federation-controller-manager has a new name.

If you are upgrading your Cluster Federation components from v1.3.x, please run this command to migrate the federation-apiserver-secret to federation-apiserver-kubeconfig serect;

$ kubectl --namespace=federation get secret federation-apiserver-secret -o json | sed 's/federation-apiserver-secret/federation-apiserver-kubeconfig/g' | kubectl create -f -

You might also want to delete the old secret using this command:

$ kubectl delete secret --namespace=federation federation-apiserver-secret
```

The current name, federation-apiserver-secret, is very similar to the
other secret we have, federation-apiserver-secrets, that delivers
somewhat similar data but in a different format. This is extremely
confusing, particularly while debugging.

This change should soothe the pain.

cc @kubernetes/sig-cluster-federation 

Automatic merge from submit-queue

Make "attach" and "exec" rejection in proxy more explicit

```release-note
kubectl proxy changed to now allow urls to pods with "attach" or "exec" in the pod name
```

The more explicit regular expression for rejection makes a possibility of accessing pods (or any other resources) which contain "attach" or "exec" in their names via proxy API. It was not possible before.

Also, the reference for "run" resource was removed, because it doesn't exist in any of k8s APIs currently.

Fixes: #21464

Automatic merge from submit-queue

Revert "Scheduler predicates tests should consider unschedulable"

Reverts kubernetes/kubernetes#28970
See https://github.com/kubernetes/kubernetes/pull/28970#issuecomment-233817280

cc @pskrzyns @rmmh

Automatic merge from submit-queue

Move ExtractPodBandwidthResources test into appropriate package

Found during #28511, this test is in the wrong package currently.

cc @kubernetes/sig-network 

Automatic merge from submit-queue

AWS: Added experimental option to skip zone check

This pull request resolves #28380.  In the vast majority of cases, it is appropriate to validate the AWS region against a known set of regions.  However, there is the edge case where this is undesirable as Kubernetes may be deployed in an AWS-like environment where the region is not one of the known regions.

By adding the optional **DisableStrictZoneCheck true** to the **[Global]** section in the aws.conf file (e.g. /etc/aws/aws.conf) one can bypass the ragion validation. 

Automatic merge from submit-queue

dnsprovider: Use route53 page functions to avoid truncated results

The List<Type>Pages functions make it pretty easy to avoid result truncation;
switch to using them

Automatic merge from submit-queue

Format apiserver options

Trivial change, code formatting only: it was hard to read long lines, and my editor was really slow when scrolling through them.

Automatic merge from submit-queue

Ubuntu: Enable ssh compression when downloading binaries during cluster creation

<!--
Checklist for submitting a Pull Request

Please remove this comment block before submitting.

1. Please read our [contributor guidelines](https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md).
2. See our [developer guide](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md).
3. If you want this PR to automatically close an issue when it is merged,
   add `fixes #<issue number>` or `fixes #<issue number>, fixes #<issue number>`
   to close multiple issues (see: https://github.com/blog/1506-closing-issues-via-pull-requests).
4. Follow the instructions for [labeling and writing a release note for this PR](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes) in the block below.
-->
resolves #20971 by using the options provided by ssh. 

Native ssh compression has existed for years, and the server is free to disregard the setting, so this should be safe.

With things like the kube binaries I see about a 2x speed increase. 

```
λ time scp kubes-bin.tar   9.30.182.251:/mnt/build/kubin
kubes-bin.tar                                                                                                            100%  344MB  10.7MB/s   00:32

real	0m32.284s
user	0m1.679s
sys	0m1.263s

λ time scp -C kubes-bin.tar   9.30.182.251:/mnt/build/kubin
kubes-bin.tar                                                                                                            100%  344MB  22.9MB/s   00:15

real	0m14.810s
user	0m12.858s
sys	0m0.994s

λ ls -lah kubes-bin.tar
-rw-r--r--  1 mhb  staff   344M Jun  2 15:29 kubes-bin.tar

λ tar -tf kubes-bin.tar
kubectl
master/
master/etcd
master/etcdctl
master/flanneld
master/kube-apiserver
master/kube-controller-manager
master/kube-scheduler
node/
node/flanneld
node/kube-proxy
node/kubelet
```