Automatic merge from submit-queue

Change some node e2e test to use the prepull image framework.

Fix https://github.com/kubernetes/kubernetes/issues/28868.

Node e2e test framework pre-pulls all images in [image_list.go](https://github.com/kubernetes/kubernetes/blob/bc2f223f5a3dcbd38cbdf36b5a18e0f596d8fea5/test/e2e_node/image_list.go)

All node e2e test should use image from the "image_list". If a test needs new image, we should update the image_list to include the new image.

/cc @kubernetes/sig-node to notice people to use `image_list` when adding test. :)

Revert "add tokenreviews endpoint to implement webhook"

Automatic merge from submit-queue

Use sharedPodInformer in ReplicaSet controller

@caesarxuchao 

Automatic merge from submit-queue

Optimise getAPIGroupVersion function in genericapiserver.go

About the newAPIGroupVersion and the getAPIGroupVersion function  in genericapiserver.go, I think they can optimise.

Automatic merge from submit-queue

Fix init container update validation for pods

Partial fix #26840

The remaining issues with `kubectl apply` on pods with init containers
are caused by temporary annotation-based representation and
will resolve themselves once init containers leave alpha state.
Also, this PR makes sure internal and external objects don't get mixed up by the
PATCH handler (see related issue #25106).

This PR is an alternative for #28557 which met criticism from @smarterclayton 
and @liggitt for working around the temporary issue with annotations.
#28557 is a full fix for #26840 and contains an e2e test that cannot pass
without the `VolumeMounts` workaround. As there appears to be no
good way to include an e2e test that's known to be failing in k8s source,
I've removed it from this PR.

Either this PR or #28557 should be applied, but not both.

Automatic merge from submit-queue

add tokenreviews endpoint to implement webhook

Wires up an API resource under `apis/authentication.k8s.io/v1beta1` to expose the webhook token authentication API as an API resource.  This allows one API server to use another for authentication and uses existing policy engines for the "authoritative" API server to controller access to the endpoint.

@cjcullen you wrote the initial type

Automatic merge from submit-queue

adds source debug build options

See issue & discussion here: #28227

Enables source debugging the Kubernetes binaries with tools like delve by providing the user with the ability to provide debug build options to the glang compiler.
Signed-off-by: Mike Brown <brownwm@us.ibm.com>

Automatic merge from submit-queue

Fix "PVC Volume not detached if pod deleted via namespace deletion" issue

Fixes #29051: "PVC Volume not detached if pod deleted via namespace deletion"

This PR:
* Fixes a bug in `desired_state_of_the_world_populator.go` to check the value of `exists` returned by the `podInformer` so that it can delete pods even if the delete event is missed (or fails).
* Reduces the desired state of the world populators sleep period from 5 min to 1 min (reducing the amount of time a volume would remain attached if a volume delete event is missed or fails).

Automatic merge from submit-queue

AWS kube-up: Fix unbound KUBE_MANIFESTS_TAR_URL variable in Salt config

It shouldn't be necessary for all distros to define this env variable (broken in 97f3f808).

This should get our e2es back creating AWS clusters, at least.

Fix broken verify-munge-docs build by running hack/update-munge-docs.sh.

Revert "Fix working_set calculation in kubelet"

Fix broken warning image link in docs

Automatic merge from submit-queue

Fix 'make test-e2e-node' example to run some tests

This PR fixes the minor problem that if you take the first example given in the Makefile and paste it verbatim,

    make test-e2e-node FOCUS=kubelet SKIP=container

it says something like "Will run 0 of 34 specs".  Several of the test suite descriptions contain "Kubernetes" with a capital K, but none match with a lower-case k.

The second part of the line is not great either, since some tests use "container" and some use "Container",   but fixing that seems to require some standardisation on how you name the tests.

Automatic merge from submit-queue

Allow mounts to run in parallel for non-attachable volumes

This PR:
* Fixes https://github.com/kubernetes/kubernetes/issues/28616
  * Enables mount volume operations to run in parallel for non-attachable volume plugins.
  * Enables unmount volume operations to run in parallel for all volume plugins.
* Renames `GoRoutineMap` to `GoroutineMap`, resolving a long outstanding request from @thockin: `"Goroutine" is a noun`

Automatic merge from submit-queue

Fix working_set calculation in kubelet

Bump cadvisor dependencies to latest head.

Fixes #28619

Automatic merge from submit-queue

ImagePuller refactoring

A plain refactoring
- Moving image pullers to a new pkg/kubelet/images directory
- Hiding image pullers inside the new ImageManager 

The next step is to consolidate the logic of the serialized and the parallel image pullers inside ImageManager

xref: #25577

Automatic merge from submit-queue

Fix node e2e (kubelet metrics) panic

On systems (rhel7, and fedora 24), the kubelet metrics node e2e panics:

```
[k8s.io] Kubelet metrics api when querying /stats/summary
  it should report resource usage through the stats api
  /root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e_node/kubelet_test.go:146
[BeforeEach] [k8s.io] Kubelet
  /root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/framework.go:132
STEP: Creating a kubernetes client
STEP: Building a namespace api object
Jul 18 10:45:45.004: INFO: Skipping waiting for service account
[It] it should report resource usage through the stats api
  /root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e_node/kubelet_test.go:146
W0718 10:45:45.036201   17112 request.go:347] Field selector: v1 - pods - metadata.name - stats-busybox-4edb73ec-4cf6-11e6-9ecc-52540041b7801: need to check if this is versioned correctly.
W0718 10:45:45.036213   17112 request.go:347] Field selector: v1 - pods - metadata.name - stats-busybox-4edb73ec-4cf6-11e6-9ecc-52540041b7800: need to check if this is versioned correctly.
E0718 10:50:45.036630   17112 streamwatcher.go:109] Unable to decode an event from the watch stream: net/http: request canceled (Client.Timeout exceeded while reading body)
Jul 18 10:50:45.036: INFO: Unexpected error occurred: timed out waiting for the condition
panic:
Your test failed.
Ginkgo panics to prevent subsequent assertions from running.
Normally Ginkgo rescues this panic so you shouldn't see it.

But, if you make an assertion in a goroutine, Ginkgo can't capture the panic.
To circumvent this, you should call

        defer GinkgoRecover()

at the top of the goroutine that caused this panic.


goroutine 68 [running]:
panic(0x1958fc0, 0xc8204ba6b0)
        /usr/lib/golang/src/runtime/panic.go:481 +0x3e6
k8s.io/kubernetes/vendor/github.com/onsi/ginkgo.Fail(0xc820302160, 0xb0, 0xc8204ba2c8, 0x1, 0x1)
        /root/upstream-code/gocode/src/k8s.io/kubernetes/vendor/github.com/onsi/ginkgo/ginkgo_dsl.go:244 +0x116
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc820820080, 0x2b2337d20268, 0x30a2da8, 0x0, 0x0, 0x0, 0x0, 0x30a2da8)
        /root/upstream-code/gocode/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:69 +0x32d
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).NotTo(0xc820820080, 0x2b2337d20268, 0x30a2da8, 0x0, 0x0, 0x0, 0x2b2337d20220)
        /root/upstream-code/gocode/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:43 +0x92
k8s.io/kubernetes/test/e2e/framework.ExpectNoError(0x2b2336968028, 0xc8200b30c0, 0x0, 0x0, 0x0)
        /root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/util.go:1811 +0x203
k8s.io/kubernetes/test/e2e/framework.(*Framework).CreatePods.func1(0xc8200a6540, 0xc820204f40, 0x33, 0xc8201576c0)
        /root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/pods.go:68 +0x68
created by k8s.io/kubernetes/test/e2e/framework.(*Framework).CreatePods
        /root/upstream-code/gocode/src/k8s.io/kubernetes/test/e2e/framework/pods.go:70 +0x213

Ginkgo ran 1 suite in 5m20.524270699s
Test Suite Failed
!!! Error in hack/make-rules/test-e2e-node.sh:132
  '"${ginkgo}" --focus=$focus --skip=$skip "${KUBE_ROOT}/test/e2e_node/" --report-dir=${report} -- --alsologtostderr --v 2 --node-name $(hostname) --disable-kubenet=true --build-services=true --start-services=true --stop-services=true "$test_args"' exited with status 1
Call stack:
  1: hack/make-rules/test-e2e-node.sh:132 main(...)
Exiting with status 1
make: *** [test-e2e-node] Error 1
```

Automatic merge from submit-queue

Kubelet: Set PruneChildren when removing image.

This is a bug introduced during switching to engine-api. https://github.com/kubernetes/kubernetes/issues/23563.

When removing image, there is an option `noprune`:
```
If prune is true, ancestor images will each attempt to be deleted quietly.
```

In go-dockerclient, the default value of the option is ["noprune=false"](https://github.com/fsouza/go-dockerclient/blob/master/image.go#L171), which means that ancestor images should be also removed. This is the expected behaviour.

However in engine-api, the option is changed to `PruneChildren`, and the default value is `PruneChildren=false`, which means that ancestor images won't be removed.
This makes `ImageRemove` only remove the first layer of the image, which causes the image garbage collection not working as expected.

This should be fixed in 1.3.
And thanks to @ronnielai for finding the bug! :)

/cc @kubernetes/sig-node 

Automatic merge from submit-queue

docker_manager: Correct determineContainerIP args

This could result in the network plugin not retrieving the pod ip in a
call to SyncPod when using the `exec` network plugin.
The CNI and kubenet network plugins ignore the name/namespace arguments,
so they are not impacted by this bug.

I verified the second included test failed prior to correcting the
argument order.

Fixes #29161 

cc @yujuhong 

Automatic merge from submit-queue

pkg/probe/http: don't compare error strings in tests

TestHTTPProbeChecker fails on the Go1.7 release candidates. The
package's history show that this was the case for Go1.5 and Go1.6
as well.

The test depend on errors holding specific string values, behavior
not guarenteed in the standard library API, and causing new test
failures every minor Go release. Just look for an error rather than
trying to inspect it using string comparison. If we feel this
impacts coverage we can add more test cases.

Fixes #15952

Automatic merge from submit-queue

Add defer

Add defer？

Automatic merge from submit-queue

Fix RBAC authorizer of ServiceAccount

RBAC authorizer assigns a role to a wrong service account.

How to reproduce

1.Create role and rolebinding to allow default user in kube-system namespace to read secrets in kube-system namespace.

```
# kubectl create -f role.yaml
# kubectl create -f binding.yaml
```

```yaml
# role.yaml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
  name: secret-reader
  namespace: kube-system
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "watch", "list"]
    nonResourceURLs: []
```

```yaml
# binding.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
  name: read-secrets
  namespace: kube-system
subjects:
  - kind: ServiceAccount
    name: default
    namespace: kube-system
roleRef:
  kind: Role
  namespace: kube-system
  name: secret-reader
  apiVersion: rbac.authorization.k8s.io/v1alpha1
```

2.Set a credential of default user

```
$ kubectl config set-credentials default_user --token=<token_of_system:serviceaccount:kube-system:default>
$ kubectl config set-context default_user-context --cluster=test-cluster --user=default_user
$ kubectl config use-context default_user-context
```

3.Try to get secrets as default user in kube-system namespace

```
$ kubectl --namespace=kube-system get secrets
the server does not allow access to the requested resource (get secrets)
```

As shown above, default user could not access to secrets.
But if I have kube-system user in default namespace, it is allowed access to secrets.


4.Create a service account and try to get secrets as kube-system user in default namespace

```
# kubectl --namespace=default create serviceaccount kube-system
serviceaccount "kube-system" created
$ kubectl config set-credentials kube-system_user --token=<token_of_system:serviceaccount:default:kube-system>
$ kubectl config set-context kube-system_user-context --cluster=test-cluster --user=kube-system_user
$ kubectl config use-context kube-system_user-context
$ kubectl --namespace=kube-system get secrets
NAME                  TYPE                                  DATA      AGE
default-token-8pyb3   kubernetes.io/service-account-token   3         4d

```

Signed-off-by: Vishnu kannan <vishnuk@google.com>

Automatic merge from submit-queue

Information is opposite to real meaning to express

master is not equal to expectedMaster, the meaning should be the master is unexpected:
	master, err := mesosCloud.Master(clusterName)
	if master != expectedMaster {
		t.Fatalf("Master returns the expected value: (expected: %#v, actual: %#v", expectedMaster, master)

Automatic merge from submit-queue

Modify err output format from %s to %v

t.Errorf err output format should be %v

Automatic merge from submit-queue

format number not consistent with real variable number

glog.Infof format number not consistent with real variable number， should add %s for second var because loadBalancerName is string:
func (c *Cloud) ensureLoadBalancer(namespacedName types.NamespacedName, loadBalancerName string, ...

Automatic merge from submit-queue

"server.go" directory error

In file "docs\devel\profiling.md", line 55:
"In 'pkg/master/server/server.go' more servers are created“
Here server.go directory is wrong, should be :pkg/kubelet/server/server.go

Automatic merge from submit-queue

Fix wrong variable of error

Should not log `err` but `cerr`.

Automatic merge from submit-queue

Proposal for ControllerReference

Proposal for including the reference pointing to the owning "collection" (controller) for objects that can be grouped. The goal is to prevent a situation when two controllers are fighting over some resources.

cc @bgrant0607 @lavalamp @caesarxuchao @davidopp @fgrzadkowski @wojtek-t @kubernetes/sig-api-machinery

Automatic merge from submit-queue

Warn when missing cloud-provider on kube controller

It shouldn't be necessary for all distros to define this env variable.

to avoid panic

Signed-off-by: Vishnu kannan <vishnuk@google.com>

Automatic merge from submit-queue

Allow handlers earlier in a request flow to inject a UID for an object

This lets admission controllers specify a stable UID for an object prior to its creation.  That lets the admission controller then record a reference to the object on another resource using that stable UID prior to the object being created.  This would be a prerequisite for supporting quota reservations.

/cc @smarterclayton @lavalamp @deads2k