Automatic merge from submit-queue

pkg/controller/node/nodecontroller: simplify mutex

Similar to #29598, we can rely on the zero-value construction behavior
to embed `sync.Mutex` into parent structs.

/CC: @saad-ali 

Automatic merge from submit-queue

Init container quota is inaccurate

Usage charged should be max of greater of init container or all regular
containers. Also, need to validate init container inputs

@derekwaynecarr

Automatic merge from submit-queue

pkg/util/goroutinemap: apply idiomatic Go cleanups

Package goroutinemap can be structurally simplified to be more
idiomatic, concise, and free of error potential.  No structural changes
are made.

It is unconventional declare `sync.Mutex` directly as a pointerized
field in a parent structure.  The `sync.Mutex` operates on pointer
receivers of itself; and by relying on that, the types that contain
those fields can be safely constructed using
https://golang.org/ref/spec#The_zero_value semantic.

The duration constants are already of type `time.Duration`, so
re-declaring that is redundant.

/CC: @saad-ali 

Automatic merge from submit-queue

Update vendored ginkgo.

Automatic merge from submit-queue

make the resource prefix in etcd configurable for cohabitation

This looks big, its not as bad as it seems.

When you have different resources cohabiting, the resource name used for the etcd directory needs to be configurable.  HPA in two different groups worked fine before.  Now we're looking at something like RC<->RS.  They normally store into two different etcd directories.  This code allows them to be configured to store into the same location.

To maintain consistency across all resources, I allowed the `StorageFactory` to indicate which `ResourcePrefix` should be used inside `RESTOptions` which already contains storage information.

@lavalamp affects cohabitation.
@smarterclayton @mfojtik prereq for our rc<->rs and d<->dc story.

Automatic merge from submit-queue

periodically reload the cni plugin configuration

Might fix #28787

Automatic merge from submit-queue

fix kubectl rolling update empty file cause panic issue

 ```release-note
Fix issue with kubectl panicing when passed files that do not exist.
```

Fix #29398 
@pwittrock @justinsb ptal. This just fix on the cmd layer, I am thinking whether we should return err from marshal&unmarshal if the reader is empty.

Automatic merge from submit-queue

Bump cadvisor dependencies to latest head. 

Fixes #28619 
Fixes #28997 

This is another try of https://github.com/kubernetes/kubernetes/pull/29153.

To update cadvisor godeps, we did:
* Bump up docker version to v1.11.2 for both cadvisor [https://github.com/google/cadvisor/pull/1388] and k8s.
* Bump up cadvisor `go-systemd` version to be the same with k8s [https://github.com/google/cadvisor/pull/1390]. Or else, a package `github.com/coreos/pkg/dlopen` will be removed by Godep, because it is used by new `go-systemd` in k8s, but not used by old `go-systemd` in cadvisor.
* Bump up runc version to be the same with docker v1.11.2 just in case.
* Add `github.com/Azure/go-ansiterm` dependency which is needed by docker v1.11.2.
* Change `pkg/util/term/`, because `SetWinsSize` is removed from windows platform in docker v1.11.2. [The first commit]

@vishh 
/cc @ncdc for the `pkg/util/term` change.

Automatic merge from submit-queue

storage error: precondition failure should return invalid object error

In introducing the preconditions by @caesarxuchao , if check preconditions failed, it returns resource version conflict error. This is the wrong error to return, and instead it should return invalid object error. We need to separate these two types of errors.
See the implementation in etcd3 [https://github.com/kubernetes/kubernetes/blob/master/pkg/storage/etcd3/store.go#L467].

Also renaming "ErrCodeResourceVersionConflicts" to "ErrCodeVersionConflicts" for simpler reading.

Automatic merge from submit-queue

kubectl container - Extract version better

1. Use --client since -c is deprecated now
2. The command (./kubectl version --client | grep -o 'GitVersion:"[^"]*"')
   now returns:
      GitVersion:"v1.4.0-alpha.1.784+ed3a29bd-dirty"
   so parse out the version better using sed

Related to #23708

Automatic merge from submit-queue

Func note is copied same as other one

Automatic merge from submit-queue

Log information wrong while wait.ForeverTestTimeout

Automatic merge from submit-queue

Fix mount collision timeout issue

Short- or medium-term workaround for #29555.  The root issue being fixed here is that the recent attach/detach work in the kubelet uses a unique volume name as a key that tracks the work that has to be done for each volume in a pod to attach/mount/umount/detach.  However, the non-attachable volume plugins do not report unique names for themselves, which causes collisions when a single secret or configmap is mounted multiple times in a pod.

This is still a WIP -- I need to add a couple E2E tests that ensure that tests break in the future if there is a regression -- but posting for early review.

cc @kubernetes/sig-storage 

Ultimately, I would like to refine this a bit further.  A couple things I would like to change:

1.  `GetUniqueVolumeName` should be a property ONLY of attachable volumes
2.  I would like to see the kubelet apparatus for attach/mount/umount/detach handle non-attachable volumes specifically to avoid things like the `WaitForControllerAttach` call that has to be done for those volume types now

Automatic merge from submit-queue

Remove unnecessary empty line.

Automatic merge from submit-queue

defer file.Close() in resource_printer.go

Automatic merge from submit-queue

Quota was not counting services with multiple nodeports properly

```release-note
If a service of type node port declares multiple ports, quota on "services.nodeports" will charge for each port in the service.
```

Fixes https://github.com/kubernetes/kubernetes/issues/29456

/cc @kubernetes/rh-cluster-infra @sdminonne 

Automatic merge from submit-queue

volume/flocker: plug time.Ticker resource leak

This commit ensures that `flockerMounter.updateDatasetPrimary` does not leak
running `time.Ticker` instances.  Upon termination of the consuming routine, we
stop the tickers.

```release-note
* flockerMounter.updateDatasetPrimary no longer leaks running time.Ticker instances.
  Upon termination of the consuming routine, we stop the tickers.
```

Automatic merge from submit-queue

Avoiding trying to gc images with no tags which are still in use

#29325

Automatic merge from submit-queue

LimitRanger and PodSecurityPolicy need to check more on init containers

Container limits not applied to init containers. HostPorts not checked on podsecuritypolicy

@pweil- @derekwaynecarr

Automatic merge from submit-queue

Give healthz more memory to mitigate #29688

This will recreate the rc but not the pods. At least on the clusters we patched, if the pods get recreated they'll ccome back up with the updated limits. 
#29688

For non-attachable volumes, do not call GetVolumeName on the plugin and instead
generate a unique name based on the identity of the pod and the name of the volume
within the pod.

Automatic merge from submit-queue

Fix wrapped volume race

**EDIT:** now covers configmap, secret, downwardapi & git_repo volume plugins.

Fixes #29297.

wrappedVolumeSpec used by configMapVolumeMounter and
configMapVolumeUnmounter contained a pointer to api.Volume which was
being patched by NewWrapperMounter/NewWrapperUnmounter, causing race
condition during configmap volume mounts.

See https://github.com/kubernetes/kubernetes/issues/29297#issuecomment-235403806 for complete explanation.
The subtle bug was introduced by #18445, it also can affect other volume plugins utilizing `wrappedVolumeSpec` technique, if this PR is correct/accepted will make more PRs for secrets etc. Although tmpfs variety of inner `emptyDir` volume appears to be less susceptible to this race, there's chance it can fail too.

The errors produced by this race look like this:
```Jul 19 17:05:21 ubuntu1604 kubelet[17097]: I0719 17:05:21.854303   17097 reconciler.go:253] MountVolume operation started for volume "kubernetes.io/configmap/foo-files"
 (spec.Name: "files") to pod "11786582-4dbf-11e6-9fc9-64cca009c636" (UID: "11786582-4dbf-11e6-9fc9-64cca009c636").
Jul 19 17:05:21 ubuntu1604 kubelet[17097]: I0719 17:05:21.854842   17097 reconciler.go:253] MountVolume operation started for volume "kubernetes.io/configmap/bar-file
s" (spec.Name: "files") to pod "117d2c22-4dbf-11e6-9fc9-64cca009c636" (UID: "117d2c22-4dbf-11e6-9fc9-64cca009c636").
Jul 19 17:05:21 ubuntu1604 kubelet[17097]: E0719 17:05:21.860796   17097 configmap.go:171] Error creating atomic writer: stat /var/lib/kubelet/pods/117d2c22-4dbf-11e6-9fc9-64cca009c636/volumes/kubernetes.io~configmap/files: no such file or directory
Jul 19 17:05:21 ubuntu1604 kubelet[17097]: E0719 17:05:21.861070   17097 goroutinemap.go:155] Operation for "kubernetes.io/configmap/bar-files" failed. No retries permitted until 2016-07-19 17:07:21.861036886 +0200 CEST (durationBeforeRetry 2m0s). error: MountVolume.SetUp failed for volume "kubernetes.io/configmap/bar-files" (spec.Name: "files") pod "117d2c22-4dbf-11e6-9fc9-64cca009c636" (UID: "117d2c22-4dbf-11e6-9fc9-64cca009c636") with: stat /var/lib/kubelet/pods/117d2c22-4dbf-11e6-9fc9-64cca009c636/volumes/kubernetes.io~configmap/files: no such file or directory
Jul 19 17:05:21 ubuntu1604 kubelet[17097]: E0719 17:05:21.861271   17097 configmap.go:171] Error creating atomic writer: stat /var/lib/kubelet/pods/11786582-4dbf-11e6-9fc9-64cca009c636/volumes/kubernetes.io~configmap/files: no such file or directory
Jul 19 17:05:21 ubuntu1604 kubelet[17097]: E0719 17:05:21.862284   17097 goroutinemap.go:155] Operation for "kubernetes.io/configmap/foo-files" failed. No retries permitted until 2016-07-19 17:07:21.862275753 +0200 CEST (durationBeforeRetry 2m0s). error: MountVolume.SetUp failed for volume "kubernetes.io/configmap/foo-files" (spec.Name: "files") pod "11786582-4dbf-11e6-9fc9-64cca009c636" (UID: "11786582-4dbf-11e6-9fc9-64cca009c636") with: stat /var/lib/kubelet/pods/11786582-4dbf-11e6-9fc9-64cca009c636/volumes/kubernetes.io~configmap/files: no such file or directory```

Note "Error creating atomic writer" errors.
This problem can be reproduced by making kubelet mount multiple config map volumes in parallel.

Usage charged should be max of greater of init container or all regular
containers. Also, need to validate init container inputs

Revert "Modified etcd manifest to support clustering."

Fix potential unbound KUBE_USER variable in gci/trusty.

Signed-off-by: Mike Danese <mikedanese@google.com>

Revert "Fix killing child sudo process in e2e_node tests"

Automatic merge from submit-queue

Fix killing child sudo process in e2e_node tests

Fixes #29211.

The context is we are trying to kill a process started as `sudo kube-apiserver`, but `sudo` ignores signals from the same process group. Applying `Setpgid` means the `sudo kill` process won't be in the same process group, so will not fall foul of this nifty feature.

I also took the liberty of removing some code setting `Pdeathsig` because it claims to be doing something  in the same area, but actually it doesn't do that at all.  The setting is applied to the forked process, i.e. `sudo`, and it means the `sudo` will get killed if we (`e2e_node.test`) die.  This (a) isn't what the comment says and (b) doesn't help because sending SIGKILL to the sudo process leaves sudo's child alive.

I didn't use the "hack for linux-only" approach because I think `Setpgid` is available on all platforms that `e2e_node` builds on.

Automatic merge from submit-queue

len(vmList) output format not correct

len(vmList) output format not correct, not "%s", is "%d".

This fixes race conditions in configmap, secret, downwardapi & git_repo
volume plugins.
wrappedVolumeSpec vars used by volume mounters and unmounters contained
a pointer to api.Volume structs which were being patched by
NewWrapperMounter/NewWrapperUnmounter, causing race condition during
volume mounts.

Automatic merge from submit-queue

Modified etcd manifest to support clustering.