Automatic merge from submit-queue (batch tested with PRs 41134, 41410, 40177, 41049, 41313)

Isolate recycler behavior in PV E2E

**What this PR does / why we need it**:
Sets the default `reclaimPolicy` for PV E2E to `Retain` and isolates `Recycle` tests to their own context.  The purpose of this is to future proof the PV test suite against the possible deprecation of the `Recycle` behavior.  This is done by consolidating recycling test code into a single Context block that can be removed en masse without affecting the test suite.

Secondly, adds a liveliness check for the NFS server pod prior to each test to avoid maxing out timeouts if the NFS server becomes unavailable.

cc @saad-ali @jeffvance

Automatic merge from submit-queue (batch tested with PRs 41134, 41410, 40177, 41049, 41313)

[Federation][Kubefed] Bug fix relating kubeconfig path in kubefed init

**What this PR does / why we need it**:
Fixes https://github.com/kubernetes/kubernetes/issues/41305

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
https://github.com/kubernetes/kubernetes/issues/41305
The kubeconfig explicit path is not updated correctly when supplied through the --kubeconfig flag in kubefed init. This leads to the details about the initialised federation control plane not getting updated in the correct kubeconfig file.

**Special notes for your reviewer**:
@madhusudancs 

**Release note**:

```
Fixed a bug that caused the kubeconfig entry for the initialized federation control plane to be not written to the supplied kubeconfig file when the file was supplied through the --kubeconfig flag.
```

Automatic merge from submit-queue (batch tested with PRs 41134, 41410, 40177, 41049, 41313)

Refactored kubemark code into provider-specific and provider-independent parts [Part-3]

Fixes #38967
Applying final part of the changes in PR #39033 (which refactored kubemark code completely). The changes included in this PR are:

- Removed `test/kubemark/common.sh` and moved relevant parts of its code to the right places in start-kubemark/stop-kubemark scripts.
- Added DOCKER_REGISTRY, PROJECT, KUBEMARK_IMAGE_MAKE_TARGET variables to `/test/kubemark/cloud-provider-config.sh` to make the kubemark image push location variable wrt provider.
- Removed get-real-pod-for-hollow-node.sh as it doesn't seem to do anything useful.

@kubernetes/sig-scalability-misc @wojtek-t @gmarek

Automatic merge from submit-queue (batch tested with PRs 41360, 41423, 41430, 40647, 41352)

move kubeadm api group testing to kubeadm package

I think this is sufficient to at least preserve round trip testing.

Automatic merge from submit-queue (batch tested with PRs 41360, 41423, 41430, 40647, 41352)

optimize NewSimpleSecretManager and cleanupOrphanedPodCgroups

Automatic merge from submit-queue (batch tested with PRs 41360, 41423, 41430, 40647, 41352)

preserve kube-master-cert metadata over upgrades

Automatic merge from submit-queue (batch tested with PRs 41360, 41423, 41430, 40647, 41352)

kubelet: reduce extraneous logging for pods using host network

For pods using the host network, kubelet/shim should not log
error/warning messages when determining the pod IP address.

Automatic merge from submit-queue

fluentd-gcp: Add kube-apiserver-audit.log.

**What this PR does / why we need it**:

Add `kube-apiserver-audit.log` from https://github.com/kubernetes/kubernetes/pull/41211 to fluentd config, so the audit log gets sent to the same place as `kube-apiserver.log`.

**Which issue this PR fixes**:

**Special notes for your reviewer**:

We would like to backport this to release-1.5 also.

**Release note**:
```release-note
The apiserver audit log (`/var/log/kube-apiserver-audit.log`) will be sent through fluentd if enabled.
```

Automatic merge from submit-queue (batch tested with PRs 41196, 41252, 41300, 39179, 41449)

Bump GCE ContainerVM to container-vm-v20170214

`container-vm-v20170214` is a re-build of the `docker-runc` in `container-vm-v20170201`, and should clear the GCE slow tests.

c.f. #40828

```release-note
Bump GCE ContainerVM to container-vm-v20170214 to address CVE-2016-9962.
```

Automatic merge from submit-queue (batch tested with PRs 41196, 41252, 41300, 39179, 41449)

record ReduceCPULimits result err info if err returned

record ReduceCPULimits result err info if err returned for debug

Automatic merge from submit-queue (batch tested with PRs 41196, 41252, 41300, 39179, 41449)

controller: cleanup workload controllers a bit

* Switches glog.Errorf to utilruntime.HandleError in DS and RC controllers
* Drops a couple of unused variables in the DS, SS, and Deployment controllers
* Updates some comments

@kubernetes/sig-apps-misc

Automatic merge from submit-queue (batch tested with PRs 41196, 41252, 41300, 39179, 41449)

Remove redundant pod helper

Automatic merge from submit-queue

Fix for Premature iSCSI logout #39202.

**What this PR does / why we need it**:

Modifies the iSCSI volume plugin code to prevent premature iSCSI logouts and the establishment of multiple iSCSI connections to the same target in certain cases.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #39202, fixes #41041, fixes #40941

**Special notes for your reviewer**:

The existing iSCSI connections are now rescanned on every AttachDisk call to discover newly created LUNs.

The disk mount points now contain an additional directory in the path corresponding to the disk iface that is later used for iSCSI logout.

The device prefixes that are used to count the existing references to the portal-target pair now contain the whole path including the mount point until the lun index.

**Release note**:
```release-note
Fixed issues #39202, #41041 and #40941 that caused the iSCSI connections to be prematurely closed when deleting a pod with an iSCSI persistent volume attached and that prevented the use of newly created LUNs on targets with preestablished connections.
```

Automatic merge from submit-queue (batch tested with PRs 40297, 41285, 41211, 41243, 39735)

fix variables in openstack.go to keep camel casing and remove unused var

Automatic merge from submit-queue (batch tested with PRs 40297, 41285, 41211, 41243, 39735)

Add LegacyHostIP as a fallback to federation api-server nodeport service

**What this PR does / why we need it**:
while deploying federation using kubefed and using NodePort type service for api-server, if the cluster does not have NodeExternalIP for nodes, then it leads to incorrect endpoint being written to kubeconfig.
So falling back to use LegacyHostIP in such cases.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #
Partly fixes an issue in this [thread](https://github.com/kubernetes/kubernetes/issues/41127#issuecomment-278888658)

**Special notes for your reviewer**:

**Release note**:
`NONE`

```release-note
```
cc @kubernetes/sig-federation-bugs @madhusudancs

Automatic merge from submit-queue (batch tested with PRs 40297, 41285, 41211, 41243, 39735)

cluster/gce: Add env var to enable apiserver basic audit log.

For now, this is focused on a fixed set of flags that makes the audit
log show up under /var/log/kube-apiserver-audit.log and behave similarly
to /var/log/kube-apiserver.log. Allowing other customization would
require significantly more complex changes.

Audit log rotation is handled the same as for `kube-apiserver.log`.

**What this PR does / why we need it**:

Add a knob to enable [basic audit logging](https://kubernetes.io/docs/admin/audit/) in GCE.

**Which issue this PR fixes**:

**Special notes for your reviewer**:

We would like to cherrypick/port this to release-1.5 also.

**Release note**:
```release-note
The kube-apiserver [basic audit log](https://kubernetes.io/docs/admin/audit/) can be enabled in GCE by exporting the environment variable `ENABLE_APISERVER_BASIC_AUDIT=true` before running `cluster/kube-up.sh`. This will log to `/var/log/kube-apiserver-audit.log` and use the same `logrotate` settings as `/var/log/kube-apiserver.log`.
```

Automatic merge from submit-queue (batch tested with PRs 40297, 41285, 41211, 41243, 39735)

Secure kube-scheduler

This PR:
* Adds a bootstrap `system:kube-scheduler` clusterrole
* Adds a bootstrap clusterrolebinding to the `system:kube-scheduler` user
* Sets up a kubeconfig for kube-scheduler on GCE (following the controller-manager pattern)
* Switches kube-scheduler to running with kubeconfig against secured port (salt changes, beware)
* Removes superuser permissions from kube-scheduler in local-up-cluster.sh
* Adds detailed RBAC deny logging

```release-note
On kube-up.sh clusters on GCE, kube-scheduler now contacts the API on the secured port.
```

Automatic merge from submit-queue (batch tested with PRs 40297, 41285, 41211, 41243, 39735)

Use fnv.New32a() in hash instead adler32

Ref: https://github.com/kubernetes/kubernetes/issues/40140

Benchmark results: https://github.com/kubernetes/kubernetes/pull/39527

NOTE: I leave  `GetPodTemplateSpecHash` as it is since we have unit test to test its un-normal behaviour.

Disable CRI in hollow-node

Some parts of the new dockershim are not properly mocked out in the
hollow-node. Disabling CRI in the hollow node until a proper fix is put
in.

Automatic merge from submit-queue

Update rbac data to v1beta1

Update RBAC fixtures to v1beta1

[Federation][Kubefed] Bug fix - kubeconfig path not updated properly when supplied through --kubeconfig flag in kubefed init

Automatic merge from submit-queue

Fix bug in status manager TerminatePod

In TerminatePod, we previously pass pod.Status to updateStatusInternal.  This is a bug, since it is the original status that we are given.  Not only does it skip updates made to container statuses, but in some cases it reverted the pod's status to an earlier version, since it was being passed a stale status initially.

This was the case in #40239 and #41095.  As shown in #40239, the pod's status is set to running after it is set to failed, occasionally causing very long delays in pod deletion since we have to wait for this to be corrected.

This PR fixes the bug, adds some helpful debugging statements, and adds a unit test for TerminatePod (which for some reason didnt exist before?).

@kubernetes/sig-node-bugs @vish @Random-Liu

Automatic merge from submit-queue

Fix comment for method SetForTests

cc @dchen1107

Automatic merge from submit-queue (batch tested with PRs 41378, 41413, 40743, 41155, 41385)

Expose the constants in pkg/controller/bootstrap and add a validate token function

**What this PR does / why we need it**: In order to hook up #36101 against kubeadm, we have to expose the constants and add a function to validate the token

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```
cc @jbeda @mikedanese @pires @dmmcquay

Automatic merge from submit-queue (batch tested with PRs 41378, 41413, 40743, 41155, 41385)

Reconcile bootstrap clusterroles on server start

Currently, on server start, bootstrap roles and bindings are only created if there are no existing roles or rolebindings.

Instead, we should look at each bootstrap role and rolebinding, and ensure it exists and has required permissions and subjects at server start. This allows seamless upgrades to new versions that define roles for new controllers, or add permissions to existing roles.

```release-note
Default RBAC ClusterRole and ClusterRoleBinding objects are automatically updated at server start to add missing permissions and subjects (extra permissions and subjects are left in place). To prevent autoupdating a particular role or rolebinding, annotate it with `rbac.authorization.kubernetes.io/autoupdate=false`.
```

Automatic merge from submit-queue (batch tested with PRs 41378, 41413, 40743, 41155, 41385)

'core' package to prevent dependency creep and isolate core functiona…

**What this PR does / why we need it**:

Solves these two problems: 

- Top level Scheduler root directory has several files in it that are needed really by the factory and algorithm implementations.  Thus they should be subpackages of scheduler.
- In addition scheduler.go and generic_scheduler.go don't naturally differentiate themselves when they are in the same package.  scheduler.go is eseentially the daemon entry point and so it should be isolated from the core 

*No release note needed*

Automatic merge from submit-queue (batch tested with PRs 41378, 41413, 40743, 41155, 41385)

move kube-aggregator to staging

Straight move of kube-aggregator to staging.  The scripts are the interesting bits.

@sttts @lavalamp

Automatic merge from submit-queue

Make EnableCRI default to true

This change makes kubelet to use the CRI implementation by default,
unless the users opt out explicitly by using --enable-cri=false.
For the rkt integration, the --enable-cri flag will have no effect
since rktnetes does not use CRI.

Also, mark the original --experimental-cri flag hidden and deprecated,
so that we can remove it in the next release. If both flags are specified,
the --enable-cri flag overrides the --experimental-cri flag.

Automatic merge from submit-queue

Make kubectl edit work with unstructured objects

Fixes https://github.com/kubernetes/kubernetes/issues/35993


1. First (before any other changes), added several test cases for complex edit scenarios:
   - [x] ensure the edit loop bails out if given the same result that already caused errors
   - [x] ensure an edited file with a syntax error is reopened preserving the input
   - [x] ensure objects with existing "caused-by" annotations get updated with the current command

2. Refactored the edit code to prep for switching to unstructured:
   - [x] made editFn operate on a slice of resource.Info objects passed as an arg, regardless of edit mode
   - [x] simplified short-circuiting logic when re-editing a file containing an error
   - [x] refactored how we build the various visitors (namespace enforcement, annotation application, patching, creating) so we could easily switch to just using a single visitor over a set of resource infos read from the updated input for all of them

3. Switched to using a resource builder to parse the stream of the user's edited output
   - [x] improve the error message you get on syntax errors
   - [x] preserve the user's input more faithfully (see how the captured testcase requests to the server changed to reflect exactly what the user edited)
   - [x] stopped doing client-side conversion (means deprecating `--output-version`)

4. Switched edit to work with generic objects
   - [x] use unstructured objects
   - [x] fall back to generic json merge patch for unrecognized group/version/kinds

5. Added new test cases
   - [x] schemaless objects falls back to generic json merge (covers TPR scenario)
   - [x] edit unknown version of known kind (version "v0" of storageclass) falls back to generic json merge

```release-note
`kubectl edit` now edits objects exactly as they were retrieved from the API. This allows using `kubectl edit` with third-party resources and extension API servers. Because client-side conversion is no longer done, the `--output-version` option is deprecated for `kubectl edit`. To edit using a particular API version, fully-qualify the resource, version, and group used to fetch the object (for example, `job.v1.batch/myjob`)
```

Automatic merge from submit-queue

delete cadvisor pod after test

tracing looks at events for pod deletion and volume teardown.  SInce the cadvisor pod has more than 1 volume, this can make results harder to analyze.
This PR moves the deletion of the cadvisor pod to after the logPodCreateThroughput call, since that marks the "end" of the test.

cc: @dchen1107 @Random-Liu

Automatic merge from submit-queue

kubeadm: add dmmcquay as a reviewer

**What this PR does / why we need it**: add dmmcquay (myself) as a reviewer to kubeadm. 

**Special notes for your reviewer**: /cc @luxas 

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue

fix comment 

**What this PR does / why we need it**:

fix comment 

Thanks.

**Special notes for your reviewer**:

**Release note**:

```release-note
```

container-vm-v20170214 is a re-build of the docker-runc in
container-vm-v20170201, and should clear the GCE slow tests.

c.f. #40828