Automatic merge from submit-queue (batch tested with PRs 41146, 41486, 41482, 41538, 41784)

fix issue #41746

**What this PR does / why we need it**:

**Which issue this PR fixes** : fixes #41746 

**Special notes for your reviewer**:

cc @feiskyer

Automatic merge from submit-queue (batch tested with PRs 41146, 41486, 41482, 41538, 41784)

Added `kubectl create clusterrole` command.

Added `kubectl create clusterrole` command.

Fixed part of #39596 

**Special notes for your reviewer**:
@deads2k, please help to review this patch, thanks

**Release note**:
```
   Added one new command `kubectl create clusterrole` to help user create a single ClusterRole from command line.
```

Automatic merge from submit-queue (batch tested with PRs 41146, 41486, 41482, 41538, 41784)

Switch statefulset controller to shared informers

Originally part of #40097 

I *think* the controller currently makes a deep copy of a StatefulSet before it mutates it, but I'm not 100% sure. For those who are most familiar with this code, could you please confirm?

@beeps @smarterclayton @ingvagabund @sttts @liggitt @deads2k @kubernetes/sig-apps-pr-reviews @kubernetes/sig-scalability-pr-reviews @timothysc @gmarek @wojtek-t

Automatic merge from submit-queue (batch tested with PRs 41146, 41486, 41482, 41538, 41784)

client-gen: independent scheme for clientsets

This PR adds a clientset internal scheme instead of using `pkg/api.Scheme`. **The clientset API stays the same.**

In detail:

- introduce a scheme for each clientset, i.e. do not use `pkg/api.Scheme+Registry+Codec+ParameterCodecs`.

  This makes it easier to compose client-go's clientset (which is rewritten in `staging/copy.sh` and therefore hardcoded to use `k8s.io/client-go/pkg/api.Scheme+Registry+Codecs+ParameterCodecs`) with third-party clientsets (kube-aggregator, openshift, federation) which are not rewritten using `copy.sh` as all of them are self-contained and therefore relocatable.

  This fixes https://github.com/kubernetes/kubernetes/pull/41403/files#diff-76edfb07dee54ff7ddeda25c33c10d29R81 and prepares client-gen for use in OpenShift.

- register types into the clientset scheme via `AddToScheme` for versioned clientsets. This decouples the client-go clients from announce+registration (internal clients continue using announce+registry and apigroup installers).

  This reduces complexity for client-go, possibly remove the necessity for the announce+register machinery for many use-cases, maybe even to delete it mid-term.
- port federation and testgroup  `install/install.go` to `announced.GroupMetaFactory` in order to have a proper `Install.Install(...)` func for registration.

With the first change it's easy to add the types of one clientset to the scheme of the other using the `clientset/scheme.AddToScheme` method. This allows to use cross-clientset `runtime.RawExtensions`:

```golang
import (
     "k8s.io/client-go/kubernetes"
     clientsetscheme "k8s.io/client-go/kuberentes/scheme"
     aggregatorclientsetscheme "k8s.io/kube-aggregator/pkg/client/clientset_generated/clientset/scheme"
)

kclientset, _ := kubernetes.NewForConfig(c)
aggregatorclientsetscheme.AddToScheme(clientsetscheme.Scheme)
```
Kubernetes types with a `RawExtension` can en/decode aggregator types after this.

TODO:
- [x] fix fake clientsets
- [x] get the `*Options` types registered correctly for core, compare DO-NOT-MERGE commit.
- [x] get prefered version right in internal client. Do we need all versions registered in the internal client to support negotiation?
- [x] run `staging/copy.sh` and run tests: https://github.com/kubernetes/kubernetes/pull/41744
- ~~[ ] fixup usage through-out the code-base~~
- **Follow-up**: move `import_known_versions.go` files somewhere such that import of the `api.Scheme` package automatically installs the apigroups. It looks like we depended on the import fo the clientset for this purpose.

Automatic merge from submit-queue (batch tested with PRs 41146, 41486, 41482, 41538, 41784)

 Add apply view-last-applied subcommand

reopen pr https://github.com/kubernetes/kubernetes/pull/40984, implement part of https://github.com/kubernetes/community/pull/287
for now unit test all pass, the output looks like:

```console
shiywang@dhcp-140-33 template $ ./kubectl apply view last-applied deployment nginx-deployment 
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  creationTimestamp: null
  name: nginx-deployment
spec:
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx:1.12.10
        name: nginx
        ports:
        - containerPort: 80
        resources: {}
status: {}
```

```release-note
Support new kubectl apply view-last-applied command for viewing the last configuration file applied
```

not sure if there is any flag I should updated or the some error handling I should changed.
will generate docs when you guys think is ok.
cc @pwittrock @jessfraz @AdoHe @ymqytw

Automatic merge from submit-queue (batch tested with PRs 38957, 41819, 41851, 40667, 41373)

Move pvutil.go from e2e package to framework package

**What this PR does / why we need it**:  

This PR moves pvutil.go to the e2e/framework package.

I am working on a PV upgrade test, and would like to use some of the wrapper functions in pvutil.go.  However, the upgrade test is in the upgrade package, and not the e2e package, and it cannot import the e2e package because it would create a circular dependency.  So pvutil.go needs to be moved out of e2e in order to break the circular dependency.  This is a refactoring name change, no logic has been modified.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: 

**Special notes for your reviewer**:

**Release note**:

NONE

Automatic merge from submit-queue (batch tested with PRs 38957, 41819, 41851, 40667, 41373)

Wrap and extract some flag description strings.

Moving on from help strings to flag descriptions.

@zen @fabianofranz @kubernetes/sig-cli-pr-reviews

Automatic merge from submit-queue (batch tested with PRs 38957, 41819, 41851, 40667, 41373)

Fix deployment helper - no assumptions on only one new ReplicaSet

#40415

**Release note**:

```release-note
NONE
```

@kubernetes/sig-apps-bugs

Automatic merge from submit-queue (batch tested with PRs 38957, 41819, 41851, 40667, 41373)

Bump GCI to gci-stable-56-9000-84-2

Changelogs since gci-beta-56-9000-80-0:

- Fixed google-accounts-daemon breaks on GCI when network is unavailable.
- Fixed iptables-restore performance regression.

cc/ @adityakali @Random-Liu @fabioy

Automatic merge from submit-queue (batch tested with PRs 38957, 41819, 41851, 40667, 41373)

Change taints/tolerations to api fields

This PR changes current implementation of taints and tolerations from annotations to API fields. Taint and toleration are now part of `NodeSpec` and `PodSpec`, respectively. The annotation keys: `scheduler.alpha.kubernetes.io/tolerations` and `scheduler.alpha.kubernetes.io/taints`  have been removed.

**Release note**:
Pod tolerations and node taints have moved from annotations to API fields in the PodSpec and NodeSpec, respectively. Pod tolerations and node taints that are defined in the annotations will be ignored. The annotation keys: `scheduler.alpha.kubernetes.io/tolerations` and `scheduler.alpha.kubernetes.io/taints`  have been removed.

Automatic merge from submit-queue

Make spiffxp an owner of test/...

Automatic merge from submit-queue

Update reviewer list for iscsi volume plugin.

Contributed nodiskconflict, multipath feature .etc to iscsi volume plugin. 
Signed-off-by: Humble Chirammal <hchiramm@redhat.com>

Automatic merge from submit-queue

Removes additional columns in test_owners.csv

**What this PR does / why we need it**:

fixes a huge collection of typos with the number of columns in the CSV file that probably has broken the auto assign bot

**Special notes for your reviewer**:

None

**Release note**:

`NONE`

list-resources: don't fail if the grep fails to match any resources

fix staging godeps

Automatic merge from submit-queue

add godep manifest files to staging repos

The staging repos should have manifests that match the godeps of kube so we know what they build against.  We don't need the actual vendored code, since a sync script on the other side needs to find the correct level of other staging directories and thus requires its own `godep restore && go get && godep save` cycle.

@sttts ptal
@lavalamp @caesarxuchao client-go needs a lot of unwinding to do something similar, but the idea is that you can run an acyclic path to get this updated by copying the types and dependencies with `go list`, then generate the clients, then generate this manifest.  Then in your sync script you can pull the proper levels and finish the actual vendoring.

Automatic merge from submit-queue (batch tested with PRs 41349, 41532, 41256, 41587, 41657)

Update dns

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 41349, 41532, 41256, 41587, 41657)

Update kubectl in addon-manager to use HPA in autoscaling/v1

Addon-manager is broken since HPA objects were removed from extensions api group.

Came across the logs from [the latest addon-manager on Jenkins](https://storage.googleapis.com/kubernetes-jenkins/logs/ci-kubernetes-e2e-gci-gce/4290/artifacts/bootstrap-e2e-master/kube-addon-manager.log):
```
INFO: == Entering periodical apply loop at 2017-02-16T17:33:37+0000 ==
error: error pruning namespaced object extensions/v1beta1, Kind=HorizontalPodAutoscaler: the server could not find the requested resource
WRN: == Failed to execute /usr/local/bin/kubectl  apply --namespace=kube-system -f /etc/kubernetes/addons     --prune=true -l kubernetes.io/cluster-service=true --recursive >/dev/null at 2017-02-16T17:33:38+0000. 2 tries remaining. ==
error: error pruning namespaced object extensions/v1beta1, Kind=HorizontalPodAutoscaler: the server could not find the requested resource
WRN: == Failed to execute /usr/local/bin/kubectl  apply --namespace=kube-system -f /etc/kubernetes/addons     --prune=true -l kubernetes.io/cluster-service=true --recursive >/dev/null at 2017-02-16T17:33:46+0000. 1 tries remaining. ==
error: error pruning namespaced object extensions/v1beta1, Kind=HorizontalPodAutoscaler: the server could not find the requested resource
WRN: == Failed to execute /usr/local/bin/kubectl  apply --namespace=kube-system -f /etc/kubernetes/addons     --prune=true -l kubernetes.io/cluster-service=true --recursive >/dev/null at 2017-02-16T17:33:53+0000. 0 tries remaining. ==
WRN: == Kubernetes addon update completed with errors at 2017-02-16T17:33:58+0000 ==
```

And notice this commit (https://github.com/kubernetes/kubernetes/commit/f66679a4e902f2c6f0f1faa7204a5f2d4b1fc6fa) came in two weeks ago, which removed HorizontalPodAutoscaler from extensions/v1beta1.

Addon-manager is now partially functioning that it could successfully create and update addons, but will fail to prune objects, which means upgrade tests may mostly fail.

Pushed another version of addon-manager with kubectl v1.6.0-alpha.2 ([release 2 days ago](https://github.com/kubernetes/kubernetes/releases/tag/v1.6.0-alpha.2)) for fixing, including below images:
- gcr.io/google-containers/kube-addon-manager:v6.4-alpha.2
- gcr.io/google-containers/kube-addon-manager-amd64:v6.4-alpha.2
- gcr.io/google-containers/kube-addon-manager-arm:v6.4-alpha.2
- gcr.io/google-containers/kube-addon-manager-arm64:v6.4-alpha.2
- gcr.io/google-containers/kube-addon-manager-ppc64le:v6.4-alpha.2
- gcr.io/google-containers/kube-addon-manager-s390x:v6.4-alpha.2

@mikedanese 

cc @wojtek-t @shyamjvs

Automatic merge from submit-queue (batch tested with PRs 41349, 41532, 41256, 41587, 41657)

Lint fixes for the master and worker Python code.

**What this PR does / why we need it**: lint fixes for the python code.

**Which issue this PR fixes** none

**Special notes for your reviewer**: This is lint fixes for the Juju python code.

**Release note**:

```release-note
NONE
```

Please consider these changes so we can pass flake8 lint tests in our build process.

Automatic merge from submit-queue (batch tested with PRs 41349, 41532, 41256, 41587, 41657)

client-go: don't import client auth provider packages

Both of these auth providers are useful for kubectl but not so much for everyone importing client-go. Let users optionally import them (example [0]) and reduce the overall number of imports that client-go requires.

Quick grep seems to imply it wont import it after.

```
$ grep -r 'client-go/plugin/pkg/client/auth' staging/
staging/src/k8s.io/client-go/plugin/pkg/client/auth/plugins.go:	_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
staging/src/k8s.io/client-go/plugin/pkg/client/auth/plugins.go:	_ "k8s.io/client-go/plugin/pkg/client/auth/oidc"
staging/src/k8s.io/client-go/examples/third-party-resources/main.go:	_ "k8s.io/client-go/plugin/pkg/client/auth/gcp"
staging/src/k8s.io/kube-aggregator/pkg/client/clientset_generated/clientset/clientset.go:	_ "k8s.io/client-go/plugin/pkg/client/auth"
staging/src/k8s.io/kube-aggregator/pkg/client/clientset_generated/internalclientset/clientset.go:	_ "k8s.io/client-go/plugin/pkg/client/auth"
```

closes https://github.com/kubernetes/client-go/issues/49
updates https://github.com/kubernetes/client-go/issues/79 (removes cloud.google.com/go import)

cc @kubernetes/sig-api-machinery-pr-reviews @kubernetes/sig-auth-pr-reviews 

```release-notes
client-go no longer imports GCP OAuth2 and OpenID Connect packages by default.
```

[0] https://github.com/kubernetes/client-go/blob/8b466d64c5da37e0d3985b907d812e1f58cb41cb/examples/third-party-resources/main.go#L34-L35

Automatic merge from submit-queue (batch tested with PRs 41349, 41532, 41256, 41587, 41657)

Enable pod level cgroups by default

**What this PR does / why we need it**:
It enables pod level cgroups by default.

**Special notes for your reviewer**:
This is intended to be enabled by default on 2/14/2017 per the plan outlined here:
https://github.com/kubernetes/community/pull/314

**Release note**:
```release-note
Each pod has its own associated cgroup by default.
```

Automatic merge from submit-queue (batch tested with PRs 41844, 41803, 39116, 41129, 41240)

NPD: Update NPD test.

For https://github.com/kubernetes/node-problem-detector/issues/58.

Update NPD e2e test based on the new behavior.

Note that before merging this PR, we need to merge all pending PRs in npd, and release the v0.3.0-alpha.1 version of NPD.

/cc @dchen1107 @kubernetes/node-problem-detector-reviewers

Automatic merge from submit-queue (batch tested with PRs 41844, 41803, 39116, 41129, 41240)

Cleanup client example

**What this PR does / why we need it**:
- Package level `config` variable in `third-party-resources/main.go` is not used, it is shadowed by the one defined in `main()`. Should probably be deleted.
- Package level `kubeconfig ` variable in `out-of-cluster/main.go` is global - make it private to `main()`.

**Which issue this PR fixes**
This fixes https://github.com/kubernetes/client-go/issues/59, except the part about global `api.Scheme`, also adds test with interface check. Supersedes https://github.com/kubernetes/client-go/pull/61.

**Special notes for your reviewer**:
This is my first PR to Kubernetes :)

Automatic merge from submit-queue (batch tested with PRs 41844, 41803, 39116, 41129, 41240)

core/v1/event_expansion.go use v1.GetReference

Fix https://github.com/kubernetes/client-go/issues/47.

Automatic merge from submit-queue (batch tested with PRs 41844, 41803, 39116, 41129, 41240)

Allow for not-ready pods in large clusters

This is to workaround issues with non-starting pods in large clusters in roughly 1/3rd of runs.