Automatic merge from submit-queue (batch tested with PRs 38433, 36245)

Allow pods to define multiple environment variables from a whole ConfigMap

Allow environment variables to be populated from ConfigMaps

- ConfigMaps represent an entire set of EnvVars
- EnvVars can override ConfigMaps


fixes #26299

Automatic merge from submit-queue (batch tested with PRs 38433, 36245)

Remove needless env var in OpenStack provider

**What this PR does / why we need it**:

If we use openstack provider to set up k8s cluster using kube-up script,
`TENANT_ID` environment variable is needed.
But to configure `TENANT_ID` is very annoying because this value is not static by each env.
This patch uses `TENANT_NAME` instead of `TENANT_ID`

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Since `TENANT_NAME` is unique if we use keystone v2 api,
so `TENANT_ID` is not needed if `TENANT_NAME` is provided
to configure OpenStack provider.
And also to set `TENANT_ID` is annoying to develop, because
`TENANT_ID` is not static by each environment.
This patch remove dependency of `TENANT_ID` and simply use
`TENANT_NAME`.

Automatic merge from submit-queue

add rules not allow message when authorize failed

old result:
```
# ./cluster/kubectl.sh --token=/test get po
Error from server (Forbidden): User "" cannot list pods in the namespace "default".: "<nil>" (get pods)
```
new result:
```
# ./cluster/kubectl.sh --token=/test get po
Error from server (Forbidden): User "" cannot list pods in the namespace "default".: "rules not allow" (get pods)
```

test.yaml
```
kind: Role
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
  name: test
rules:
- apiGroups: ["*"]
  verbs: ["create"]
  resources: ["*"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
  name: admin-resource-binding
subjects:
  - kind: Group
    name: test
roleRef:
  kind: Role
  name: test
```

Automatic merge from submit-queue (batch tested with PRs 39284, 39367)

e2e: Use sudo to start/stop kubelet service

It seems that user 'jenkins' does not have permissions to manage services. sudo is already used to e.g. check iptables in https://github.com/kubernetes/kubernetes/blob/master/test/e2e/service.go#L402

Fixes #37956

Automatic merge from submit-queue (batch tested with PRs 39284, 39367)

Remove HostRecord annotation (beta feature)

The annotation has made it to GA so this code should be deleted.

**Release note**:
```release-note
The 'endpoints.beta.kubernetes.io/hostnames-map' annotation is no longer supported.  Users can use the 'Endpoints.subsets[].addresses[].hostname' field instead.
```

Automatic merge from submit-queue (batch tested with PRs 39280, 37350, 39389, 39390, 39313)

Moves e2e service util functions into service_util.go and cleans up

Basically moves codes into a central place for service util functions.

Some other codes are touched mostly only due to this migration. Also put a bunch of network reachability utils functions into network_utils.go. They seem somehow redundant, may consider combine they later.

@bowei @freehan

Automatic merge from submit-queue (batch tested with PRs 39280, 37350, 39389, 39390, 39313)

Fix AWS break injected by #39020

Shuffle the `download-cfssl` to `cluster/common.sh` (broken in #39020)

Automatic merge from submit-queue (batch tested with PRs 39280, 37350, 39389, 39390, 39313)

kubeadm: readd --api-port, accidentally removed during refactor.

this broke stuff

Automatic merge from submit-queue (batch tested with PRs 39280, 37350, 39389, 39390, 39313)

delete meaningless judgments

What this PR does / why we need it：
      Whether "err" is nil or not, "err" can be return, so the judgment "err !=nil " is unnecessary

Automatic merge from submit-queue (batch tested with PRs 39280, 37350, 39389, 39390, 39313)

Refactor the certificate and kubeconfig code in the kubeadm binary into two phases

**What this PR does / why we need it**:

First stab at refactoring kubeadm code into logically independent phases.
This defines two phases in the kubeadm init process:
 - certs: Takes some API values as input (the API will be refactored in a later PR), and generates certificates in the pki directory
 - kubeconfig: Takes the pki directory and the endpoint where the master is located and produces two kubeconfig files: admin.conf and kubelet.conf

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
Required long-term for graduating our API

**Special notes for your reviewer**:

### Old sample output
The earlier kubeconfig code had a bug in it; see this example:
_admin.conf:_
```yaml
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <data>
    server: https://192.168.200.x:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: admin
  name: admin@kubernetes
- context:
    cluster: kubernetes
    user: kubelet
  name: kubelet@kubernetes
current-context: admin@kubernetes
kind: Config
preferences: {}
users:
- name: admin
  user:
    client-certificate-data: <data>
    client-key-data: <data>
- name: kubelet
  user:
    client-certificate-data: <data>
    client-key-data: <data>
```
kubelet.conf:
```yaml
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <data>
    server: https://192.168.200.x:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: admin
  name: admin@kubernetes
- context:
    cluster: kubernetes
    user: kubelet
  name: kubelet@kubernetes
current-context: admin@kubernetes
kind: Config
preferences: {}
users:
- name: admin
  user:
    client-certificate-data: <data>
    client-key-data: <data>
- name: kubelet
  user:
    client-certificate-data: <data>
    client-key-data: <data>
```
```console
$ shasum /etc/kubernetes/*.conf
2b22b25cc4c97e5619ece6c43badf42b87c4970a  /etc/kubernetes/admin.conf
2b22b25cc4c97e5619ece6c43badf42b87c4970a  /etc/kubernetes/kubelet.conf
```

#### New output
admin.conf
```yaml
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <data>
    server: https://192.168.200.x:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: admin
  name: admin@kubernetes
current-context: admin@kubernetes
kind: Config
preferences: {}
users:
- name: admin
  user:
    client-certificate-data: <data>
    client-key-data: <data>
```
kubelet.conf
```yaml
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <data>
    server: https://192.168.200.x:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubelet
  name: kubelet@kubernetes
current-context: kubelet@kubernetes
kind: Config
preferences: {}
users:
- name: kubelet
  user:
    client-certificate-data: <data>
    client-key-data: <data>
```

**Release note**:

```release-note
Refactor the certificate and kubeconfig code in the kubeadm binary into two phases
```

PTAL @dgoodwin @jbeda @mikedanese @errordeveloper @pipejakob @lukemarsden

Automatic merge from submit-queue (batch tested with PRs 39001, 39104, 35978, 39361, 39273)

delete SetNodeStatus() function and fix some function notes words

Automatic merge from submit-queue (batch tested with PRs 39001, 39104, 35978, 39361, 39273)

Log error modification

controller_utils.go:  Log content is wrong, modify

Automatic merge from submit-queue (batch tested with PRs 39001, 39104, 35978, 39361, 39273)

Allow PATCH in an API CORS setup

Allows the PATCH method to be used in a REST API CORS setup.

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 39001, 39104, 35978, 39361, 39273)

refactored admission to avoid internal client references

Refactored admission to avoid internal client references.  This required switching to plugin initializers for them.  And that required some rewiring of the plugin initializers. 

Technically I can decouple from the other two commits, but I'm optimistic that those will go through easy. This is slightly move invasive, but I'd like to shoot for pre-christmas to avoid new admission plugins coming through and breaking bits.

@sttts @derekwaynecarr

Automatic merge from submit-queue

fix group in create clusterrolebinding

group lost.

```
#kubectl create clusterrolebinding test --clusterrole=a --group=b
clusterrolebinding "test" created

#kubectl get clusterrolebinding test -o yaml
apiVersion: rbac.authorization.k8s.io/v1alpha1
kind: ClusterRoleBinding
metadata:
  creationTimestamp: 2016-12-20T03:13:24Z
  name: test
  resourceVersion: "284"
  selfLink: /apis/rbac.authorization.k8s.io/v1alpha1/clusterrolebindingstest
  uid: 44d3476d-c662-11e6-a439-7427ea6f0fe3
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: a
subjects: null
```

Automatic merge from submit-queue

Service with type=NodePort and clusterIP=None is not allowed

Fixes https://github.com/kubernetes/kubernetes/issues/30908

Automatic merge from submit-queue

Add json,yaml output format support to kubectl create, kubectl apply

Fixes: https://github.com/kubernetes/kubernetes/issues/37390

**Release note**:
```release-note
Added support for printing in all supported `--output` formats to `kubectl create ...` and `kubectl apply ...`
```

This patch adds the ability to specify an output format other than
"name" to `kubectl create ...`. It can be used in conjunction with the
`--dry-run` option. Converts unstructured objects into known types in
order to support all `--output` values.

The patch prints `*resource.Info`s returned by the server. If a resource does not yet exist (and the `--dry-run` option is not set), the resource is created and printed in the specified format.

@kubernetes/cli-review @fabianofranz

Refactor the pki, cert, kubeconfig code in the kubeadm binary into two separate and logically independent phases

Automatic merge from submit-queue

Register necessary objects in internal groups

Found during working on #39305

Automatic merge from submit-queue

Update New Relic sample docs with dynamic hostname info

This PR shows how to set extra environment variables for the New Relic DaemonSet that can be generated at runtime. I'm using this technique to prepend cluster names to host names as they are reported to New Relic.

Automatic merge from submit-queue (batch tested with PRs 39022, 39331, 39070, 39344)

Add a build rule for the boilerplate unit test.

We recently added unit tests that just run whenever you run `hack/verify_boilerplate.sh`, which really isn't the right time to do that. This adds a build rule instead.

Automatic merge from submit-queue (batch tested with PRs 39022, 39331, 39070, 39344)

remove unused parameter

Automatic merge from submit-queue (batch tested with PRs 39022, 39331, 39070, 39344)

Makefile speedup on generated code

This has been languishing in a  branch for a long time.  It makes the build more consistent wrt GOPATH (I still hope to enforce GOPATH at some point) and it removes a `go list` from each codegen.

I verified manually that the files that this emits as part of the make are only change in safe ways (ordering, _test files removed, etc).

Automatic merge from submit-queue

genericapiserver: extract CA cert from server cert and SNI cert chains

Without this PR a matching server cert or SNI cert is directly used as CA cert in the loopback client config. This fails if the cert is no CA cert.

With this PR the loopback client setup code walks through the chains of the server cert and the SNI certs to find a `CA:TRUE` cert. This is then used as the CA in the loopback client config.

Automatic merge from submit-queue (batch tested with PRs 39092, 39126, 37380, 37093, 39237)

Add unit test function to test DeletePod

Automatic merge from submit-queue (batch tested with PRs 39092, 39126, 37380, 37093, 39237)

Endpoints with TolerateUnready annotation, should list Pods in state terminating

**What this PR does / why we need it**:

We are using preStop lifecycle hooks to gracefully remove a node from a cluster. This hook is potentially long running and after the preStop hook is fired, the DNS resolution of the soon to be stopped Pod is failing, which causes a failure there.

**Special notes for your reviewer**:

Would be great to backport that to 1.4, 1.3 

**Release note**:

```release-note
Endpoints, that tolerate unready Pods, are now listing Pods in state Terminating as well
```

@bprashanth

Automatic merge from submit-queue (batch tested with PRs 39092, 39126, 37380, 37093, 39237)

Improve error reporting in Ceph RBD provisioner.

- We should report an error when user references a secret that cannot be found
- We should report output of rbd create/delete commands, logging "exit code 1"
  is not enough.


Before:
```
Events:
  FirstSeen     LastSeen        Count   From                            SubobjectPath   Type            Reason                  Message
  ---------     --------        -----   ----                            -------------   --------        ------                  -------
  33m           33m             1       {persistentvolume-controller }                  Warning         ProvisioningFailed      Failed to provision volume with StorageClass "cephrbdprovisioner": rbd: create volume failed, err: exit status 1
```

After:

```
Events:
  FirstSeen     LastSeen        Count   From                            SubobjectPath   Type            Reason                  Message
  ---------     --------        -----   ----                            -------------   --------        ------                  -------
  33m           33m             1       {persistentvolume-controller }                  Warning         ProvisioningFailed      Failed to provision volume with StorageClass "cephrbdprovisioner": failed to create rbd image: exit status 1, command output: rbd: couldn't connect to the cluster
```


@rootfs, PTAL

Automatic merge from submit-queue (batch tested with PRs 39092, 39126, 37380, 37093, 39237)

update prompt in create rolebinding/clusterrolebinding

Automatic merge from submit-queue

remove abac

Remove the abac authorizer as an authorizer for e2e.

Automatic merge from submit-queue

retry RBAC initialization for up to 30 seconds, kill server on failure

RBAC initialization needs to complete in order to bootstrap a cluster.  When the bootstrapping fails (etcd not ready has happened in e2e runs), things fail badly and we don't even kill the API server to force it to retry.  This retries for up to 30 seconds and kills the server if it never succeeds.

Fixes https://github.com/kubernetes/kubernetes/issues/39108

Automatic merge from submit-queue (batch tested with PRs 39075, 39350, 39353)

Move pkg/api.{Context,RequestContextMapper} into pkg/genericapiserver/api/request

**Based on #39350**