Automatic merge from submit-queue

Ensure invalid token returns 401 error, not 403

fixes #39267

If a user attempts to use a bearer token, and the token is rejected, the authenticator should return an error. This distinguishes requests that did not provide a bearer token (and are unauthenticated without error) from ones that attempted to, and failed.

Automatic merge from submit-queue

snip links from genericapiserver to api/validation

We had a trip dependency on `pkg/api/validation` through `pkg/api/rest` which is used for RESTStorage.  This splits the generic validation required for a genericapiserver out from the rest of validation.  For now, functions still delegate.

@sttts

Automatic merge from submit-queue

Adding test-federation-cmd.sh to test kubectl with federation apiserver

There are 2 parts to the PR:
* Adding ability to run kubectl tests for a subset of resources.
* Adding test-federation-cmd.sh that runs kubectl tests for resources that are supported by federation-apiserver.

cc @kubernetes/sig-federation @kubernetes/sig-api-machinery 

```release-note
Adding kubectl tests for federation
```

Automatic merge from submit-queue

Start moving genericapiserver to staging

This moves `pkg/auth/user` to `staging/k8s.io/genericapiserver/pkg/authentication/user`.  I'll open a separate pull into the upstream gengo to support using `import-boss` on vendored folders to support staging.

After we agree this is the correct approach and see everything build, I'll start moving other packages over which don't have k8s.io/kubernetes deps.

@kubernetes/sig-api-machinery-misc @lavalamp 
@sttts @caesarxuchao ptal

Automatic merge from submit-queue

pkg/labels: fix the godoc for Parse()

previously the formatting was broken for labels.Parse() function. I
fixed the spacing so it will format correctly for the generated go docs.
There was also a dangling "]" at the end of the first line so i deleted
it.



**What this PR does / why we need it**:

This fixes a formatting issue with the generated go documentation.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

make discovery static when extensions/thirdpartyresources is not enabled

this should be a bug fix, if `extensions/thirdpartyresources` is enabled, the result of `Discovery().ServerPreferredNamespacedResources` will be dynamic then, so we are making the `discoverResourcesFn` static only when the `extensions/thirdpartyresources` is not enabled.

Automatic merge from submit-queue

add some specifications on the top command

**What this PR does / why we need it**:
the top command is available when heapster works fine on the server, but the specifications doesn't mention about this. So I added some specifications here.

Automatic merge from submit-queue (batch tested with PRs 39409, 39400)

fix wrong help message for taint

fix #39381

Automatic merge from submit-queue (batch tested with PRs 39409, 39400)

remove newAPIGroupVersion() and getAPIGroupVersion() second return value

Automatic merge from submit-queue (batch tested with PRs 38084, 39306)

Small improve for GetContainerOOMScoreAdjust

In `GetContainerOOMScoreAdjust`, make logic more clear for the case `oomScoreAdjust >= besteffortOOMScoreAdj`. If `besteffortOOMScoreAdj`  is defined to another value(e.g. 996), suppose `oomScoreAdjust` is 999, the function will return 998(which equals 999 - 1) instead of 995(996 -1).

Automatic merge from submit-queue

Fix comment and optimize code

**What this PR does / why we need it**:

Fix comment and optimize code.

Thanks.

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Automatic merge from submit-queue

Share rc cache from the rc manager

@kubernetes/sig-apps-misc @hodovska

Automatic merge from submit-queue

Remove jobs that do not exist from active list of CronJob

**What this PR does / why we need it**: This PR modifies the controller for CronJob to remove from the active job list any job that does not exist anymore, to avoid staying blocked in active state forever. See #37957.

**Which issue this PR fixes**: fixes #37957 

**Special notes for your reviewer**:

**Release note**:

```
```

Automatic merge from submit-queue (batch tested with PRs 39405, 39371)

hack/local-up-cluster.sh: fix typo in error message

This commit fixes typo in error message and also removes stale comment from `hack/local-up-cluster.sh`.

Automatic merge from submit-queue (batch tested with PRs 36087, 39405)

federation-apiserver doesn't needs cluster-ip range

This option has been removed from genericserver options
and it is not being used by federation-apiserver anyways.

Fixes - #39388

Automatic merge from submit-queue

oidc auth-n plugin: enforce email_verified claim

This change causes the OpenID Connect authenticator to start
enforcing the 'email_verified' claim.

https://openid.net/specs/openid-connect-core-1_0.html#StandardClaims

If the OIDC authenticator uses the 'email' claim as a user's username
and the 'email_verified' is not set to `true`, reject that authentication attempt.

cc @erictune @kubernetes/sig-auth @mlbiam

```release-note
When using OIDC authentication and specifying --oidc-username-claim=email, an `"email_verified":true` claim must be returned from the identity provider.
```

Automatic merge from submit-queue

Include swaggerapi urls in system:discovery role

Used by client side API validation and for client schema generation

Automatic merge from submit-queue

Remove DockerLegacyService comment from kubelet

Automatic merge from submit-queue (batch tested with PRs 38433, 36245)

Allow pods to define multiple environment variables from a whole ConfigMap

Allow environment variables to be populated from ConfigMaps

- ConfigMaps represent an entire set of EnvVars
- EnvVars can override ConfigMaps


fixes #26299

Automatic merge from submit-queue (batch tested with PRs 38433, 36245)

Remove needless env var in OpenStack provider

**What this PR does / why we need it**:

If we use openstack provider to set up k8s cluster using kube-up script,
`TENANT_ID` environment variable is needed.
But to configure `TENANT_ID` is very annoying because this value is not static by each env.
This patch uses `TENANT_NAME` instead of `TENANT_ID`

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
```

Since `TENANT_NAME` is unique if we use keystone v2 api,
so `TENANT_ID` is not needed if `TENANT_NAME` is provided
to configure OpenStack provider.
And also to set `TENANT_ID` is annoying to develop, because
`TENANT_ID` is not static by each environment.
This patch remove dependency of `TENANT_ID` and simply use
`TENANT_NAME`.

Since newAPIGroupVersion() and getAPIGroupVersion() will never return err,we do not need that

Automatic merge from submit-queue

add rules not allow message when authorize failed

old result:
```
# ./cluster/kubectl.sh --token=/test get po
Error from server (Forbidden): User "" cannot list pods in the namespace "default".: "<nil>" (get pods)
```
new result:
```
# ./cluster/kubectl.sh --token=/test get po
Error from server (Forbidden): User "" cannot list pods in the namespace "default".: "rules not allow" (get pods)
```

test.yaml
```
kind: Role
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
  name: test
rules:
- apiGroups: ["*"]
  verbs: ["create"]
  resources: ["*"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1alpha1
metadata:
  name: admin-resource-binding
subjects:
  - kind: Group
    name: test
roleRef:
  kind: Role
  name: test
```

This option has been removed from genericserver options
and it is not being used by federation-apiserver anyways.

Fixes - #39388

Automatic merge from submit-queue (batch tested with PRs 39284, 39367)

e2e: Use sudo to start/stop kubelet service

It seems that user 'jenkins' does not have permissions to manage services. sudo is already used to e.g. check iptables in https://github.com/kubernetes/kubernetes/blob/master/test/e2e/service.go#L402

Fixes #37956

Automatic merge from submit-queue (batch tested with PRs 39284, 39367)

Remove HostRecord annotation (beta feature)

The annotation has made it to GA so this code should be deleted.

**Release note**:
```release-note
The 'endpoints.beta.kubernetes.io/hostnames-map' annotation is no longer supported.  Users can use the 'Endpoints.subsets[].addresses[].hostname' field instead.
```

Automatic merge from submit-queue (batch tested with PRs 39280, 37350, 39389, 39390, 39313)

Moves e2e service util functions into service_util.go and cleans up

Basically moves codes into a central place for service util functions.

Some other codes are touched mostly only due to this migration. Also put a bunch of network reachability utils functions into network_utils.go. They seem somehow redundant, may consider combine they later.

@bowei @freehan

Automatic merge from submit-queue (batch tested with PRs 39280, 37350, 39389, 39390, 39313)

Fix AWS break injected by #39020

Shuffle the `download-cfssl` to `cluster/common.sh` (broken in #39020)

Automatic merge from submit-queue (batch tested with PRs 39280, 37350, 39389, 39390, 39313)

kubeadm: readd --api-port, accidentally removed during refactor.

this broke stuff

Automatic merge from submit-queue (batch tested with PRs 39280, 37350, 39389, 39390, 39313)

delete meaningless judgments

What this PR does / why we need it：
      Whether "err" is nil or not, "err" can be return, so the judgment "err !=nil " is unnecessary

Automatic merge from submit-queue (batch tested with PRs 39280, 37350, 39389, 39390, 39313)

Refactor the certificate and kubeconfig code in the kubeadm binary into two phases

**What this PR does / why we need it**:

First stab at refactoring kubeadm code into logically independent phases.
This defines two phases in the kubeadm init process:
 - certs: Takes some API values as input (the API will be refactored in a later PR), and generates certificates in the pki directory
 - kubeconfig: Takes the pki directory and the endpoint where the master is located and produces two kubeconfig files: admin.conf and kubelet.conf

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*:
Required long-term for graduating our API

**Special notes for your reviewer**:

### Old sample output
The earlier kubeconfig code had a bug in it; see this example:
_admin.conf:_
```yaml
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <data>
    server: https://192.168.200.x:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: admin
  name: admin@kubernetes
- context:
    cluster: kubernetes
    user: kubelet
  name: kubelet@kubernetes
current-context: admin@kubernetes
kind: Config
preferences: {}
users:
- name: admin
  user:
    client-certificate-data: <data>
    client-key-data: <data>
- name: kubelet
  user:
    client-certificate-data: <data>
    client-key-data: <data>
```
kubelet.conf:
```yaml
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <data>
    server: https://192.168.200.x:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: admin
  name: admin@kubernetes
- context:
    cluster: kubernetes
    user: kubelet
  name: kubelet@kubernetes
current-context: admin@kubernetes
kind: Config
preferences: {}
users:
- name: admin
  user:
    client-certificate-data: <data>
    client-key-data: <data>
- name: kubelet
  user:
    client-certificate-data: <data>
    client-key-data: <data>
```
```console
$ shasum /etc/kubernetes/*.conf
2b22b25cc4c97e5619ece6c43badf42b87c4970a  /etc/kubernetes/admin.conf
2b22b25cc4c97e5619ece6c43badf42b87c4970a  /etc/kubernetes/kubelet.conf
```

#### New output
admin.conf
```yaml
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <data>
    server: https://192.168.200.x:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: admin
  name: admin@kubernetes
current-context: admin@kubernetes
kind: Config
preferences: {}
users:
- name: admin
  user:
    client-certificate-data: <data>
    client-key-data: <data>
```
kubelet.conf
```yaml
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: <data>
    server: https://192.168.200.x:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubelet
  name: kubelet@kubernetes
current-context: kubelet@kubernetes
kind: Config
preferences: {}
users:
- name: kubelet
  user:
    client-certificate-data: <data>
    client-key-data: <data>
```

**Release note**:

```release-note
Refactor the certificate and kubeconfig code in the kubeadm binary into two phases
```

PTAL @dgoodwin @jbeda @mikedanese @errordeveloper @pipejakob @lukemarsden

Automatic merge from submit-queue (batch tested with PRs 39001, 39104, 35978, 39361, 39273)

delete SetNodeStatus() function and fix some function notes words