Automatic merge from submit-queue (batch tested with PRs 48583, 48605, 48601)

apimachinery+apiserver: separate test types in their own packages

Preparation for static deepcopy https://github.com/kubernetes/kubernetes/pull/48544
and its use of package-global deepcopy-gen tags for all runtime.Objects.

- [x] wait for #48497

Automatic merge from submit-queue (batch tested with PRs 48583, 48605, 48601)

support json output for log backend of advanced audit

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```
Add json format support for advanced audit in apiserver. Use --audit-log-format=json to emit json to log backend.
```

Automatic merge from submit-queue

Record 429 and timeout errors to prometheus

Allows gathering of load being shed.

Fixes #48559

@deads2k please review, there was a logic error in apiserver RequestInfo (minor, fortunately)

```release-note
Requests with the query parameter `?watch=` are treated by the API server as a request to watch, but authorization and metrics were not correctly identifying those as watch requests, instead grouping them as list calls.
```

Automatic merge from submit-queue

[Federation] Remove flakey and redundant replicaset unit test

The coverage provided by this test duplicates coverage provided in e2e.  As well, the test is 'unit' in name only, as evidenced by the dependency on sleep statements to try to coordinate timing between the test and the controller.  If low-level coverage of replicaset scheduling is desired, it can be implemented in a more targeted fashion that is not prone to race conditions.

Fixes #48368

cc: @kubernetes/sig-federation-pr-reviews

Automatic merge from submit-queue

hack/OWNERS: add myself (sttts)

Automatic merge from submit-queue (batch tested with PRs 48497, 48604, 48599, 48560, 48546)

remove dead code

This removes the dead code cruft since we stopped serving TPRs.

ref #48152

Automatic merge from submit-queue (batch tested with PRs 48497, 48604, 48599, 48560, 48546)

GCE: Use network project id for firewall/route mgmt and zone listing

- Introduces a new environment variable for plumbing the network project id which will be used for firewall and route management. fixes #48515
- onXPN is determined by metadata if config is not specified
- Split `if` conditions: fixes #48521
- Remove `getNetworkNameViaAPICall` which was used as a last resort for the `networkURL` (if empty) which was previously filled with the metadata network project & name.

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 48497, 48604, 48599, 48560, 48546)

audit: fix deepcopy registration

Remove manual registration into scheme by fixing the deepcopy tag of the api group.

Automatic merge from submit-queue (batch tested with PRs 48497, 48604, 48599, 48560, 48546)

client-go: remove TPR example

Now that the TPR api is gone (https://github.com/kubernetes/kubernetes/issues/48152).

Automatic merge from submit-queue (batch tested with PRs 48497, 48604, 48599, 48560, 48546)

Move pkg/apimachinery/test to apimachinery

Fixes https://github.com/kubernetes/kubernetes/issues/48265

for circular dependency reason, all the test file contain both `"k8s.io/apimachinery/pkg/apis/meta/v1"` and `"k8s.io/apimachinery/pkg/runtime"` is hard to move to a ideal location, so I create a separated test package for those files

I also bump the example package :https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/apiserver/pkg/apis/example here for apimachinery, @sttts told me it's ok if to bump file if only for test   EDIT: seems it's no need to bump, will update soon
```
NONE
```

Automatic merge from submit-queue

Fix deleting empty monitors

Fix #48094
When create-monitor of cloud-config is false, pool has not monitor
and can not delete empty monitor.

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue

Deleting kubectl.ServiceReaper since there is no special service deletion logic

Ref https://github.com/kubernetes/kubernetes/pull/46471 #42594

ServiceReaper does not have any special deletion logic so we dont need it. The generic deletion logic should be enough.
By removing this reaper, service deletion also gets the new wait logic from https://github.com/kubernetes/kubernetes/pull/46471

cc @kubernetes/sig-cli-misc

Automatic merge from submit-queue

Delete unused return

**What this PR does / why we need it**:
We do not use the function return, it's better not to write the return.

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 47234, 48410, 48514, 48529, 48348)

Check opts of cloud config file

Fix #48347
Check opts when register OpenStack CloudProvider rather than
returning error when use opts to create/use cloud resource.

**Release note**:
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 47234, 48410, 48514, 48529, 48348)

eliminate kubectl dependency on kubelet

```
ConfigMirrorAnnotationKey    = v1.MirrorPodAnnotationKey
```
`k8s.io/kubernetes/pkg/kubelet/types.ConfigMirrorAnnotationKey` is defined as `k8s.io/api/core/v1.MirrorPodAnnotationKey`

partially addresses: kubernetes/community#598

```release-note
NONE
```

/assign @monopole @apelisse

Automatic merge from submit-queue (batch tested with PRs 47234, 48410, 48514, 48529, 48348)

expose error lock release failure from iptables util

ref: https://github.com/kubernetes/kubernetes/issues/48107

Automatic merge from submit-queue (batch tested with PRs 47234, 48410, 48514, 48529, 48348)

Remove unused sub-pkgs in pkg/util

**What this PR does / why we need it**:
Remove no longer used sug-pkgs in pkg/util

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #48386

**Special notes for your reviewer**:
/cc @deads2k 

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 47234, 48410, 48514, 48529, 48348)

servicecontroller: trivial fix error message naming

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:

```release-note
NONE
```

Automatic merge from submit-queue

scheduler: fix validation test
Signed-off-by: sakeven <jc5930@sina.cn>



**What this PR does / why we need it**:

Without setting `Weight`, `ValidatePolicy` will report
```
Priority for extender http://127.0.0.1:8081/extender should have a positive weight applied to it
```

**Besides**, it seems it's not a good way to test ValidatePolicy by```if ValidatePolicy(extenderPolicy) == nil```, because we can't determine specific reason which causes error.

**Special notes for your reviewer**:

**Release note**:

```
NONE
```

Automatic merge from submit-queue

Allow NetworkPolicy.spec updates

ValidateNetworkPolicyUpdate currently prohibits changes to `spec` in an existing NetworkPolicy. We were going to fix this for 1.7 but I forgot to submit this PR after the main PR merged. Too late for 1.7? @thockin @caseydavenport @cmluciano 

This only changes networking.NetworkPolicy validation at the moment... Should I change extensions.NetworkPolicy validation too?

Fixes #35911

We should add a test to the e2e NetworkPolicy test for this too if this is going to merge.

**Release note**:
```release-note
As part of the NetworkPolicy "v1" changes, it is also now
possible to update the spec field of an existing
NetworkPolicy. (Previously you had to delete and recreate a
NetworkPolicy if you wanted to change it.)
```

Automatic merge from submit-queue (batch tested with PRs 48374, 48524, 48519, 42548, 48615)

godep-save.sh: add sanity checks

Might prevent https://github.com/kubernetes/kubernetes/issues/48593.

Automatic merge from submit-queue (batch tested with PRs 48374, 48524, 48519, 42548, 48615)

Use Secrets for files that self-hosted pods depend on

**What this PR does / why we need it**:
See https://github.com/kubernetes/kubeadm/issues/194

**Release note**:

```release-note
NONE
```

```
bash-4.2# kubectl --kubeconfig /etc/kubernetes/admin.conf get pods -n kube-system
NAME                                                   READY     STATUS    RESTARTS   AGE
dummy-1628042694-6ghbq                                 1/1       Running   0          42m
kube-dns-1853130399-4nzx4                              3/3       Running   0          9m
kube-flannel-ds-mnq10                                  2/2       Running   2          12m
kube-flannel-ds-n3tl8                                  2/2       Running   0          42m
kube-proxy-lqpcb                                       1/1       Running   0          42m
kube-proxy-pw0pw                                       1/1       Running   0          12m
self-hosted-kube-apiserver-fkkwd                       1/1       Running   1          42m
self-hosted-kube-controller-manager-1387498942-mzg41   1/1       Running   1          42m
self-hosted-kube-scheduler-2588609441-cwhqb            1/1       Running   1          42m
```

### API Server
```
bash-4.2# kubectl --kubeconfig /etc/kubernetes/admin.conf exec self-hosted-kube-apiserver-fkkwd -n kube-system -- ls /etc/pki
ca-trust
java
nssdb
rpm-gpg
tls
bash-4.2# kubectl --kubeconfig /etc/kubernetes/admin.conf exec self-hosted-kube-apiserver-fkkwd -n kube-system -- ls /etc/ssl
certs
bash-4.2# kubectl --kubeconfig /etc/kubernetes/admin.conf exec self-hosted-kube-apiserver-fkkwd -n kube-system -- ls /etc/kubernetes/
pki
bash-4.2# kubectl --kubeconfig /etc/kubernetes/admin.conf exec self-hosted-kube-apiserver-fkkwd -n kube-system -- ls /etc/kubernetes/pki/
apiserver-kubelet-client.crt
apiserver-kubelet-client.key
apiserver.crt
apiserver.key
ca.crt
front-proxy-ca.crt
sa.pub
```

### Scheduler
```
bash-4.2# kubectl --kubeconfig /etc/kubernetes/admin.conf exec self-hosted-kube-scheduler-2588609441-cwhqb -n kube-system -- ls /etc/kubernetes/
scheduler.conf
```

### Controller Manager
```
bash-4.2# kubectl --kubeconfig /etc/kubernetes/admin.conf exec self-hosted-kube-controller-manager-1387498942-mzg41 -n kube-system -- ls /etc/ssl
certs
bash-4.2# kubectl --kubeconfig /etc/kubernetes/admin.conf exec self-hosted-kube-controller-manager-1387498942-mzg41 -n kube-system -- ls /etc/pki
ca-trust
java
nssdb
rpm-gpg
tls
bash-4.2# kubectl --kubeconfig /etc/kubernetes/admin.conf exec self-hosted-kube-controller-manager-1387498942-mzg41 -n kube-system -- ls /etc/kubernetes/
controller-manager.conf
pki
bash-4.2# kubectl --kubeconfig /etc/kubernetes/admin.conf exec self-hosted-kube-controller-manager-1387498942-mzg41 -n kube-system -- ls /etc/kubernetes/pki/
ca.crt
ca.key
sa.key
```

/cc @luxas @liggitt @pires @timothysc

Automatic merge from submit-queue (batch tested with PRs 48374, 48524, 48519, 42548, 48615)

Kubelet run() should accept partial KubeletDeps

This is used by integrators that want to perform partial overrides of
key interfaces. Refactors the run() method to fit the existing style and
preserve the existing behavior, but allow (for instance) client
bootstrap and cert refresh even when some dependencies are injected.

@derekwaynecarr hit while reviewing client bootstrap code

Automatic merge from submit-queue (batch tested with PRs 48374, 48524, 48519, 42548, 48615)

flush conntrack for udp service when # of backend changes from 0

fixes: #48370

Automatic merge from submit-queue (batch tested with PRs 48374, 48524, 48519, 42548, 48615)

Add a node upgrade test for etcd statefulset with a PDB.

Tests for #38336

Automatic merge from submit-queue

Small fix for number of pods and nodes in test function

**What this PR does / why we need it**:
Small fix to have correct pods and nodes in test function.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
Scheduler perf, nodes and pods number fix for 100 nodes 3k pods.
**Release note**:

```release-note
NONE
```

Allows gathering of load being shed.

Fix #48347
Check opts when register OpenStack CloudProvider rather than
returning error when use opts to create/use cloud resource.