Automatic merge from submit-queue

Increase single logfile size in kubemark

Automatic merge from submit-queue

Fix services in load test

Automatic merge from submit-queue (batch tested with PRs 38597, 38570)

Logging request / response body should not allocate

Putting string(data) in the critical path causes an allocation which can
be very large, and happens regardless of whether logging is enabled.

Also, use the more sophisticated logging of body output that avoids
causing quoting of JSON output (current behavior is "{\"key\":...}" and
should be {"key":...} for readability).

@wojtek-t reduces large heap allocations on updates

Automatic merge from submit-queue

generic_scheduler: Schedule() pod param should never be nil

This code doesn't make sense.
Schedule() pod param should never be nil.

Automatic merge from submit-queue

fix typo

**What this PR does / why we need it**:
    fix typo.

**Release note**:

```NONE
```

Automatic merge from submit-queue (batch tested with PRs 38527, 37934)

improve the result of checking role name

Get all error messages before return.

Automatic merge from submit-queue

Fail kubelet is runtime is unresponsive for 30 seconds

Fixes https://github.com/kubernetes/kubernetes/issues/30534

Automatic merge from submit-queue

Register metav1.GetOptions with federation APIs.

PR #37693 missed a spot.

cc @kubernetes/sig-federation @nikhiljindal

Automatic merge from submit-queue (batch tested with PRs 38277, 36361, 38452)

local-up: resolve sudo -E'd paths better

This problem won't affect everyone, but for people who have `--with-secure-path` configured for their sudo binaries, `sudo -E` won't actually preserve PATH, which means the sub-bash won't necessarily be able to find the binary which `test` found to exist.
This fixes that.

Ubuntu [used to](https://bugs.launchpad.net/ubuntu/+source/sudo/+bug/50797) configure it that way, and they might still.
It's configured as such on my (Gentoo) machine.

Automatic merge from submit-queue (batch tested with PRs 38277, 36361, 38452)

Add options for build container rsync optimization

KUBE_RSYNC_COMPRESS env var sets rsync compression level.
KUBE_RSYNC_GENERATED_TO_BUILD_CONTAINER env var disables rsyncing
generated files to build containers.

Why KUBE_RSYNC_COMPRESS is needed -- from rsync manual on `--compress` option (implied by non-zero `--compress-level`):
> Note that this option typically achieves better compression ratios than can be achieved by using a compressing remote shell or a compressing transport because it takes advantage of the implicit information in the matching data blocks that are not explicitly sent over  the connection.

Use case for `KUBE_RSYNC_GENERATED_TO_BUILD_CONTAINER`: when you sometimes build stuff locally (e.g. `make WHAT=cmd/kubectl`) and sometimes do it on remote docker (`build-tools/run.sh make WHAT=cmd/hyperkube`), local builds touch generated files which causes them to be rsynced to the build data container, which may slow down the builds. Still, I'm not sure whether local->remote rsync of generated files is useful (e.g. someone may want to edit generated files for debugging purposes?), so I made not rsyncing these files an option instead of forcing such behavior.

Automatic merge from submit-queue

fix set advertise address in local cluster

Automatic merge from submit-queue

Keeps addon manager yamls in sync

From #38437.

We should have kept all addon manager YAML files in sync. This does not fix the release scripts issue, but we should still have this.

@mikedanese @ixdy

Automatic merge from submit-queue (batch tested with PRs 38058, 38523)

Renames kube-dns configure files from skydns* to kubedns*

`skydns-` prefix and `-rc` suffix are confusing and misleading. Renaming it to `kubedns` in existing yaml files and scripts.

@bowei @thockin

Automatic merge from submit-queue (batch tested with PRs 38058, 38523)

Move unstructured types under `meta/v1` to indicate they are typed

Add a few abstractions that make them simpler to use from generic code that does not need accessors. Move OwnerReference to meta/v1 and remote metatypes.go

Part of #37530

PR #37693 missed a spot.

Putting string(data) in the critical path causes an allocation which can
be very large, and happens regardless of whether logging is enabled.

Also, use the more sophisticated logging of body output that avoids
causing quoting of JSON output (current behavior is "{\"key\":...}" and
should be {"key":...} for readability).

Automatic merge from submit-queue

Don't require docker or other tools when running `make clean`

**What this PR does / why we need it**: we don't require `docker` for a local build, so we shouldn't require it for `make clean` either.

**Which issue this PR fixes**: fixes #37690

cc @sebgoa

Also ensure meta/v1 swagger docs are generated.

Avoid directly accessing an unstructured type if it is not required.

OwnerReference is common.

It is a versioned type.

Automatic merge from submit-queue

fix duplicate validation/field/errors

**Release note**:

``` release-note
release-note-none
```

Related PR: https://github.com/kubernetes/kubernetes/pull/30313

PR #30313 fixed duplicate errors for invalid aggregate errors in
https://github.com/kubernetes/kubernetes/blob/master/pkg/kubectl/cmd/util/helpers.go

However, duplicate aggregate errors that went through
https://github.com/kubernetes/kubernetes/blob/master/pkg/util/validation/field/errors.go
were not affected by that patch.

This patch adds duplicate aggregate error checking to
`pkg/util/validation/field/errors.go`
##### Before

`$ kubectl set env rc/idling-echo-1 test-abc=1234`

```
error: ReplicationController "idling-echo-1" is invalid:
[spec.template.spec.containers[0].env[0].name: Invalid value:
"test-abc": must be a C identifier (matching regex
[A-Za-z_][A-Za-z0-9_]*): e.g. "my_name" or "MyName",
spec.template.spec.containers[1].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName", spec.template.spec.containers[0].env[0].name:
Invalid value: "test-abc": must be a C identifier (matching regex
[A-Za-z_][A-Za-z0-9_]*): e.g. "my_name" or "MyName",
spec.template.spec.containers[1].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName"]
```

`$ kubectl set env rc/node-1 test-abc=1234`

```
error: ReplicationController "idling-echo-1" is invalid:
[spec.template.spec.containers[0].env[0].name: Invalid value:
"test-abc": must be a C identifier (matching regex
[A-Za-z_][A-Za-z0-9_]*): e.g. "my_name" or "MyName",
spec.template.spec.containers[1].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName"]
```
##### After

`$ kubectl set env rc/idling-echo-1 test-abc=1234`

```
error: ReplicationController "idling-echo-1" is invalid:
[spec.template.spec.containers[0].env[0].name: Invalid value:
"test-abc": must be a C identifier (matching regex
[A-Za-z_][A-Za-z0-9_]*): e.g. "my_name" or "MyName",
spec.template.spec.containers[1].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName"]
```

`$ kubectl set env rc/node-1 test-abc=1234`

```
error: ReplicationController "node-1" is invalid:
spec.template.spec.containers[0].env[0].name: Invalid value: "test-abc":
must be a C identifier (matching regex [A-Za-z_][A-Za-z0-9_]*): e.g.
"my_name" or "MyName"
```

Automatic merge from submit-queue

Remove json serialization annotations from internal types

fixes #3933

Internal types should never be serialized, and including json serialization tags on them makes it possible to accidentally do that without realizing it.

fixes in this PR:

* types
  * [x] remove json tags from internal types
  * [x] fix references from serialized types to internal ObjectMeta
* generation
  * [x] remove generated json codecs for internal types (they should never be used)
* kubectl
  * [x] fix `apply` to operate on versioned object
  * [x] fix sorting by field to operate on versioned object
  * [x] fix `--record` to build annotation patch using versioned object
* hpa
  * [x] fix unmarshaling to internal CustomMetricTargetList in validation
* thirdpartyresources
  * [x] fix encoding API responses using internal ObjectMeta
* tests
  * [x] fix tests to use versioned objects when checking encoded content
  * [x] fix tests passing internal objects to generic printers

follow ups (will open tracking issues or additional PRs):
- [ ] remove json tags from internal kubeconfig types (`kubectl config set` pathfinding needs to work against external type)
- [ ] HPA should version CustomMetricTargetList serialization in annotations
- [ ] revisit how TPR resthandlers encoding objects
- [ ] audit and add tests for printer use (human-readable printer requires internal versions, generic printers require external versions)
- [ ] add static analysis tests preventing new internal types from adding tags
- [ ] add static analysis tests requiring json tags on external types (and enforcing lower-case first letter)
- [ ] add more tests for `kubectl get` exercising known and unknown types with all output options

Automatic merge from submit-queue

providers: Remove long-deprecated Instances.List()