Automatic merge from submit-queue

+optional tag for OpenAPI spec

OpenAPI rely on "omitempty" json tag to determine if a field is optional or not. This change will add "+optional" tag to all fields with "omitempty" json tag and support the tag in OpenAPI spec generator.

Automatic merge from submit-queue

Do not cache hyperkube package installation

**What this PR does / why we need it**:

The hyperkube build process could use a cached layer containing out of date packages. For example, the v1.4.0 image contains packages with security vulnerabilities, which should have been available as of the release build date.

This was surfaced from quay.io/clair scanning the hyperkube images:

https://quay.io/repository/coreos/hyperkube-dev/image/17bc61b54ebb627b01da4d04e725c3d73646ef3e047da8cd1b23316aac6961db?tab=vulnerabilities

This patch adds a cache-busting comment to the RUN command which installs/updates packages.

Automatic merge from submit-queue

remove all instances of docker tag -f

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**: There is no flag `-f` for newer versions of `docker tag`

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->

Automatic merge from submit-queue

Escape special characters in jsonpath field names.

There may be a better way to do this, but this seemed like the simplest possible version.

Example: `{.items[*].metadata.labels.kubernetes\.io/hostname}`

[Resolves #31984]

Automatic merge from submit-queue

Add test_list command, to enumerate unit and e2e tests.

This uses go/parser and go/ast to analyze all test files in ~1 second.
It only recognizes a few simple structures that the tests all have, and
modifies a few tests to fit expected structure better.

This is part of an effort to ensure all tests have owners, by having a
verify check to catch new tests being added without an owner.

Automatic merge from submit-queue

Revert "Add e2e tests for storageclass"

Reverts kubernetes/kubernetes#32485

This PR broke the multizone tests on GCE/GKE, and broke the AWS tests, e.g.:
* https://k8s-testgrid.appspot.com/google-aws
* https://k8s-testgrid.appspot.com/google-gke#gke-multizone
* https://k8s-testgrid.appspot.com/google-gke#gci-gke-multizone
Suggest revert. 

Fixes #34794

cc @k8s-oncall

Automatic merge from submit-queue

Fix federated-image.tag written with empty string

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**: Fix federation-up

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #34575

**Special notes for your reviewer**: This issue is happening because there is no kubectl available now when we are trying to write federated-image.tag file. Instead of using kubectl we can use KUBE_GIT_VERSION to get the version.

**Release note**: `NONE`
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
```

@nikhiljindal @madhusudancs @colhom

Automatic merge from submit-queue

Add LimitRange informer

This is part of the goal of reducing duplicate watches for resources in admission/controller code paths.

This uses go/parser and go/ast to analyze all test files in ~1 second.
It only recognizes a few simple structures that the tests all have, and
modifies a few tests to fit expected structure better.

This is part of an effort to ensure all tests have owners, by having a
verify check to catch new tests being added without an owner.

Automatic merge from submit-queue

Make E2E tests easier to debug

This PR removes deferred deletion calls in E2E tests in order to make test easier to debug.  Some of these tests predate namespace finalization; we should just rely on that mechanism to ensure that framework test artifacts are deleted.

Automatic merge from submit-queue

local-up: Add option to guess binary path

This adds a `-O` flag which guesses the right output directory.

The reason for having two flags, not one, is because bash's `getopt` doesn't let you do optional arguments easily, so having both makes the code simpler.

I also removed the redundant empty check; the bash argument check meant that was never hit.

cc @jayunit100 and @jbeda (arbitrary people from the git log)

Automatic merge from submit-queue

fix duplicated changelog from release script

Fixes the duplicated changelog details from the release script.

Automatic merge from submit-queue

Fixed kibana image

@piosz

Automatic merge from submit-queue

test: wait for complete rollouts in WaitForDeploymentStatusValid

@kubernetes/deployment should fix https://github.com/kubernetes/kubernetes/issues/34816 once and forever

Signed-off-by: Jess Frazelle <acidburn@google.com>

Automatic merge from submit-queue

add ownerref permission checks

Adds an admission plugin that ensures that anyone adding an `ownerReference` to a resource has delete rights on the resource they're setting up a delete for.

@caesarxuchao example admission plugin that tests for ownerReference diffs and uses an authorizer to drive the decision.

@liggitt @ncdc we've talked about this before

Automatic merge from submit-queue

Reduce amount of annoying logs in cacher

Automatic merge from submit-queue

controller: set minReadySeconds in deployment's replica sets

* Estimate available pods for a deployment by using minReadySeconds on
the replica set.
* Stop requeueing deployments on pod events, superseded by following the
replica set status.
* Cleanup redundant deployment utilities

Fixes https://github.com/kubernetes/kubernetes/issues/26079

@kubernetes/deployment ptal

Automatic merge from submit-queue

Add monitoring architecture

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**: This adds a description of monitoring architecture.

**Special notes for your reviewer**: This proposal have been already extensively discussed in [this doc](https://docs.google.com/document/d/1z7R44MUz_5gRLwsVH0S9rOy8W5naM9XE5NrbeGIqO2k); this is just a copy&paste so that it's in our repo.

@kubernetes/autoscaling @kubernetes/sig-instrumentation @DirectXMan12 @davidopp @piosz @derekwaynecarr @thockin 

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
NONE
```

Automatic merge from submit-queue

add admin,edit,view roles

Adds some more basic roles to bootstrapping and associated tests.

Automatic merge from submit-queue

Add flags for alternative API and discovery ports

**What this PR does / why we need it**:

We had many issues reported due to the fact that we are using port 443 by default, and we should allow users to override whatever defaults we pick. This doesn't touch on `localhost:8080` yet, which should just generally get rid of.

**Which issue this PR fixes**: fixes #34311 #34307 #33638

**Special notes for your reviewer**: cc @pires 

**Release note**:
```release-note
Add `kubeadm` flags `--api-port` and `--discovery-port`, change default API port to 6443
```

Automatic merge from submit-queue

clientgen: allow to pass custom apiPath when generating client sets

This PR allow to pass the `--clientset-api-path` parameter to clientgen that allows to customize the default API path set in clients. This allows projects like OpenShift to re-use the client with different API path (`/oapi` in our case).

Automatic merge from submit-queue

Add logs generator

@piosz 

Tool to test logging mechanisms, mentioned in #34310