Automatic merge from submit-queue

hack/: update cherry-pick script to have settings to override remote names

**What this PR does / why we need it**: 
it allows you to override the remote names for the cherry-pick script in case you have it named differently

Automatic merge from submit-queue

Improvements to CLI usability and maintainability

Improves `kubectl` from an usability perspective by

1. Fixing how we handle terminal width in help. Some sections like the flags use the entire available width, while others like long descriptions breaks lines but don't follow a well established max width (screenshot below). This PR adds a new responsive writer that will adjust to terminal width and set 80, 100, or 120 columns as the max width, but not more than that given POSIX best practices and recommendations for better readability.
![terminal_width](https://cloud.githubusercontent.com/assets/158611/19253184/b23a983e-8f1f-11e6-9bae-667dd5981485.png)
2. Adds our own normalizers for long descriptions and cmd examples which allows us better control about how things like lists, paragraphs, line breaks, etc are printed. Features markdown support. Looks like `templates.LongDesc` and `templates.Examples` instead of `dedent.Dedend`.
3. Allows simple reordering and reuse of help and usage sections.
3. Adds `verify-cli-conventions.sh` which intends to run tests to make sure cmd developers are using what we propose as [kubectl conventions](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/kubectl-conventions.md). Just a couple simple tests for now but the framework is there and it's easy to extend.
4. Update [kubectl conventions](https://github.com/kubernetes/kubernetes/blob/master/docs/devel/kubectl-conventions.md) to use our own normalizers instead of `dedent.Dedent`.

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
Improves how 'kubectl' uses the terminal size when printing help and usage.
```

@kubernetes/kubectl

Automatic merge from submit-queue

Avoid unnecessary allocation

This is supposed to avoid unnecessary memory allocations.

PodToSelectableFields seems to be the biggest contributor to memory allocations:
```
Showing top 10 nodes out of 247 (cum >= 83166442)
      flat  flat%   sum%        cum   cum%
1796823715 31.09% 31.09% 1796823715 31.09%  k8s.io/kubernetes/pkg/registry/core/pod.PodToSelectableFields
 530856268  9.19% 40.28%  530856268  9.19%  k8s.io/kubernetes/pkg/storage.NamespaceKeyFunc
 241505351  4.18% 44.46%  241505351  4.18%  reflect.unsafe_New
...
```

Automatic merge from submit-queue

+optional tag for OpenAPI spec

OpenAPI rely on "omitempty" json tag to determine if a field is optional or not. This change will add "+optional" tag to all fields with "omitempty" json tag and support the tag in OpenAPI spec generator.

Automatic merge from submit-queue

Do not cache hyperkube package installation

**What this PR does / why we need it**:

The hyperkube build process could use a cached layer containing out of date packages. For example, the v1.4.0 image contains packages with security vulnerabilities, which should have been available as of the release build date.

This was surfaced from quay.io/clair scanning the hyperkube images:

https://quay.io/repository/coreos/hyperkube-dev/image/17bc61b54ebb627b01da4d04e725c3d73646ef3e047da8cd1b23316aac6961db?tab=vulnerabilities

This patch adds a cache-busting comment to the RUN command which installs/updates packages.

Automatic merge from submit-queue

remove all instances of docker tag -f

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**: There is no flag `-f` for newer versions of `docker tag`

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->

Automatic merge from submit-queue

Escape special characters in jsonpath field names.

There may be a better way to do this, but this seemed like the simplest possible version.

Example: `{.items[*].metadata.labels.kubernetes\.io/hostname}`

[Resolves #31984]

Automatic merge from submit-queue

Add test_list command, to enumerate unit and e2e tests.

This uses go/parser and go/ast to analyze all test files in ~1 second.
It only recognizes a few simple structures that the tests all have, and
modifies a few tests to fit expected structure better.

This is part of an effort to ensure all tests have owners, by having a
verify check to catch new tests being added without an owner.

Automatic merge from submit-queue

Revert "Add e2e tests for storageclass"

Reverts kubernetes/kubernetes#32485

This PR broke the multizone tests on GCE/GKE, and broke the AWS tests, e.g.:
* https://k8s-testgrid.appspot.com/google-aws
* https://k8s-testgrid.appspot.com/google-gke#gke-multizone
* https://k8s-testgrid.appspot.com/google-gke#gci-gke-multizone
Suggest revert. 

Fixes #34794

cc @k8s-oncall

Automatic merge from submit-queue

Fix federated-image.tag written with empty string

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**: Fix federation-up

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #34575

**Special notes for your reviewer**: This issue is happening because there is no kubectl available now when we are trying to write federated-image.tag file. Instead of using kubectl we can use KUBE_GIT_VERSION to get the version.

**Release note**: `NONE`
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
```

@nikhiljindal @madhusudancs @colhom

Automatic merge from submit-queue

Add LimitRange informer

This is part of the goal of reducing duplicate watches for resources in admission/controller code paths.

Signed-off-by: Jess Frazelle <acidburn@google.com>

This uses go/parser and go/ast to analyze all test files in ~1 second.
It only recognizes a few simple structures that the tests all have, and
modifies a few tests to fit expected structure better.

This is part of an effort to ensure all tests have owners, by having a
verify check to catch new tests being added without an owner.

Automatic merge from submit-queue

Make E2E tests easier to debug

This PR removes deferred deletion calls in E2E tests in order to make test easier to debug.  Some of these tests predate namespace finalization; we should just rely on that mechanism to ensure that framework test artifacts are deleted.

Automatic merge from submit-queue

local-up: Add option to guess binary path

This adds a `-O` flag which guesses the right output directory.

The reason for having two flags, not one, is because bash's `getopt` doesn't let you do optional arguments easily, so having both makes the code simpler.

I also removed the redundant empty check; the bash argument check meant that was never hit.

cc @jayunit100 and @jbeda (arbitrary people from the git log)

Automatic merge from submit-queue

fix duplicated changelog from release script

Fixes the duplicated changelog details from the release script.

Automatic merge from submit-queue

Fixed kibana image

@piosz

Automatic merge from submit-queue

test: wait for complete rollouts in WaitForDeploymentStatusValid

@kubernetes/deployment should fix https://github.com/kubernetes/kubernetes/issues/34816 once and forever

Signed-off-by: Jess Frazelle <acidburn@google.com>

Automatic merge from submit-queue

add ownerref permission checks

Adds an admission plugin that ensures that anyone adding an `ownerReference` to a resource has delete rights on the resource they're setting up a delete for.

@caesarxuchao example admission plugin that tests for ownerReference diffs and uses an authorizer to drive the decision.

@liggitt @ncdc we've talked about this before

Automatic merge from submit-queue

Reduce amount of annoying logs in cacher

Automatic merge from submit-queue

controller: set minReadySeconds in deployment's replica sets

* Estimate available pods for a deployment by using minReadySeconds on
the replica set.
* Stop requeueing deployments on pod events, superseded by following the
replica set status.
* Cleanup redundant deployment utilities

Fixes https://github.com/kubernetes/kubernetes/issues/26079

@kubernetes/deployment ptal

Automatic merge from submit-queue

Add monitoring architecture

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**: This adds a description of monitoring architecture.

**Special notes for your reviewer**: This proposal have been already extensively discussed in [this doc](https://docs.google.com/document/d/1z7R44MUz_5gRLwsVH0S9rOy8W5naM9XE5NrbeGIqO2k); this is just a copy&paste so that it's in our repo.

@kubernetes/autoscaling @kubernetes/sig-instrumentation @DirectXMan12 @davidopp @piosz @derekwaynecarr @thockin 

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
NONE
```

Automatic merge from submit-queue

add admin,edit,view roles

Adds some more basic roles to bootstrapping and associated tests.