Automatic merge from submit-queue

Move generated openAPI specs out of genericapiserver and make it configurable

A follow up to #31468

Generated OpenAPI Spec does not belong to genericapiserver package. A new package "generated" created in hope of all generated codes goes into it in future. Openapi folder of that package contains generated definitions and generic API server will accept the definition map as a configuration parameter.

Reference: #13414

Automatic merge from submit-queue

stop etcd watcher when watch chan is closed

When startWatching() stops due to the watch chan is closed, the watcher could still keeps running in the background. When this case happen, we should stop the watcher entirely and close ResultChan() too.

Automatic merge from submit-queue

federation ingress controller: Fixing a panic by initialising the map before using it

Ref https://github.com/kubernetes/kubernetes/issues/32733#issuecomment-248428635

cc @kubernetes/sig-cluster-federation @quinton-hoole

Automatic merge from submit-queue

Only approve evictions when budgets would stay enforced after.

Prior to this, we would approve eviction as long as the current state of
the pods matched the budget.  The new version requires that after the
eviction, the pods would still match the budget.

Also update tests to match.

Prior to this, we would approve eviction as long as the current state of
the pods matched the budget.  The new version requires that after the
eviction, the pods would still match the budget.

Also update tests to match.

Automatic merge from submit-queue

Stop referring to loop variable inside closure.

Prior to this, I was actually running the same (last) test 5 times in a row.  :-(

Fixes #33137

Automatic merge from submit-queue

Fix the properties file for node e2e cri validation.

I fixed this locally before, but accidentally missed in the PR. Sorry about that.

This time, I've tried myself, it should work.

@yujuhong

Automatic merge from submit-queue

In error, the first letter is low-case letter

Automatic merge from submit-queue

Fix backward compatibility issue caused by promoting initcontainers f…

#31026 moves init-container feature from alpha to beta, but only took care the backward compatibility for pod specification, not deal with status. For status, it simply moved from `pods.beta.kubernetes.io/init-container-statuses` to
`pods.beta.kubernetes.io/init-container-statuses` instead of introducing one more pods.beta.kubernetes.io/init-container-statuses. This breaks when the cluster is running with 1.4, but the user is still running with kubectl 1.3.x. 

Fixed #32711

Automatic merge from submit-queue

Bump up GCI version.

```release-note
   Upgrading Container-VM base image for k8s on GCE. Brief changelog as follows:
    - Fixed performance regression in veth device driver
    - Docker and related binaries are statically linked
    - Fixed the issue of systemd being oom-killable
```

Fixes #32596

This needs a cherrypick into v1.4 release branch because it is fixing v1.4 release blocking issues. This patch is easy and safe to rollback in case of emergencies.

@vishh can you please review?

Fixes #32596 and many other issues.
cc/ @kubernetes/goog-image  FYI

Automatic merge from submit-queue

Move HighWaterMark to the top of the struct in order to fix arm

I haven't tested this yet, but let's see how e2e tests react.
It should be no difference at all except for that it will fix arm.

etcd has had to do this some times (and I think there are some fixes like this that are needed for etcd as well)

For reference see: https://golang.org/pkg/sync/atomic/

This should be a cherrypick-candidate for v1.4.1 (as I understand it, v1.4.0 has clearly left the cherrypickable state)

@lavalamp @pwittrock @xiang90 @smarterclayton

Automatic merge from submit-queue

use groupmetafactoryregistry

Switches all groups to the new API registration mechanism.

@ncdc ptal

Automatic merge from submit-queue

Viper direct bindings to TestContext struct with hierarchichal suppor…

Part of #31453 to support hierarchichal parameters.  This one does so for density, paves way for other tests as well.

Automatic merge from submit-queue

CRI: Add CRI validation suite

Based on https://github.com/kubernetes/kubernetes/pull/32903.

This PR added a separate test suite for CRI. I'll enable this in test-infra and make this publicly visible via https://k8s-testgrid.appspot.com/.

@yujuhong @yifan-gu @feiskyer 
/cc @kubernetes/sig-node

Brief changelog compared to gci-dev-54-8743-3-0:
- Fixed performance regression in veth device driver
- Docker and related binaries are statically linked
- Fixed the issue of systemd being oom-killable
- Updated built-in kubelet version to 1.3.7
- add ethtool and ebtables binaries expected by kubelet

Fixes #32596

Automatic merge from submit-queue

[Controller Manager] Fix job controller hot loop

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it:**

Fix Job controller hotloop on unexpected API server rejections.

**Which issue this PR fixes**

Related issue is #30629

**Special notes for your reviewer:**

@deads2k @derekwaynecarr PTAL.

Automatic merge from submit-queue

fix a spell mistake

**What this PR does / why we need it**:
this should be "every" not 'ever'

**Which issue this PR fixes**: 

**Special notes for your reviewer**:

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
```

Kubernetes HEAD and docker v1.11.2 with CRI enabled.

Automatic merge from submit-queue

Work around the etcd watch issue

fix #32770

Underlyingly, differentiating context will unshare the grpc stream.

Automatic merge from submit-queue

Add myself to job/scheduledjob tests

@fejta updated the test owners again :)

Automatic merge from submit-queue

kubelet: Add a hidden flag to enable docker CRI integration for testing

Automatic merge from submit-queue

Add additional delay to the es logging e2e test to make it stable

Fix #32804

Automatic merge from submit-queue

typo fixes

**What this PR does / why we need it**: Fixes typographical errors.

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: None

**Special notes for your reviewer**: None

**Release note**: None

Automatic merge from submit-queue

Add storage api group to hack/update-codegen.sh

Add it to the script then run it in the second commit. 1.4 needs the same change, I'll do it in my other PR.

Automatic merge from submit-queue

Extend kubemark cluster startup timeout to account for log route crea…

Fix #32441 (see https://github.com/kubernetes/kubernetes/issues/32441#issuecomment-248308161)

cc @wojtek-t

Automatic merge from submit-queue

separate RESTStorage by API group

This demonstrates how we could start separating the description of `RESTStorage` from `pkg/master`.  Each API group owns constructing its own RESTStorage.  Since `pkg/registry` is where we place all packages that create RESTStorage, it seems reason to split the package by `pkg/registry/<api group>/storage` to create RESTStorage and `pkg/registry/<api group>/<resource>` to support each individual resource.

The interface to construct this RESTStorage is dependent upon some `genericapiserver` types now, but we could (should?) move those items to a different location.

@kubernetes/sig-api-machinery 
@lavalamp can you weigh in on how I've split these packages?  I think we agree that we need better segregation, but we haven't spoken about how to spell it.

Automatic merge from submit-queue

Node E2E: Add image white list

This is part of #29081. Fixes #29155.

As is discussed with @yujuhong in #29155, it is difficult to maintain the prepull image list if it is not enforced. 

This PR added an image white list in the test framework, only images in the white list could be used in the test. If the image is not in the white list, the test will fail with reason:
```
Image "XXX" is not in the white list, consider adding it to CommonImageWhiteList in test/e2e/common/util.go or NodeImageWhiteList in test/e2e_node/image_list.go
```

Notice that if image pull policy is `PullAlways`, the image is not necessary to be in the white list or prepulled, because the test expects the image to be pulled during the test.

Currently, the image white list is only enabled in node e2e, because the image puller in e2e test is not integrated with the image white list yet.

/cc @kubernetes/sig-node

Change-Id: I55ce706381f1494e5cd2064177b938f56d9c356a

Automatic merge from submit-queue

Node E2E: Add image pull retry in image pulling test.

Fixes #29259, #28047.

This test added image pull retry in image pulling node e2e test. It will retry for 3 times until test successes.

This should be able to make the image pulling test less flaky.

@yujuhong