Automatic merge from submit-queue

Add quantity benchmarks

Fix proto configs

Inter pod topological affinity and anti-affinity implementation

Fix codecgen sort

Automatic merge from submit-queue

Jenkins: Clean up even if we can't bring cluster up

We're gathering all the cluster logs, go ahead and clean up the
resources.

Automatic merge from submit-queue

fix #24937:  flake pod not found

https://github.com/kubernetes/kubernetes/issues/24937
Split from #24191
Signed-off-by: liang chenye <liangchenye@huawei.com>

Automatic merge from submit-queue

Delete pod with uid as precondition.

Addressed https://github.com/kubernetes/kubernetes/issues/25169#issuecomment-217033202.

Fix #25169 
Fix #24937

This PR change status manager to delete pods with uid as a precondition, so that kubelet won't delete pods with different uid but the same name and namespace accidentally.

/cc @yujuhong 

Automatic merge from submit-queue

start etcd compactor in background

ref: #22448

What's in this PR?
- StartCompactor starts a compactor in the background in order to compact keys older than fixed time. We need to compact keys because we can't let on disk data grow forever. We save the most recent 10 minutes data. It should be enough for slow watchers and to tolerate burst. We might keep a longer history (12h) in the future once storage API can take advantage of multi-version key.
- Have only one compaction job for each cluster. Use endpoints from user input to differentiate clusters.

Automatic merge from submit-queue

API changes for Cascading deletion 

This PR includes the necessary API changes to implement cascading deletion with finalizers as proposed is in #23656. Comments are welcome.

@lavalamp @derekwaynecarr @bgrant0607 @rata @hongchaodeng 

Automatic merge from submit-queue

Allow etcd to store protobuf objects

Split storage serialization from client negotiation, and allow API server to take flag controlling serialization.

TODO:

* [x] API server still doesn't start - range allocation object doesn't seem to round trip correctly to etcd
* [ ] Verify that third party resources are ignoring protobuf (add a test)
* [ ] Add integration tests that verify storage is correctly protobuf
* [ ] Add a global default for which storage format to prefer?

Automatic merge from submit-queue

fix log message for self-signed cert generation

Automatic merge from submit-queue

Add a better error message to run.sh

I ran build/run.sh w/o any args (by mistake) and it just said
   `Invalid input.`
after several other steps. I had no idea whether I was doing something
wrong or if my env was busted. Clearly, I just forget to include the
command that run.sh was to invoke in the Docker container.  But it took
me time to go track down where this error came from and why. So to help
others I just tweaked the error message to be:
   `Invalid input - please specify a command to run.`

Very minor thing,I know, but if it helps others...
Signed-off-by: Doug Davis <dug@us.ibm.com>

Automatic merge from submit-queue

Reduce kubelet LOC: extract getters

Step 1 of #25028 as discussed in @kubernetes/sig-node meeting

Automatic merge from submit-queue

Map secret files into dockerized e2e

Rather than copying the GCE and AWS private keys/credentials to each Jenkins VM, we can put them in credentials and map them through.

This is one half of the change; if the relevant environment variables are set, we'll mount the files in.

cc @fejta @rmmh @apelisse 

Don't surround node-tags list with square brackets

That's not how yaml list parsing works...

Actually populate nodeTags field in GCE provider struct

I ran build/run.sh w/o any args (by mistake) and it just said
   `Invalid input.`
after several other steps. I had no idea whether I was doing something
wrong or if my env was busted. Clearly, I just forget to include the
command that run.sh was to invoke in the Docker container.  But it took
me time to go track down where this error came from and why. So to help
others I just tweaked the error message to be:
   `Invalid input - please specify a command to run.`

Very minor thing,I know, but if it helps others...
Signed-off-by: Doug Davis <dug@us.ibm.com>

Automatic merge from submit-queue

Fix v1.3.0-alpha.3 CHANGELOG entry

We got a double entry in #24942; my bad.  Another example of why automation > humans.  👍 

Automatic merge from submit-queue

GCI: Add two GCI specific metadata pairs

This PR adds two GCI specific metadata pairs when using GCI image.

(1) "gci-update-strategy": by default the GCI in-place updater is enabled. It means that when a new image is released, the instance on the old image will be upgraded to the new image. In this change, we turn it off;

(2) "gci-ensure-gke-docker": GCI is built with two versions of docker. When this metadata is set to "true", the version satisfying kubernetes qualification will be used. Setting this metadata prevents from using incorrect docker version.

Automatic merge from submit-queue

e2e/framework/util.StartPods: don't wait for pods that are not created

When running ``[k8s.io] SchedulerPredicates [Serial] validates resource limits of pods that are allowed to run [Conformance]`` pods can be created in a way in which additional pods have to be create to fully saturate node's capacity CPU in a cluster. Additional pods are created by calling ``framework.StartPods``. The function creates pods with a given label and waits for them (if ``waitForRunning`` is ``true``). This is fine as long as the number of pods to created is non-zero. If there are zero pods to be created and ``waitForRunning`` is ``true``, the function waits forever as there is not going to be any pods with requested label. Thus, resulting in ``Error waiting for 0 pods to be running - probably a timeout``. Causing the e2e test to fail even if it should not.

Adding condition to return from the function immediately if there is not pod to create.

The codec factory should support two distinct interfaces - negotiating
for a serializer with a client, vs reading or writing data to a storage
form (etcd, disk, etc). Make the EncodeForVersion and DecodeToVersion
methods only take Encoder and Decoder, and slight refactoring elsewhere.

In the storage factory, use a content type to control what serializer to
pick, and use the universal deserializer. This ensures that storage can
read JSON (which might be from older objects) while only writing
protobuf. Add exceptions for those resources that may not be able to
write to protobuf (specifically third party resources, but potentially
others in the future).

YAML is not guaranteed to be recognizable, so we need to bump JSON and
protobuf above it in the decoding order. Add a unit test.

Automatic merge from submit-queue

Petset controller

Took longer than I expected. Main parts of this pr are:
1. Identity generation based on petset spec (volumes are mapped per discussion in #18016)
2. Ensure that we create/delete pets in sequence
3. Ensuring that we create, wait for healthy, create; or delete, wait for terminationGrace, delete
4. Controller that watches apiserver and drives actual -> desired

PVCs are not deleted, yet.

Automatic merge from submit-queue

Deleting duplicate code from federated-apiserver.Run()

This removes most of duplicate code from federated-apiserver.Run().
The code remaining is related to storage or authz and authn.
https://github.com/kubernetes/kubernetes/pull/24787 refactors the storage related code.
I am still figuring out authz and authn.

cc @jianhuiz 

Automatic merge from submit-queue

Fix units in kubemark resource gatherer

Automatic merge from submit-queue

Configure proto for test clusters

ref #25132

@smarterclayton - FYI

Automatic merge from submit-queue

Use v1.6.2-1 tag for build.

Is there any reason these don't use the VERSION file like everything else? cc @luxas @ixdy