Automatic merge from submit-queue

Add deployments to federation apiserver

cc: @quinton-hoole @nikhiljindal @kubernetes/sig-cluster-federation

Automatic merge from submit-queue

Fix hack/dev-push-hyperkube.sh

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**: This fixes `hack/dev-push-hyperkube.sh`. It simplifies/standardizes its usage and fixes it for the changes to the build system.

Automatic merge from submit-queue

Ubuntu: download client and server tarballs instead of mondo-tarball

Part of #28629 / #28435.

This should be functionally the same, except that you will download ~1/3 the bytes.

cc @WIZARD-CXY

Automatic merge from submit-queue

Update godeps for libcontainer+cadvisor

Needed to unblock more progress on pod cgroup.

/cc @vishh @dchen1107 @timstclair

Automatic merge from submit-queue

etcd3: use PrevKV to remove additional get

ref: #https://github.com/kubernetes/kubernetes/issues/33653

We are trying to test using PrevKV feature and see if it improves performance.
In order to test this, we will need etcd v3.1 (alpha) image.

Blockers:
- update gcr.io image (version v3.0.12)

Automatic merge from submit-queue

Fixup in kubelet.go: Package short names should be [a-z0-9]

Automatic merge from submit-queue

Kubelet: Use RepoDigest for ImageID when available

```release-note
Use manifest digest (as `docker-pullable://`) as ImageID when available (exposes a canonical, pullable image ID for containers).
```

Previously, we used the docker config digest (also called "image ID"
by Docker) for the value of the `ImageID` field in the container status.
This was not particularly useful, since the config manifest is not
what's used to identify the image in a registry, which uses the manifest
digest instead.  Docker 1.12+ always populates the RepoDigests field
with the manifest digests, and Docker 1.10 and 1.11 populate it when
images are pulled by digest.

This commit changes `ImageID` to point to the the manifest digest when
available, using the prefix `docker-pullable://` (instead of
`docker://`)

Related to #32159

Automatic merge from submit-queue

DaemonSets in federated apiserver

cc: @quinton-hoole @nikhiljindal

Automatic merge from submit-queue

Add hint message for go-bindata

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**:

Add hint message for go-bindata

Automatic merge from submit-queue

Fix upgrade.sh image setup

The original fix (#33147) sourced the correct `node-helper.sh` but set
`node_os_distribution` instead of `NODE_OS_DISTRIBUTION`. The
`set-node-image` function is imported indirectly via `source
"${KUBE_ROOT}/cluster/kube-util.sh"`, which in turn (in the GCE case)
sources `cluster/gce/util.sh`. Since the `set-node-image` function
relies on the `NODE_OS_DISTRIBUTION` variable, the original fix
did not have the entire intended effect.

I have confirmed that cherry-picking #33147 into the `release-1.4`
branch and layering this commit on top of it make for a successful
upgrade from a GCI based K8s 1.3 cluster to a GCI based K8s 1.4 cluster.


**NOTE:** This, along with #33147, should be cherry-picked into the `release-1.4` branch.

Automatic merge from submit-queue

Add default cluster role bindings

Add default cluster roles bindings to rbac bootstrapping.  Also adds a case for allowing escalation when you have no authenticator.

@liggitt I expect you may need to make peace with this.

Automatic merge from submit-queue

Update sftp package

In support of #34328

Looks like some folks need:
https://github.com/pkg/sftp/commit/4d0e916071f68db74f8a73926335f809396d6b42

Automatic merge from submit-queue

Enable service account signing key rotation

fixes #21007

```release-note
The kube-apiserver --service-account-key-file option can be specified multiple times, or can point to a file containing multiple keys, to enable rotation of signing keys.
```

This PR enables the apiserver authenticator to verify service account tokens signed by different private keys. This can be done two different ways:
* including multiple keys in the specified keyfile (e.g. `--service-account-key-file=keys.pem`)
* specifying multiple key files (e.g. `--service-account-key-file current-key.pem --service-account-key-file=old-key.pem`)

This is part of enabling signing key rotation:

1. update apiserver(s) to verify tokens signed with a new public key while still allowing tokens signed with the current public key (which is what this PR enables)
2. give controllermanager the new private key to sign new tokens with
3. remove old service account tokens (determined by verifying signature or by checking creationTimestamp) once they are no longer in use (determined using garbage collection or magic) or some other algorithm (24 hours after rotation, etc). For the deletion to immediately revoke the token, `--service-account-lookup` must be enabled on the apiserver.
4. once all old tokens are gone, update apiservers again, removing the old public key.

Automatic merge from submit-queue

Run flaky tests in parallel

We should try to emulate the main CI environment in the flaky test suite so that it is clear when a test can be moved out of the flaky suite. Since a common source of flakes is unintended interactions between tests running in parallel, we should run the flaky suite in parallel to better detect such flakes.

Automatic merge from submit-queue

Clarify MinAvailable approval semantics.

Automatic merge from submit-queue

Fix panic in #34459

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**:
Use the correct variable in the thirdpartyresourcedata codec so it doesn't panic on a nil pointer reference

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #34459

**Special notes for your reviewer**:

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
```

Automatic merge from submit-queue

Bump kubeadm to use kubernetes v1.4.1

v1.4.1 is going to be released tomorrow, so make kubeadm use it by default

This will make it possible to run kubeadm on Raspberry Pi's OOTB

We should strive towards releasing a second kubeadm release this week that's stable and has the small new tweaks we've coded these two weeks.
@errordeveloper @mikedanese

Automatic merge from submit-queue

log-dump.sh: Add a LOG_DUMP_USE_KUBECTL mode, refactor

**What this PR does / why we need it**: This refactors log-dump.sh so that it can optionally just use `"kubectl get node"` output, for dumping logs off a cluster that was brought up using something other than kube-up.sh (e.g. kops or kubeadm).

Note that for obvious reasons, if the cluster is fried, it'll probably fail to hit the apiserver and all bets are off, but for the 95% case where you're debugging tests, this is good. To cover the remaining 5%, you could actually squirrel away the relevant external IPs after the e2e.go:Up stage (because we better be able to hit the apiserver then) and fetch them from cache later.

**Special notes for your reviewer**: Sorry for so many changes. :)

Automatic merge from submit-queue

Clean up the link in cassandra

**What this PR does / why we need it**:
Clean up the link in  README for cassandra

**Which issue this PR fixes** :
 fixes #34431
Signed-off-by: yuexiao-wang <wang.yuexiao@zte.com.cn>

Automatic merge from submit-queue

Remove onlyLocal NodePort e2e till pr #33957

We were basically testing this bug: https://github.com/kubernetes/kubernetes/issues/30809
We fixed the bug: https://github.com/kubernetes/kubernetes/pull/33587, but forgot to remove the "test". 
This pr adds a test for the new feature: https://github.com/kubernetes/kubernetes/pull/33957 (ensure that nodePort with onlyLocal works only on nodes with endpoints and fails otherwise)

fixes https://github.com/kubernetes/kubernetes/issues/34124

This refactors log-dump.sh so that it can optionally just use "kubectl
get node" output, for dumping logs off a cluster that was brought up
using sometihng other than kube-up.sh (e.g. kops or kubeadm).

Note that for obvious reasons, if the cluster is fried, it'll probably
fail to hit the apiserver and all bets are off, but for the 95% case
where you're debugging tests, this is good. To cover the remaining 5%,
you could actually squirrel away the relevant external IPs after the
e2e.go:Up stage (because we better be able to hit the apiserver then)
and fetch them from cache later.

Automatic merge from submit-queue

Move periodic backoff gc to federation utils

Plus put it into a for loop.

cc: @quinton-hoole

Automatic merge from submit-queue

openstack: Support config-drive and improve CurrentNodeName, GetZone

This PR adds support for fetching local instance metadata via config-drive (as well as querying metadata service), and surfaces some additional metadata information (from either source):

- `CurrentNodeName` now returns the OpenStack instance name, rather than the current hostname (they might not be the same)
- `GetZone` includes availability zone label in `FailureDomain`

Thanks to @kiall for a WIP implementation of the latter.

Previously, we used the docker config digest (also called "image ID"
by Docker) for the value of the `ImageID` field in the container status.
This was not particularly useful, since the config manifest is not
what's used to identify the image in a registry, which uses the manifest
digest instead.  Docker 1.12+ always populates the RepoDigests field
with the manifest digests, and Docker 1.10 and 1.11 populate it when
images are pulled by digest.

This commit changes `ImageID` to point to the the manifest digest when
available, using the prefix `docker-pullable://` (instead of
`docker://`)

Previously, the `InspectImage` method of the Docker interface expected a
"pullable" image ref (name, tag, or manifest digest).  If you tried to
inspect an image by its ID (config digest), the inspect would fail to
validate the image against the input identifier.  This commit changes
the original method to be named `InspectImageByRef`, and introduces a
new method called `InspectImageByID` which validates that the input
identifier was an image ID.

Automatic merge from submit-queue

hack: update cherry-pick script to show subject when patch is split

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**:

This is a continuation of #34049, which worked, but then I realized some patch files have more than one subject, see example: https://github.com/kubernetes/kubernetes/pull/34228

This cleans the output so it looks like this:

```
Automated cherry pick of #32593

Cherry pick of #32593 on release-1.4.

#32593: Fix audit_test regex for iso8601 timestamps
```

pretty!!!

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
Signed-off-by: Jess Frazelle <acidburn@google.com>

Automatic merge from submit-queue

Use nodeutil.GetHostIP consistently when talking to nodes

Most of our communications from apiserver -> nodes used
    nodutil.GetNodeHostIP, but a few places didn't - and this meant that the
    node name needed to be resolvable _and_ we needed to populate valid IP
    addresses.

```release-note
The apiserver now uses addresses reported by the kubelet in the Node object's status for apiserver->kubelet communications, rather than the name of the Node object. The address type used defaults to `InternalIP`, `ExternalIP`, and `LegacyHostIP` address types, in that order.
```

The original fix (#33147) sourced the correct `node-helper.sh` but set
`node_os_distribution` instead of `NODE_OS_DISTRIBUTION`. The
`set-node-image` function is imported indirectly via `source
"${KUBE_ROOT}/cluster/kube-util.sh"`, which in turn (in the GCE case)
sources `cluster/gce/util.sh`. Since the `set-node-image` function
relies on the `NODE_OS_DISTRIBUTION` variable, the original fix
did not have the entire intended effect.

I have confirmed that cherry-picking #33147 into the `release-1.4`
branch and layering this commit on top of it make for a successful
upgrade from a GCI based K8s 1.3 cluster to a GCI based K8s 1.4 cluster.

Fixes #34459

Fixes doc comments to be in line with #33143

Automatic merge from submit-queue

Federated deployment controller - part 1

Based on federated replicaset controller (copy + find/replace).

Remaining stuff:
- refacing out common elements to libs
- using owerref in pod analysis
- e2e tests
- renaming concurrency flag for rs and reusing it in deployment
- updating only one cluster at a time if rollingupdate strategy is used.

cc: @quinton-hoole @kubernetes/sig-cluster-federation

**Release note**:

```release-note
Federated deployment controller that supports the same api as the regular kubernetes deployment controller.
```