Automatic merge from submit-queue (batch tested with PRs 37325, 38313, 38141, 38321, 38333)

Fix running e2e with 'Completed' kube-system pods

As of now, e2e runner keeps waiting for pods in `kube-system` namespace to be "Running and Ready" if there are any pods in `Completed` state in that namespace.
This for example happens after following [Kubernetes Hosted Installation](http://docs.projectcalico.org/v2.0/getting-started/kubernetes/installation/#kubernetes-hosted-installation) instructions for Calico, making it impossible to run conformance tests against the cluster. It's also to possible to reproduce the problem like that:
```
$ cat testjob.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: tst
  namespace: kube-system
spec:
  template:
    metadata:
      name: tst
    spec:
      containers:
      - name: tst
        image: busybox
        command: ["echo",  "test"]
      restartPolicy: Never
$ kubectl create -f testjob.yaml
$ go run hack/e2e.go -v --test --test_args='--ginkgo.focus=existing\s+RC'
```

Automatic merge from submit-queue

type HttpProxyCheck should be HTTPProxyCheck

**What this PR does / why we need it**:
Change HttpProxyCheck to HTTPProxyCheck
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
golint
**Release note**:

```release-note
```
Signed-off-by: yupeng <yu.peng36@zte.com.cn>

Automatic merge from submit-queue (batch tested with PRs 38318, 38258)

kernel memcg notification enabled via experimental flag

Kubelet integrates with kernel memcg notification API if and only if enabled via experimental flag.

Automatic merge from submit-queue

Adds docs for dns-horizontal-autoscaler and kube-dns

Although we have separate docs on kubernetes.io, we should have a short description about the dns-horizontal-autoscaler addon in folder.

Also updates kube-dns README with example command to scale kube-dns Deployment. This is needed because Addon Manager v6 has stricter reconcile behavior.

@bowei @bprashanth @thockin

Automatic merge from submit-queue

Delete regional static-ip instead of global for type=lb

Global vs region is the difference between 
```
$ gcloud compute addresses delete foo --global
$ gcloud compute addresses delete foo --region us-central1
```

Type=LoadBalancer users the second type and were were doing the first. 
Also adds some logging.

Automatic merge from submit-queue (batch tested with PRs 36140, 38304)

remove validation dependency on version negotiation

Things that just need a REST client or a bit of discovery information should not attempt to perform negotiation.  Doing so just limits the genericness of the client.

@kubernetes/sig-cli 
@fabianofranz this blocks the kubernetes-discovery work.

Automatic merge from submit-queue

make invocation ReadDockerConfigFile can handle .dockerconfigjson file

**What this PR does / why we need it**:

When **.docker/config.json** is used to authenticate docker registry, the data key **.dockerconfigjson** should be used if we want to save this kind of docker auth data into a secret. So this PR is mainly to make invocation `ReadDockerConfigFile `have ability to read  **.dockerconfigjson** file.
@liggitt

Automatic merge from submit-queue

Cover port_allocator_test with more conditions

The test cases of port_allocator_test should cover more conditions, such as `rangeAllocator.used.Bit`.

Automatic merge from submit-queue (batch tested with PRs 36543, 38189, 38289, 38291, 36724)

context.Context should be the first parameter of a function in vsphere

**What this PR does / why we need it**:
Change the position of the context.Context parameter.
**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:
golint
**Release note**:

```release-note
```
Signed-off-by: yupeng <yu.peng36@zte.com.cn>

Automatic merge from submit-queue (batch tested with PRs 36543, 38189, 38289, 38291, 36724)

kube-up: Only specify ETCD_QUORUM_READ if non-empty

Automatic merge from submit-queue (batch tested with PRs 36543, 38189, 38289, 38291, 36724)

add authentication/authorization to kubernetes-discovery

Wires authentication and authorization into `kubernetes-discovery` and re-enables the `local-up-cluster.sh` along with proper permission granting for RBAC cases.

@sttts @liggitt

Automatic merge from submit-queue (batch tested with PRs 36543, 38189, 38289, 38291, 36724)

Kubelet: only check podUID when it is actually set

Fixes #38188.

cc/ @timstclair @yujuhong

Automatic merge from submit-queue

Fix scheduler_perf test so that QPS is non-zero even if there is a scheduling "cold start" 

@gmarek  ... is something like this more realistic as an expectation ?  That is, wait till scheduling starts, then wait 1 second before any polling attempts....  i.e. 

- gaurantee uniform sleep before measure, by doing it first rather then last
- print the initial status so that its easier to debug in case of issues

#36532

Automatic merge from submit-queue (batch tested with PRs 38294, 37009, 36778, 38130, 37835)

Add a cloudprovider validator flag to kubeadm and update the DNS spec

Broken out from: https://github.com/kubernetes/kubernetes/pull/37568

This PR creates a flag for `cloud-provider` that validates the value before `RunInit()` is run, which makes it now act as a "real" flag
Then it removes the `k8s.io/kubernetes/pkg/cloudprovider` dependency, which makes the binary ~40MB smaller! That's _really_ worth it!

In the second commit, the DNS addon is updated to the latest version: https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/skydns-rc.yaml.base

@kubernetes/sig-cluster-lifecycle

Automatic merge from submit-queue (batch tested with PRs 38294, 37009, 36778, 38130, 37835)

Bump github.com/docker/go-units dependency

<!--  Thanks for sending a pull request!  Here are some tips for you:
1. If this is your first time, read our contributor guidelines https://github.com/kubernetes/kubernetes/blob/master/CONTRIBUTING.md and developer guide https://github.com/kubernetes/kubernetes/blob/master/docs/devel/development.md
2. If you want *faster* PR reviews, read how: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/faster_reviews.md
3. Follow the instructions for writing a release note: https://github.com/kubernetes/kubernetes/blob/master/docs/devel/pull-requests.md#release-notes
-->

**What this PR does / why we need it**: Bump [github.com/docker/go-units](https://github.com/docker/go-units/) to include https://github.com/docker/go-units/pull/22 which improves the precision of human durations ("about an hour ago", "2 days ago", etc).

**Which issue this PR fixes**: Issue detected in OpenShift: https://github.com/openshift/origin/issues/11967

**Release note**:
<!--  Steps to write your release note:
1. Use the release-note-* labels to set the release note state (if you have access) 
2. Enter your extended release note in the below block; leaving it blank means using the PR title as the release note. If no release note is required, just write `NONE`. 
-->
```release-note
NONE
```

Automatic merge from submit-queue (batch tested with PRs 38294, 37009, 36778, 38130, 37835)

Only configure basic auth on gci if KUBE_USER and KUBE_PASSWORD are specified.

This should not change the existing flow when KUBE_USER/KUBE_PASSWORD are specified.
It makes not specifying those a valid option that means "don't turn on basic auth".
I only did it for cluster/gce/gci for now, but others should be somewhat similar.

Automatic merge from submit-queue (batch tested with PRs 38294, 37009, 36778, 38130, 37835)

fix permissions when using fsGroup

Currently, when an fsGroup is specified, the permissions of the defaultMode are not respected and all files created by the atomic writer have mode 777.  This is because in `SetVolumeOwnership()` the `filepath.Walk` includes the symlinks created by the atomic writer.  The symlinks have mode 777 when read from `info.Mode()`.  However, when the are chmod'ed later, the chmod applies to the file the symlink points to, not the symlink itself, resulting in the wrong mode for the underlying file.

This PR skips chmod/chown for symlinks in the walk since those operations are carried out on the underlying file which will be included elsewhere in the walk.

xref https://bugzilla.redhat.com/show_bug.cgi?id=1384458

@derekwaynecarr @pmorie

Automatic merge from submit-queue (batch tested with PRs 38294, 37009, 36778, 38130, 37835)

Re-use tested ratelimiter

The ratelimiter introduced in #35583 is not working correctly when called from multiple threads
This reverts to the tested ratelimiter we were previously using to unblock 1.5 (automated revert wasn't possible)
Ref #38273

reproducing test case:
```
func TestMultiThreadedBlocking(t *testing.T) {
	done := make(chan bool)

	// 100 QPS, burst of 100
	b := NewBucketWithRate(100, 100)

	go func() {
		defer close(done)
		fmt.Println(time.Now(), "Waiting for 1000 (should block for ~9-10 seconds)")
		b.Wait(1000)
		fmt.Println(time.Now(), "Got 1000")
	}()

	// give the request for 1000 plenty of time to take the tokens
	time.Sleep(2 * time.Second)
	fmt.Println(time.Now(), "Waiting for 1 (should wait until 1000 block is refilled)")
	b.Wait(1)
	fmt.Println(time.Now(), "Got 1 (should happen right after the wait for 1000 completes)")

	<-done
}

$ go test ./pkg/util/ratelimit/ -v -run TestMultiThreadedBlocking
=== RUN   TestMultiThreadedBlocking
2016-12-07 12:15:36.222133049 -0500 EST Waiting for 1000 (should block for ~9-10 seconds)
2016-12-07 12:15:38.222797752 -0500 EST Waiting for 1 (should wait until 1000 block is refilled)
2016-12-07 12:15:38.222897951 -0500 EST Got 1 (should happen right after the wait for 1000 completes)
2016-12-07 12:15:45.223125234 -0500 EST Got 1000
```

in contrast, the same test run against juju/ratelimit:
```
go test ./pkg/util/flowcontrol/ -v -run TestMultiThreadedBlocking
=== RUN   TestMultiThreadedBlocking
2016-12-07 12:32:56.796077782 -0500 EST Waiting for 1000 (should block for ~9-10 seconds)
2016-12-07 12:32:58.799159059 -0500 EST Waiting for 1 (should wait until 1000 block is refilled)
2016-12-07 12:33:05.801076002 -0500 EST Got 1000
2016-12-07 12:33:05.807510387 -0500 EST Got 1 (should happen right after the wait for 1000 completes)
--- PASS: TestMultiThreadedBlocking (9.01s)
```

Reverts changes in cebfc821

Automatic merge from submit-queue (batch tested with PRs 35101, 38215, 38092)

fix alias conflict of clusterrolebinding

create_configmap alias is "cm"

Automatic merge from submit-queue (batch tested with PRs 35101, 38215, 38092)

fix informer generation

Informer generation doesn't work for informers from a different clientset.  This updates the generator to generate the internal interfaces required to break the cycle.

@ncdc take a look at the last two commits.

Automatic merge from submit-queue (batch tested with PRs 35101, 38215, 38092)

auth duplicate detect

I think we should not allow people set duplicate tokens in token file or set duplicate usernames in password file. because the default action overwriting the old data may let people misunderstand.

Fix #38290

Create a new cloud-provider flag in order to catch wrong args directly and don't import the cloudprovider package and save ~50% in binary size

Automatic merge from submit-queue

Disable kubernetes-discovery in local-up-cluster.sh

fix #38257

Fixes local-up-cluster until kubernetes-discovery flags are hooked up

Automatic merge from submit-queue

add default label to rbac bootstrap policy

allow people to retrieve information of bootstrap policy by label :

`kubectl get clusterroles -l key=value` 
`kubectl get clusterrolebindings -l key=value`

Automatic merge from submit-queue

update local-up-cluster to allow full authentication proxying

Adds group and header information in auth proxy authenticator options for `local-up-cluster.sh`.  Must have been missed in the rebase madness.

Automatic merge from submit-queue (batch tested with PRs 38282, 38281)

Fix skip logic in e2e framework

Fixes 37792

cc: @wojtek-t

Automatic merge from submit-queue

e2e: mark 3rd portforwarding test as flaky

Follow-up of https://github.com/kubernetes/kubernetes/pull/38194 and https://github.com/kubernetes/kubernetes/issues/27680#issuecomment-265392033.

cc @kubernetes/sig-testing