Automatic merge from submit-queue

Move deployment functions to deployment/util.go not widely used

If the function is not used in multiple areas, move it to deployment/util.go

fixes #26750

Automatic merge from submit-queue

Some scheduler optimizations

Ref #28590

This PR doesn't do anything fancy - it is just reducing amount of memory allocations in scheduler, which in turn significantly speeds up scheduler.

Automatic merge from submit-queue

Enable extensions/v1beta1/NetworkPolicy by default

Fixes https://github.com/kubernetes/kubernetes/issues/28401

For some reason this also triggered an update to the swagger spec (which apparently hadn't been done before but wasn't failing validation...)

Automatic merge from submit-queue

Prep for not checking in generated, part 2/2

This PR is extracted from #25978 and build on #28578 - it is just the conversion related parts. All the Makefile stuff is excluded.

@wojtek-t this is literally branched, a bunch of commits deleted, and a very small number of manual fixups applied. If you think this is easier to review (and if it passes CI) you can feel free to go over it again. I will follow this with a conversion-related PR to build on this.

Or if you prefer, just close this and let the mega-PR ride.

@lavalamp

Automatic merge from submit-queue

Fix DNS verbosity 

fixes #28515

By default kube-dns logs all queries against the internal API at Info level. In a larger cluster this is noisy. Log DNS queries at verbosity level 2 by default. 

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

This fixes PodSpec to generate cleanly.  No other types only half-generate (so
now we Fatalf), though several fail to generate at all (only Errorf for now).

There are ample opportunities to optimize and streamline here.  For example,
there's no reason to have a function to convert IntStr to IntStr. Removing the
function does generate the right assignment, but it is unclear whether the
registered function is needed or not.  I opted to leave it alone for now.
Another example is Convert_Slice_byte_To_Slice_byte, which just seems silly.

This drives conversion generation from file tags like:
  // +conversion-gen=k8s.io/my/internal/version
.. rather than hardcoded lists of packages.

The only net change in generated code can be explained as correct.  Previously
it didn't know that conversion was available.

This is used subsequently to simplify the conversion generation, so each
package can declare what peer-packages it uses, and have those imported
dynamically, rather than having one mega list of packages to import and not
really being clear why, for any given list item.

This is to clarify the code.  No actual effect at the moment, but I manually
verified this in the case of identical types.

This is to clarify the intent of the code for new readers (me).

This was causing us to process packages we didn't really want, which was only
visible when debugging was enabled.

This is just comment clarity.

Automatic merge from submit-queue

Prep for not checking in generated, part 1/2

This PR is extracted from #25978 - it is just the deep-copy related parts.  All the Makefile and conversion stuff is excluded.

@wojtek-t this is literally branched, a bunch of commits deleted, and a very small number of manual fixups applied.  If you think this is easier to review (and if it passes CI) you can feel free to go over it again. I will follow this with a conversion-related PR to build on this.

Or if you prefer, just close this and let the mega-PR ride.

@lavalamp 

Automatic merge from submit-queue

Follow up to PR 25640 - Cleanup newline and tweak help text

Follow up to PR https://github.com/kubernetes/kubernetes/pull/25640

* Remove redundant newline below Aliases:
* Renaming "Available Commands:" to "Available Sub-commands:"

Automatic merge from submit-queue

Check for valid serviceaccount JWT token before inspecting claims

Moved claims check after the error check that ensures we have a valid JWT token

Automatic merge from submit-queue

Node E2E: Use waiting reason to figure out image pulling error.

Addresses https://github.com/kubernetes/kubernetes/pull/28323#issuecomment-230002158, using `Waiting` reason to check whether the image pulling failures as expected.

@yujuhong 

[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Node E2E: Disable kubenet for local node e2e test.

After https://github.com/kubernetes/kubernetes/pull/28196, we must manually setup cni and nsenter in local node to run `make test_e2e_node`, which may not be necessary for local development.

I've tried to move cni downloading logic into `BeforeSuite`, however it is still hard to figure out who should install nsenter, manually installed by every developer? in the `setup_host.sh` script? in `BeforeSuite`?

This PR:
* Added a flag to disable kubenet and disabled kubenet in local test.
* Cleaned up the CNI installation logic a bit.

/cc @yujuhong @freehan 
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/.github/PULL_REQUEST_TEMPLATE.md?pixel)]()

Automatic merge from submit-queue

Remove reflection path in meta.Accessor

Callers are required to implement their interfaces, removes the
potential for mistakes. We have a reflective test
pkg/api/meta_test.go#TestAccessorImplementations that verifies that all
objects registered to the scheme properly implement their interfaces.

@wojtek-t this may need an additional test or two, but I think this is a
net win over the current state (where people fallback to reflection without
being aware of it).

Automatic merge from submit-queue

don't migrate files you can't access

If you can't access a file, you shouldn't try to migrate it.

Ref https://github.com/openshift/origin/issues/9581

@fabianofranz 

Fix mungedocs TOC generation for duplicate headers

Automatic merge from submit-queue

Autoscaling should be part of clientset.Adaption

@mfojtik

This make it clearer where "is copyable" is needed and where "is in bounds" is
needed.  No effect on generated code.

This is closer to standard Go semantics and obsoletes our custom --recursive
flag.

This minimizes the number of functions generated.  It also breaks the chain of
symbol dependencies to pkg/conversion, so we can remove now-unreferenced
(previously unused but referenced) conversion deep-copies.

This is the last piece of Clayton's #26179 to be implemented with file tags.
All diffs are accounted for.  Followup will use this to streamline some
packages.

Also add some V(5) debugging - it was helpful in diagnosing various issues, it
may be helpful again.

This drives most of the logic of deep-copy generation from tags like:
  // +deepcopy-gen=package
..rather than hardcoded lists of packages.  This will make it possible to
subsequently generate code ONLY for packages that need it *right now*, rather
than all of them always.

Also remove pkgs that really do not need deep-copies (no symbols used
anywhere).

This will be used when generated files are no longer checked in.  I include it
as a standalone commit for ease of review.

It's simpler and clearer without them, and it leaves room for per-use-case
parsing of the tag lines.  If anything we should be splitting on whitespace.

This is in prep to simplify tag logic.  Don't rely on processing commas as new
tag delimiters.  Put new tags on new lines.  This had zero effect on generated
code (as intended).

Previously we just tracked comments on the 'package' declaration.  Treat all
file comments as one comment-block, for simplicity.  Can be revisited if
needed.

This means that tags like:
  // +foo=bar
  // +foo=bat
..will produce []string{"bar", "bat"}.  This is needed for later commits which
will want to use this to make code generation more self contained.

This makes subsequent comment-tag PRs more consistent.

In bringing back Clayton's PR piece-by-piece this was almost as easy to
implement as his version, and is much more like what I think we should be
doing.

Specifically, any time which defines a .DeepCopy() method will have that method
called preferentially.  Otherwise we generate our own functions for
deep-copying.  This affected exactly one type - resource.Quantity.  In applying
this heuristic, several places in the generated code were simplified.

To achieve this I had to convert types.Type.Methods from a slice to a map,
which seems correct anyway (to do by-name lookups).

This re-institutes some of the rolled-back logic from previous commits. It
bounds the scope of what the deepcopy generator is willing to do with regards
to generating and calling generated functions.

His PR cam during the middle of this development cycle, and it was easier to
burn it down and recreate it than try to patch it into an existing series and
re-test every assumption.  This behavior will be re-introduced in subsequent
commits.