• Kubernetes Submit Queue's avatar
    Merge pull request #49593 from gunjan5/np-e2e · c7f970c5
    Kubernetes Submit Queue authored
    Automatic merge from submit-queue (batch tested with PRs 53567, 53197, 52944, 49593). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.
    
    Refactor network policy e2e tests, and add additional logging when tests fail
    
    **What this PR does / why we need it**:
    - Restructure tests, clean up duplicate code
    - Add additional logging, so it's easier to debug when the tests fail
    - Add retry on the client side pods because sometimes kubelet will start the pod when the network is not ready, resulting in flaky tests
    - Add readiness probe for the server pods to further reduce the flakiness.
    
    Current failure messages when a test fails:
    ```
    checking client-can-connect could communicate with server.
      Expected error:
          <*errors.errorString | 0xc420868820>: {
              s: "pod \"client-can-connect\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2017-07-25 15:54:48 -0700 PDT Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2017-07-25 15:54:57 -0700 PDT Reason:ContainersNotReady Message:containers with unready status: [client-can-connect-container]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2017-07-25 15:54:48 -0700 PDT Reason: Message:}] Message: Reason: HostIP:10.138.0.2 PodIP:10.60.1.180 StartTime:2017-07-25 15:54:48 -0700 PDT InitContainerStatuses:[] ContainerStatuses:[{Name:client-can-connect-container State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2017-07-25 15:54:49 -0700 PDT,FinishedAt:2017-07-25 15:54:57 -0700 PDT,ContainerID:docker://c91c33139672c20a5231bcba9437d74e12d35a6d19275b41bb8e571148b4ab57,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:gcr.io/google_containers/redis:e2e ImageID:docker://sha256:e5e67996c442f903cda78dd983ea6e94bb4e542950fd2eba666b44cbd303df42 ContainerID:docker://c91c33139672c20a5231bcba9437d74e12d35a6d19275b41bb8e571148b4ab57}] QOSClass:BestEffort}",
          }
          pod "client-can-connect" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2017-07-25 15:54:48 -0700 PDT Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2017-07-25 15:54:57 -0700 PDT Reason:ContainersNotReady Message:containers with unready status: [client-can-connect-container]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2017-07-25 15:54:48 -0700 PDT Reason: Message:}] Message: Reason: HostIP:10.138.0.2 PodIP:10.60.1.180 StartTime:2017-07-25 15:54:48 -0700 PDT InitContainerStatuses:[] ContainerStatuses:[{Name:client-can-connect-container State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2017-07-25 15:54:49 -0700 PDT,FinishedAt:2017-07-25 15:54:57 -0700 PDT,ContainerID:docker://c91c33139672c20a5231bcba9437d74e12d35a6d19275b41bb8e571148b4ab57,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:gcr.io/google_containers/redis:e2e ImageID:docker://sha256:e5e67996c442f903cda78dd983ea6e94bb4e542950fd2eba666b44cbd303df42 ContainerID:docker://c91c33139672c20a5231bcba9437d74e12d35a6d19275b41bb8e571148b4ab57}] QOSClass:BestEffort}
      not to have occurred
    ```
    
    Failure message with this PR:
    ```
     should support a 'default-deny' policy [Feature:NetworkPolicy] [It]
        /Users/gunjan/go/src/k8s.io/kubernetes/_output/local/go/src/k8s.io/kubernetes/test/e2e/network/network_policy.go:81
    
        Jul 25 14:35:55.978: Pod client-cannot-connect should be able to connect to service svc-server, but was not able to connect.
        Pod logs:
        Connecting to svc-server.e2e-tests-network-policy-tbzm8:80 (10.63.254.124:80)
        wget: download timed out
        Connecting to svc-server.e2e-tests-network-policy-tbzm8:80 (10.63.254.124:80)
        wget: download timed out
        Connecting to svc-server.e2e-tests-network-policy-tbzm8:80 (10.63.254.124:80)
        wget: download timed out
        Connecting to svc-server.e2e-tests-network-policy-tbzm8:80 (10.63.254.124:80)
        wget: download timed out
        Connecting to svc-server.e2e-tests-network-policy-tbzm8:80 (10.63.254.124:80)
        wget: download timed out
    
    
       Current NetworkPolicies:
        	[{{ } {deny-all  e2e-tests-network-policy-tbzm8 /apis/networking.k8s.io/v1/namespaces/e2e-tests-network-policy-tbzm8/networkpolicies/deny-all 20fa92ab-7181-11e7-855b-42010a8a0003 87694 1 2017-07-25 14:35:07 -0700 PDT <nil> <nil> map[] map[] [] nil [] } {{map[] []} []}}]
    
       Pods:
        	[Pod: server, Status: &PodStatus{Phase:Running,Conditions:[{Initialized True 0001-01-01 00:00:00 +0000 UTC 2017-07-25 14:34:52 -0700 PDT  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2017-07-25 14:35:00 -0700 PDT  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2017-07-25 14:34:52 -0700 PDT  }],Message:,Reason:,HostIP:10.138.0.2,PodIP:10.60.1.145,StartTime:2017-07-25 14:34:52 -0700 PDT,ContainerStatuses:[{server-container-80 {nil ContainerStateRunning{StartedAt:2017-07-25 14:34:53 -0700 PDT,} nil} {nil nil nil} true 0 gcr.io/google_containers/porter:4524579c docker://sha256:711ccbc1c74f3417f2a0f403c194017b3bde09eec3c3b42dc9c838db3894c223 docker://3fa73da57380ef28520f8bbf53b3540f523ab76d45a1b42cc9a1e4573eb45639}],QOSClass:BestEffort,InitContainerStatuses:[],}
         Pod: server, Status: &PodStatus{Phase:Running,Conditions:[{Initialized True 0001-01-01 00:00:00 +0000 UTC 2017-07-25 14:34:52 -0700 PDT  } {Ready True 0001-01-01 00:00:00 +0000 UTC 2017-07-25 14:35:00 -0700 PDT  } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2017-07-25 14:34:52 -0700 PDT  }],Message:,Reason:,HostIP:10.138.0.2,PodIP:10.60.1.145,StartTime:2017-07-25 14:34:52 -0700 PDT,ContainerStatuses:[{server-container-80 {nil ContainerStateRunning{StartedAt:2017-07-25 14:34:53 -0700 PDT,} nil} {nil nil nil} true 0 gcr.io/google_containers/porter:4524579c docker://sha256:711ccbc1c74f3417f2a0f403c194017b3bde09eec3c3b42dc9c838db3894c223 docker://3fa73da57380ef28520f8bbf53b3540f523ab76d45a1b42cc9a1e4573eb45639}],QOSClass:BestEffort,InitContainerStatuses:[],}
        ]
    ```
    **Release note**:
    
    ```release-note
    NONE
    ```
    c7f970c5
Name
Last commit
Last update
..
e2e Loading commit data...
e2e_node Loading commit data...
fixtures Loading commit data...
images Loading commit data...
integration Loading commit data...
kubemark Loading commit data...
list Loading commit data...
soak Loading commit data...
utils Loading commit data...
BUILD Loading commit data...
OWNERS Loading commit data...
test_owners.csv Loading commit data...
test_owners.json Loading commit data...