Merge pull request #59005 from hyperbolic2346/mwilson/node-name-fix

Automatic merge from submit-queue (batch tested with PRs 59053, 59005). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>. Forcing get_node_name to continue searching for a node name. There was a race condition where the kubelet was restarting and we were querying the api server for this node. In that case, we may get a valid list of nodes that doesn't include our node. This would cause the code to just raise an exception. Now we wait the full timeout before raising the exception. **What this PR does / why we need it**: Fixes a race condition on the get_node_name function inside the kubernetes-worker charm. **Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*: Fixes # **Special notes for your reviewer**: **Release note**: ```release-note Fixed a race condition inside kubernetes-worker that would result in a temporary error situation. ```

Merge pull request #59005 from hyperbolic2346/mwilson/node-name-fix
7fe25af6 · Kubernetes Submit Queue · GitHub · 25173455 · e87b5376 · 7fe25af6
Unverified Commit 7fe25af6 authored Jan 30, 2018 by Kubernetes Submit Queue Committed by GitHub Jan 30, 2018
Hide whitespace changes
Inline Side-by-side

Showing with 20 additions and 21 deletions

kubernetes_worker.py ...ju/layers/kubernetes-worker/reactive/kubernetes_worker.py +20 -21

No files found.
--- a/cluster/juju/layers/kubernetes-worker/reactive/kubernetes_worker.py
+++ b/cluster/juju/layers/kubernetes-worker/reactive/kubernetes_worker.py
@@ -979,31 +979,30 @@ def get_node_name():
    while time.time() < deadline:
        try:
            raw = check_output(cmd)
-            break
        except CalledProcessError:
            hookenv.log('Failed to get node name for node %s.'
                        ' Will retry.' % (gethostname()))
            time.sleep(1)
-    else:
+            continue
-        msg = 'Failed to get node name for node %s' % gethostname()
-        raise GetNodeNameFailed(msg)
+        result = json.loads(raw.decode('utf-8'))
+        if 'items' in result:
-    result = json.loads(raw.decode('utf-8'))
+            for node in result['items']:
-    if 'items' in result:
+                if 'status' not in node:
-        for node in result['items']:
+                    continue
-            if 'status' not in node:
+                if 'addresses' not in node['status']:
-                continue
+                    continue
-            if 'addresses' not in node['status']:
-                continue
+                # find the hostname
+                for address in node['status']['addresses']:
-            # find the hostname
+                    if address['type'] == 'Hostname':
-            for address in node['status']['addresses']:
+                        if address['address'] == gethostname():
-                if address['type'] == 'Hostname':
+                            return node['metadata']['name']
-                    if address['address'] == gethostname():
-                        return node['metadata']['name']
+                        # if we didn't match, just bail to the next node
+                        break
-                    # if we didn't match, just bail to the next node
+        time.sleep(1)
-                    break
    msg = 'Failed to get node name for node %s' % gethostname()
    raise GetNodeNameFailed(msg)