Skip to content

K
k3s
Commit c47693c0 authored by Eric Tune's avatar Eric Tune
Browse files
Options

Separated user, dev, and design docs.

Renamed: logging.md -> devel/logging.m Renamed: access.md -> design/access.md Renamed: identifiers.md -> design/identifiers.md Renamed: labels.md -> design/labels.md Renamed: namespaces.md -> design/namespaces.md Renamed: security.md -> design/security.md Renamed: networking.md -> design/networking.md Added abbreviated user user-focused document in place of most moved docs. Added docs/README.md explains how docs are organized. Added short, user-oriented documentation on labels Added a glossary. Fixed up some links.
parent a18cdac6
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 546 additions and 347 deletions
docs/README.md 0 → 100644
View file @ c47693c0
# Kubernetes Documentation
Kubernetes documentation is organized into several categories.
- **Getting Started Guides**
- for people who want to create a kubernetes cluster
- in [docs/getting-started-guides](./getting-started-guides)
- **User Documentation**
- in [docs](./overview.md)
- for people who want to run programs on kubernetes
- describes current features of the system (with brief mentions of planned features)
- **Developer Documentation**
- in [docs/devel](./devel)
- for people who want to contribute code to kubernetes
- covers development conventions
- explains current architecture and project plans
- **Design Documentation**
- in [docs/design](./design)
- for people who want to understand the design choices made
- describes tradeoffs, alternative designs
- descriptions of planned features that are too long for a github issue.
- **Walkthroughs and Examples**
- in [examples](../examples)
- Hands on introduction and example config files
- **API documentation**
- in [api](../api)
- automatically generated REST API documentation
- **Wiki**
- in [wiki](https://github.com/GoogleCloudPlatform/kubernetes/wiki)
docs/design/identifiers.md 0 → 100644
View file @ c47693c0
# Identifiers and Names in Kubernetes
A summarization of the goals and recommendations for identifiers in Kubernetes. Described in [GitHub issue #199](https://github.com/GoogleCloudPlatform/kubernetes/issues/199).
## Definitions
UID
: A non-empty, opaque, system-generated value guaranteed to be unique in time and space; intended to distinguish between historical occurrences of similar entities.
Name
: A non-empty string guaranteed to be unique within a given scope at a particular time; used in resource URLs; provided by clients at creation time and encouraged to be human friendly; intended to facilitate creation idempotence and space-uniqueness of singleton objects, distinguish distinct entities, and reference particular entities across operations.
[rfc1035](http://www.ietf.org/rfc/rfc1035.txt)/[rfc1123](http://www.ietf.org/rfc/rfc1123.txt) label (DNS_LABEL)
: An alphanumeric (a-z, A-Z, and 0-9) string, with a maximum length of 63 characters, with the '-' character allowed anywhere except the first or last character, suitable for use as a hostname or segment in a domain name
[rfc1035](http://www.ietf.org/rfc/rfc1035.txt)/[rfc1123](http://www.ietf.org/rfc/rfc1123.txt) subdomain (DNS_SUBDOMAIN)
: One or more rfc1035/rfc1123 labels separated by '.' with a maximum length of 253 characters
[rfc4122](http://www.ietf.org/rfc/rfc4122.txt) universally unique identifier (UUID)
: A 128 bit generated value that is extremely unlikely to collide across time and space and requires no central coordination
## Objectives for names and UIDs
1. Uniquely identify (via a UID) an object across space and time
2. Uniquely name (via a name) an object across space
3. Provide human-friendly names in API operations and/or configuration files
4. Allow idempotent creation of API resources (#148) and enforcement of space-uniqueness of singleton objects
5. Allow DNS names to be automatically generated for some objects
## General design
1. When an object is created via an API, a Name string (a DNS_SUBDOMAIN) must be specified. Name must be non-empty and unique within the apiserver. This enables idempotent and space-unique creation operations. Parts of the system (e.g. replication controller) may join strings (e.g. a base name and a random suffix) to create a unique Name. For situations where generating a name is impractical, some or all objects may support a param to auto-generate a name. Generating random names will defeat idempotency.
* Examples: "guestbook.user", "backend-x4eb1"
2. When an object is created via an api, a Namespace string (a DNS_SUBDOMAIN? format TBD via #1114) may be specified. Depending on the API receiver, namespaces might be validated (e.g. apiserver might ensure that the namespace actually exists). If a namespace is not specified, one will be assigned by the API receiver. This assignment policy might vary across API receivers (e.g. apiserver might have a default, kubelet might generate something semi-random).
* Example: "api.k8s.example.com"
3. Upon acceptance of an object via an API, the object is assigned a UID (a UUID). UID must be non-empty and unique across space and time.
* Example: "01234567-89ab-cdef-0123-456789abcdef"
## Case study: Scheduling a pod
Pods can be placed onto a particular node in a number of ways. This case
study demonstrates how the above design can be applied to satisfy the
objectives.
### A pod scheduled by a user through the apiserver
1. A user submits a pod with Namespace="" and Name="guestbook" to the apiserver.
2. The apiserver validates the input.
1. A default Namespace is assigned.
2. The pod name must be space-unique within the Namespace.
3. Each container within the pod has a name which must be space-unique within the pod.
3. The pod is accepted.
1. A new UID is assigned.
4. The pod is bound to a node.
1. The kubelet on the node is passed the pod's UID, Namespace, and Name.
5. Kubelet validates the input.
6. Kubelet runs the pod.
1. Each container is started up with enough metadata to distinguish the pod from whence it came.
2. Each attempt to run a container is assigned a UID (a string) that is unique across time.
* This may correspond to Docker's container ID.
### A pod placed by a config file on the node
1. A config file is stored on the node, containing a pod with UID="", Namespace="", and Name="cadvisor".
2. Kubelet validates the input.
1. Since UID is not provided, kubelet generates one.
2. Since Namespace is not provided, kubelet generates one.
1. The generated namespace should be deterministic and cluster-unique for the source, such as a hash of the hostname and file path.
* E.g. Namespace="file-f4231812554558a718a01ca942782d81"
3. Kubelet runs the pod.
1. Each container is started up with enough metadata to distinguish the pod from whence it came.
2. Each attempt to run a container is assigned a UID (a string) that is unique across time.
1. This may correspond to Docker's container ID.
docs/design/labels.md 0 → 100644
View file @ c47693c0
# Labels
_Labels_ are key/value pairs identifying client/user-defined attributes (and non-primitive system-generated attributes) of API objects, which are stored and returned as part of the [metadata of those objects](api-conventions.md). Labels can be used to organize and to select subsets of objects according to these attributes.
Each object can have a set of key/value labels set on it, with at most one label with a particular key.
```
"labels": {
"key1" : "value1",
"key2" : "value2"
}
```
Unlike [names and UIDs](identifiers.md), labels do not provide uniqueness. In general, we expect many objects to carry the same label(s).
Via a _label selector_, the client/user can identify a set of objects. The label selector is the core grouping primitive in Kubernetes.
Label selectors may also be used to associate policies with sets of objects.
We also [plan](https://github.com/GoogleCloudPlatform/kubernetes/issues/560) to make labels available inside pods and [lifecycle hooks](container-environment.md).
[Namespacing of label keys](https://github.com/GoogleCloudPlatform/kubernetes/issues/1491) is under discussion.
Valid labels follow a slightly modified RFC952 format: 24 characters or less, all lowercase, begins with alpha, dashes (-) are allowed, and ends with alphanumeric.
## Motivation
Service deployments and batch processing pipelines are often multi-dimensional entities (e.g., multiple partitions or deployments, multiple release tracks, multiple tiers, multiple micro-services per tier). Management often requires cross-cutting operations, which breaks encapsulation of strictly hierarchical representations, especially rigid hierarchies determined by the infrastructure rather than by users. Labels enable users to map their own organizational structures onto system objects in a loosely coupled fashion, without requiring clients to store these mappings.
## Label selectors
Label selectors permit very simple filtering by label keys and values. The simplicity of label selectors is deliberate. It is intended to facilitate transparency for humans, easy set overlap detection, efficient indexing, and reverse-indexing (i.e., finding all label selectors matching an object's labels - https://github.com/GoogleCloudPlatform/kubernetes/issues/1348).
Currently the system supports selection by exact match of a map of keys and values. Matching objects must have all of the specified labels (both keys and values), though they may have additional labels as well.
We are in the process of extending the label selection specification (see [selector.go](../blob/master/pkg/labels/selector.go) and https://github.com/GoogleCloudPlatform/kubernetes/issues/341) to support conjunctions of requirements of the following forms:
```
key1 in (value11, value12, ...)
key1 not in (value11, value12, ...)
key1 exists
```
LIST and WATCH operations may specify label selectors to filter the sets of objects returned using a query parameter: `?labels=key1%3Dvalue1,key2%3Dvalue2,...`. We may extend such filtering to DELETE operations in the future.
Kubernetes also currently supports two objects that use label selectors to keep track of their members, `service`s and `replicationController`s:
- `service`: A [service](services.md) is a configuration unit for the proxies that run on every worker node. It is named and points to one or more pods.
- `replicationController`: A [replication controller](replication-controller.md) ensures that a specified number of pod "replicas" are running at any one time. If there are too many, it'll kill some. If there are too few, it'll start more.
The set of pods that a `service` targets is defined with a label selector. Similarly, the population of pods that a `replicationController` is monitoring is also defined with a label selector.
For management convenience and consistency, `services` and `replicationControllers` may themselves have labels and would generally carry the labels their corresponding pods have in common.
In the future, label selectors will be used to identify other types of distributed service workers, such as worker pool members or peers in a distributed application.
Individual labels are used to specify identifying metadata, and to convey the semantic purposes/roles of pods of containers. Examples of typical pod label keys include `service`, `environment` (e.g., with values `dev`, `qa`, or `production`), `tier` (e.g., with values `frontend` or `backend`), and `track` (e.g., with values `daily` or `weekly`), but you are free to develop your own conventions.
Sets identified by labels and label selectors could be overlapping (think Venn diagrams). For instance, a service might target all pods with `tier in (frontend), environment in (prod)`. Now say you have 10 replicated pods that make up this tier. But you want to be able to 'canary' a new version of this component. You could set up a `replicationController` (with `replicas` set to 9) for the bulk of the replicas with labels `tier=frontend, environment=prod, track=stable` and another `replicationController` (with `replicas` set to 1) for the canary with labels `tier=frontend, environment=prod, track=canary`. Now the service is covering both the canary and non-canary pods. But you can mess with the `replicationControllers` separately to test things out, monitor the results, etc.
Note that the superset described in the previous example is also heterogeneous. In long-lived, highly available, horizontally scaled, distributed, continuously evolving service applications, heterogeneity is inevitable, due to canaries, incremental rollouts, live reconfiguration, simultaneous updates and auto-scaling, hardware upgrades, and so on.
Pods (and other objects) may belong to multiple sets simultaneously, which enables representation of service substructure and/or superstructure. In particular, labels are intended to facilitate the creation of non-hierarchical, multi-dimensional deployment structures. They are useful for a variety of management purposes (e.g., configuration, deployment) and for application introspection and analysis (e.g., logging, monitoring, alerting, analytics). Without the ability to form sets by intersecting labels, many implicitly related, overlapping flat sets would need to be created, for each subset and/or superset desired, which would lose semantic information and be difficult to keep consistent. Purely hierarchically nested sets wouldn't readily support slicing sets across different dimensions.
Pods may be removed from these sets by changing their labels. This flexibility may be used to remove pods from service for debugging, data recovery, etc.
Since labels can be set at pod creation time, no separate set add/remove operations are necessary, which makes them easier to use than manual set management. Additionally, since labels are directly attached to pods and label selectors are fairly simple, it's easy for users and for clients and tools to determine what sets they belong to (i.e., they are reversible). OTOH, with sets formed by just explicitly enumerating members, one would (conceptually) need to search all sets to determine which ones a pod belonged to.
## Labels vs. annotations
We'll eventually index and reverse-index labels for efficient queries and watches, use them to sort and group in UIs and CLIs, etc. We don't want to pollute labels with non-identifying, especially large and/or structured, data. Non-identifying information should be recorded using [annotations](annotations.md).
docs/design/namespaces.md 0 → 100644
View file @ c47693c0
# Kubernetes Proposal - Namespaces
**Related PR:**
| Topic | Link |
| ---- | ---- |
| Identifiers.md | https://github.com/GoogleCloudPlatform/kubernetes/pull/1216 |
| Access.md | https://github.com/GoogleCloudPlatform/kubernetes/pull/891 |
| Indexing | https://github.com/GoogleCloudPlatform/kubernetes/pull/1183 |
| Cluster Subdivision | https://github.com/GoogleCloudPlatform/kubernetes/issues/442 |
## Background
High level goals:
* Enable an easy-to-use mechanism to logically scope Kubernetes resources
* Ensure extension resources to Kubernetes can share the same logical scope as core Kubernetes resources
* Ensure it aligns with access control proposal
* Ensure system has log n scale with increasing numbers of scopes
## Use cases
Actors:
1. k8s admin - administers a kubernetes cluster
2. k8s service - k8s daemon operates on behalf of another user (i.e. controller-manager)
2. k8s policy manager - enforces policies imposed on k8s cluster
3. k8s user - uses a kubernetes cluster to schedule pods
User stories:
1. Ability to set immutable namespace to k8s resources
2. Ability to list k8s resource scoped to a namespace
3. Restrict a namespace identifier to a DNS-compatible string to support compound naming conventions
4. Ability for a k8s policy manager to enforce a k8s user's access to a set of namespaces
5. Ability to set/unset a default namespace for use by kubecfg client
6. Ability for a k8s service to monitor resource changes across namespaces
7. Ability for a k8s service to list resources across namespaces
## Proposed Design
### Model Changes
Introduce a new attribute *Namespace* for each resource that must be scoped in a Kubernetes cluster.
A *Namespace* is a DNS compatible subdomain.
```
// TypeMeta is shared by all objects sent to, or returned from the client
type TypeMeta struct {
Kind string `json:"kind,omitempty" yaml:"kind,omitempty"`
Uid string `json:"uid,omitempty" yaml:"uid,omitempty"`
CreationTimestamp util.Time `json:"creationTimestamp,omitempty" yaml:"creationTimestamp,omitempty"`
SelfLink string `json:"selfLink,omitempty" yaml:"selfLink,omitempty"`
ResourceVersion uint64 `json:"resourceVersion,omitempty" yaml:"resourceVersion,omitempty"`
APIVersion string `json:"apiVersion,omitempty" yaml:"apiVersion,omitempty"`
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
Name string `json:"name,omitempty" yaml:"name,omitempty"`
}
```
An identifier, *UID*, is unique across time and space intended to distinguish between historical occurences of similar entities.
A *Name* is unique within a given *Namespace* at a particular time, used in resource URLs; provided by clients at creation time
and encouraged to be human friendly; intended to facilitate creation idempotence and space-uniqueness of singleton objects, distinguish
distinct entities, and reference particular entities across operations.
As of this writing, the following resources MUST have a *Namespace* and *Name*
* pod
* service
* replicationController
* endpoint
A *policy* MAY be associated with a *Namespace*.
If a *policy* has an associated *Namespace*, the resource paths it enforces are scoped to a particular *Namespace*.
## k8s API server
In support of namespace isolation, the Kubernetes API server will address resources by the following conventions:
The typical actors for the following requests are the k8s user or the k8s service.
| Action | HTTP Verb | Path | Description |
| ---- | ---- | ---- | ---- |
| CREATE | POST | /api/{version}/ns/{ns}/{resourceType}/ | Create instance of {resourceType} in namespace {ns} |
| GET | GET | /api/{version}/ns/{ns}/{resourceType}/{name} | Get instance of {resourceType} in namespace {ns} with {name} |
| UPDATE | PUT | /api/{version}/ns/{ns}/{resourceType}/{name} | Update instance of {resourceType} in namespace {ns} with {name} |
| DELETE | DELETE | /api/{version}/ns/{ns}/{resourceType}/{name} | Delete instance of {resourceType} in namespace {ns} with {name} |
| LIST | GET | /api/{version}/ns/{ns}/{resourceType} | List instances of {resourceType} in namespace {ns} |
| WATCH | GET | /api/{version}/watch/ns/{ns}/{resourceType} | Watch for changes to a {resourceType} in namespace {ns} |
The typical actor for the following requests are the k8s service or k8s admin as enforced by k8s Policy.
| Action | HTTP Verb | Path | Description |
| ---- | ---- | ---- | ---- |
| WATCH | GET | /api/{version}/watch/{resourceType} | Watch for changes to a {resourceType} across all namespaces |
| LIST | GET | /api/{version}/list/{resourceType} | List instances of {resourceType} across all namespaces |
The legacy API patterns for k8s are an alias to interacting with the *default* namespace as follows.
| Action | HTTP Verb | Path | Description |
| ---- | ---- | ---- | ---- |
| CREATE | POST | /api/{version}/{resourceType}/ | Create instance of {resourceType} in namespace *default* |
| GET | GET | /api/{version}/{resourceType}/{name} | Get instance of {resourceType} in namespace *default* |
| UPDATE | PUT | /api/{version}/{resourceType}/{name} | Update instance of {resourceType} in namespace *default* |
| DELETE | DELETE | /api/{version}/{resourceType}/{name} | Delete instance of {resourceType} in namespace *default* |
The k8s API server verifies the *Namespace* on resource creation matches the *{ns}* on the path.
The k8s API server will enable efficient mechanisms to filter model resources based on the *Namespace*. This may require
the creation of an index on *Namespace* that could support query by namespace with optional label selectors.
The k8s API server will associate a resource with a *Namespace* if not populated by the end-user based on the *Namespace* context
of the incoming request. If the *Namespace* of the resource being created, or updated does not match the *Namespace* on the request,
then the k8s API server will reject the request.
TODO: Update to discuss k8s api server proxy patterns
## k8s storage
A namespace provides a unique identifier space and therefore must be in the storage path of a resource.
In etcd, we want to continue to still support efficient WATCH across namespaces.
Resources that persist content in etcd will have storage paths as follows:
/registry/{resourceType}/{resource.Namespace}/{resource.Name}
This enables k8s service to WATCH /registry/{resourceType} for changes across namespace of a particular {resourceType}.
Upon scheduling a pod to a particular host, the pod's namespace must be in the key path as follows:
/host/{host}/pod/{pod.Namespace}/{pod.Name}
## k8s Authorization service
This design assumes the existence of an authorization service that filters incoming requests to the k8s API Server in order
to enforce user authorization to a particular k8s resource. It performs this action by associating the *subject* of a request
with a *policy* to an associated HTTP path and verb. This design encodes the *namespace* in the resource path in order to enable
external policy servers to function by resource path alone. If a request is made by an identity that is not allowed by
policy to the resource, the request is terminated. Otherwise, it is forwarded to the apiserver.
## k8s controller-manager
The controller-manager will provision pods in the same namespace as the associated replicationController.
## k8s Kubelet
There is no major change to the kubelet introduced by this proposal.
### kubecfg client
kubecfg supports following:
```
kubecfg [OPTIONS] ns {namespace}
```
To set a namespace to use across multiple operations:
```
$ kubecfg ns ns1
```
To view the current namespace:
```
$ kubecfg ns
Using namespace ns1
```
To reset to the default namespace:
```
$ kubecfg ns default
```
In addition, each kubecfg request may explicitly specify a namespace for the operation via the following OPTION
--ns
When loading resource files specified by the -c OPTION, the kubecfg client will ensure the namespace is set in the
message body to match the client specified default.
If no default namespace is applied, the client will assume the following default namespace:
* default
The kubecfg client would store default namespace information in the same manner it caches authentication information today
as a file on user's file system.
docs/devel/logging.md 0 → 100644
View file @ c47693c0
Logging Conventions
===================
The following conventions for the glog levels to use. glog is globally prefered to "log" for better runtime control.
* glog.Errorf() - Always an error
* glog.Warningf() - Something unexpected, but probably not an error
* glog.Infof() has multiple levels:
* glog.V(0) - Generally useful for this to ALWAYS be visible to an operator
* Programmer errors
* Logging extra info about a panic
* CLI argument handling
* glog.V(1) - A reasonable default log level if you don't want verbosity.
* Information about config (listening on X, watching Y)
* Errors that repeat frequently that relate to conditions that can be corrected (pod detected as unhealthy)
* glog.V(2) - Useful steady state information about the service and important log messages that may correlate to significant changes in the system. This is the recommended default log level for most systems.
* Logging HTTP requests and their exit code
* System state changing (killing pod)
* Controller state change events (starting pods)
* Scheduler log messages
* glog.V(3) - Extended information about changes
* More info about system state changes
* glog.V(4) - Debug level verbosity (for now)
* Logging in particularly thorny parts of code where you may want to come back later and check it
As per the comments, the practical default level is V(2). Developers and QE environments may wish to run at V(3) or V(4). If you wish to change the log level, you can pass in `-v=X` where X is the desired maximum level to log.
docs/glossary.md 0 → 100644
View file @ c47693c0
# Glossary and Concept Index
**Authorization**
:Kubernetes does not currently have an authorization system. Anyone with the cluster password can do anything. We plan
to add sophisticated authorization, and to make it pluggable. See the [access control design doc](./devel/access.md) and
[this issue](https://github.com/GoogleCloudPlatform/kubernetes/issue/1430).
**Annotation**
: A key/value pair that can hold large (compared to a Label), and possibly not human-readable data. Intended to store
non-identifying metadata associated with an object, such as provenance information. Not indexed.
**Image**
: A [Docker Image](https://docs.docker.com/userguide/dockerimages/). See [images](./images.md).
**Label**
: A key/value pair conveying user-defined identifying attributes of an object, and used to form sets of related objects, such as
pods which are replicas in a load-balanced service. Not intended to hold large or non-human-readable data. See [labels](./labels.md).
**Name**
: A user-provided name for an object. See [identifiers](identifiers.md).
**Namespace**
: A namespace is like a prefix to the name of an object. You can configure your client to use a particular namespace,
so you do not have to type it all the time. Namespaces allow multiple projects to prevent naming collisions between unrelated teams.
**Pod**
: A collection of containers which will be scheduled onto the same node, which share and an IP and port space, and which
can be created/destroyed together. See [pods](./pods.md).
**Replication Controller**
: A _replication controller_ ensures that a specified number of pod "replicas" are running at any one time. Both allows
for easy scaling of replicated systems, and handles restarting of a Pod when the machine it is on reboots or otherwise fails.
**Resource**
: CPU, memory, and other things that a pod can request. See [resources](resources.md).
**Selector**
: An expression that matches Labels. Can identify related objects, such as pods which are replicas in a load-balanced
service. See [labels](labels.md).
**Service**
: A load-balanced set of `pods` which can be accessed via a single stable IP address. See [services](./services.md).
**UID**
: An identifier on all Kubernetes objects that is set by the Kubernetes API server. Can be used to distinguish between historical
occurrences of same-Name objects. See [identifiers](identifiers.md).
**Volume**
: A directory, possibly with some data in it, which is accessible to a Container as part of its filesystem. Kubernetes
Volumes build upon [Docker Volumes](https://docs.docker.com/userguide/dockervolumes/), adding provisioning of the Volume
directory and/or device. See [volumes](volumes.md).
docs/identifiers.md
View file @ c47693c0
# Identifiers and Names in Kubernetes
# Identifiers
All objects in the Kubernetes REST API are identified by a Name and a UID.
A summarization of the goals and recommendations for identifiers in Kubernetes. Described in [GitHub issue #199](https://github.com/GoogleCloudPlatform/kubernetes/issues/199).
## Names
Names are user-provided. Only one object of a given kind can have a given name at a time. But if you delete an object, you can make a new object with the same name. Names are the used to refer to an object in a resource URL, such as `/api/v1beta3/pods/some.name`. Names may be up to maximum length of 253 characters and consist of alphanumeric characters, `-`, and `.`. See the [identifiers design doc](design/identifiers.md) for the precise syntax rules for names.
## Definitions
UID
: A non-empty, opaque, system-generated value guaranteed to be unique in time and space; intended to distinguish between historical occurrences of similar entities.
Name
: A non-empty string guaranteed to be unique within a given scope at a particular time; used in resource URLs; provided by clients at creation time and encouraged to be human friendly; intended to facilitate creation idempotence and space-uniqueness of singleton objects, distinguish distinct entities, and reference particular entities across operations.
[rfc1035](http://www.ietf.org/rfc/rfc1035.txt)/[rfc1123](http://www.ietf.org/rfc/rfc1123.txt) label (DNS_LABEL)
: An alphanumeric (a-z, A-Z, and 0-9) string, with a maximum length of 63 characters, with the '-' character allowed anywhere except the first or last character, suitable for use as a hostname or segment in a domain name
[rfc1035](http://www.ietf.org/rfc/rfc1035.txt)/[rfc1123](http://www.ietf.org/rfc/rfc1123.txt) subdomain (DNS_SUBDOMAIN)
: One or more rfc1035/rfc1123 labels separated by '.' with a maximum length of 253 characters
[rfc4122](http://www.ietf.org/rfc/rfc4122.txt) universally unique identifier (UUID)
: A 128 bit generated value that is extremely unlikely to collide across time and space and requires no central coordination
## Objectives for names and UIDs
1. Uniquely identify (via a UID) an object across space and time
2. Uniquely name (via a name) an object across space
3. Provide human-friendly names in API operations and/or configuration files
4. Allow idempotent creation of API resources (#148) and enforcement of space-uniqueness of singleton objects
5. Allow DNS names to be automatically generated for some objects
## General design
1. When an object is created via an API, a Name string (a DNS_SUBDOMAIN) must be specified. Name must be non-empty and unique within the apiserver. This enables idempotent and space-unique creation operations. Parts of the system (e.g. replication controller) may join strings (e.g. a base name and a random suffix) to create a unique Name. For situations where generating a name is impractical, some or all objects may support a param to auto-generate a name. Generating random names will defeat idempotency.
* Examples: "guestbook.user", "backend-x4eb1"
2. When an object is created via an api, a Namespace string (a DNS_SUBDOMAIN? format TBD via #1114) may be specified. Depending on the API receiver, namespaces might be validated (e.g. apiserver might ensure that the namespace actually exists). If a namespace is not specified, one will be assigned by the API receiver. This assignment policy might vary across API receivers (e.g. apiserver might have a default, kubelet might generate something semi-random).
* Example: "api.k8s.example.com"
3. Upon acceptance of an object via an API, the object is assigned a UID (a UUID). UID must be non-empty and unique across space and time.
* Example: "01234567-89ab-cdef-0123-456789abcdef"
## Case study: Scheduling a pod
Pods can be placed onto a particular node in a number of ways. This case
study demonstrates how the above design can be applied to satisfy the
objectives.
### A pod scheduled by a user through the apiserver
1. A user submits a pod with Namespace="" and Name="guestbook" to the apiserver.
2. The apiserver validates the input.
1. A default Namespace is assigned.
2. The pod name must be space-unique within the Namespace.
3. Each container within the pod has a name which must be space-unique within the pod.
3. The pod is accepted.
1. A new UID is assigned.
4. The pod is bound to a node.
1. The kubelet on the node is passed the pod's UID, Namespace, and Name.
5. Kubelet validates the input.
6. Kubelet runs the pod.
1. Each container is started up with enough metadata to distinguish the pod from whence it came.
2. Each attempt to run a container is assigned a UID (a string) that is unique across time.
* This may correspond to Docker's container ID.
### A pod placed by a config file on the node
1. A config file is stored on the node, containing a pod with UID="", Namespace="", and Name="cadvisor".
2. Kubelet validates the input.
1. Since UID is not provided, kubelet generates one.
2. Since Namespace is not provided, kubelet generates one.
1. The generated namespace should be deterministic and cluster-unique for the source, such as a hash of the hostname and file path.
* E.g. Namespace="file-f4231812554558a718a01ca942782d81"
3. Kubelet runs the pod.
1. Each container is started up with enough metadata to distinguish the pod from whence it came.
2. Each attempt to run a container is assigned a UID (a string) that is unique across time.
1. This may correspond to Docker's container ID.
## UIDs
UID are generated by Kubernetes. Every object created over the whole lifetime of a Kubernetes cluster has a distinct UID.
docs/labels.md
View file @ c47693c0
# Labels
_Labels_ are key/value pairs identifying client/user-defined attributes (and non-primitive system-generated attributes) of API objects, which are stored and returned as part of the [metadata of those objects](api-conventions.md). Labels can be used to organize and to select subsets of objects according to these attributes.
Each object can have a set of key/value labels set on it, with at most one label with a particular key.
_Labels_ are key/value pairs that are attached to objects, such as pods.
Labels can be used to organize and to select subsets of objects. They are
created by users at the same time as an object. Each object can have a set of
key/value labels set on it, with at most one label with a particular key.
```
"labels": {
"key1" : "value1",
......@@ -14,55 +15,28 @@ Unlike [names and UIDs](identifiers.md), labels do not provide uniqueness. In ge
Via a _label selector_, the client/user can identify a set of objects. The label selector is the core grouping primitive in Kubernetes.
Label selectors may also be used to associate policies with sets of objects.
We also [plan](https://github.com/GoogleCloudPlatform/kubernetes/issues/560) to make labels available inside pods and [lifecycle hooks](container-environment.md).
[Namespacing of label keys](https://github.com/GoogleCloudPlatform/kubernetes/issues/1491) is under discussion.
Valid labels follow a slightly modified RFC952 format: 24 characters or less, all lowercase, begins with alpha, dashes (-) are allowed, and ends with alphanumeric.
## Motivation
Service deployments and batch processing pipelines are often multi-dimensional entities (e.g., multiple partitions or deployments, multiple release tracks, multiple tiers, multiple micro-services per tier). Management often requires cross-cutting operations, which breaks encapsulation of strictly hierarchical representations, especially rigid hierarchies determined by the infrastructure rather than by users. Labels enable users to map their own organizational structures onto system objects in a loosely coupled fashion, without requiring clients to store these mappings.
Labels let you categorize objects in a complex service deployment or batch processing pipelines along multiple
dimensions, such as:
- `release=stable`, `release=canary`, ...
- `environment=dev`, `environment=qa`, `environment=production`
- `tier=frontend`, `tier=backend`, ...
- `partition=customerA`, `partition=customerB`, ...
- `track=daily`, `track=weekly`
These are just examples; you are free to develop your own conventions.
## Label selectors
Label selectors permit very simple filtering by label keys and values. The simplicity of label selectors is deliberate. It is intended to facilitate transparency for humans, easy set overlap detection, efficient indexing, and reverse-indexing (i.e., finding all label selectors matching an object's labels - https://github.com/GoogleCloudPlatform/kubernetes/issues/1348).
Currently the system supports selection by exact match of a map of keys and values. Matching objects must have all of the specified labels (both keys and values), though they may have additional labels as well.
We are in the process of extending the label selection specification (see [selector.go](../blob/master/pkg/labels/selector.go) and https://github.com/GoogleCloudPlatform/kubernetes/issues/341) to support conjunctions of requirements of the following forms:
Label selectors permit very simple filtering by label keys and values. Currently, label selectors only support these forms:
```
key1
key1 = value11
key1 != value11
key1 in (value11, value12, ...)
key1 not in (value11, value12, ...)
key1 exists
```
LIST and WATCH operations may specify label selectors to filter the sets of objects returned using a query parameter: `?labels=key1%3Dvalue1,key2%3Dvalue2,...`. We may extend such filtering to DELETE operations in the future.
Kubernetes also currently supports two objects that use label selectors to keep track of their members, `service`s and `replicationController`s:
- `service`: A [service](services.md) is a configuration unit for the proxies that run on every worker node. It is named and points to one or more pods.
- `replicationController`: A [replication controller](replication-controller.md) ensures that a specified number of pod "replicas" are running at any one time. If there are too many, it'll kill some. If there are too few, it'll start more.
The set of pods that a `service` targets is defined with a label selector. Similarly, the population of pods that a `replicationController` is monitoring is also defined with a label selector.
For management convenience and consistency, `services` and `replicationControllers` may themselves have labels and would generally carry the labels their corresponding pods have in common.
In the future, label selectors will be used to identify other types of distributed service workers, such as worker pool members or peers in a distributed application.
Individual labels are used to specify identifying metadata, and to convey the semantic purposes/roles of pods of containers. Examples of typical pod label keys include `service`, `environment` (e.g., with values `dev`, `qa`, or `production`), `tier` (e.g., with values `frontend` or `backend`), and `track` (e.g., with values `daily` or `weekly`), but you are free to develop your own conventions.
Sets identified by labels and label selectors could be overlapping (think Venn diagrams). For instance, a service might target all pods with `tier in (frontend), environment in (prod)`. Now say you have 10 replicated pods that make up this tier. But you want to be able to 'canary' a new version of this component. You could set up a `replicationController` (with `replicas` set to 9) for the bulk of the replicas with labels `tier=frontend, environment=prod, track=stable` and another `replicationController` (with `replicas` set to 1) for the canary with labels `tier=frontend, environment=prod, track=canary`. Now the service is covering both the canary and non-canary pods. But you can mess with the `replicationControllers` separately to test things out, monitor the results, etc.
Note that the superset described in the previous example is also heterogeneous. In long-lived, highly available, horizontally scaled, distributed, continuously evolving service applications, heterogeneity is inevitable, due to canaries, incremental rollouts, live reconfiguration, simultaneous updates and auto-scaling, hardware upgrades, and so on.
Pods (and other objects) may belong to multiple sets simultaneously, which enables representation of service substructure and/or superstructure. In particular, labels are intended to facilitate the creation of non-hierarchical, multi-dimensional deployment structures. They are useful for a variety of management purposes (e.g., configuration, deployment) and for application introspection and analysis (e.g., logging, monitoring, alerting, analytics). Without the ability to form sets by intersecting labels, many implicitly related, overlapping flat sets would need to be created, for each subset and/or superset desired, which would lose semantic information and be difficult to keep consistent. Purely hierarchically nested sets wouldn't readily support slicing sets across different dimensions.
Pods may be removed from these sets by changing their labels. This flexibility may be used to remove pods from service for debugging, data recovery, etc.
Since labels can be set at pod creation time, no separate set add/remove operations are necessary, which makes them easier to use than manual set management. Additionally, since labels are directly attached to pods and label selectors are fairly simple, it's easy for users and for clients and tools to determine what sets they belong to (i.e., they are reversible). OTOH, with sets formed by just explicitly enumerating members, one would (conceptually) need to search all sets to determine which ones a pod belonged to.
LIST and WATCH operations may specify label selectors to filter the sets of objects returned using a query parameter: `?labels=key1%3Dvalue1,key2%3Dvalue2,...`.
## Labels vs. annotations
The `service` and `replicationController` kinds of objects use selectors to match sets of pods that they operate on.
We'll eventually index and reverse-index labels for efficient queries and watches, use them to sort and group in UIs and CLIs, etc. We don't want to pollute labels with non-identifying, especially large and/or structured, data. Non-identifying information should be recorded using [annotations](annotations.md).
See the [Labels Design Document](./design/labels.md) for more about how we expect labels and selectors to be used, and planned features.
docs/logging.md
View file @ c47693c0
Logging Conventions
===================
# Logging
## Logging by Kubernetes Components
Kubernetes components, such as kubelet and apiserver, use the [glog](https://godoc.org/github.com/golang/glog) logging library. Developer conventions for logging severity are described in [devel/logging.md](devel/logging.md).
## Logging in Containers
There are no Kubernetes-specific requirements for logging from within containers. A
[search](https://www.google.com/?q=docker+container+logging) will turn up any number of articles about logging and
Docker containers. However, we do provide an example of how to collect, index, and view pod logs [using Elasicsearch and Kibana](./getting-started-guides/logging.md)
The following conventions for the glog levels to use. glog is globally prefered to "log" for better runtime control.
* glog.Errorf() - Always an error
* glog.Warningf() - Something unexpected, but probably not an error
* glog.Infof() has multiple levels:
* glog.V(0) - Generally useful for this to ALWAYS be visible to an operator
* Programmer errors
* Logging extra info about a panic
* CLI argument handling
* glog.V(1) - A reasonable default log level if you don't want verbosity.
* Information about config (listening on X, watching Y)
* Errors that repeat frequently that relate to conditions that can be corrected (pod detected as unhealthy)
* glog.V(2) - Useful steady state information about the service and important log messages that may correlate to significant changes in the system. This is the recommended default log level for most systems.
* Logging HTTP requests and their exit code
* System state changing (killing pod)
* Controller state change events (starting pods)
* Scheduler log messages
* glog.V(3) - Extended information about changes
* More info about system state changes
* glog.V(4) - Debug level verbosity (for now)
* Logging in particularly thorny parts of code where you may want to come back later and check it
As per the comments, the practical default level is V(2). Developers and QE environments may wish to run at V(3) or V(4). If you wish to change the log level, you can pass in `-v=X` where X is the desired maximum level to log.
docs/namespaces.md
View file @ c47693c0
# Kubernetes Proposal - Namespaces
# Namespaces
**Related PR:**
Namespaces help different projects, teams, or customers to share a kubernetes cluster. First, they provide a scope for [Names](identifiers.md). Second, as our access control code develops, it is expected that it will be convenient to attach authorization and other policy to namespaces.
| Topic | Link |
| ---- | ---- |
| Identifiers.md | https://github.com/GoogleCloudPlatform/kubernetes/pull/1216 |
| Access.md | https://github.com/GoogleCloudPlatform/kubernetes/pull/891 |
| Indexing | https://github.com/GoogleCloudPlatform/kubernetes/pull/1183 |
| Cluster Subdivision | https://github.com/GoogleCloudPlatform/kubernetes/issues/442 |
## Background
High level goals:
* Enable an easy-to-use mechanism to logically scope Kubernetes resources
* Ensure extension resources to Kubernetes can share the same logical scope as core Kubernetes resources
* Ensure it aligns with access control proposal
* Ensure system has log n scale with increasing numbers of scopes
## Use cases
Actors:
1. k8s admin - administers a kubernetes cluster
2. k8s service - k8s daemon operates on behalf of another user (i.e. controller-manager)
2. k8s policy manager - enforces policies imposed on k8s cluster
3. k8s user - uses a kubernetes cluster to schedule pods
User stories:
1. Ability to set immutable namespace to k8s resources
2. Ability to list k8s resource scoped to a namespace
3. Restrict a namespace identifier to a DNS-compatible string to support compound naming conventions
4. Ability for a k8s policy manager to enforce a k8s user's access to a set of namespaces
5. Ability to set/unset a default namespace for use by kubecfg client
6. Ability for a k8s service to monitor resource changes across namespaces
7. Ability for a k8s service to list resources across namespaces
## Proposed Design
### Model Changes
Introduce a new attribute *Namespace* for each resource that must be scoped in a Kubernetes cluster.
A *Namespace* is a DNS compatible subdomain.
```
// TypeMeta is shared by all objects sent to, or returned from the client
type TypeMeta struct {
Kind string `json:"kind,omitempty" yaml:"kind,omitempty"`
Uid string `json:"uid,omitempty" yaml:"uid,omitempty"`
CreationTimestamp util.Time `json:"creationTimestamp,omitempty" yaml:"creationTimestamp,omitempty"`
SelfLink string `json:"selfLink,omitempty" yaml:"selfLink,omitempty"`
ResourceVersion uint64 `json:"resourceVersion,omitempty" yaml:"resourceVersion,omitempty"`
APIVersion string `json:"apiVersion,omitempty" yaml:"apiVersion,omitempty"`
Namespace string `json:"namespace,omitempty" yaml:"namespace,omitempty"`
Name string `json:"name,omitempty" yaml:"name,omitempty"`
}
```
An identifier, *UID*, is unique across time and space intended to distinguish between historical occurences of similar entities.
A *Name* is unique within a given *Namespace* at a particular time, used in resource URLs; provided by clients at creation time
and encouraged to be human friendly; intended to facilitate creation idempotence and space-uniqueness of singleton objects, distinguish
distinct entities, and reference particular entities across operations.
As of this writing, the following resources MUST have a *Namespace* and *Name*
* pod
* service
* replicationController
* endpoint
A *policy* MAY be associated with a *Namespace*.
If a *policy* has an associated *Namespace*, the resource paths it enforces are scoped to a particular *Namespace*.
## k8s API server
In support of namespace isolation, the Kubernetes API server will address resources by the following conventions:
The typical actors for the following requests are the k8s user or the k8s service.
| Action | HTTP Verb | Path | Description |
| ---- | ---- | ---- | ---- |
| CREATE | POST | /api/{version}/ns/{ns}/{resourceType}/ | Create instance of {resourceType} in namespace {ns} |
| GET | GET | /api/{version}/ns/{ns}/{resourceType}/{name} | Get instance of {resourceType} in namespace {ns} with {name} |
| UPDATE | PUT | /api/{version}/ns/{ns}/{resourceType}/{name} | Update instance of {resourceType} in namespace {ns} with {name} |
| DELETE | DELETE | /api/{version}/ns/{ns}/{resourceType}/{name} | Delete instance of {resourceType} in namespace {ns} with {name} |
| LIST | GET | /api/{version}/ns/{ns}/{resourceType} | List instances of {resourceType} in namespace {ns} |
| WATCH | GET | /api/{version}/watch/ns/{ns}/{resourceType} | Watch for changes to a {resourceType} in namespace {ns} |
The typical actor for the following requests are the k8s service or k8s admin as enforced by k8s Policy.
| Action | HTTP Verb | Path | Description |
| ---- | ---- | ---- | ---- |
| WATCH | GET | /api/{version}/watch/{resourceType} | Watch for changes to a {resourceType} across all namespaces |
| LIST | GET | /api/{version}/list/{resourceType} | List instances of {resourceType} across all namespaces |
The legacy API patterns for k8s are an alias to interacting with the *default* namespace as follows.
| Action | HTTP Verb | Path | Description |
| ---- | ---- | ---- | ---- |
| CREATE | POST | /api/{version}/{resourceType}/ | Create instance of {resourceType} in namespace *default* |
| GET | GET | /api/{version}/{resourceType}/{name} | Get instance of {resourceType} in namespace *default* |
| UPDATE | PUT | /api/{version}/{resourceType}/{name} | Update instance of {resourceType} in namespace *default* |
| DELETE | DELETE | /api/{version}/{resourceType}/{name} | Delete instance of {resourceType} in namespace *default* |
The k8s API server verifies the *Namespace* on resource creation matches the *{ns}* on the path.
The k8s API server will enable efficient mechanisms to filter model resources based on the *Namespace*. This may require
the creation of an index on *Namespace* that could support query by namespace with optional label selectors.
The k8s API server will associate a resource with a *Namespace* if not populated by the end-user based on the *Namespace* context
of the incoming request. If the *Namespace* of the resource being created, or updated does not match the *Namespace* on the request,
then the k8s API server will reject the request.
TODO: Update to discuss k8s api server proxy patterns
## k8s storage
A namespace provides a unique identifier space and therefore must be in the storage path of a resource.
In etcd, we want to continue to still support efficient WATCH across namespaces.
Resources that persist content in etcd will have storage paths as follows:
/registry/{resourceType}/{resource.Namespace}/{resource.Name}
This enables k8s service to WATCH /registry/{resourceType} for changes across namespace of a particular {resourceType}.
Upon scheduling a pod to a particular host, the pod's namespace must be in the key path as follows:
/host/{host}/pod/{pod.Namespace}/{pod.Name}
## k8s Authorization service
This design assumes the existence of an authorization service that filters incoming requests to the k8s API Server in order
to enforce user authorization to a particular k8s resource. It performs this action by associating the *subject* of a request
with a *policy* to an associated HTTP path and verb. This design encodes the *namespace* in the resource path in order to enable
external policy servers to function by resource path alone. If a request is made by an identity that is not allowed by
policy to the resource, the request is terminated. Otherwise, it is forwarded to the apiserver.
## k8s controller-manager
The controller-manager will provision pods in the same namespace as the associated replicationController.
## k8s Kubelet
There is no major change to the kubelet introduced by this proposal.
### kubecfg client
kubecfg supports following:
```
kubecfg [OPTIONS] ns {namespace}
```
To set a namespace to use across multiple operations:
```
$ kubecfg ns ns1
```
To view the current namespace:
```
$ kubecfg ns
Using namespace ns1
```
To reset to the default namespace:
```
$ kubecfg ns default
```
In addition, each kubecfg request may explicitly specify a namespace for the operation via the following OPTION
--ns
When loading resource files specified by the -c OPTION, the kubecfg client will ensure the namespace is set in the
message body to match the client specified default.
If no default namespace is applied, the client will assume the following default namespace:
* default
The kubecfg client would store default namespace information in the same manner it caches authentication information today
as a file on user's file system.
Use of multiple namespaces is optional. For small teams, they may not be needed.
Namespaces are still under development. For now, the best documentation is the [Namespaces Design Document](design/namespaces.md).
docs/overview.md 0 → 100644
View file @ c47693c0
# Kubernetes User Documentation
## Resources
The Kubernetes API currently manages 3 main resources: `pods`,
`replicationControllers`, and `services`. Pods correspond to colocated groups
of [Docker containers](http://docker.io) with shared volumes, as supported by
[Google Cloud Platform container-vm
images](https://developers.google.com/compute/docs/containers). Singleton pods
can be created directly via the `/pods` endpoint. Sets of pods may created,
maintained, and scaled using replicationControllers. Services create
load-balanced targets for sets of pods.
Each resource has a two [identifiers](identifiers.md): a string `Name` and a
string `UID`. The name is provided by the user. The UID is generated by the
system and is guaranteed to be unique in space and time across all resources.
`labels` is a map of string (key) to string (value).
Each resource has list of key-value [labels](labels.md).
Individual labels are used to specify identifying metadata that can be used to define sets of resources by
specifying required labels.
## Creation and Updates
Object creation is idempotent when the client remembers the name of the object it wants to create.
Resources have a `desiredState` for the user provided parameters and a
`currentState` for the actual system state. When a new version of a resource
is PUT the `desiredState` is updated and available immediately. Over time the
system will work to bring the `currentState` into line with the `desiredState`.
The system will drive toward the most recent `desiredState` regardless of
previous versions of that stanza. In other words, if a value is changed from 2
to 5 in one PUT and then back down to 3 in another PUT the system is not
required to 'touch base' at 5 before making 3 the `currentState`.
When doing an update, we assume that the entire `desiredState` stanza is
specified. If a field is omitted it is assumed that the user is looking to
delete that field. It is viable for a user to GET the resource, modify what
they like in the `desiredState` or labels stanzas and then PUT it back. If the
`currentState` is included in the PUT it will be silently ignored.
Concurrent modification should be accomplished with optimistic locking of
resources. All resources have a `ResourceVersion` as part of their metadata.
If this is included with the PUT operation the system will verify that there
have not been other successful mutations to the resource during a
read/modify/write cycle. The correct client action at this point is to GET the
resource again, apply the changes afresh and try submitting again.
docs/ux.md
View file @ c47693c0
......@@ -24,7 +24,7 @@ The Kubernetes user interface is a query-based visualization of the Kubernetes A
_GroupBy_ takes a label ```key``` as a parameter, places all objects with the same value for that key within a single group. For example ```/groups/host/selector``` groups pods by host. ```/groups/name/selector``` groups pods by name. Groups are hiearchical, for example ```/groups/name/host/selector``` first groups by pod name, and then by host.
#### Select
Select takes a [label selector](docs/labels.md) and uses it to filter, so only resources which match that label selector are displayed. For example, ```/groups/host/selector/name=frontend```, shows pods, grouped by host, which have a label with the name `frontend`.
Select takes a [label selector](./labels.md) and uses it to filter, so only resources which match that label selector are displayed. For example, ```/groups/host/selector/name=frontend```, shows pods, grouped by host, which have a label with the name `frontend`.
## Rebuilding the UX
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment