Skip to content

K
k3s
Commit b18bad15 authored by Kubernetes Submit Queue's avatar Kubernetes Submit Queue Committed by GitHub
Browse files
Options

Merge pull request #40634 from Crassirostris/use-docker-log-rotation

Automatic merge from submit-queue Use docker log rotation mechanism instead of logrotate This is a solution for https://github.com/kubernetes/kubernetes/issues/38495. Instead of rotating logs using logrotate tool, which is configured quite rigidly, this PR makes docker responsible for the rotation and makes it possible to configure docker logging parameters. It solves the following problems: * Logging agent will stop loosing lines upon rotation * Container's logs size will be more strictly constrained. Instead of checking the size hourly, size will be checked upon write, preventing https://github.com/kubernetes/kubernetes/issues/27754 It's still far from ideal, for example setting logging options per pod, as suggested in https://github.com/kubernetes/kubernetes/issues/15478 would be much more flexible, but latter approach requires deep changes, including changes in API, which may be in vain because of CRI and long-term vision for logging. Changes include: * Change in salt. It's possible to configure docker log parameters, using variables in pillar. They're exported from env variables on `gce`, but for different cloud provider they have to be exported first. * Change in `configure-helper.sh` scripts for those os on `gce` that don't use salt + default values exposed via env variables This change may be problematic for kubelet logs functionality with CRI enabled, that will be tackled in the follow-up PR, if confirmed. CC @piosz @Random-Liu @yujuhong @dashpole @dchen1107 @vishh @kubernetes/sig-node-pr-reviews ```release-note On GCI by default logrotate is disabled for application containers in favor of rotation mechanism provided by docker logging driver. ```
parents 7265908e cabb9890
Hide whitespace changes
Inline Side-by-side
Showing with 20 additions and 15 deletions
cluster/common.sh
View file @ b18bad15
......@@ -709,6 +709,21 @@ EOF
DOCKER_TEST_LOG_LEVEL: $(yaml-quote ${DOCKER_TEST_LOG_LEVEL})
EOF
fi
if [ -n "${DOCKER_LOG_DRIVER:-}" ]; then
cat >>$file <<EOF
DOCKER_LOG_DRIVER: $(yaml-quote ${DOCKER_LOG_DRIVER})
EOF
fi
if [ -n "${DOCKER_LOG_MAX_SIZE:-}" ]; then
cat >>$file <<EOF
DOCKER_LOG_MAX_SIZE: $(yaml-quote ${DOCKER_LOG_MAX_SIZE})
EOF
fi
if [ -n "${DOCKER_LOG_MAX_FILE:-}" ]; then
cat >>$file <<EOF
DOCKER_LOG_MAX_FILE: $(yaml-quote ${DOCKER_LOG_MAX_FILE})
EOF
fi
if [ -n "${ENABLE_CUSTOM_METRICS:-}" ]; then
cat >>$file <<EOF
ENABLE_CUSTOM_METRICS: $(yaml-quote ${ENABLE_CUSTOM_METRICS})
......
cluster/gce/gci/configure-helper.sh
View file @ b18bad15
......@@ -95,21 +95,6 @@ function ensure-local-ssds() {
# Installs logrotate configuration files
function setup-logrotate() {
mkdir -p /etc/logrotate.d/
cat >/etc/logrotate.d/docker-containers <<EOF
/var/lib/docker/containers/*/*-json.log {
rotate 5
copytruncate
missingok
notifempty
compress
maxsize 10M
daily
dateext
dateformat -%Y%m%d-%s
create 0644 root root
}
EOF
# Configure log rotation for all logs in /var/log, which is where k8s services
# are configured to write their log files. Whenever logrotate is ran, this
# config will:
......@@ -568,6 +553,11 @@ function assemble-docker-flags {
docker_opts+=" --registry-mirror=${DOCKER_REGISTRY_MIRROR_URL}"
fi
# Configure docker logging
docker_opts+=" --log-driver=${DOCKER_LOG_DRIVER:-json-file}"
docker_opts+=" --log-opt=max-size=${DOCKER_LOG_MAX_SIZE:-10m}"
docker_opts+=" --log-opt=max-file=${DOCKER_LOG_MAX_FILE:-5}"
echo "DOCKER_OPTS=\"${docker_opts} ${EXTRA_DOCKER_OPTS:-}\"" > /etc/default/docker
if [[ "${use_net_plugin}" == "true" ]]; then
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment