Redirect Docker/Kubernete log out of /var/log/message for Cloudera Data Science Workbench

After you installed Cloudera Data Science Workbench (CDSW) in a cluster, by default, the log messages for Docker and Kubernete all go into /var/log/messages. This is OK in most cases, but some user might not want the logs to be shared with other processes, so separating those logs outside of /var/log/messages might make sense.

follow url To do this, please follow steps below:

1. Create the file under /etc/rsyslog.d/10-docker.conf
2. Put the below contents into the above file:

http://www.ilplanetario.net/?paper-term-writer # Docker logging *.* { /var/log/docker.log stop }

3. Configure logrotate to roll and archive these files by creating file: /etc/logrotate.d/docker
4. Put the following contents into file /etc/logrotate.d/docker:

/var/log/docker.log { size 100M rotate 2 missingok compress }

go to link 5. Finally, restart rsyslog:

service rsyslog restart

After all above steps, the CDSW logging should be redirected to /var/log/docker.log, no restart of CDSW is required.

Note: this blog was based on steps from Mark Wolfe’s Blog. The difference is that instead of “ see url daemon.*” in step 2, I need to use “ get *.*“, otherwise only kubernete log will go into /var/log/docker.log, not dockerd logs.