Docker logs provide essential information about the commands and processes that are being executed inside the container. This is helpful in cases when your containers fail to work or gets crashed. You can tail Docker logs to find the exact set of commands that were responsible for the failure. Docker logs also help you to monitor the processes inside the container by live-streaming the process details.
Docker provides us with logging mechanisms that can be used to perform debugging at the daemon as well as container level. In this article, we will discuss how to display the container logs and tail Docker logs to get only the specific lines. You can check out our complete free Docker tutorial.
What are Docker logs and how do they work?
Every Docker container running processes inside it produces logs with valuable information. In simpler terms, logs are the data that are written by containers to the STDOUT or STDERR streams. When we run a container in detached mode, we can’t see the logs in the console. This is obvious since the containers run in the background when we start them in detached mode. However, we can use the Docker logs command with various options to monitor the container logs and debug issues.
For any developer, logs are useful especially when they try to debug or analyze the reasons for failures. In addition to this, to detect anomalies, developers can apply trend analysis using tools such as Loggly, Logstash, etc. to change the approach from reactive to proactive. This allows them to not only figure out the points of failure but also predict them beforehand.
Inside a Docker container, when an application emits logs, they are sent to the STDOUT and STDERR output streams. These streams are accessed by the logging drivers of the containers. They send these streams to a file or a collector running on the host or any type of management service endpoint.
Where are Docker logs stored by default?
We know that the Docker containers are stateless. This means that as soon as we stop a container, all the information is lost. For this reason, the container logs are stored in the Docker host machine itself. They use the JSON (JavaScript Object Notation) file driver. The JSON log files are specific to containers and the file driver writes JSON-formatted logs to such files that reside in the host machine itself.
Docker has several in-built logging drivers that we can choose from. This determines how the logs are formatted by the containers and where they are sent. In addition to the default JSON logging driver, there are several other drivers that we can use such as syslog, fluentd, logagent, journald, etc.
Each log file has information about the respective container. We can find these log files inside the following directory –
/var/lib/docker/containers/<ID of Container>/<ID of Container>-json.logThis directory resides in the host machine where we access the Docker.
You can use the following command to find the default logging driver in your host.
$ docker info | grep Logging
Docker Container Logs Command
We can use the Docker logs command to batch-retrieve the logs present during the execution of the containers. However, this command is functional only when you use the journald or json-file logging drivers. The syntax of the Docker logs command is –
$ docker logs [OPTIONS] CONTAINERYou can use several options to modify the output.
| Name | Shorthand | Description |
--details | It is used to display extra information provided to the logs. | |
--follow | -f | You can use this option to follow the log output. |
--since | The since option can be used to display the logs since the mentioned timestamp. It can be absolute such as 2021-05-04T13:23:37Z) or relative values such as (e.g. 21m for 21 minutes). | |
--tail | -n | The tail option can be used to specify the number of lines that you want to display starting from the end of the logs. |
--timestamps | -t | It is used to show the timestamps along with other information. |
--until | It can be used to display the logs before a given timestamp. |
Let’s create a container and perform the following processes inside it.
$ docker run -it --name=myubuntu ubuntu bash
Create a file along with some content inside it.
# echo "Welcome to TechTutorialSite" > TechTutorialSite.txt
# ls
# cat TechTutorialSite.txt
# mkdir Demo
# ls
Next, keeping this container running, open a new terminal and execute the Docker logs command as mentioned below.
$ docker logs myubuntu
You can see that the exact sequence of the processes has been displayed.
Tail Docker Logs
Let’s use the tail option along with the Docker logs command to print only the last two lines.
$ docker logs --tail 2 myubuntu
How to choose a Docker Logging Driver?
We can also use the built-in drivers that Docker provides to forward logs to various endpoints. You can forward logs to logging services like journald or syslog, you can send them to a log shipper like fluentd, or forward them to any centralized management service for logs.
We can also use logging driver plugins supported by Docker. Using them, we can create our own logging drivers and share them using registries like Docker hub, etc. By default, Docker uses the json-file logging driver and it is also the recommended ones. We can override this default setting using two ways.
The first method is to use the –log-driver option while running the container using the Docker run command. Let’s create an Ubuntu container with journald driver.
$ docker run --name=myubuntu --log-driver journald ubuntu
To check the driver used by the myubuntu container, we can use the Docker inspect command.
$ docker inspect myubuntu
Another method to change the logging driver is by modifying the daemon.json file located on /etc/docker/daemon.json. If the file does not exist, you can create one. You can add the following snippet inside the curly braces.
After saving the file, you can check the default logging driver using the following command.
$ docker info | grep LoggingBest Practices for Docker Logging
You can follow the practices mentioned below for the best result.
- Always export logs to persistent storage – When we stop a container, all the information related to logs is deleted. Hence, we should never store any application-specific data inside Docker containers. Instead, we can use persistent storage volumes mounted on the host to store logs. We can also pipe the logs to a log management service or hard drives.
- You can use a Logging Container – You can pipe log outputs of all your containers to another logging container. This way, you can save your storage and scale logging services easily.
- Use Docker Logging Drivers – You can use the logging drivers of the platform. You can use them to forward all the log events to an isolated syslog instance that is running on the same host machine. The logging driver will read all the log events straight from the stdout and stderr output streams of the container. This will help you to eliminate the need to read or write from log files and help in a performance gain.
- You can also use a sidecar – A sidecar is a service that intends to run alongside your own container service acting as a secondary process. They provide infrastructure features that are exposed through a homogeneous interface. An example of such an interface is a REST-like API over HTTP. The advantage of using a sidecar approach for logging is that you can link each container to its own logging container
Final Thoughts!
You can use Docker logs to debug issues and monitor the processes running inside your container. Docker provides options to choose from various logging drivers. However, the Docker logs command only works with json-file and journald logging drivers. The json-file driver is the default logging driver and also the recommended ones. You can tail Docker logs using the –tail option in the Docker logs command.
If you have any queries or suggestions, please mention them in the comment box and we will have our expert get back to you as soon as possible. Also, check out our free and complete Docker Tutorial.
One comment