Understand the logs and the art of Log Rotation.
Efficient Linux log management using logrotate. Learn the essentials for automating log management and to save disk space.
Published May 25, 2025
Logs are records of events and activities generated by software applications, managed in a structured way. This structure of logging may differ application to application but there are few details which can be common. Applications generates logs and keep generating 24*7, however as logs are very important part of applications its crucial task to manage them on day to day basis.
As we understood what are the logs, lets now understand what is there in the logs? and why logs are generated and managed?
As we understood what are the logs, lets now understand what is there in the logs? and why logs are generated and managed?
What is there in the logs :
- Errors and Warnings:
Such as a internal error, bug, an invalid input, or a system failure. example: NullPointerException. - User Interactions:
When a user logs in, submits a form, or performs other actions within the application. - System Events:
To record the when the application starts, stops, or encounters other system-level events. - Application-Specific Data:
Logging database queries, network requests, or custom variables important for monitoring or debugging.
Why logs:
- Troubleshooting:
Troubleshoot the cause of errors and issues in the application. - Performance Monitoring:
To capture performance metrics, like as response times and resource utilization. - Security Auditing:
To track the user access, identify security breaches, and monitor unauthorized activities. - Understanding User Behavior:
To know how the user interacts with application, what are the user interests.
Common Logs :
There are below logs available in linux:
1. auth.log
2. system.log
3. boot.log
1. auth.log
2. system.log
3. boot.log
and there are other logs also which can be found o path:
also, below are the logs in nginx web server.
/var/log/also, below are the logs in nginx web server.
/var/log/nginx/access.log/var/log/nginx/error.logthe default location for nginx logs is :
/var/log/nginx/As we know that these logs keeps generating on every user interaction or system events, the size of these log files keep increasing and eventually occupying more disk space. which can cause harm to continuous running of application. To avoid the disk space utilization due to logs we have to manage logs in such a way that those can be accessible for some period of time and later archived or purged (deleted) when not required. this activity of archiving or deleting the logs on pre-defined condition is called as "Log Rotation".
There is a tool available in the linux for rotating the logs. which is
logrotate. this tool is usually installed by default on most Linux distributions. However if it is not, then can be installed using below command:On Debian/Ubuntu:
On CentOS/RHEL:
sudo apt install logrotate On CentOS/RHEL:
sudo yum install logrotate logrotate is consist of 4 integral parts:
- The logs which need to be rotated.
- The config file which has the rules of log rotation for given log path.
- logrotate service which apply and work on the logs as per the config file.
- logrotate timer which keeps invoking the given logrotate service periodically.
Lets deep dive into all of these with an example:
Suppose I have my application running 24*7 and generating log files in my linux server path :
/home/zain/samlogs
and I want these logs to be compressed and archived on daily basis when a log file crosses 100 MB size. for that I will be writing the below logrotate config file for all the logs in the location:
/etc/logrotate.d/
here
log-rotation is my config file written specifically for rotating the logs on my log path i.e. /home/zain/samlogs
Next, the service file located in
This file contains the command to run the config file created above.
/etc/systemd/system/This file contains the command to run the config file created above.
and at last but very important the timer, which keep invoking the logrotate service on given time period.
This file contains the time logrotation service created above step along with the timer value for the period for invoking that service.
You can check the timer service status if it is running.
This service has to be running continuously.
This service has to be running continuously.
sudo systemctl status log-rotation.timerto check if the log rotation is service is working properly or not and to check the logs use this above command.
you can see the current log files and archive/compressed log files in the log directory.
