How To Deal With Huge Apache Web Server Log Files
I must admit that I have not been paying attention to the size of some of the Apache web server log files on my dedicated server. For those that do not know, the Apache logs are the log files of the Apache web server. They log every hit to a website and provide excellent material that can be processed by a log file analysis software.
This log files can become huge if you have a popular website. The current size of the Ghacks log file is 45 Gigabytes. If it continues growing in that pace the web server will run out of hard disk space pretty soon.
The first problem was to figure out how to backup the 45 Gigabyte file on the web server since I did not want to loose all the data stored within. The idea was born to compress the Apache web server log file first and then reduce it to a few Kilobytes with the echo command.
The Apache web server log files are stored in the /var/log/apache2 directory on the Debian dedicated server. The first command to issue would compress the contents of the apache2 directory into multiple tar files if the compressed file reaches a size maximum. This makes it easier to transfer files to another computer.
tar -cz directory | split -d -b 1024m - destination.tgz
Directory is the source directory where the Apache web server log files reside and destination.tgz is the name of the to be created compressed file. It is important to note that you might need to stop the Apache web server for the operation to succeed because the compression utility will stop if files get changed during procession.
The last thing that is to be done is to clear the original Apache web server log files in the log files directory. This can be done with the following command even if Apache is running.
echo " " > /var/log/apache2/filename.log
This simply overwrites the contents of the file. Please note again that some Linux distributions use another directory structure and that the log files can be located in another directory on the server.
This takes only a few seconds and the log file will be cleared from all contents afterwards.
Then it is just a matter of transferring the various compressed archives to another computer.
Advertisement
In fact deleting the logs (often proposed), as well as truncating them like that is not the best way.
Instead files should be MOVED, then apache gracefully restarted (send signal HUP) then wait (1 minute or more) and then delete the MOVED files.
Otherwise you could end up with lots of ghost files – deleted by still taking disk space.
I found out when fixing this problem four our customer, invisible files filled up entire partition.
You see this files in lsof | grep deleted
To delete files by logrotate just call it with -f (force) flag and provide config file directly like /etc/logrotate.d/apache (debian stable 6).
> It is important to note that you might need to stop
> the Apache web server for the operation to succeed
> because the compression utility will stop if files
> get changed during procession.
How about copying the logfile to another location before compressing? This will of course only be practical if the logile is less than half of your available space, though if that is _not_ your situation then you are in trouble!
This is why logfiles are usually stored in /var and why /var is recommended to be on it’s own partition. When there is a traffic spike Apache logs can grow very fast, and when there are other problems (hardware failure, bad software update, breach) other logfiles can grow very quickly as well. If they are on their own partition they won’t take down the server.
Yes logrotate is a very good option which would have saved me the hassle of dealing with a 45 Gigabyte log file in first place.
just curious… how about using logrotate?