Recently on one of our new servers, we had an issue where the mysql daemon had halted on bus error. We couldn’t stop the mysql daemon using

/etc/init.d/mysqld stop

Then I looked at the error-log file for mysql in /var/log/mysql-error.log and found that the disk didn’t had any empty space left and correctly enough, when I did

df -h

I saw that the /var mounted partition didn’t had any diskspace remaining. To find the exact culprit I looked around in the /var directory and found that the ‘mysqld.log’ file was 9GB large. I was puzzled as to how come this log grew so big. Thoughts came that the server would have been compromised but the actual culprit was the my.cnf file for the mysql. During the setup of the server, by mistake I had put a statement

[mysqld]

log=/var/log/mysqld.log

This configuration was causing to save all the queries that were performed on the mysql server since it booted. Naturally on a high traffic site, you will see this file grow huge quickly. I learned the lesson hard way, but always RTFM when you are configuring something on the production server. So for solution, all I had to was remove the mysqld.log file and comment out the above “log” configuration in the my.cnf.

I thought to post this out there to help other people in case they have the same issue.