Arch Linux - A stop job is running for MariaDB database server

The message in question as shown below made its appearance while I was trying to shut down my laptop. The job went on for about a minute before completing its task.

A stop job is running for MariaDB database server

A stop job is running for MariaDB database server

Thinking Arch had finally thrown med a curveball, I booted up the machine to have a look at the systemd journal.

# View logs by service:
journalctl -u mysqld.service

# Output
- mysqld.service: State 'stop-sigterm' timed out. Killing.
- mysqld.service: Main process exited, code=killed
- mysqld[5894]: 150718 12:20:54 [Warning] mysqld: Disk is full writing './mysql-bin.~rec~' (Errcode: 28 "No space left on device")....

So yeah, the “Disk is full” warning did not require me to be sysadmin of the year to decipher. By looking at the file system I could confirm that / was using 100% of available space. To find out where all the space had gone, I issued the command:

du -Pshx /* 2>/dev/null

# Output
20K     /tmp
5.8G    /usr
13G     /var

13G under /var doesn’t seem right. That can’t be all logs…
Let’s see whats taking up most of the space:

du -Sh /var | sort -rh | head -n 5 

# Output
11G     /var/cache/pacman/pkg
2.0G    /var/log/journal/43a7d67b84eb4c05bc77f4ab75096597
174M    /var/tmp/kdecache-master
112M    /var/lib/mysql
58M     /var/tmp/kdecache-master/http

To quote the pacman section of the Arch wiki:

pacman stores its downloaded packages in /var/cache/pacman/pkg/ and does not remove the old or uninstalled versions automatically, therefore it is necessary to deliberately clean up that folder periodically to prevent such folder to grow indefinitely in size.

In short, I had unknowingly kept a years worth of  package updates on disk, which in turn had accumulated to 11G and eventually hit the roof.

Following the book by using the paccache command (instead of deleting everything) freed up some much needed space.

I was actually aware of this “feature” but had forgotten all about it. I wont be making the same mistake again as filling up / will eventually cause serious harm to the system.

Roger Comply avatar
Roger Comply
Thank you for reading!
Feel free to waste more time by subscribing to my RSS feed.