This morning Gregor Samsa woke up… oh, pardon! This morning I woke up and found myself puzzled, because my home server was eating up all of my 4 cores CPU cycles. Especially mysqld was high on CPU load. 100% CPU load for the mysql-server instance and 100% CPU load for akonadiservers own mysqld instance. Restarting KDE and mysql-server didn’t help on my Debian unstable machine. Next step was upgrading the system. Sometimes this helps indeed, but not today.
Looking at bugs.debian.org for mysql-server didn’t reveal any help as well. So my next logical step was to ask on #debian-devel in IRC. And my question was very quick answered:
11:28 < ij> since tonight I’ve got two mysqld processes running at 100% CPU, one spawned by akonadi and
the other is the mysqld from mysql-server (unstable that is). is this an already known issue?
haven’t found anything on b.d.o for mysql-server, though
11:29 < mrvn> ij: topic
11:29 < mrvn> you need to set the time
11:30 < ij> waaaaah!
11:30 < mrvn> ij: indeed.
The topic was at that time:
100% CPU? Reset leap second http://xrl.us/bnde4w
So, it was caused by the leap second. Although you might suspect mysql doing some nasty things (which, IMHO, is always a good guess ;)), the issue is this time within the Linux kernel itself, as a commit on git.kernel.org clearifies.
To fix this issue you need to set the time manually using the following command or just reboot:
date -s "`date`"
So far I found these applications being hit by this kernel bug:
- akonadi (as it uses own mysql instances)
- Openfire Jabber server (because it’s using Java, which seems to trigger the problem as mysql does)
- Virtualbox’ VBoxSVC process
- puppetmaster from package puppet, reported by Michael
- mythfrontend, reported by pos on #debian-devel
- Jetty, Hudson, Puppet agent and master, reported by Christian
- milter-greylist, reported by E. Recio
- dovecot, reported by Diogo Resende
- Google Chrome, reported by Erik B. Andersen
- if you find more apps, please comment and I’ll include them here…
So, hope this helps and many thanks to mrvn and infinity on #debian-devel for the help!