Russel Coker blogs about a problem, which concerned me as well: Killing Servers with Virtualisation and Swap, i.e.: what happens when one domU is happily swapping the whole day?
Luckily this didn't happen to me yet, but as Russel do I tend as well to give a small sized swap area to domUs. If they're doing extensive swapping, something is wrong and most probably a task has a memory leaking problem or the virtual machine is undersized with its RAM.
Anyway, Russel had some thoughts and ideas how to deal with the problem, such as giving dedicated swap areas on single disks to some domUs to seperate disk I/O for the filesystem and the I/O for swap. He proceeds with:
Now if you have a server with 8 or 12 disks (both of which seem to be reasonably common capacities of modern 2RU servers) and if you decide that RAID is not required for the swap space of DomUs then it would be possible to assign single disks for swap spaces for groups of virtual machines. So if one client had several virtual machines they could have them share the same single disk for the swap, so a thrashing server would only affect the performance of other VMs from the same client. One possible configuration would be a 12 disk server that has a four disk RAID-5 array for main storage and 8 single disks for swap. 8 CPU cores is common for a modern 2RU server, so it would be possible to lock 8 groups of DomUs so that they share CPUs and swap spaces. Another possibility would be to have four groups of DomUs where each group had a RAID-1 array for swap and two CPU cores.
I would myself consider a different approach:
Of course you can mix up both approaches or even use swap over NFS or NBD/DRBD. There are many ways and possible solutions, but the best way to deal with swapping is to take care that it doesn't happen, of course... ;)