Let’s say you want to create a lot of snapshots for a single VM inside VMware ESX. How many snapshots can you create before your ESX server’s performance goes to crap? Furthermore, is this maximum number relative, depending on how the snapshots are organized in the tree? (For example, can I have more snapshots whose of depth is 1 versus less snapshots of depth 3?)
Those were just some of the questions I’ve been dealing with, in porting the HoneyClient code to ESX.
Here’s the short answer: If your snapshots are getting stored as regular files (which is the default), then UNDER NO CIRCUMSTANCES should you exceed 32 snapshots per VM.
The longer answer: Regardless of how the snapshots are organized, VMware ESX appears to seek through ALL snapshots (in chronological order) when performing ANY snapshot-related operation (e.g., renaming a given snapshot). So, if you create 128 snapshots and just want to rename/revert/alter/delete the 3rd snapshot, then guess what? The ESX server will still iterate through all 128 snapshots before performing the requested operation.
You’d think ESX would be smart enough to perform some sort of depth-first (or even breadth-first search), but no, it doesn’t. This limitation comes from the fact that the snapshot metadata is stored as a FLAT FILE in chronological order.
Wait, it gets better. So it appears that as the number of snapshots increases linearly, the amount of time ESX needs to seek through these snapshots increases in a geometric rate (not quite exponential, but bad enough). And so, this delay becomes noticeable when the number of snapshots is greater than 32.
So, here are my questions to VMware and other ESX users:
1) I’m guessing that this 32 snapshot limit still exists, even if the disks and snapshot data were mapped to a physical iSCSI volume via RDM. Can anyone confirm?
2) VMware: Are you ever planning on improving this snapshotting implementation, for those who want more than 32 snapshots?
3) Anyone aware of similar limitations in other virualization products? (VirtualBox, Xen, etc.?)