I receive a lot of questions lately about ESX memory management. Things that are very obvious to me seem to be not so obvious at all for some other people. So I’ll try to explain these things from my point of view.
First let’s have a look at the virtual machine settings available to us. On the vm setting page we have several options we can configure for memory assignment.
- Allocated memory: This is the amount of memory we assign to the vm and is also the amount of memory the guest OS will see as its physical memory. This is a hard limit and the vm cannot exceed this limit if it demands more memory. It is configured on the hardware tab of the vm’s settings.
- Reservations: A reservation is a guaranteed amount of memory assigned to the vm. This is a way of ensuring that the vm gets a minimal amount of memory assigned. When this reservation cannot be met, you will be unable to start the vm. This is known as “Admission Control”. Reservations are set on the resources tab of the vm’s settings and by default there is no reservation set.
- Limits: A limit is a restriction on the vm, so it cannot use more memory than this limit. If you would set this limit lower than the allocated memory value, the ballooning driver will start to inflate as soon as the vm demands more memory than this limit. Limits are set on the resources tab of the vm’s settings and by default the limit is set to “unlimited”.Now that we know of limits and reservations, we need to have a quick look at the VMkernel swap file. This swap file is used by the VMkernel to swap out the vm’s memory as a last resort to free up memory when the host is running out of it. When we set a reservation, that memory is guaranteed and cannot be swapped out to disk. So whenever a vm starts up, the VMkernel creates a swap file which has a size of the limit minus the reservation. For example we have a vm with a 1024MB limit and a reservation of 512MB. The swap file created will be 1024MB – 512MB = 512MB. If we would set the reservation to 1024MB there won’t be a swap file created at all. Remember that by default there are no reservations and no limits set, so the swap file created for each vm will be the same size as the allocated memory.
- Shares: With shares you set a relative importance on a vm. Unlike limits and reservation which are fixed, shares can change dynamically. Remember that the share system only comes into play when memory resources are scarce and contention is occurring. Shares are set on the resources tab of the vm’s settings and can be set to “low”, “normal”, “high” or a custom value.
low = 5 shares per 1MB allocated to the vm
normal = 10 shares per 1MB allocated to the vm
high = 20 shares per 1MB allocated to the vm
It is important to note that the more memory you assign to a vm, the more shares it receives.Let’s look at an example to show you how this share system works. Say you have 5 vms with each 2,000MB memory allocated and the share value set to “normal”. The ESX host only has 4,000MB of physical machine memory available for virtual machines. Each vm receives 20,000 shares according to the “normal” setting (10 * 2,000). The sum of all shares is 5 * 20,000 = 100,000. Every vm will receive an equal share of 20,000/100,000 = 1/5th of the resources available = 4,000/5 = 800MB.Now we change the shares setting on 1 vm to “High”, which results in this vm receiving 40,000 shares instead of 20,000. The sum of all shares is now increased to 120,000. This vm will receive 40,000/120,000 = 1/3rd of the resources available. Thus 4,000/3 = 1333 MB. All the other vms will receive only 20,000/120,000 = 1/6th of the available resources = 4,000/6 = 666 MB
Instead of configuring these settings on a vm basis, it is also possible to configure these settings on a resource pool. A VMware ESX resource pool is a pool of CPU and memory resources. I always look to the resource pool as a group of VMs.
This concludes the memory settings we can configure on a vm. Next time I will go into ESX memory management techniques.
Continue reading Part2
- VMware Storage Sudoku Tweet Last Friday I was brainstorming with Gabrie van Zanten about the optimal placement of the VMDKs across our LUNs. We tried to come up with an algorithm that could...
- Unable to login to your ESX server Tweet Ivo Beerens posted this article last week on the defunct cimservera processes that render an ESX Host unmanageable. See also this VMWare KB Article. Symptoms include: Unable to log...
- Unattended upgrade of HP management agents Tweet After upgrading to ESX 3.5 to update3, I found out that the HP management agents needed to be upgraded to version 8.1.1, since this version supports ESX3.5 update3. So...
- HOW-TO: Recover from failed Storage VMotion Tweet A while ago I received a request from the storage department to move a whole ESX cluster to another storage I/O-Group. This would be a disruptive action. I was...
- VMware ESX(i) 3.5 Update4 released Tweet VMware has released ESX(i) 3.5 update4. Do not forget to read the release notes here or you can go to the download page here While going through the release notes...