Recently a good article named “Is this VM actively swapping?” was posted by Duncan Epping. Coincidently, in a recent engagement I stumbled upon an interesting swapping scenario.
Just to put in context:
- DRS Cluster with 8 hosts across 2 different blade enclosures
- Cluster is not memory overcommitted
- All VMs have latest VMware Tools installed
- TPS is enabled
- All VMs hosted in shared storage with low latency
- No memory reservations for VMs individually
16078 Free -> Currently 16GB memory available
High State -> Hypervisor is not under memory pressure
SWAP /MB 7222 Cur -> 7GB has been swapped
0.00 r/s -> No reads from swap currently
0.00 w/s -> No writes to swap currently
Interesting circumstance here, where even with staggering 16GB (PMEM/MB 16078 free) of free memory resources available, Memory Overcommit at AVG 0.49, and no VM memory reservations the host was swapping. The host swapped a total of 7GB (SWAP/MB 7222).
Paying attention to column MCTL? you will notice that the Ballooning Driver is not running for two VMs.
Balloon drivers not running or disabled. If the balloon driver is not running, or has been deliberately disabled in some VMs, the amount of balloonable memory on the host will be decreased. As a result, the host may be forced to swap even though there is idle memory that could have been reclaimed through ballooning.
Two of the VMs with respectively memory size of 16GB and 4GB were not ballooning, therefore not freeing up memory resources to satisfy host’s memory overcommitment levels. When host’s memory is scarce or when a VM hits a Limit, the kernel needs to reclaim memory and prefers ballooning over swapping. The balloon driver is installed inside the guest OS as part of the VMware Tools installation and is also known as the vmmemctl driver.
The ballooning driver was disabled by a previous administrator.
It seems that some administrators believe that ballooning will not guarantee the total amount of RAM allocated to a given VM and add extra disk I/O. That might be true in an overcommitted environment but the gains from being able to overcommit memory in conjunction with DRS functionality IMHO outcast any benefits of disabling ballooning. A good alternative to disabling memory ballooning is to set memory reservations at VM level to guarantee the minimum amount of RAM available to the VMs.
In my opinion the important here is learn how to read the information provided by performance analysis tools such as esxtop, vCenter performance graphs, and pay attention to details.
Good readings on this topic:
Performance Troubleshooting for VMware vSphere 4
Is this VM actively swapping? (helping @heiner_hardt) from Duncan Epping
Swapping? from Duncan Epping