«

»

Dec 15 2011

Windows 7 Transparent Page Sharing and the ASLR story…

“Common knowledge is knowledge that is known by everyone or nearly everyone, usually with reference to the community in which the term is used.” – Source: Wikipedia.

It is typical to have sets of common knowledge that guide best practices. However, common knowledge isn’t always correct and people tend to embrace common knowledge as the only and absolute truth.

I have been longing this article for a long time and I have finally found time (I’m stranded at SFO airport) and resources to prove that many times common knowledge is not correct.

In the VDI context, Windows XP has been used as Guest OS for a long time. Over this period vendors, architects and customers were able to familiarize themselves with the behavior of vSphere TPS (Transparent Page Sharing) and Windows XP. Common knowledge quantifies Windows XP TPS savings of ~40% in VDI environments.

Windows 7 came along and with it a new feature called ASLR (Address Space Layout Randomization). Address space randomization hinders some types of security attacks by making it more difficult for an attacker to predict target addresses.

ASLR is an important security feature when combined with NX (DEP), and by disabling one or the other will make Windows 7 VMs more easily exploitable by security vulnerabilities.

While it is true that on Windows ASLR requires some code page contents to be modified, the common knowledge tell us that the effects on TPS are empirical. This means, it’s not possible to predict the amount of TPS savings at a given point in time – even if we are trying to calculate the average. Because of that, consultants, vendors and architects have been using safety numbers around 10% – 15% as ballpark numbers for designing VDI solutions with Windows 7 and vSphere.

With the OS change there is a loss of approximately 30% on TPS savings. In business values this means that VDI infrastructures will run ~30% less sessions before the hypervizor start to run into contention (memory compression or host swap).

There are many blog references regarding ASLR and how it impacts TPS. Meanwhile, there are many references of how to disable Windows ASLR feature by setting the following registry key to 0.

\HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\MoveImages

MoveImages key is described in “Windows Internals”, but that doesn’t make it a documented feature supported by Microsoft.

Is it a good practice to disable ASLR?

Despite I’ve not had a great deal of time to determine the difference with or without ASLR, the short answer is No. Unless you are pushing very high levels of memory overcommit in a 32-bit desktop VDI environment, you have a lot more to lose than to gain from disabling ASLR. On 64-bit platforms the loss of opportunities to share pages is much less due to the large memory page nature.

Having set baseline for this discussion I would like to mention that I have not seen any solid test or research that quantifies the amount of savings provided by transparent page sharing when in use with Windows 7 virtual desktops, with ALSR enabled (default).

To create valid tests it is important that a baseline is always maintained. In this case, during tests, the host used was a Cisco UCS B200 with 48GB RAM and two Quad-Core CPU’s at 2.526MHz. The virtual desktops are running Windows 7 with 1GB RAM.

The workload was generated using LoginVSI with the Default Heavy profile. Due to the nature of LoginVSI workload the results presented below are valid to demonstrate virtual desktops that run similar applications on the same host, such as task workers in call center or office branch.

To maintain host affinity during tests a DRS group (Must run on) was created and pinned to the server host. The next step was to add virtual desktops to the DRS group and watch VMs being VMotion(ed) to the chosen host. DRS was set to aggressive to make sure VM not belonging to the DRS group were properly evacuated.

Out of 100 VMs a total of 54 VMs were VMotion(ed) to the server host until the point where the the host started to run into memory contention. At this point the total amount of memory saving due to TPS was 13760/MB, or 28%.

clip_image001

clip_image003

The next step was to watch TPS behavior during the next one hour for which LoginVSI was continuously running workload tests. I did not notice at any point in time during the tests a behavior where memory footprint increased due to ASLR functionality (memory randomization).

As time went by TPS started to collapse more small memory blocks and reduced the total memory footprint in use. After one hour the savings due to TPS were 16364/MB, or 33%. The savings provided by TPS would probably allow me to add additional four or five virtual desktops. See stats below.

clip_image004

At this point in time I was convinced that Windows ASLR does impact how TPS collapse memory blocks, however not the way Common Knowledge tell us. For this constant and similar workload the savings averaged 30%.

 

Going Beyond…

VDI is in a very cost sensitive solution. The higher the consolidation ratio more cost effective the solution will present itself. So, why not go further and see what kind of awesomeness is TPS likely to deliver if settings are tweaked a little bit.

In the past I have written about TPS, Large Memory Pages and your VDI environment and Increase VDI consolidation ratio with TPS tuning on ESX. I recommend tweaking Mem.AllocGuestLargePage and Mem.ShareScanTime for higher consolidation rates. After the changes, vSphere started to collapse more memory blocks and more aggressively.

Collapsing memory blocks more aggressively does put an extra overhead on CPU. However most VDI deployments are memory bound, not CPU – so is this one. Therefore, using more CPU was not really an issue here.

As time went by the savings provided by TPS freed up a total of 11267MB RAM that would allow me increase consolidation ratio from 54 to probably 64 virtual desktops per host (17%).

clip_image006

I run those tests a while back and unfortunately I misplaced the screenshot that would demonstrate the stats on PSHARE/MB. I thought the results were remarkable and decided to share the results anyway.

Conclusion

Windows 7 ASLR does impact vSphere’s ability to collapse memory blocks via Transparent Page Sharing. However, the numbers attained during tests are better than Common Knowledge. With a constant workload generated via LoginVSI, TPS achieved a 30% average memory saving, without vSphere tweaking.

With tweaking of vSphere hosts mentioned in TPS, Large Memory Pages and your VDI environment and Increase VDI consolidation ratio with TPS tuning on ESX, TPS savings should average roughly 40% for this specific workload.

Every workload is different, and even similar workloads may behave differently. I recommended anyone planning to implement the mentioned host changes to execute tests with a single host before changing entire vSphere clusters.

If you are architecting a VDI solution on vSphere you may now decide go go further than 10%-15%, but as always, run your own tests before implementing the solution. No architecture or calculation is substitute for a good PILOT.

Similar Posts:

Permanent link to this article: http://myvirtualcloud.net/?p=2545

10 comments

1 ping

Skip to comment form

  1. Richard Bell

    A good article, Andre. One of those ‘elephant in the room’ discussions that seems to be overlooked, or not spoken about by virtualisation vendors.
    A couple of questions…
    – do you see any difference in the TPS savings between x86 and x64 Win7 on vSphere5.0?
    – 1 hour before TPS really starts to make an impact. Is that any different to how WinXP performs under TPS? Do you see this 1 hour delay as something of a risk to VDI environments? In that, if your modus operandi is to startup virtual desktops 1 hour before users start arriving for work, you would need to take this 1 hour into account so that the environment ‘stabilses’ before users start logging on.
    – I wonder how Xen Server and Hyper-V Dynamic Memory compares with TPS. Maybe they can’t be compared. Too dissimilar?

    Richard

  2. Duncan

    You can’t compare Dynamic Memory with TPS. Totally different concept imho.

    I think it is a shame that there’s no test showing what the TPS savings would be with ASLR disabled. Now you mention that 40% is what people usually get and then show that with ASLR this is roughly 30% and mention in the conclusion you could hit 40% with tweaking, but there’s no data supporting this. Would be nice to see add that in my opinion,

  3. Andre Leibovici

    @Richard Bell
    I have not tested TPS savings between x86 and x64 on Win7 therefore I can’t really comment. I did however compare Windows XP 32b and 64b in my article “Windows XP x64 in virtual environments might go beyond appearances” http://myvirtualcloud.net/?p=900. However, I did not look into TPS.

    1 hour was the time it took to get to stabilize at ~30% TPS saving. So, if you power on VMs in the morning I would expect to take a little while untill memory blocks are colapsed. However, accelerating the process with Mem.ShareScanTime should provide some benefits.

    If ylou power on all VMs at same time the host will run into memory pressure then compression and swap will be activated. Therefor, I sugest to be carefull with the total overcomitt in use.

    Unfortunately, I have no information on Xen Server and Hyper-V Dynamic Memory behaviour.

    Thanks for your comments and questions,
    Andre

  4. Andre Leibovici

    @Duncan
    I don’t think there is point to test Win7 with ASLR disabled since it’s not supported by MS anyway. In WinXP VDI world the TPS saving would commonly get up to 40%. These tests proved that Win7 TPS savings can actually be predictable, and not empirical as people thought previously.

    I agree with you that my article is missing data to validate the 40% TPS savings after tweaking vSphere with Mem.AllocGuestLargePage and Mem.ShareScanTime. I misplaced the screenshot that would demonstrate the stats on PSHARE/MB.

    For now 30% TPS savings is the number I have validated and have the number and stats to back up. Still good improvement from the previous 10%-15% ballpark. As soon I have some extra cycles I will re-visit the subject.

    Thanks for commenting,
    Andre

  5. ferfebles

    While studying for VCP5, started to think about ASLR and TPS. After the exam, looked for information, found this post, and couldn’t resist the temptation of doing my own tests but with Ubuntu.

    The results are quite similar: ASLR reduces shared memory, but only by small margin: http://testandset.posterous.com/transparent-page-sharing-vs-address-space-lay

  6. Andre Leibovici

    @ferfebles
    Thanks for your comment and article. I am interested to know if Ubuntu ASLR works with the same algorithms Windows ASLR does. Would you know?

    Regards,
    Andre

  7. ferfebles

    Do not know about Windows ASLR, but what Linux ASLR basically does, based on kernel.randomize_va_space value, is:

    0 – do not randomize
    1 – randomize stack, vdso page and mmap (shared libraries will be loaded to random addresses)
    2 – also randomize brk base address (heap randomization).

    All start memory addresses are aligned to memory pages boundaries: 4k or bigger values if large pages are used (they’re called “hugepages” in linux http://wiki.debian.org/Hugepages).

    By the way, I’m curious about how large pages could affect ASLR security. I suppose that with large pages, the number of selectable random addresses gets reduced, but couldn’t find information about this.

    The information about ASLR comes from Kernel docs http://www.kernel.org/doc/Documentation/sysctl/kernel.txt , and a detailed implementation description can be found here http://xorl.wordpress.com/2011/01/16/linux-kernel-aslr-implementation/

  8. Anonymous

    IE10 doesn’t work with disabled ASLR.

  9. Andre Leibovici

    @Anonymous,

    Interesting. Thanks for Sharing.
    Yet another reason to use Chrome.

    Andre

  10. Ryan Wood

    Adobe reader XI also doesn’t work with MoveImages set to 0

  1. vSphere 5.1 & Hyper-V 2012 memory concepts explained and compared - viktorious.nl - Virtualization & Cloud Management

    […] Windows 7 Transparent Page Sharing and the ASLR story – Andre Leibovici […]

Leave a Reply