Powershell Script to Mass Clone and Power VMs via vCenter using VAAI

After publishing the Cloning and Powering VMs on Nutanix with VAAI video I was asked if I could share the PowerShell scripts used for the demo. I want to thank Steven Poitras (@StevenPoitras) for creating and allowing me to share the scripts with the broader community.
The scripts utilize vCenter Server to create clones and it’s likely that vCenter will be the bottleneck during the provisioning process. I recommend assigning more RAM and vCPU as needed. My vCenter server ended up with 8GB RAM and 4 vCPU.
Additionally, I recommend increasing the maximum number of concurrent operations vCenter will allow. To do that one must increase the maxCostPerHost advanced setting to 24.

1 – Open the vpxd.cfg file using the text editor.
2 – Add the following entries within the <vpxd> and </vpxd> tags:


3 – Restart vCenter services.


To change the amount of clones modify $clone_count = 400
To change the base clone modify $base_clone = “Win7-View-Gold”
To enable or disable VAAI modify [bool] $vaai = $true
Finally, scripts can always be improved. Share any modification with us.

param( [Parameter(Mandatory=$True)][String]$vcenter, [Parameter(Mandatory=$True)][String]$cluster, [String] $base_clone = "Win7-View-Gold", [Int] $clone_count = 400, [bool] $vaai = $true, [bool] $wait_for_ips = $false)
Add-PSSnapin VMware.VimAutomation.Core
Connect-VIServer $vcenter

$cluster = Get-Cluster | where {$_.name -eq $cluster}
$hosts = Get-VMHost -Location $cluster
$source_vm = Get-VM -Location $cluster | where {$_.name -like $base_clone } | Get-View
$clone_folder = $source_vm.parent
$clone_spec = new-object Vmware.Vim.VirtualMachineCloneSpec
$clone_spec.Location = new-object Vmware.Vim.VirtualMachineRelocateSpec
$clone_spec.Location.Transform = [Vmware.Vim.VirtualMachineRelocateTransformation]::flat
if ($vaai) {
 Write-Host "Cloning VM $base_clone using VAAI."
 $clone_spec.Location.DiskMoveType = [Vmware.Vim.VirtualMachineRelocateDiskMoveOptions]::moveAllDiskBackingsAndAllowSharing
 }else {
 Write-Host "Cloning VM $base_clone without VAAI."
 $clone_spec.Location.DiskMoveType = [Vmware.Vim.VirtualMachineRelocateDiskMoveOptions]::createNewChildDiskBacking
 $clone_spec.Snapshot = $source_vm.Snapshot.CurrentSnapshot

Write-Host "Creating $clone_count VMs from VM: $base_clone"

# Create VMs.
$global:creation_start_time = Get-Date
for($i=1; $i -le $clone_count; $i++){
 $clone_name = "Windows7-bootstorm-$i"
 $clone_spec.Location.host = $hosts[$i % $hosts.count].Id
 $source_vm.CloneVM_Task( $clone_folder, $clone_name, $clone_spec ) | Out-Null

# Wait for all VMs to finish being cloned.
$VMs = Get-VM -Location $cluster -Name "Windows7-bootstorm-*"
while($VMs.count -lt $clone_count){
 $count = $VMs.count
 Write-Host "Waiting for VMs to finish creation. Only $count have been created so far..."
 Start-Sleep -s 5
 $VMs = Get-VM -Location $cluster -Name "Windows7-bootstorm-*"

Write-Host "Powering on VMs"
# Power on newly created VMs.
$global:power_on_start_time = Get-Date
Start-VM -RunAsync "Windows7-bootstorm-*" | Out-Null

$booted_clones = New-Object System.Collections.ArrayList
#$waiting_clones = New-Object System.Collections.ArrayList
while($booted_clones.count -lt $clone_count){
 # Wait until all VMs are booted.
 $clones = Get-VM -Location $cluster -Name "Windows7-bootstorm-*"
 foreach ($clone in $clones){
 if((-not $booted_clones.contains($clone.Name)) -and ($clone.PowerState -eq "PoweredOn")){
 $ip = $clone.Guest.IPAddress[0]
 if ($ip){
 Write-Host "$clone.Name started with ip: $ip"

$global:total_runtime = $(Get-Date) - $global:creation_start_time
$global:power_on_runtime = $(Get-Date) - $global:power_on_start_time

Write-Host "Total time elapsed to boot $clone_count VMs: $global:power_on_runtime"
Write-Host "Total time elapsed to clone and boot $clone_count VMs: $global:total_runtime"
<span style="font-family: Consolas, Monaco, monospace;">

This following script is responsible for deleting all VMs created with the previous script.

param([Parameter(Mandatory=$True)][String]$vcenter, [String] $base_clone = "Win7-View-Gold", [Int] $clone_count = 400, [bool] $vaai = $true)
Add-PSSnapin VMware.VimAutomation.Core
Connect-VIServer $vcenter
Write-Host "Cleaning up after test run"
Write-Host "Powering off $clone_count VMs"
Stop-VM -RunAsync -Confirm:$false "Windows7-bootstorm-*" | Out-Null
Write-Host "Deleting $clone_count VMs"
Remove-VM -RunAsync -Confirm:$false -DeletePermanently:$true "Windows7-bootstorm-*" | Out-Null
Write-Host "Cleanup complete"

This article was first published by Andre Leibovici (@andreleibovici) at myvirtualcloud.net.


1 ping

Skip to comment form

  1. After running this script about 10 times, 4 of my disks were taken offline with the error:

    Disk 4 on node C of block XXXXXXX is marked offline due to IO errors. Serial number of the disk is XXXXX in host 192.168.XXX.XXX of block XXXXXXX.

    I called support and they said there is actually a bug they are tracking:

    “Given your scenario, it is likely the number of delete operations resulted in the higher latencies leading to the disks being marked offline. If you would like us to confirm we will need the /home/nutanix/data/logs/stargate*INFO* logs from the controllers to verify. Engineering is aware of this behavior where a large number of delete operations lead to disks being marked offline and we are tracking a fix for this in bug id ENG-12486.”

    They gave me a command to run to mark the disks back online… at least I could have run it if I hadn’t thought I could run the disk remove / install procedure to fix it.

  2. Hi Josh,

    First of all I am glad that support was able to help you and everything is back to normal. It’s important to know you have good technical support when you need the most. I have run the script multiple times without any issues on a 4 node cluster, but I will track the bug.


  3. This was a test cluster anyway so I wasn’t worried about nuking it. t don’t think it was a bug in the script as much as the IO. I was modifying the script trying to put increasing amounts of IO, rather than finding the quickest clone time. I changed the script so it would make a clone, power it on and run the sys prep guest OS customization. I got up to 16000 IOPS and 375MB CVM bandwidth when the disks were placed offline. This was on a 3 node 1350 block.

    I have another 8 node 3460 cluster that I’m going to try next.

  4. Nice, I envy you… you have more nodes than I will ever have for my tests, tricks and validations.
    Nice you pulling that kind of performance on a 1350 with 3 nodes. Imagine if was a 3651.

  5. This script really stresses the vcenter server so make sure you have plenty of resources on your vcenter server to handle it. This also represents the worse case scenario in a VDI type environment. Actually worse than worse case since I don’t think any of the VDI brokers launch that many concurrent jobs

    • Michael on 02/24/2014 at 12:31 pm

    @ John Sinclair : Can you share the modified script?

  6. I just modified the Create VMs section to create a clone using my Customization Specification, which runs sysprep when the clone fires up.

    # Create VMs.
    $global:creation_start_time = Get-Date
    for($i=1; $i -le $clone_count; $i++){
    if ($i -lt 10) {$clone_name = “CloneTEST-00$i”}
    elseif ($i -lt 100) {$clone_name = “CloneTEST-0$i”}
    else {$clone_name = “CloneTEST-$i”}
    New-VM -vmhost ($hosts[$i % $hosts.count]) -Name $clone_name -Template Win7-Template -Datastore Nutanix01 -OSCustomizationSpec joshsinclair.com -RunAsync | Out-Null
    # $clone_spec.Location.host = $hosts[$i % $hosts.count].Id
    # $source_vm.CloneVM_Task( $clone_folder, $clone_name, $clone_spec ) | Out-Null

  7. Andrei,

    I added a Clone-VM and Unclone-VM cmdlet to my repo in a branch at https://github.com/rnelson0/powercli-modules/tree/Clone/Unclone-VM. Do you or Steven Poitras have any issues if I merge this into the Master branch? I credited the both of you and only tweaked it to make it a more general purpose cmdlet. Thanks!

  8. rnelson0, as long you refer back to the article and provide credits it’a all good.

  1. […] http://myvirtualcloud.net/?p=5924 (@StevenPoitras and @andreleibovici) are a pair of cmdlets useful for stress-testing your storage […]

Leave a Reply

Your email address will not be published.