VMGuru.com - Home of the Virtualization Gurus

Updated to fix a minor bug on April 16, 2010

I took it as a personal challenge this week to leverage PowerShell, PowerCLI and the vSphere API to track the amount of data that changes in a VM over a regular interval window.  It turns out VMware does make it quite simple to query what blocks in a VMDK file have changed and what the length of data is as long as you know how to structure the API call.

The following script can be run as a scheduled task in Windows on a regular interval such as every 15 or 60 minutes, or even just once a day (depending on how granular you want your data) against a single VM.  This is perfect for profiling the amount of data a particular VM will need to send (uncompressed) over a network connection for backup or replication purposes. 

There are a few quick things to note about this script:

1. It requires vSphere and CBT to be enabled for the selected virtual machine.  This also means the VM Hardware version must be set to 7.  There is an optional flag of "-enableCBT $true" that can be set as a script parameter to enable CBT for the specified VM.  This executes an additional function in the script to turn CBT on.  If CBT is already enabled, the function simply exits cleanly, no harm, no foul.
2. I don't recommend running this script against a VM if there is potential overlap with other regular processes such as backup or replication that add snapshots to virtual machines.  I don't have advanced snapshot debugging in here and simply don't know how all scenarios of snapshots being added/removed from multiple programs against a single VM will play out.
3. I have it limited so a single instance of the script can be run against a single VM only.  I started off with this being a simple proof-of-concept script, and haven't built in proper multi-VM intelligence yet.  If there is enough demand, I'll definitely consider it.

You should create a standalone directory somewhere on an available hard drive of a Windows system.  Create a new PS1 file using the built in script editor in Virtualization EcoShell. Copy and paste the following code block and save the file as "CBT_Tracker.ps1" in the newly created directory:

#(c)2010 Scott Herold - VMGuru.com
#Quest Software
Param (
    [string] $vmName,
    [string] $vimHost,
    [string] $vimUser,
    [string] $vimPass,
    [Boolean] $enableCBT = $false

)
#Use the optional $enableCBT paramater at input to enable CBT on
#a compatible virtual machine (HW Version 7). In order for CBT flags to take
#effect, a Snapshot must be added and removed.
function Add-CBTFlag {
    if ($vmv.Config.ChangeTrackingEnabled){
        "CBT is already enabled on $vm"
        return
    }
    $vmConfigSpec = New-Object VMware.Vim.VirtualMachineConfigSpec
    $vmConfigSpec.changeTrackingEnabled = $true
    $vmv.ReconfigVM($vmConfigSpec)
    sleep 3
    $vm | New-Snapshot -Name "Enable CBT"
    sleep 5
    $vm | Get-Snapshot -Name "Enable CBT" | Remove-Snapshot -Confirm:$false
    "Enabled CBT for $vm"
}

#Add a snapshot specific to the purposes of this script
function Add-CBTSnap {
    $newSnap = $vm | New-Snapshot -Name "CBT Change Tracker"
    return $newSnap
}

#Remove the snapshot created for the purpose of this script
function Remove-CBTSnap {
    param (
        $vmSnap
    )
    $vmSnap | Remove-Snapshot -Confirm:$false
}

#Captures the ChangeID of the snapshot, which is required for comparisson
#tracking purposes.  The current run of the script will always compare
#current state against the previously logged ChangeID for differences.
function Get-ChangeID {
    param (
        $vmSnap
    )
    $vmSnapv.Config.Hardware.Device | where {($_.GetType()).Name -eq "VirtualDisk"} | ForEach-Object {
        $vzCBT = New-Object PSObject
        $vzCBT.PSObject.TypeNames.Clear()
        $vzCBT.PSObject.TypeNames.Insert(0,"vzCBTTracker")
        $vzCBT `
            | Add-Member -MemberType NoteProperty -Name VMName -Value $vm.Name -PassThru `
            | Add-Member -MemberType NoteProperty -Name DeviceID -Value $_.Key -PassThru `
            | Add-Member -MemberType NoteProperty -Name ChangeID -Value $_.Backing.Get_ChangeID()
        [array] $vzCBTColl += $vzCBT
    }
    return $vzCBTColl
}

#Writes the historic ChangeID and VMDK device information to a tracking file
#This file is read on each iteration of the script to capture the previos
#iterations ChangeID inforation.
function Write-TrackerLog {
    $output = $null
    $cid | ForEach-Object {
        $output += $_.VMName + "," + $_.DeviceID + "," + $_.ChangeID + "`n"
    }
    $output | Out-File "$vm-CBTTracker.cfg"
}

#Reads previous state information of the VM from the Tracker.cfg file
#Captures the list of changed blocks for the specified VM for each VMDK file
#calculates the total size in bytes of changed block data across all VMDK
#files associated with the VM.
function Get-ChangedBlockSize {
    $chSize = 0
    Get-Content "$vm-CBTTracker.cfg" | ForEach-Object {
        if ($_ -ne "") {
            $TrackerRef = $_.split(",")
            $changes = $vmv.QueryChangedDiskAreas($vmSnapv.MoRef,$TrackerRef[1],0,$TrackerRef[2])
            $changes.ChangedArea | ForEach-Object {
                $chSize += $_.Length
            }
        }
    }
    $chSize = $chSize/1024/1024
    return $chSize
}

#Writes the actual iterations change data out to a CSV file
function Write-TrackerCSV {
    $longDate = Get-Date
    $shortDate = $longDate.toShortDateString() + " - " + $longdate.toShortTimeString()
    Add-Content "$vm-CBTTracker.csv" "$vm,$shortDate,$amtChanged"
}

#Loads VMware's PowerCLI PowerShell Cmdlets if necessary
[void](Get-PSSnapin VMWare.VimAutomation.Core -ErrorVariable getVmwareSnapinErr 2> $null)
if ($getVmwareSnapinErr.Count -gt 0) {    Add-PSSnapin VMware.VimAutomation.Core }

#Establish a connection to the VI Server based on input parameters
Connect-VIServer $vimHost -User $vimUser -Password $vimPass

#Load some VM variables
$vm = Get-VM $vmName
$vmv = $vm | Get-View

#We can only run against a single VM at this point in time.  This makes sure
#the object being passed is a single VirtualMachineImpl object, and not a
#collection.
if (($vm.GetType()).Name -ne "VirtualMachineImpl"){
    "Script can only run against a single VM at a time"
    return
}

#Checks to make sure VM Hardware version is 7.  Terminates if not.
if ($vmv.Config.Version -ne "vmx-07"){
    "The Virtual Machine $vm must be VM Hardware Version 7 to support CBT"
    return
}

#If the "-enableCBT $true" flag is set as an input parameter, enables CBT
#for specified VM if necessary
if ($enableCBT){
    Add-CBTFlag
}

#If all other checks fail, this is a final failsafe to ensure CBT is enabled
#on the VM before continuing the script.
if ($vmv.Config.ChangeTrackingEnabled -eq $false){
    "CBT is not enabled on $vm.  Script cannot proceed."
    return
}

#Load up snapshot variables
$vmSnap = Add-CBTSnap
$vmSnapv = $vmSnap | Get-View
$amtChanged = 0

#Checks to see if the script has previous written out a CBTTracker.cfg file. This
#would indicate file deletion, or first script run.  If cfg file doesn't exist,
#initializes CSV headings.  If this is an iterative run, Gets the amount of
#data that has changed since previous iteration.  Writes to CSV file.
if (Test-Path "$vm-CBTTracker.cfg") {
    $amtChanged = Get-ChangedBlockSize
    Write-TrackerCSV
}
else {
    Add-Content "$vm-CBTTracker.csv" "VMName,Time,ChangedMB"
}

#Capsture the current ChangeID to use against next iteration, Remove Snapshot
#Write cfg data.
$cid = Get-ChangeID ($vmSnap)
Remove-CBTSnap ($vmSnap)
Write-TrackerLog
 

With the script file created, use notepad or other simple text editor to create a new CMD file called "CBT_Tracker.cmd".  This is what you will execute within Windows Task Scheduler.  Paste the following line in the CMD file and edit with the proper parameters for your environment.  Make sure you copy exactly, including the quotes.

powershell -command "& '.\CBT_Tracker.ps1' -VMName VMName -vimHost vCenter.domain.com -vimuser domain\user -vimpass Passw0rd"

You can now fire up Windows Task Scheduler and point to the CMD file.  After the script runs for the first time, you will see 2 new files per VM that you set this to run against.  $VMName-CBT_Tracker.cfg is used for tracking previous snapshot history for reference in the current script iteration.  $VMName-CBT_Tracker.csv is the actual data file that contains the VMName, Timestamp, and Amount of data changed.

The CSV/Excel combination will let you do some creative things with the data output.