Automated addition of VMware Disks and Controllers

Automated addition of VMware Disks and Controllers

Bradley Bishop

8 minute read

When adding disks to VMware VM using Python we need to use the pyVmomi VMware SOAP API to communicate with vCenter. pyVmomi gives us a lot of very helpful features that take out a lot of the guess work in adding new disks to a VMware VM.

You can install pyVmomi from pip into a virtualenv like the following:

virtualenv ~/vmware
source ~/vmware/bin/activate
pip install pyvmomi

VMware Connection

pyVmomi makes it really easy to connect to the VMware vCenter environment to query information and perform all the necessary tasks needed.

Example of connecting to VMware vCenter unsecured:

from pyVim.connect import SmartConnect, Disconnect
import requests
import ssl

requests.packages.urllib3.disable_warnings()
context = ssl._create_unverified_context()

si = SmartConnect(host="test.dev",
                  port=443,
                  user="username",
                  pwd="password",
                  sslContext=context)

client = si.RetrieveContent()

Virtual Machine

The client can then be used to perform many different tasks, such as querying for a VM. Once you retrieve a VM object you can get all the information about the VM as well as make changes to the VM. There are many different ways to query for a VM but since the VM names can be duplicated we will use the uuid.

Example of querying for a VMware VM using the client from above:

vm = client.searchIndex.FindByUuid(uuid="0000-1111-2222", vmSearch=True)

VMware Disks

pyVmomi can generate a whole object for you. For a disk we can generate a disk object that has all the default keys that are needed and any values that can be autogenerated. With this we can set the parameters that we know that we want to change and let VMware auto-generate the rest for us. This takes out all of the guess work when creating new disks.

Blank Disk

You can generate a basic disk for a VM using the following code:

from pyVmomi import vim

disk = vim.vm.device.VirtualDisk()
disk.backing = vim.vm.device.VirtualDisk.FlatVer2BackingInfo()
print disk

This produces the following output:

(vim.vm.device.VirtualDisk) {
   dynamicType = <unset>,
   dynamicProperty = (vmodl.DynamicProperty) [],
   key = 0,
   deviceInfo = <unset>,
   backing = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) {
      dynamicType = <unset>,
      dynamicProperty = (vmodl.DynamicProperty) [],
      fileName = '',
      datastore = <unset>,
      backingObjectId = <unset>,
      diskMode = '',
      split = <unset>,
      writeThrough = <unset>,
      thinProvisioned = <unset>,
      eagerlyScrub = <unset>,
      uuid = <unset>,
      contentId = <unset>,
      changeId = <unset>,
      parent = <unset>,
      deltaDiskFormat = <unset>,
      digestEnabled = <unset>,
      deltaGrainSize = <unset>,
      deltaDiskFormatVariant = <unset>,
      sharing = <unset>,
      keyId = <unset>
   },
   connectable = <unset>,
   slotInfo = <unset>,
   controllerKey = <unset>,
   unitNumber = <unset>,
   capacityInKB = 0L,
   capacityInBytes = <unset>,
   shares = <unset>,
   storageIOAllocation = <unset>,
   diskObjectId = <unset>,
   vFlashCacheConfigInfo = <unset>,
   iofilter = (str) [],
   vDiskId = <unset>
}

This gives you all the available options that are needed to successfully create a new disk for a VM. Not all the information is required to be filled out, a lot of the information has the ability to be auto generated when the disk is added to a VMware VM. Some of the options need to be whole objects as well.

Example Disk

Here is an example of basic information needed to create a VMware Disk:

from pyVmomi import vim

disk = vim.vm.device.VirtualDisk()
disk.backing = vim.vm.device.VirtualDisk.FlatVer2BackingInfo()
disk.backing.diskMode = "persistent"
disk.capacityInKB = 1024

# This needs to be the current controller on the VM.
disk.controllerKey = 1

# This must be a VMware datastore object
disk.backing.datastore = vim.Datastore("testname")

# This number is a unique number for the disk on the current controller
disk.unitNumber = 1
print disk

This produces the following output:

(vim.vm.device.VirtualDisk) {
   dynamicType = <unset>,
   dynamicProperty = (vmodl.DynamicProperty) [],
   key = 0,
   deviceInfo = <unset>,
   backing = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) {
      dynamicType = <unset>,
      dynamicProperty = (vmodl.DynamicProperty) [],
      fileName = '',
      datastore = 'vim.Datastore:datastore',
      backingObjectId = <unset>,
      diskMode = 'persistent',
      split = <unset>,
      writeThrough = <unset>,
      thinProvisioned = <unset>,
      eagerlyScrub = <unset>,
      uuid = <unset>,
      contentId = <unset>,
      changeId = <unset>,
      parent = <unset>,
      deltaDiskFormat = <unset>,
      digestEnabled = <unset>,
      deltaGrainSize = <unset>,
      deltaDiskFormatVariant = <unset>,
      sharing = <unset>,
      keyId = <unset>
   },
   connectable = <unset>,
   slotInfo = <unset>,
   controllerKey = 1,
   unitNumber = 1,
   capacityInKB = 1024,
   capacityInBytes = <unset>,
   shares = <unset>,
   storageIOAllocation = <unset>,
   diskObjectId = <unset>,
   vFlashCacheConfigInfo = <unset>,
   iofilter = (str) [],
   vDiskId = <unset>
}

Notes:

  • The disk.backing.datstore needs to be a pyvmomi.vim.Datastore object for the disk to be added properly

  • The controller key can be found by querying for the current controller and getting the key.

    from pyVmomi import vim

# Using the vm object we found above we can query for the controllers.
# This was we can look at all the controllers and get the one we want from the array.
controllers = []
for device in self.vm.config.hardware.device:
if (isinstance(device, vim.vm.device.VirtualSCSIController) or
        isinstance(device, vim.vm.device.ParaVirtualSCSIController)):
    controllers.append(device)

  • There can be a max of 15 disks per controller.

  • The disk.unitNumber must in the range [0, 6] and [8, 15] 7 is reserved.

  • The disk.unitNumber must be unique with respect to all other disks on the controller (ex: there can only be on disk with disk.unitNumber == 5 on a controller).

VMware Controller

We also have a need to add a new controller to a VMware VM when the current controller is full. The process to add a controller is very similar to adding a disk thanks to pyVmomi.

Blank Controller

Example of a blank controller:

from pyVmomi import vim

controller = vim.vm.device.VirtualSCSIController()
print controller

This produces the following output:

(vim.vm.device.VirtualSCSIController) {
   dynamicType = <unset>,
   dynamicProperty = (vmodl.DynamicProperty) [],
   key = 0,
   deviceInfo = <unset>,
   backing = <unset>,
   connectable = <unset>,
   slotInfo = <unset>,
   controllerKey = <unset>,
   unitNumber = <unset>,
   busNumber = 0,
   device = (int) [],
   hotAddRemove = <unset>,
   sharedBus = <unset>,
   scsiCtlrUnitNumber = <unset>
}

This is very similar to the Disks configuration in that most of the options do not need to be filled out. Something to keep in mind is that while the busNumber is initialized at 0 this is already taken by the first controller on the VM. This busNumber can be 0 to 3 since there can only be 4 controllers per VM.

Example Controller

Here is an example of a controller that has been filled in to be added.

from pyVmomi import vim

controller = vim.vm.device.VirtualSCSIController()
controller.sharedBus = 'noSharing'
controller.busNumber = 1
print controller

This produces the following output:

(vim.vm.device.VirtualSCSIController) {
   dynamicType = <unset>,
   dynamicProperty = (vmodl.DynamicProperty) [],
   key = 0,
   deviceInfo = <unset>,
   backing = <unset>,
   connectable = <unset>,
   slotInfo = <unset>,
   controllerKey = <unset>,
   unitNumber = <unset>,
   busNumber = 1,
   device = (int) [],
   hotAddRemove = <unset>,
   sharedBus = 'noSharing',
   scsiCtlrUnitNumber = <unset>
}

Reconfigure VM Task

Once you have your configuration for the addition that you want to make to the VMware VM you need to generate a configuration specification to be passed into a reconfigure VM task to send to VMware. This accepts a list of configurationss that can be done at the same time if you wish. Keep in mind that if you need to generate a new controller to add a disk then that will need to be done before so that you can get auto generated information unless you specify it.

Example of how to add a disk using the VM object from above:

from pyVmomi import vim

# Generate basic Disk information
disk = vim.vm.device.VirtualDisk()
disk.backing = vim.vm.device.VirtualDisk.FlatVer2BackingInfo()
disk.backing.diskMode = "persistent"
disk.capacityInKB = 1024
disk.controllerKey = 1
disk.backing.datastore = vim.Datastore("datastore")
disk.unitNumber = 1

# Generate virtual device Specification
disk_spec = vim.vm.device.VirtualDeviceSpec()
disk_spec.fileOperation = "create"
disk_spec.operation = vim.vm.device.VirtualDeviceSpec.Operation.add

# Add disk to the device spec
disk_spec.device = disk

config_spec = vim.vm.ConfigSpec()
config_spec.deviceChange = [disk_spec]

# This submits the configspec and performs all the tasks contained
vm.ReconfigVM_Task(config_spec)

Config Spec output:

(vim.vm.ConfigSpec) {
   dynamicType = <unset>,
   dynamicProperty = (vmodl.DynamicProperty) [],
   changeVersion = <unset>,
   name = <unset>,
   version = <unset>,
   uuid = <unset>,
   instanceUuid = <unset>,
   npivNodeWorldWideName = (long) [],
   npivPortWorldWideName = (long) [],
   npivWorldWideNameType = <unset>,
   npivDesiredNodeWwns = <unset>,
   npivDesiredPortWwns = <unset>,
   npivTemporaryDisabled = <unset>,
   npivOnNonRdmDisks = <unset>,
   npivWorldWideNameOp = <unset>,
   locationId = <unset>,
   guestId = <unset>,
   alternateGuestName = <unset>,
   annotation = <unset>,
   files = <unset>,
   tools = <unset>,
   flags = <unset>,
   consolePreferences = <unset>,
   powerOpInfo = <unset>,
   numCPUs = <unset>,
   numCoresPerSocket = <unset>,
   memoryMB = <unset>,
   memoryHotAddEnabled = <unset>,
   cpuHotAddEnabled = <unset>,
   cpuHotRemoveEnabled = <unset>,
   virtualICH7MPresent = <unset>,
   virtualSMCPresent = <unset>,
   deviceChange = (vim.vm.device.VirtualDeviceSpec) [
      (vim.vm.device.VirtualDeviceSpec) {
         dynamicType = <unset>,
         dynamicProperty = (vmodl.DynamicProperty) [],
         operation = 'add',
         fileOperation = 'create',
         device = (vim.vm.device.VirtualDisk) {
            dynamicType = <unset>,
            dynamicProperty = (vmodl.DynamicProperty) [],
            key = 0,
            deviceInfo = <unset>,
            backing = (vim.vm.device.VirtualDisk.FlatVer2BackingInfo) {
               dynamicType = <unset>,
               dynamicProperty = (vmodl.DynamicProperty) [],
               fileName = '',
               datastore = 'vim.Datastore:datastore',
               backingObjectId = <unset>,
               diskMode = 'persistent',
               split = <unset>,
               writeThrough = <unset>,
               thinProvisioned = <unset>,
               eagerlyScrub = <unset>,
               uuid = <unset>,
               contentId = <unset>,
               changeId = <unset>,
               parent = <unset>,
               deltaDiskFormat = <unset>,
               digestEnabled = <unset>,
               deltaGrainSize = <unset>,
               deltaDiskFormatVariant = <unset>,
               sharing = <unset>,
               keyId = <unset>
            },
            connectable = <unset>,
            slotInfo = <unset>,
            controllerKey = 1,
            unitNumber = 1,
            capacityInKB = 1024,
            capacityInBytes = <unset>,
            shares = <unset>,
            storageIOAllocation = <unset>,
            diskObjectId = <unset>,
            vFlashCacheConfigInfo = <unset>,
            iofilter = (str) [],
            vDiskId = <unset>
         },
         profile = (vim.vm.ProfileSpec) [],
         backing = <unset>
      }
   ],
   cpuAllocation = <unset>,
   memoryAllocation = <unset>,
   latencySensitivity = <unset>,
   cpuAffinity = <unset>,
   memoryAffinity = <unset>,
   networkShaper = <unset>,
   cpuFeatureMask = (vim.vm.ConfigSpec.CpuIdInfoSpec) [],
   extraConfig = (vim.option.OptionValue) [],
   swapPlacement = <unset>,
   bootOptions = <unset>,
   vAppConfig = <unset>,
   ftInfo = <unset>,
   repConfig = <unset>,
   vAppConfigRemoved = <unset>,
   vAssertsEnabled = <unset>,
   changeTrackingEnabled = <unset>,
   firmware = <unset>,
   maxMksConnections = <unset>,
   guestAutoLockEnabled = <unset>,
   managedBy = <unset>,
   memoryReservationLockedToMax = <unset>,
   nestedHVEnabled = <unset>,
   vPMCEnabled = <unset>,
   scheduledHardwareUpgradeInfo = <unset>,
   vmProfile = (vim.vm.ProfileSpec) [],
   messageBusTunnelEnabled = <unset>,
   crypto = <unset>,
   migrateEncryption = <unset>
}

With this information, we can now effectively add many disks to a VM without worrying about running out of controller space since we can effectively add more controllers when needed.

-Bradley Bishop

comments powered by Disqus