Category: VMware

VMware is a virtualization and cloud computing software vendor based in Palo Alto, California. Founded in 1998, VMware is now a subsidiary of Dell Technologies. VMware bases its virtualization technologies on its bare-metal hypervisor ESX/ESXi in x86 architecture.

VMware vExpert 2024 Award

Hey everyone, let me share some awesome news, I’ve just been awarded the VMware vExpert 2024 for the eighth year in a row! 🏆🎉

Now, if you’re scratching your head wondering what exactly that means, don’t worry, I’ve got you covered. Essentially, the VMware vExpert program is like the club for folks who are really into VMware technologies. And guess what? I’m super honored to be part of that club, especially for the eighth time running!

First things first, I gotta give a massive shoutout to VMware / Broadcom and the amazing vExpert Team. Seriously, these folks are like the unsung heroes behind the scenes, making sure the VMware community stays vibrant and buzzing with excitement. Without their hard work and dedication, none of this would be possible.

I’ve met some incredible folks along the way, fellow vExperts who have become friends, mentors, and colleagues at ITQ, and most importantly, we’ve learned from each other.

So, what does eight years of vExpert status mean to me? Well, for starters, it’s a reminder to never stop pushing myself, to keep learning, growing, and evolving as a technologist. Whether I’m writing blog posts, giving talks at conferences, or just hanging out in the VMware community Slack channel, I’m always looking for ways to give back and pay it forward.

And of course, none of this would be possible without the support of my family, friends, and colleagues. You guys are the real MVPs, always cheering me on, even when I’m knee-deep in a particularly gnarly ESXi upgrade.

So here’s to another year of virtualization adventures, of pushing the boundaries of what’s possible with VMware technologies, and most importantly, of being part of an incredible community that I’m proud to call home.

Thanks again to VMware and the vExpert Team for this amazing honor. Let’s make the next eight years even more epic!

See you next time! 🙂 Thanks for reading my blog.

Aria Orchestrator – Add CD-ROM to a Virtual Machine

In this blog post, we will add a CD-ROM device to a vSphere Virtual Machine in an automated way. This will be done with vRO (vRealize Orchestrator/Aria Automation Orchestrator). The action is used for creating a CD-ROM drive when provisioning a new machine with vRO.

I am doing this blog post because, after a lot of Googling, I could not find a good example or solution online. So it was time to do a blog post after figuring out what I needed to do!

So let’s start the blog post about adding a CD-ROM to a virtual machine.

vRO – Action Code

Here is the vRealize Orchestrator/Aria Automation Orchestrator code for an action. This action creates the specification for adding a CD-ROM to an already running or a new virtual machine. It’s a lot of code for a “simple” CD-ROM drive because, in the vCenter Server interface, it feels like a couple of easy clicks. In the backend it is another story, see the code below. You need to attach a lot of specifications together to add a CD-ROM to a virtual machine.

Action details:

  • Name: createCdDvdDriveSpecification
  • Version: 1.0.0
  • Description: Create the specification for a vSphere CD/DVD drive to add a CD/DVD drive to a virtual machine with the VMware vCenter SDK.
  • Inputs: None
  • Return Type: Any
  • Location: com.bv.vsphere.vm.spec
// Set variable
var deviceConfigSpecs = new Array();
var deviceConfigSpec;

// Add CD-ROM connect spec
var connectInfo = new VcVirtualDeviceConnectInfo();
    connectInfo.allowGuestControl = true;
    connectInfo.connected = false;
    connectInfo.startConnected = true;

// Add CD-ROM backing spec
var backingInfo = null;
    backingInfo = new VcVirtualCdromRemotePassthroughBackingInfo();
    backingInfo.deviceName = "";

// Add Virtual CD-ROM
var cdrom = new VcVirtualCdrom();
    cdrom.backing = backingInfo;
    cdrom.controllerKey = 200;
    cdrom.key = 0;
    cdrom.unitNumber = 0;
    cdrom.connectable = connectInfo;

// Create CD-ROM configuration spec
var deviceConfigSpec = new VcVirtualDeviceConfigSpec();
    deviceConfigSpec.device = cdrom;
    deviceConfigSpec.operation = VcVirtualDeviceConfigSpecOperation.add;
    deviceConfigSpecs[0] = deviceConfigSpec;

// Troubleshooting generated configuration specification
// System.debug(deviceConfigSpec);

// Return specification
return deviceConfigSpec;

vRO – Workflow

This is a part of a larger workflow but it will help you get started. I have listed the most important parts of creating a virtual machine and how to get started. This code is quite identical to changing a virtual machine to add a CD-ROM drive.

// Load module
var vsphereVmSpec = System.getModule("com.bv.vsphere.vm.spec");

// Set variable
var actionName = arguments.callee.name.substr(6);
var deviceConfigSpecs = [];
var deviceConfigSpec;

// Virtual machine spec
var vmConfigSpec = new VcVirtualMachineConfigSpec();
// Lot more stuff here like VM name, resource pool, host etc

// Add CD-ROM
deviceConfigSpec = vsphereVmSpec.createCdDvdDriveSpecification();
deviceConfigSpecs[ii++] = deviceConfigSpec;

// Combine configuration
vmConfigSpec.deviceChange = deviceConfigSpecs;

// Start Virtual Machine creation
try {
    System.log("[" + actionName + "] Starting Virtual Machine creation (" + virtualMachineName +")");
    task = vmFolder.createVM_Task(vmConfigSpec, vmResourcePool, vmHost);
}
catch (exception) {
    throw "[" + actionName + "] exception";
}

// Return VC:Task
return task;

Wrap-up

So this is my technical blog post about adding a CD-ROM to a virtual machine with vRealize Orchestrator (vRO). Hopefully, it is useful for somebody, please respond below if you have any comments or additional information! See you next time! 🙂

NSX Advanced Load Balancer Data Network Issue

When configuring the NSX Advanced Load Balancer for some testing in my Home Lab. I noticed something odd related to the service engines management network and data network settings. After thinking I was crazy… I saw an issue in the interface surrounding the data network configuration which caused the issue. In this short blog post, I will explain what was happening and how to resolve the issue.

Environment

My Home Lab environment was running the following products:

Note: Licenses are provided by the vExpert program (this also includes the NSX Advanced Load Balancer licenses for lab usage).

Data Network Issue

At first, we will go to the location in the interface that causes the issue:

  1. Log in on the web interface.
  2. Navigate to “Infrastructure > Cloud Resources > Service Engine Group“.
  3. Click for example on the “Default-Group” (depending on your configuration).
  4. Go to the section “Placement“.
  5. Check the following setting “Override Data Network“.
  6. Select a network that you want…

Sounds all good so far… but look at the description popup on the last screenshot. Are we configuring the management or data network for the service engines? Because the description and the field tell something different.

Management Network or Data Network?

After verifying what happened to the service engines in the group the management network for the Service Engines is changed. This was noticeable to me because the service engines were not reachable anymore on the management network for the controller.

My conclusion after some testing was that the description field is correct. This setting changes the management network!

How can you verify the changes to the service engine group?

Option 01:

  1. Open a command prompt.
  2. Run the following command “ping %management-ip-address service engine%“.
  3. They are probably not available anymore because they are on the wrong network.

Option 02:

  1. Navigate to the vCenter Server.
  2. Login with your account.
  3. Select the Service Engine virtual machine belonging to the group where you configured this setting.
  4. Check the virtual network cards.
  5. There the management network card is assigned to the “override data network” network.

Wrapup

So that was my blog post about the service engine group data network issue. I hope it was useful for somebody because it took me some hours to figure it out…

This wraps up the blog article hopefully it is useful for somebody, please respond below if you have any comments or additional information! See you next time! 🙂

vRealize Log Insight Domain Login Problem

Today a blog post about vRealize Log Insight domain login problems. After some time I wanted to check my vRealize Log Insight instance but somehow the domain authentication was not working anymore. So it was time for a new blog post about this issue.

Environment

The vRealize Log Insight instance is deployed in my 24×7 Lab environment that is running vExpert licenses. I am analyzing my logging and checking my login attempts in vRealize Log Insight.

So I am running the following version but I have seen this issue reoccurring in the last years:

  • VMware vRealize Log Insight 8.X (vRLI) (my screenshots are from version 8.8.2).
  • Windows Server 2022 Domain Controllers (as domain connected to vRealize Log Insight)

Problem

So let’s start with the actual problem. At first, the domain login is working for months… nothing strange and everything is fine but then one day it doesn’t work anymore. A vRealize Log Insight cluster restart doesn’t help nor do other basic troubleshooting steps. The local account like the admin account is still working, SSH access is also still working. So it is related to the domain accounts that are provided by my Active Directory.

Here is my configuration and screenshots of the domain login problem issue:

Fix

Now it is time to fix the issue. As always not difficult unless you know what is going wrong.

So let’s start with the actual problem, the problem is that every year the certificates of the domain controllers are replaced by my Certificate Authority and extended by a year. At that moment vRealize Log Insight doesn’t trust the domain controllers anymore as a login source.

So how do we fix it? By adding and trusting the new certificate on the vRealize Log Insight appliance. Luckily this can be done with the GUI so it is straightforward.

Procedure

  1. Log in as the local “admin” account on the vRealize Log Insight appliance web interface (TCP 443).
  2. Go to the following location “Configuration > Authentication > Active Directory”.
  3. Click on the following button “Test Connection“.
  4. Accept all the offered certificates.
  5. This must result in the green text “Succeeded“.

Now try to log in again with a domain account, this should be working again (for another year in my case).

Wrapup

So that was my blog post about the vRealize Log Insight domain login problems. I hope it was useful for somebody because I always run into this issue. It happens in my Lab but also with customers that are using the Active Directory integration.

This wraps up the blog article hopefully it is useful for somebody, please respond below if you have any comments or additional information! See you next time! 🙂

HPE ProLiant Removing SD Card iLO Degraded

Recently I was removing an SD card from one of my lab servers but after removing it, the server kept complaining about it. The HPE ProLiant is equipped with HPE Integrated Lights-Out (iLO). This is an out-of-band management system to manage and configure the server. It also is responsible for monitoring the components inside the server.

This means it also monitors the health state of the SD card that is located on the motherboard slot. So when I removed the SD card it just kept checking the health of the component and causing health alerts.

In this blog post, I going to explain what I did to reset the HPE iLO to stop it from monitoring the SD card after permanent removal.

Environment

Here is a short list of information about the HPE ProLiant system that I used for this blog post:

  • Hardware: HPE Proliant DL360e Gen 8
  • HPE ilO version: 4
  • HPE SD card: HP 32GB SD card / Part nr: 700135-001
  • Firmware: HPE iLO version: 2.78
  • Software: VMware ESXi 7.0.3

Location – SD Card

To make the blog post complete I added the motherboard drawing from the HPE manual. The SD card slot is located on the HPE ProLiant DL360e Gen 8 motherboard and the slot is located at number 29 in the drawing below.

Problem – Removing SD card causes degraded state

The issue occurred when the SD card failed. After the SD card failure, I removed the SD card from the system and moved to an SSD-based boot media for VMware ESXi.

I performed some basic troubleshooting like removing the power from the server and restarting the HPE iLO but the health status was still degraded and it was still searching for the SD card.

Here are the error messages in the interface:

  • Error message on the login page: iLO Self-Test report a problem with: Embedded Flash/SD-CARD. View details on Diagnostics page.
  • Error message on diagnostics: Controller firmware revision 2.10.00 NAND read failure

Here are some screenshots related to the error messages:

Resolving – Resetting the SD card slot

Resolving the issue isn’t partially hard… if you know which buttons to push and in what order ;). Before starting, make sure the SD card is removed from the system and that the iLO has been rebooted.

To make sure that everything just works directly… open a clean browser and login into the iLO and directly follow the procedure described below.

Closing words

In the end, it cost me about three hours to get it fixed. The reason why I wanted it so badly fixed was that it kept triggering my monitoring system and that drove me crazy. This server in particular powers on and powers off regularly and during every power cycle, the health state resets and triggers monitoring alerts.

This wraps up the blog article hopefully it is useful for somebody, please respond below if you have any comments or additional information! See you next time! 🙂

NLVMUG 2022 Event

Today a blog about the NLVMUG 2022 event, which I attended on the fourteenth of June 2022. The event took place in the DeFabrique in Utrecht The Netherlands.

Lots of parking spaces and easy to reach as it is centrally located in the country. The location was the same location used for the NLVMUG in 2019 (the last physical VMUG before COVID).

For me personally, this was my first physical industry event since VMworld 2019 in Barcelona Spain. So it was great to be back at a physical conference and to actually meet up with a lot of familiar faces such as ITQ/VMware colleges and community members.

There was a great lineup and a lot of sessions you could choose from. Here is an overview of the available sessions of the NLVMUG 2022:

I personally attended a couple of sessions, as you can see in the photos below. Overall the sessions were good quality and with great content! Very nice to see that a lot of people love to share and spend time on prepare those sessions!

NLVMUG 2022 Photos:

I took some photos of the event to share on my blog to give people an idea of what a VMUG is and maybe inspire them to join the next NLVMUG in 2023 (if there isn’t a new … virus or something else).

So this wraps up my short blog post about the NLVMUG 2022. I would like to thank the NLVMUG team and my employer (ITQ) for making it possible for me to be there!

For me the next physical event will be VMworld … VMware Explore so maybe see you there? See you next time!

VMware Tanzu HAProxy Unattended Deployment

In this blog post, I am going to share my script to automate the HAProxy deployment for Tanzu with vSphere or in short TKGs. Because of my interest in the Tanzu product family, I ended up testing and redeploying parts of TKG many times. To reduce the change of mistakes and improve my speed I automated the HAProxy deployment part. To start with a special thanks to William Lam for this blog post which pointed me in the right direction for automating the HAProxy OVA file.

Goal

The reason behind the creation of the code was the following:

  • Automate as much as possible
  • Standardize deployment
  • Streamline the process
  • Improved the speed of the deployment process

Environment

My environment for performing this unattended deployment of HAProxy is listed below. All additional requirements can be found in the README.md file in the GIT Repository like DNS records etc.

  • Server:
    • VMware ESXi 7.0 Update 3
    • VMware vCenter 7.0 Update 3
  • Workstation:
    • OS: Windows 10
    • Components required: PowerShell and PowerCLI

Recording

Here is a recording of the HAProxy unattended deployment in my lab environment. I have changed the variables in the script to match my environment. You must change the variables in a way so that it matches your environment to perform a successful deployment.

Code

Here is an overview of the code and a link to the GIT repository. Keep in mind to always use the GIT repository version of the code because there could be new improvements.

HAProxy Tanzu Deployment:

<#
    Script: HAProxy Tanzu Deployment
    Author: M. Buijs
    Original concept developed by: William Lam - https://github.com/lamw/vmware-scripts/blob/master/powershell/deploy_3nic_haproxy.ps1
    version: 1.0 - 2021-12-17
    Execution: HAProxy_Deployment.ps1
#>

# Set variables

	# Script variables
	$global:script_name = "HAProxy_Tanzu_Deployment"
	$global:script_version = "v1.0"
	$global:debug = 0
    $global:temp_directory = "C:\Temp\"

    # vSphere
    $vCenter = "LAB-VC01.Lab.local"
    $ClusterName = "Lab"
    $DatastorePrefix = "iSCSI - Production - *" # datastore prefix
    $DiskProvisioning = "thin" # thin or thick
    $Hardware = "v14" # Virtual hardware

    # HAProxy General
    $HAProxyDisplayName = "LAB-HAProxy01"
    $HAProxyHostname = "lab-haproxy01.lab.local"
    $HAProxyDNS = "192.168.126.21, 192.168.126.22"
    $HAProxyPort = "5556" # 5556 default port

    # HAProxy Management
    $HAProxyManagementNetwork = "Management"
    $HAProxyManagementIPAddress = "192.168.151.40/24" # Format is IP Address/CIDR Prefix
    $HAProxyManagementGateway = "192.168.151.254"

    # HAProxy Frontend
    $HAProxyFrontendNetwork = "TKG - Frontend"
    $HAProxyFrontendIPAddress = "192.168.127.40/24" # Format is IP Address/CIDR Prefix
    $HAProxyFrontendGateway = "192.168.27.254"
    $HAProxyLoadBalanceIPRange = "192.168.127.128/26" # Format is Network CIDR Notation

    # HAProxy Workload
    $HAProxyWorkloadNetwork = "TKG - Workload"
    $HAProxyWorkloadIPAddress = "192.168.128.40/24" # Format is IP Address/CIDR Prefix
    $HAProxyWorkloadGateway = "192.168.128.254"

    # HAProxy Users
    $HAProxyUsername = "haproxy_api"

# Functions
function banner {
    # Clear
	Clear-Host

	# Clear errors
	$Error.clear()

    # Message
    Write-Host "`n---------------------------------------------------------" -foreground Red
    Write-Host "               $script_name - $script_version" -foreground Red
    Write-Host "---------------------------------------------------------" -foreground Red
}

function script_exit {
	Write-Host -Foreground Yellow ""
	Write-Host -Foreground Yellow "ERROR Message: $($Error[0].Exception.Message)"
	Write-Host -Foreground Yellow ""
	Write-Host -Foreground Cyan "Exiting PowerShell Script..."
	exit
}

function validate_media {
    ##### Message
    Write-Host "`nValidating media:"

        #### Locate temp directory
        If (-not (Test-Path "$($Temp_Directory)")) {
            Write-Host -ForegroundColor Red "- The temp directory is not created ($Temp_Directory)"
            script_exit
        }
        else {
            Write-Host -ForegroundColor Green "- Located the temp directory ($Temp_Directory)"
        }

        #### Locate OVA file
        Try {
            Write-Host -ForegroundColor Green  "- Searching for OVA file"
            $script:OVF_HAProxy = $(Get-ChildItem -Path "$Temp_Directory" -Include haproxy-v*.ova -File -Recurse -ErrorAction Stop | Sort-Object LastWriteTime | Select-Object -last 1)

            ### In case of no results
            if ([string]::IsNullOrEmpty($OVF_HAProxy.name)) {
                throw
            }
            #### Message
            Write-Host -ForegroundColor Green "- Located HAProxy OVA file ($($OVF_HAProxy.Name))"
        }
        Catch {
            Write-Host -ForegroundColor Red  "- Could not find HAProxy OVA file in location ($Temp_Directory)"
            script_exit
        }
}

function ask_passwords {
    # Banner
    Write-Host "`nPasswords:"

    # Ask passwords
    $script:HAProxyOSPassword = Read-Host -asSecureString "- Enter the HAProxy user password (root)"
    $script:HAProxyPassword = Read-Host -asSecureString "- Enter the HAProxy user password ($HAProxyUsername)"

    # Validation
    If ($HAProxyOSPassword.Length -eq 0) {
        Write-Host -ForegroundColor Red "- HAProxy root account password is empty"
        script_exit
    }
    # Validation
    If ($HAProxyPassword.Length -eq 0) {
        Write-Host -ForegroundColor Red "- HAProxy user account password is empty"
        script_exit
    }
}

function connect_vcenter {
    # Banner
    Write-Host "`nvCenter connection:"

        # Disable vCenter deprecation warnings
        Set-PowerCLIConfiguration -DisplayDeprecationWarnings $false -Confirm:$false | Out-Null

        # Disable vCenter certification errors
        Set-PowerCLIConfiguration -InvalidCertificateAction "ignore" -Confirm:$false | Out-Null

        # Determine script or user input
        if ($vCenter) {
            Write-Host -ForegroundColor Green "- Connecting with vCenter server ($vCenter)"
        }
        else {
            # Ask required vCenter information
            $script:vCenter = Read-Host "- Enter the vCenter IP address or hostname"
        }

        if ($global:DefaultVIServers.Count -gt 0) {
            Write-Host -ForegroundColor Green "- Session already established ($vCenter)"
        }
        else {
            # Check IP address for connectivity
            if (test-connection -computername $vCenter -count 1 -quiet -ErrorAction SilentlyContinue) {
                Write-Host -ForegroundColor Green "- Host is alive ($vCenter)"
            }
            else {
                Write-Host -ForegroundColor Red "- Host is not responding ($vCenter)"
                $vCenter = ""
                Break
            }

            # Connect with vCenter
            try {
                Write-host -ForegroundColor Green "- Connecting to vCenter, please wait..."

                # Connect to vCenter
                Connect-ViServer -server $vCenter -ErrorAction Stop | Out-Null
            }
            catch [Exception]{
                $status = 1
                $exception = $_.Exception
                Write-Host "- Could not connect to vCenter, exiting script" -foreground Yellow
                Write-Host ""
                Write-Host "Exit code: $status" -foreground Yellow
                Write-Host "Output: $exception" -foreground Yellow
                Break
            }
        }

        # Message
        Write-Host -ForegroundColor Green "- Connection successful"
}

function ovf_config {
    # Banner
    Write-Host "`nOVF Configuration:"

    # Start
    Write-Host -ForegroundColor Green "- Creating OVF Configuration"

    $script:ovfconfig = Get-OvfConfiguration $OVF_HAProxy

    # Three nic configuration
    $script:ovfconfig.DeploymentOption.value = "frontend"

    # General
    $script:ovfconfig.network.hostname.value = $HAProxyHostname
    $script:ovfconfig.network.nameservers.value = $HAProxyDNS
    $script:ovfconfig.loadbalance.dataplane_port.value = $HAProxyPort

    # Network port groups
    $script:ovfconfig.NetworkMapping.Management.value = $HAProxyManagementNetwork
    $script:ovfconfig.NetworkMapping.Frontend.value = $HAProxyFrontendNetwork
    $script:ovfconfig.NetworkMapping.Workload.value = $HAProxyWorkloadNetwork

    # Management
    $script:ovfconfig.network.management_ip.value = $HAProxyManagementIPAddress
    $script:ovfconfig.network.management_gateway.value = $HAProxyManagementGateway

    # Workload
    $script:ovfconfig.network.workload_ip.value = $HAProxyWorkloadIPAddress
    $script:ovfconfig.network.workload_gateway.value = $HAProxyWorkloadGateway
    $script:ovfconfig.loadbalance.service_ip_range.value = $HAProxyLoadBalanceIPRange

    # Accounts
    $script:ovfconfig.loadbalance.haproxy_user.value = $HAProxyUsername

    # Password root
    $BSTR1 = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($HAProxyOSPassword)
    $HAProxyOSPassword = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($BSTR1)
    $script:ovfconfig.appliance.root_pwd.value = $HAProxyOSPassword

    # Password user
    $BSTR2 = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($HAProxyPassword)
    $HAProxyPassword = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($BSTR2)
    $script:ovfconfig.loadbalance.haproxy_pwd.value = $HAProxyPassword

    # Finish
    Write-Host -ForegroundColor Green "- Completed OVF Configuration"
}

function pre_deployment {
    # Banner
    Write-Host "`nPre-deployment:"

    # Cluster
    $script:Cluster = Get-Cluster $ClusterName
    Write-Host -ForegroundColor Green "- Selected cluster ($Cluster)"

    # VMhost
    $script:VMHost = Get-VMHost | Where-Object { $_.ConnectionState -eq "Connected" } | Get-Random
    Write-Host -ForegroundColor Green "- Selected ESXi Host ($VMHost)"

    # Datastore
    $script:Datastore = Get-VMhost -Name $VMHost | Get-Datastore -Name $DatastorePrefix | Select-Object Name, FreeSpaceGB | Sort-Object FreeSpaceGB -Descending | Select-Object -first 1 | Select-Object Name -expandproperty name
    Write-Host -ForegroundColor Green "- Selected datatore ($Datastore)"

    # Check virtual machine name exists
    $VMname_check_query = Get-Cluster -Name $ClusterName | Get-VM -name $HAProxyDisplayName -ErrorAction SilentlyContinue

    if (! $VMname_check_query) {
        Write-Host -ForegroundColor Green "- Virtual machine name is not in use ($HAProxyDisplayName)"
    }
    else {
        Write-Host -ForegroundColor Red "- Virtual Machine with name ($HAProxyDisplayName) already exists. Exiting script cannot continue!"
        script_exit
    }

	#### Ask for conformation
	Write-Host "`nThis task is going to build the HAProxy virtual machine for TKGs."
	$confirmation = Read-Host "Are you sure you want to proceed? [y/n]"

	if ($confirmation -eq 'n') {
		Write-Host "Operation cancelled by user!" -Foreground Red
		base_exit
	}

	if (!$confirmation) {
		Write-Host -Foreground Red "No input detected!"
	    base_exit
	}
}

function deployment {
    # Banner
    Write-Host "`nDeployment:"

	# HAProxy deployment of OVF
	try {
		### Message
		Write-Host -ForegroundColor Green "- Starting HAProxy Deployment ($HAProxyHostname / $HAProxyManagementIPAddress)"

        $script:vm = Import-VApp -Source $OVF_HAProxy -OvfConfiguration $ovfconfig -Name $HAProxyDisplayName -Location $Cluster -VMHost $VMHost -Datastore $Datastore -DiskStorageFormat $DiskProvisioning

        ### Message
		Write-Host -ForegroundColor Green "- Finished HAProxy Deployment ($HAProxyHostname / $HAProxyManagementIPAddress)"
    }
	catch [Exception]{
		Write-Host -ForegroundColor Red "- HAProxy Deployment Failed ($HAProxyHostname / $HAProxyManagementIPAddress)"
		script_exit
	}
}

function post_deployment {
    # Banner
    Write-Host "`nPost-deployment:"

	# Configure OVF
	try {
		### Message
		Write-Host -ForegroundColor Green "- Starting HAProxy OVF Configuration ($HAProxyHostname / $HAProxyManagementIPAddress)"

        $vappProperties = $vm.ExtensionData.Config.VAppConfig.Property
        $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
        $spec.vAppConfig = New-Object VMware.Vim.VmConfigSpec

        $ovfChanges = @{
            "frontend_ip"=$HAProxyFrontendIPAddress
            "frontend_gateway"=$HAProxyFrontendGateway
        }

        ### Message
		Write-Host -ForegroundColor Green "- Finished HAProxy OVF Configuration ($HAProxyHostname / $HAProxyManagementIPAddress)"
    }
	catch {
		Write-Host -ForegroundColor Red "- HAProxy OVF Configuration failed ($HAProxyHostname / $HAProxyManagementIPAddress)"
		script_exit
	}

    try {
        # Message
		Write-Host -ForegroundColor Green "- Starting HAProxy Update Specification ($HAProxyHostname / $HAProxyManagementIPAddress)"

        # Retrieve existing OVF properties from VM
        $vappProperties = $VM.ExtensionData.Config.VAppConfig.Property

        # Create a new Update spec based on the # of OVF properties to update
        $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
        $spec.vAppConfig = New-Object VMware.Vim.VmConfigSpec
        $propertySpec = New-Object VMware.Vim.VAppPropertySpec[]($ovfChanges.count)

        # Find OVF property Id and update the Update Spec
        foreach ($vappProperty in $vappProperties) {
            if($ovfChanges.ContainsKey($vappProperty.Id)) {
                $tmp = New-Object VMware.Vim.VAppPropertySpec
                $tmp.Operation = "edit"
                $tmp.Info = New-Object VMware.Vim.VAppPropertyInfo
                $tmp.Info.Key = $vappProperty.Key
                $tmp.Info.value = $ovfChanges[$vappProperty.Id]
                $propertySpec+=($tmp)
            }
        }
        $spec.VAppConfig.Property = $propertySpec

        # Message
		Write-Host -ForegroundColor Green "- Finished HAProxy Update Specification ($HAProxyHostname / $HAProxyManagementIPAddress)"
    }

    catch {
        # Message
        Write-Host -ForegroundColor Red "- HAProxy Update Specification failed ($HAProxyHostname / $HAProxyManagementIPAddress)"
		script_exit
    }

    # HAProxy reconfigure task for virtual machine
    try {
        # Message
        Write-Host -ForegroundColor Green "- Start Reconfigure VM task ($HAProxyHostname / $HAProxyManagementIPAddress)"
        $task = $vm.ExtensionData.ReconfigVM_Task($spec)
        $task1 = Get-Task -Id ("Task-$($task.value)")
        $task1 | Wait-Task | Out-Null
    }
    catch {
        Write-Host -ForegroundColor Red "- Reconfigure VM task failed ($HAProxyHostname / $HAProxyManagementIPAddress)"
        script_exit
    }

    # Message
    Write-Host -ForegroundColor Green "- Completed the reconfigure VM task ($HAProxyHostname / $HAProxyManagementIPAddress)"
}

function boot {
    # Banner
    Write-Host "`nBoot:"

	# Upgrade Virtual Hardware
	Try {
		Write-Host -ForegroundColor Green "- Upgrade Virtual Hardware ($HAProxyHostname / $HAProxyManagementIPAddress)";
		Get-VM -Name $vm | Set-VM -Version $Hardware -Confirm:$false | Out-Null
	}
	Catch {
		Write-Host -ForegroundColor Red "- Upgrade Virtual Hardware failed ($HAProxyHostname / $HAProxyManagementIPAddress)";
		script_exit
	}

	# Power-On Virtual Machine
	Try {
		Write-Host -ForegroundColor Green "- Power-on HAProxy started ($HAProxyHostname / $HAProxyManagementIPAddress)"
		Get-VM $vm | Start-VM | Out-Null
	}
	Catch {
		Write-Host -ForegroundColor Red "- Starting HAProxy failed ($HAProxyHostname / $HAProxyManagementIPAddress)"
		script_exit
	}

    Write-Host -ForegroundColor Green "- Power-on HAProxy completed ($HAProxyHostname / $HAProxyManagementIPAddress)"
}

function check {
    # Banner
    Write-Host "`nCheck:"

    # Set total of retries
    $TOTAL = "10"

    # Host retry interval (seconds)
	$HOST_WAIT = "10";

    # Start loop
    For ($i=0; $i -le $TOTAL; $i++) {

        # Number conversion to 2 digit:
        $NUMBER = [INT]$i + 1
        $NUMBER = "{0:D2}" -f $NUMBER

        # Check Host
        $Host_check_query = Test-Connection -computername $HAProxyHostname -count 1 -quiet -ErrorAction SilentlyContinue

        # Validate, else retry after a wait
        if ($Host_check_query -eq $false) {
            Write-Host -Foregroundcolor green "- [$NUMBER/$TOTAL] Checking HAProxy availability ($HAProxyHostname)"
            Start-Sleep $HOST_WAIT
        }
        else {
            Write-Host -Foregroundcolor green "- [$NUMBER/$TOTAL] Checking HAProxy availability ($HAProxyHostname)"
            Write-Host -Foregroundcolor green "- [Ready] HAProxy is available ($HAProxyHostname)"
            break
        }
    }
}

function retrieve_certificate {
    # Banner
    Write-Host "`nRetrieve certificate:"

    # Build URL
    $script:url = "https://${HAProxyHostname}:${HAProxyPort}/v2/info"

    # Configure local system
    try {
        # Message
        Write-Host -ForegroundColor Green "- Disable certificate checking on local system"

        # Disable certificate check
        [System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
    }
	catch {
		Write-Host -ForegroundColor Red "- Could not disable certificate checking on local system"
		script_exit
	}

    # Download certificate
    try {
        # Message
        Write-Host -ForegroundColor Green "- Get HAProxy certificate ($url)"

        $req = [Net.HttpWebRequest]::Create($url)
        $req.ServicePoint | Out-Null

        # Authentication
        $req.Credentials = New-Object Net.NetworkCredential($HAProxyUsername, $HAProxyPassword);
    }
	catch {
		Write-Host -ForegroundColor Red "- Could not get HAProxy Certificate ($url)"
		script_exit
	}

    # Store error messages in variable to not crash a try and catch statement.
    $GetResponseResult = $req.GetResponse()

    # Store certificate as X.509 file
    try {
        # Message
        Write-Host -ForegroundColor Green "- Store HAProxy certificate as X.509 ($url)"

        $cert = $req.ServicePoint.Certificate
        $bytes = $cert.Export([Security.Cryptography.X509Certificates.X509ContentType]::Cert)
        set-content -value $bytes -encoding byte -path "$pwd\$HAProxyHostname.cer"
    }
    catch {
        Write-Host -ForegroundColor Red "- HAProxy X.509 certificate could not be saved ($url)"
        Write-Host -ForegroundColor Red "- Result from GetResponse: ($GetResponseResult)";
        script_exit
    }

    # Convert certificate to Base-64 file
    try {
        # Message
        Write-Host -ForegroundColor Green "- Store HAProxy certificate as Base-64 ($url)"

        $InsertLineBreaks=1
        $sMyCert="$pwd\$HAProxyHostname.cer"
        $oMyCert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($sMyCert)
        $oPem = New-Object System.Text.StringBuilder
        $oPem.AppendLine("-----BEGIN CERTIFICATE-----") | Out-Null
        $oPem.AppendLine([System.Convert]::ToBase64String($oMyCert.RawData,$InsertLineBreaks)) | Out-Null
        $oPem.AppendLine("-----END CERTIFICATE-----") | Out-Null
        $oPem.ToString() | out-file "$pwd\$HAProxyHostname.pem"
    }
    catch {
        Write-Host -ForegroundColor Red "- HAProxy Base-64 certificate could not be saved ($url)"
        script_exit
    }
}

function complete_banner {
    # Message
    Write-Host -ForegroundColor Green "- HAProxy deployment completed successfully! ($HAProxyHostname / $HAProxyManagementIPAddress)"
}

##### Main
banner
validate_media
connect_vcenter
ask_passwords
ovf_config
pre_deployment
deployment
post_deployment
boot
check
retrieve_certificate
complete_banner

Wrap-up

I hope this blog about HAProxy unattended deployment was useful for some people or that the PowerShell code inspires people to deploy other types of OVA appliances.

If you got any improvements please commit them to the GIT repository and if you got any questions please respond below. Thank you for reading my blog post and see you next time!

VMware Tanzu HAProxy Troubleshooting

This blog post is dedicated to HAProxy Troubleshooting for vSphere with Tanzu or also known as TKGs. Based on your configuration and deployment and the various items you need to configure you can make mistakes or items are not correctly configured. In my case, there were multiple problems at different deployments with parameters and reachability related to the network. In the end, after all the hours of troubleshooting, I ended up with a list of commands that might help others out. So that is the topic of this blog post.

HAProxy Background

First an introduction about the product HAProxy. HAProxy is a load balancer that is used by vSphere with Tanzu. This is not mandatory but is a product to choose from. The main reason for HAProxy compared to the others is that it is completed free/open-source. The HAProxy OVA is packaged and delivered by VMware and can be found in the following repository. All commands below have been tested against the HAProxy v0.2.0 version (haproxy-v0.2.0.ova) that is at the moment of writing the most recent version available.

Appliance access (SSH)

After a successful deployment, you can access the HAProxy appliance with an SSH session. This session can be established with a tool like PuTTY. The user account that should be used in the root account.

Keep in mind: Do not change configuration unless you absolutely know what you are doing. Almost all the issues I ran into were related to entering incorrect information into the deployment wizard or firewall issues.

Troubleshooting Services

One of the first things to check at first is that all services are running on the HAProxy appliance. When services are not started this is mostly caused by an invalid/incomplete configuration that is filled by the deployment wizard of the OVA.

### Check failed services
systemctl list-units --state=failed

### Check primary services for HAProxy and Tanzu Integration
systemctl status anyip-routes.service
systemctl status haproxy.service

### Restart services
systemctl restart haproxy

Troubleshooting Configuration Files

There are multiple configuration files in use by HAProxy here are the most important ones. Also, keep in mind what I already said before… do not change anything unless…

### Anyip-routes configuration file
cat /etc/vmware/anyip-routes.cfg

### HAProxy configuration file
cat /etc/haproxy/haproxy.cfg

### HAProxy dataplane api configuration file
cat /etc/haproxy/dataplaneapi.cfg

### Validation of configuration file
haproxy -c -f /etc/haproxy/haproxy.cfg

Troubleshooting HAProxy process output

Sometimes it is good to check the latest messages generated by the HAProxy process. There will be information about the startup of the process and the pool members.

### Show logging
journalctl -u haproxy.service --since today --no-pager

Troubleshooting IP Settings

By entering wrong IP information in the deployment wizard the configuration files surrounding the IP address settings, gateway, etc can be configured incorrectly. What I noticed is there is not really a check inside the deployment that verifies if the address that is entered is valid in any sort of way.

### List IP Settings
ifconfig

### Config files (incase of three NIC configuration)
cat /etc/systemd/network/10-frontend.network
cat /etc/systemd/network/10-workload.network
cat /etc/systemd/network/10-management.network

### Routing check
route
ip route

Troubleshooting Certificates

Certificates files used by the HAProxy application are inside the HAProxy directory on the local system. The certificates are BASE-64 encoded!

### Certificate authority file:
cat /etc/haproxy/ca.crt

### Certificate server file:
cat /etc/haproxy/server.crt

### Certificate URL by default:
https://%HAProxy-Management-IP%:5556

Troubleshooting NTP

One of the all-time favorites that are notorious for disrupting IT systems is off course NTP. Here are some commands for troubleshooting on Photon OS.

### Check service status
systemctl status systemd-timesyncd

### Show NTP peers
ntpq -p

### Restart service
systemctl restart systemd-timesyncd

### Configuration file
cat /etc/systemd/timesyncd.conf

Troubleshooting the HAProxy API

The HAProxy API is used by Tanzu to configure HAProxy for the management and workload components. Authentication is set up when deploying the OVA and the credentials are entered in the wizard. With the second URL you can verify those credentials:

### Info page
https://%IP-address%:5556/v2/info

### Authentication should work with the HAProxy user account (specified in the deployment wizard)
https://%IP-address%:5556/v2/cluster

Wrapup

Thank you for reading this blog post about HAProxy troubleshooting for vSphere with Tanzu or in short TKGs. I hope it was useful to you! If you got something to add? Have additional tips or remarks please respond in the comment section below.

Have a nice day and see you next time.

Source

PowerCLI Update Fails on Certificate Issue

Today I was updating my PowerShell modules on my local system but I ran into an issue related to certificates. After some investigation, it appeared that the PowerCLI Update was failing. Every PowerCLI module that it tried to update returned an error and did not update to a newer version. In this blog post, I am showing you how to get rid of this error and you will be able to update PowerCLI again to the latest version.

Environment

Here is a quick summary of my environment where the issue occurred:

  • Current PowerCLI: 12.1.0.16997004
  • The system tried to update PowerCLI to: 12.4.0.18627054
  • Operating System: Windows 10 Pro (21H1)

Problem

Here is an overview of the PowerShell command I used to update my modules. Also, the error message is listed below in the code box. As you can see PowerShell is complaining about a DigiCert certificate and that it is not save to update because of the certificate change.

# Open a PowerShell command prompt with administrative permissions

# Enter the command to update the PowerShell modules
Update-Module

# The following error message appears
Install-Package: Authenticode issuer 'CN=DigiCert Trusted Root G4, OU=www.digicert.com, O=DigiCert INc, C=US' of the new module 
'VMware.VimAutomation.Sdk' with version '12.4.0.18627054' is not matching with the authenticode issuer 'CN=VeriSign Class 3 Public Primary Certification 
Authority - G5, OU="(c) 2006 VeriSign, Inc. - for authorized use only", OU=VeriSign Trust Network, O="VeriSign, Inc.", C=US' of the previously-installed
 module 'VMware.VimAutomation.Sdk' with version '12.1.0.16997004'.

Here is a screenshot of the error message, error message is listed in red text. It also shows the commands I used for the PowerCLI update on my local system.

Solution

The solution for fixing the issue is quite simple. Just reinstall the VMware PowerCLI modules on your local system. This needs to be done in a forced way but after that, you are done.

# Open a PowerShell command prompt with administrative permissions

# Enter the command to re-install the PowerShell modules
Install-Module VMware.PowerCLI -Force -SkipPublisherCheck

# Wait a couple of minutes and everything should be upgraded.
Get-Module -ListAvailable VMware.PowerCLI | Select-Object Name, Version

As you can see in the screenshots below the PowerCLI update is working and is returning no errors after the upgrade.

Wrap-up

This wraps up this small blog post about a PowerCLI update issue in PowerShell. Thank you for reading and I hope it helped you out. Please respond in the comment section below if you have any questions or remarks!

vRealize Orchestrator 8.X – Input Form Dropdown

Today a basic tutorial on vRealize Orchestrator 8.X drop-down boxes in a form. With a basic drop-down box, you can improve the user experience in selecting and requesting items from your cloud management portal (CMP). By using drop-down boxes you can leverage easy validation and responses based on other drop-down boxes in your form.

In this tutorial, we are going to create dropdown boxes that respond to each other based on the user’s selection. This can be handy for improving the user experience. Sometimes the list can become very big with numerous options. By sub-selecting a group and filtering to a smaller list of options the user can easier make his decision.

Keep in mind:

  • This tutorial is focused on vRealize Orchestrator 8.X but can still be leveraged in vRealize Orchestrator 7.X with some minor modifications.
  • This tutorial is also usable for vRealize Automation 8.X forms. This can be leveraged by the Service Broker component by importing vRealize Orchestrator workflows.

Use Case

To give you some background around the code and usability. Let’s assume we are developing a workflow for creating Virtual Machines in vSphere. Based on user input surrounding the Operating System information we can determine the type of virtual machine that will be created when the request is submitted. We can also limit the user to some standard options like only Windows 10 or Windows Server 2019.

Keep in mind: this blog post is only focused on the form part, not on the actual creation of the virtual machine in vSphere.

vRO Actions

The first action we are going to create is called “formVmOsFamily“. This will display three values in the form. Based on what you select here the second action will be triggered.

/*
Script name: formVmOsFamily

Inputs:
- None

Return Type:
- vRO 8.X: string:array

Description field:
Author: M. Buijs - ITQ
Developed by: M. Buijs - ITQ
Date: 2021-08-17
Version: 1.0.0

Description: This action returns the available Guest Family of the Operating Systems.
*/


// Operating System Family list
return [
    "Linux",
    "VMware",
    "Windows"
];

Here is the second action that is called “formVmOsGuest“. This will respond to the input provided by the operating system family in the interface.


/*
Script name: formVmOsGuest

Inputs:
- osFamily (string) = Operating System Family

Return Type:
- vRO 8.X: string:array

Description field:
Author: M. Buijs - ITQ
Developed by: M. Buijs - ITQ
Date: 2021-08-05
Version: 1.0.0

Description: This action returns the available Guest Operating Systems.
*/

// Input validation
if (osFamily == "" || osFamily == null) {
    return ["Please select the Operating System family first"];
}

// Linux
if (osFamily == "Linux")
return [
    "CentOS 6 (64-Bit)",
    "CentOS 7 (64-Bit)",
    "CentOS 8 (64-Bit)",
    "Debian 10 (64-Bit)"
];

// VMware
if (osFamily == "VMware")
return [
    "VMware ESXi 6.0",
    "VMware ESXi 6.5",
    "VMware ESXi 6.7",
    "VMware ESXi 7.0"
];

// Windows
if (osFamily == "Windows")
return [
    "Windows 10 (64-Bit)",
    "Windows Server 2016 (64-Bit)",
    "Windows Server 2019 (64-Bit)",
    "Windows Server 2022 (64-Bit)"
];

Here is an overview of screenshots of how it should look like when created the actions in vRealize Orchestrator:

vRO Workflow

Here is the vRealize Orchestrator workflow, I have created an empty workflow and only configured the input form dropdown part! This will help you to set up the workflow so that the actions will work in your environment. The important part is not to forget to configure the workflow inputs and listed below:

Inputs:

  • virtualMachineOsFamily (string)
  • virtualMachineOsGuest (string)

Recording

Here is a recorded video of the input form dropdown boxes in action. The video demonstrates the capability of dropdown boxes and what they can deliver for a customer. It also gives you an idea of what you will get after creating the workflow and actions.

Improvements/Guidance

If you are going to use this code in production… You might need to consider some important points:

  • Always create a list of supported Operating Systems that are used and allowed to be used in your Company. More options will not always simplify the deployment for an end-user.
  • You could store the values in a vRealize Orchestrator Configuration Element depending on how frequently this list is changed.

Summary

So this concludes my blog post about creating dropdown boxes in vRealize Orchestrator and reacting on the input. Hopefully, this was useful for somebody getting started with interfaces in vRealize Automation (vRA) or vRealize Orchestrator (vRO). Please respond in the comment section below if you have any questions or remarks!