Author: Mischa Buijs

Ubiquiti UniFi Inform/Adopt Access Point

Today we will look at the Ubiquiti UniFi inform/adopt access point process. Recently I bought some new UniFi equipment for my home environment but I had some issues with assigning them to a new UniFi Controller. Also, I received a secondhand unit that was already configured to an existing UniFi Controller. Because it took me some time to figure out the complete process and some items were not directly clear to me it was time for a quick write-up.

This blog post will cover three scenarios depending on the status of your UniFi access point.

Some important notes:

  • In case your UniFi Access Point management network is not in the same network as the UniFi Controller it will not be found automatically by the UniFi Controller.
  • The network configuration is a little bit more complex than the standard for home environments:
    • Dedicated management network for the management interface of the UniFi APs.
    • Two separate SSIDs with two separate networks.


I created an example configuration for this blog post based on my home environment. Below is also a basic network diagram to give you some insight into the configuration.

Network configuration:

  • UniFi Controller IP address: (VLAN 10)
  • UniFi Access Point Management IP address: (VLAN 11)
  • Two separate WIFI SSIDs backed by different networks for isolation of devices:
    • SSID: Guest-Wifi / VLAN 12 / 192.168.12.X/24
    • SSID: Home-Wifi / VLAN 13 / 192.168.13.X/24

Device types:

  • UniFi Controller: version 7.3 (docker image) running on a Synology NAS.
  • UniFi Access Point: Ubiquiti UniFi 6 Pro

Components explained:

  • UniFi Controller: The UniFi Controller is the management suite/management interface for controlling the UniFi environment. This is a free downloadable software package from the Ubiquiti website or you can pay for the cloud service.
  • UniFi Access Point: The access point is the physical component you buy from Ubiquiti which after a successful configuration a WIFI client connects to.


Here is a basic network diagram of the UniFi setup of the environment and also some basic screenshots to give you an idea.


Default credentials with a new or reset UniFi access point.

  • Username: ubnt
  • Password: ubnt

Procedure: New UniFi Access Point

This is the procedure for a brand new access point or an access point that had a factory reset. With this command, you will let the UniFi Controller know the UniFi access point is available for adoption.

  1. Power up the access point
  2. Check the DHCP server for its lease
  3. Connect with SSH (for example use Putty)
  4. Login with the following default credentials:
    • Username: ubnt
    • Password: ubnt
  5. Run the following command on the UniFi Access Point:
# Syntax
set-inform http://%UniFi-Controller-IP-Address%:8080/inform

# Command with an example IP address to let the UniFi Controller know the AP is ready for adoption

Procedure: Physical Device Reset

This is the procedure for when the device is unreachable or the login credentials are unknown:

  1. Locate the access point, and remove it from its bracket (it might differ per type of UniFi access point).
  2. Make sure the access point has power and is booted before starting the procedure.
  3. Hold the reset button on the underside with a paperclip for 10 seconds (it might differ per type of UniFi access point).
  4. Continue with the procedure “New UniFi Access Point”.

Procedure: Software Reset

This is the procedure for when the device is accessible but needs to be moved to another UniFi Controller instance.

  1. Power up the access point
  2. Check the DHCP server for lease
  3. Connect with SSH (for example use Putty)
  4. Log in with the known configured credentials by the current UniFi Controller.
  5. Run the following commands on the UniFi Access Point:
# Run the following command to issue a factory reset on the device
or restore-default

# Syntax
set-inform http://%UniFi-Controller-IP-Address%:8080/inform

# Command with an example IP address to let the UniFi Controller know the AP is ready for adoption

Wrap up

So this is my blog post about informing and adopting an UniFi access point by an UniFi Controller. Hopefully, it is useful for somebody, please respond below if you have any comments or additional information! See you next time! 🙂

NSX Advanced Load Balancer Data Network Issue

When configuring the NSX Advanced Load Balancer for some testing in my Home Lab. I noticed something odd related to the service engines management network and data network settings. After thinking I was crazy… I saw an issue in the interface surrounding the data network configuration which caused the issue. In this short blog post, I will explain what was happening and how to resolve the issue.


My Home Lab environment was running the following products:

Note: Licenses are provided by the vExpert program (this also includes the NSX Advanced Load Balancer licenses for lab usage).

Data Network Issue

At first, we will go to the location in the interface that causes the issue:

  1. Log in on the web interface.
  2. Navigate to “Infrastructure > Cloud Resources > Service Engine Group“.
  3. Click for example on the “Default-Group” (depending on your configuration).
  4. Go to the section “Placement“.
  5. Check the following setting “Override Data Network“.
  6. Select a network that you want…

Sounds all good so far… but look at the description popup on the last screenshot. Are we configuring the management or data network for the service engines? Because the description and the field tell something different.

Management Network or Data Network?

After verifying what happened to the service engines in the group the management network for the Service Engines is changed. This was noticeable to me because the service engines were not reachable anymore on the management network for the controller.

My conclusion after some testing was that the description field is correct. This setting changes the management network!

How can you verify the changes to the service engine group?

Option 01:

  1. Open a command prompt.
  2. Run the following command “ping %management-ip-address service engine%“.
  3. They are probably not available anymore because they are on the wrong network.

Option 02:

  1. Navigate to the vCenter Server.
  2. Login with your account.
  3. Select the Service Engine virtual machine belonging to the group where you configured this setting.
  4. Check the virtual network cards.
  5. There the management network card is assigned to the “override data network” network.


So that was my blog post about the service engine group data network issue. I hope it was useful for somebody because it took me some hours to figure it out…

This wraps up the blog article hopefully it is useful for somebody, please respond below if you have any comments or additional information! See you next time! 🙂

vRealize Log Insight Domain Login Problem

Today a blog post about vRealize Log Insight domain login problems. After some time I wanted to check my vRealize Log Insight instance but somehow the domain authentication was not working anymore. So it was time for a new blog post about this issue.


The vRealize Log Insight instance is deployed in my 24×7 Lab environment that is running vExpert licenses. I am analyzing my logging and checking my login attempts in vRealize Log Insight.

So I am running the following version but I have seen this issue reoccurring in the last years:

  • VMware vRealize Log Insight 8.X (vRLI) (my screenshots are from version 8.8.2).
  • Windows Server 2022 Domain Controllers (as domain connected to vRealize Log Insight)


So let’s start with the actual problem. At first, the domain login is working for months… nothing strange and everything is fine but then one day it doesn’t work anymore. A vRealize Log Insight cluster restart doesn’t help nor do other basic troubleshooting steps. The local account like the admin account is still working, SSH access is also still working. So it is related to the domain accounts that are provided by my Active Directory.

Here is my configuration and screenshots of the domain login problem issue:


Now it is time to fix the issue. As always not difficult unless you know what is going wrong.

So let’s start with the actual problem, the problem is that every year the certificates of the domain controllers are replaced by my Certificate Authority and extended by a year. At that moment vRealize Log Insight doesn’t trust the domain controllers anymore as a login source.

So how do we fix it? By adding and trusting the new certificate on the vRealize Log Insight appliance. Luckily this can be done with the GUI so it is straightforward.


  1. Log in as the local “admin” account on the vRealize Log Insight appliance web interface (TCP 443).
  2. Go to the following location “Configuration > Authentication > Active Directory”.
  3. Click on the following button “Test Connection“.
  4. Accept all the offered certificates.
  5. This must result in the green text “Succeeded“.

Now try to log in again with a domain account, this should be working again (for another year in my case).


So that was my blog post about the vRealize Log Insight domain login problems. I hope it was useful for somebody because I always run into this issue. It happens in my Lab but also with customers that are using the Active Directory integration.

This wraps up the blog article hopefully it is useful for somebody, please respond below if you have any comments or additional information! See you next time! 🙂

HPE ProLiant Removing SD Card iLO Degraded

Recently I was removing an SD card from one of my lab servers but after removing it, the server kept complaining about it. The HPE ProLiant is equipped with HPE Integrated Lights-Out (iLO). This is an out-of-band management system to manage and configure the server. It also is responsible for monitoring the components inside the server.

This means it also monitors the health state of the SD card that is located on the motherboard slot. So when I removed the SD card it just kept checking the health of the component and causing health alerts.

In this blog post, I going to explain what I did to reset the HPE iLO to stop it from monitoring the SD card after permanent removal.


Here is a short list of information about the HPE ProLiant system that I used for this blog post:

  • Hardware: HPE Proliant DL360e Gen 8
  • HPE ilO version: 4
  • HPE SD card: HP 32GB SD card / Part nr: 700135-001
  • Firmware: HPE iLO version: 2.78
  • Software: VMware ESXi 7.0.3

Location – SD Card

To make the blog post complete I added the motherboard drawing from the HPE manual. The SD card slot is located on the HPE ProLiant DL360e Gen 8 motherboard and the slot is located at number 29 in the drawing below.

Problem – Removing SD card causes degraded state

The issue occurred when the SD card failed. After the SD card failure, I removed the SD card from the system and moved to an SSD-based boot media for VMware ESXi.

I performed some basic troubleshooting like removing the power from the server and restarting the HPE iLO but the health status was still degraded and it was still searching for the SD card.

Here are the error messages in the interface:

  • Error message on the login page: iLO Self-Test report a problem with: Embedded Flash/SD-CARD. View details on Diagnostics page.
  • Error message on diagnostics: Controller firmware revision 2.10.00 NAND read failure

Here are some screenshots related to the error messages:

Resolving – Resetting the SD card slot

Resolving the issue isn’t partially hard… if you know which buttons to push and in what order ;). Before starting, make sure the SD card is removed from the system and that the iLO has been rebooted.

To make sure that everything just works directly… open a clean browser and login into the iLO and directly follow the procedure described below.

Closing words

In the end, it cost me about three hours to get it fixed. The reason why I wanted it so badly fixed was that it kept triggering my monitoring system and that drove me crazy. This server in particular powers on and powers off regularly and during every power cycle, the health state resets and triggers monitoring alerts.

This wraps up the blog article hopefully it is useful for somebody, please respond below if you have any comments or additional information! See you next time! 🙂

Digitus Patch Cabinet Covering Fan Holes

For the last couple of months, I was busy making my new patch cabinets ready for use. One of the things I noticed with the Digitus patch cabinet is that there are openings on the top by default for the fan installation (fan holes/mounting holes). This is great for people that install the optional fan kit from Digitus but not everyone needs active fans and it is about 210/230 euros.

When you do not install the optional fan kit you will run the following problem… dust! The openings and holes on top of the cabinet allow dust to fall into the cabinet on top of your patch panels and switches. When cleaning the cabinet you need to be careful so you do not swipe the dust into large fan mounting holes and bolt hols.

So it was time to look for a solution without buying the Digitus expensive optional fan kit for basically no reason in my case… One of my requirements was no drilling or modifying of the cabinet because that would inevitably block me from installing the fan kit (in case I ever need them).

Part List

So let’s get started here is an overview of the part that I ordered online for one enclosure:

  • 2x Fan Filter (AABCOOLING Plastic Filter 120 Black OFG016 @ Amazon)
  • 8x Bolt (Bolt m5x12mm @ RVSpaleis)
  • 8x Nut (Nut with locking @ RVSpaleis)
  • 12x Plastic covers for 5 mm holes (Premium Afdekdopjes voor Boorgaten (specifications: 5 mm / RAL 9005 black) @
  • Total costs: ~ 30 euro
  • Saving: ~ 220 – 30 = 190 euro
  • Percentage: ~ 86%

Keep in mind:

  • The items were ordered in The Netherlands (Yes, it is a country on planet earth).
  • The items were ordered in June 2022.
  • The prices might change over time.


In my home, I have two of the same Digitus Patch Cabinets mounted next to each other. That is why some of the pictures look double ;). No worries you do not have to go to the optometrist!

So here is a photo of the top side of the patch cabinet before the installation, as you can see large holes that attract a lot of dust. As you can see there are bolt holes and fan holes. Both of those we are going to cover up.

Here is a list of the parts that I ordered: Some basic product images to give you an idea of what you need. The links to the webshop are listed in the parts list.

Here are the photos when installing the parts for covering the fan holes in the cabinet. The last picture is the final result of both patch cabinets in a row.

So here is the result after installing all the components on both of the patch cabinets, the fan holes and bolt holes are covered and no dust gets in anymore:


In my opinion quite a nice retrofit for the Digitus patch cabinet and not too expensive. The nice thing is you can just remove the fan covering and everything is back to stock. This makes it possible to install the Digitus fan kit unit at a later stage without limiting yourself.

This wraps up the blog article hopefully it is useful for somebody, please respond below if you have any comments or additional information! See you next time! 🙂

NLVMUG 2022 Event

Today a blog about the NLVMUG 2022 event, which I attended on the fourteenth of June 2022. The event took place in the DeFabrique in Utrecht The Netherlands.

Lots of parking spaces and easy to reach as it is centrally located in the country. The location was the same location used for the NLVMUG in 2019 (the last physical VMUG before COVID).

For me personally, this was my first physical industry event since VMworld 2019 in Barcelona Spain. So it was great to be back at a physical conference and to actually meet up with a lot of familiar faces such as ITQ/VMware colleges and community members.

There was a great lineup and a lot of sessions you could choose from. Here is an overview of the available sessions of the NLVMUG 2022:

I personally attended a couple of sessions, as you can see in the photos below. Overall the sessions were good quality and with great content! Very nice to see that a lot of people love to share and spend time on prepare those sessions!

NLVMUG 2022 Photos:

I took some photos of the event to share on my blog to give people an idea of what a VMUG is and maybe inspire them to join the next NLVMUG in 2023 (if there isn’t a new … virus or something else).

So this wraps up my short blog post about the NLVMUG 2022. I would like to thank the NLVMUG team and my employer (ITQ) for making it possible for me to be there!

For me the next physical event will be VMworld … VMware Explore so maybe see you there? See you next time!

Domain Controller NTP Server Configuration

This blog post is about Microsoft Domain Controllers and NTP Server configuration for the FSMO PDC emulator role. In this blog, we are going to configure NTP and make sure it works correctly for all the clients.


In this case, we were deploying an entire VMware environment at a customer for testing purposes including domain controllers. This was working fine until we wanted to configure the vCenter Server with domain authentication. It was not possible to login into VMware vCenter with domain accounts only with local accounts. After some looking and comparing we realized that the time synchronization in the environment was not working correctly.

I have been sitting on these simple commands for a long time without actually putting them on my blog but the customer that was suffering from this issue encouraged me to do this. So here we are ;).

Why an NTP Server is important

Short introduction on the Network Time Protocol or in short NTP. NTP is used for clock syncronization between IT systems. The NTP protocol uses port 123 and sends UDP packages. Now going back to Microsoft Domain Controllers ;).

One of the main things, when you are setting up a Domain Controller is that you need to make sure that time synchronization is working. This can be done with an NTP source or NTP GPS source. In most cases, an internet source is used, for example

The domain controllers that are controlling the domain act as NTP sources for the domain members. This is the default setting by Microsoft. Keep in mind: that this can be changed with domain group policies.

So what is acceptable and when are things falling apart? By default a 300 seconds difference is allowed by Kerberos, this sounds a lot but when NTP is not working things quickly go beyond that limit. This 300 seconds interval means that you can be 300 seconds behind or 300 seconds in front of the domain controllers. When you are beyond this limit, Kerberos tickets are not working anymore and then there is a real problem: account login, log entry timestamps are not valid, permissions not working, etc.

Command support

I have tested these commands on the following Operating System versions over the years. It doesn’t matter if this is a server with a GUI or core. As long as the PowerShell terminal is accessible!

  • Windows Servers 2008
  • Windows Servers 2008 R2
  • Windows Servers 2012
  • Windows Servers 2012 R2
  • Windows Servers 2016
  • Windows Servers 2019
  • Windows Servers 2022


Here is an overview of the NTP commands for a domain controller. The most common way is to run this in a PowerShell terminal with administrative rights. I have listed the steps with a simple explanation:

# 1. Stop the NTP service
net stop w32time

# 2. Run the following command
w32tm /config /syncfromflags:manual /manualpeerlist:",,,"

# 3. Trust the time server
w32tm /config /reliable:yes

# 4. Start the NTP service
net start w32time

# 5. Check the configuration
w32tm /query /configuration

# 6. Force time resync
w32tm /resync


Here are some troubleshooting commands to validate the working and additional settings:

# Show timezone
w32tm /tz

# NTP Peers/Stratum
w32tm /query /peers

# Sync information
w32tm /query /peers /verbose 

# Monitor status
w32tm /monitor

# Read registry values
reg query HKLM\system\CurrentControlSet\Services\W32Time\TimeProviders\NtpServer


So this is my blog post about Microsoft Domain Controllers in combination with an NTP server. I have explained the importance and the issues that can be caused by an invalid NTP Server configuration and provided the commands related to a good configuration.

Do you have additional tips or remarks please respond in the comment section below.

Have a nice day and see you next time!

VMware Tanzu HAProxy Unattended Deployment

In this blog post, I am going to share my script to automate the HAProxy deployment for Tanzu with vSphere or in short TKGs. Because of my interest in the Tanzu product family, I ended up testing and redeploying parts of TKG many times. To reduce the change of mistakes and improve my speed I automated the HAProxy deployment part. To start with a special thanks to William Lam for this blog post which pointed me in the right direction for automating the HAProxy OVA file.


The reason behind the creation of the code was the following:

  • Automate as much as possible
  • Standardize deployment
  • Streamline the process
  • Improved the speed of the deployment process


My environment for performing this unattended deployment of HAProxy is listed below. All additional requirements can be found in the file in the GIT Repository like DNS records etc.

  • Server:
    • VMware ESXi 7.0 Update 3
    • VMware vCenter 7.0 Update 3
  • Workstation:
    • OS: Windows 10
    • Components required: PowerShell and PowerCLI


Here is a recording of the HAProxy unattended deployment in my lab environment. I have changed the variables in the script to match my environment. You must change the variables in a way so that it matches your environment to perform a successful deployment.


Here is an overview of the code and a link to the GIT repository. Keep in mind to always use the GIT repository version of the code because there could be new improvements.

HAProxy Tanzu Deployment:

    Script: HAProxy Tanzu Deployment
    Author: M. Buijs
    Original concept developed by: William Lam -
    version: 1.0 - 2021-12-17
    Execution: HAProxy_Deployment.ps1

# Set variables

	# Script variables
	$global:script_name = "HAProxy_Tanzu_Deployment"
	$global:script_version = "v1.0"
	$global:debug = 0
    $global:temp_directory = "C:\Temp\"

    # vSphere
    $vCenter = "LAB-VC01.Lab.local"
    $ClusterName = "Lab"
    $DatastorePrefix = "iSCSI - Production - *" # datastore prefix
    $DiskProvisioning = "thin" # thin or thick
    $Hardware = "v14" # Virtual hardware

    # HAProxy General
    $HAProxyDisplayName = "LAB-HAProxy01"
    $HAProxyHostname = "lab-haproxy01.lab.local"
    $HAProxyDNS = ","
    $HAProxyPort = "5556" # 5556 default port

    # HAProxy Management
    $HAProxyManagementNetwork = "Management"
    $HAProxyManagementIPAddress = "" # Format is IP Address/CIDR Prefix
    $HAProxyManagementGateway = ""

    # HAProxy Frontend
    $HAProxyFrontendNetwork = "TKG - Frontend"
    $HAProxyFrontendIPAddress = "" # Format is IP Address/CIDR Prefix
    $HAProxyFrontendGateway = ""
    $HAProxyLoadBalanceIPRange = "" # Format is Network CIDR Notation

    # HAProxy Workload
    $HAProxyWorkloadNetwork = "TKG - Workload"
    $HAProxyWorkloadIPAddress = "" # Format is IP Address/CIDR Prefix
    $HAProxyWorkloadGateway = ""

    # HAProxy Users
    $HAProxyUsername = "haproxy_api"

# Functions
function banner {
    # Clear

	# Clear errors

    # Message
    Write-Host "`n---------------------------------------------------------" -foreground Red
    Write-Host "               $script_name - $script_version" -foreground Red
    Write-Host "---------------------------------------------------------" -foreground Red

function script_exit {
	Write-Host -Foreground Yellow ""
	Write-Host -Foreground Yellow "ERROR Message: $($Error[0].Exception.Message)"
	Write-Host -Foreground Yellow ""
	Write-Host -Foreground Cyan "Exiting PowerShell Script..."

function validate_media {
    ##### Message
    Write-Host "`nValidating media:"

        #### Locate temp directory
        If (-not (Test-Path "$($Temp_Directory)")) {
            Write-Host -ForegroundColor Red "- The temp directory is not created ($Temp_Directory)"
        else {
            Write-Host -ForegroundColor Green "- Located the temp directory ($Temp_Directory)"

        #### Locate OVA file
        Try {
            Write-Host -ForegroundColor Green  "- Searching for OVA file"
            $script:OVF_HAProxy = $(Get-ChildItem -Path "$Temp_Directory" -Include haproxy-v*.ova -File -Recurse -ErrorAction Stop | Sort-Object LastWriteTime | Select-Object -last 1)

            ### In case of no results
            if ([string]::IsNullOrEmpty($ {
            #### Message
            Write-Host -ForegroundColor Green "- Located HAProxy OVA file ($($OVF_HAProxy.Name))"
        Catch {
            Write-Host -ForegroundColor Red  "- Could not find HAProxy OVA file in location ($Temp_Directory)"

function ask_passwords {
    # Banner
    Write-Host "`nPasswords:"

    # Ask passwords
    $script:HAProxyOSPassword = Read-Host -asSecureString "- Enter the HAProxy user password (root)"
    $script:HAProxyPassword = Read-Host -asSecureString "- Enter the HAProxy user password ($HAProxyUsername)"

    # Validation
    If ($HAProxyOSPassword.Length -eq 0) {
        Write-Host -ForegroundColor Red "- HAProxy root account password is empty"
    # Validation
    If ($HAProxyPassword.Length -eq 0) {
        Write-Host -ForegroundColor Red "- HAProxy user account password is empty"

function connect_vcenter {
    # Banner
    Write-Host "`nvCenter connection:"

        # Disable vCenter deprecation warnings
        Set-PowerCLIConfiguration -DisplayDeprecationWarnings $false -Confirm:$false | Out-Null

        # Disable vCenter certification errors
        Set-PowerCLIConfiguration -InvalidCertificateAction "ignore" -Confirm:$false | Out-Null

        # Determine script or user input
        if ($vCenter) {
            Write-Host -ForegroundColor Green "- Connecting with vCenter server ($vCenter)"
        else {
            # Ask required vCenter information
            $script:vCenter = Read-Host "- Enter the vCenter IP address or hostname"

        if ($global:DefaultVIServers.Count -gt 0) {
            Write-Host -ForegroundColor Green "- Session already established ($vCenter)"
        else {
            # Check IP address for connectivity
            if (test-connection -computername $vCenter -count 1 -quiet -ErrorAction SilentlyContinue) {
                Write-Host -ForegroundColor Green "- Host is alive ($vCenter)"
            else {
                Write-Host -ForegroundColor Red "- Host is not responding ($vCenter)"
                $vCenter = ""

            # Connect with vCenter
            try {
                Write-host -ForegroundColor Green "- Connecting to vCenter, please wait..."

                # Connect to vCenter
                Connect-ViServer -server $vCenter -ErrorAction Stop | Out-Null
            catch [Exception]{
                $status = 1
                $exception = $_.Exception
                Write-Host "- Could not connect to vCenter, exiting script" -foreground Yellow
                Write-Host ""
                Write-Host "Exit code: $status" -foreground Yellow
                Write-Host "Output: $exception" -foreground Yellow

        # Message
        Write-Host -ForegroundColor Green "- Connection successful"

function ovf_config {
    # Banner
    Write-Host "`nOVF Configuration:"

    # Start
    Write-Host -ForegroundColor Green "- Creating OVF Configuration"

    $script:ovfconfig = Get-OvfConfiguration $OVF_HAProxy

    # Three nic configuration
    $script:ovfconfig.DeploymentOption.value = "frontend"

    # General
    $ = $HAProxyHostname
    $ = $HAProxyDNS
    $script:ovfconfig.loadbalance.dataplane_port.value = $HAProxyPort

    # Network port groups
    $script:ovfconfig.NetworkMapping.Management.value = $HAProxyManagementNetwork
    $script:ovfconfig.NetworkMapping.Frontend.value = $HAProxyFrontendNetwork
    $script:ovfconfig.NetworkMapping.Workload.value = $HAProxyWorkloadNetwork

    # Management
    $ = $HAProxyManagementIPAddress
    $ = $HAProxyManagementGateway

    # Workload
    $ = $HAProxyWorkloadIPAddress
    $ = $HAProxyWorkloadGateway
    $script:ovfconfig.loadbalance.service_ip_range.value = $HAProxyLoadBalanceIPRange

    # Accounts
    $script:ovfconfig.loadbalance.haproxy_user.value = $HAProxyUsername

    # Password root
    $BSTR1 = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($HAProxyOSPassword)
    $HAProxyOSPassword = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($BSTR1)
    $script:ovfconfig.appliance.root_pwd.value = $HAProxyOSPassword

    # Password user
    $BSTR2 = [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($HAProxyPassword)
    $HAProxyPassword = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto($BSTR2)
    $script:ovfconfig.loadbalance.haproxy_pwd.value = $HAProxyPassword

    # Finish
    Write-Host -ForegroundColor Green "- Completed OVF Configuration"

function pre_deployment {
    # Banner
    Write-Host "`nPre-deployment:"

    # Cluster
    $script:Cluster = Get-Cluster $ClusterName
    Write-Host -ForegroundColor Green "- Selected cluster ($Cluster)"

    # VMhost
    $script:VMHost = Get-VMHost | Where-Object { $_.ConnectionState -eq "Connected" } | Get-Random
    Write-Host -ForegroundColor Green "- Selected ESXi Host ($VMHost)"

    # Datastore
    $script:Datastore = Get-VMhost -Name $VMHost | Get-Datastore -Name $DatastorePrefix | Select-Object Name, FreeSpaceGB | Sort-Object FreeSpaceGB -Descending | Select-Object -first 1 | Select-Object Name -expandproperty name
    Write-Host -ForegroundColor Green "- Selected datatore ($Datastore)"

    # Check virtual machine name exists
    $VMname_check_query = Get-Cluster -Name $ClusterName | Get-VM -name $HAProxyDisplayName -ErrorAction SilentlyContinue

    if (! $VMname_check_query) {
        Write-Host -ForegroundColor Green "- Virtual machine name is not in use ($HAProxyDisplayName)"
    else {
        Write-Host -ForegroundColor Red "- Virtual Machine with name ($HAProxyDisplayName) already exists. Exiting script cannot continue!"

	#### Ask for conformation
	Write-Host "`nThis task is going to build the HAProxy virtual machine for TKGs."
	$confirmation = Read-Host "Are you sure you want to proceed? [y/n]"

	if ($confirmation -eq 'n') {
		Write-Host "Operation cancelled by user!" -Foreground Red

	if (!$confirmation) {
		Write-Host -Foreground Red "No input detected!"

function deployment {
    # Banner
    Write-Host "`nDeployment:"

	# HAProxy deployment of OVF
	try {
		### Message
		Write-Host -ForegroundColor Green "- Starting HAProxy Deployment ($HAProxyHostname / $HAProxyManagementIPAddress)"

        $script:vm = Import-VApp -Source $OVF_HAProxy -OvfConfiguration $ovfconfig -Name $HAProxyDisplayName -Location $Cluster -VMHost $VMHost -Datastore $Datastore -DiskStorageFormat $DiskProvisioning

        ### Message
		Write-Host -ForegroundColor Green "- Finished HAProxy Deployment ($HAProxyHostname / $HAProxyManagementIPAddress)"
	catch [Exception]{
		Write-Host -ForegroundColor Red "- HAProxy Deployment Failed ($HAProxyHostname / $HAProxyManagementIPAddress)"

function post_deployment {
    # Banner
    Write-Host "`nPost-deployment:"

	# Configure OVF
	try {
		### Message
		Write-Host -ForegroundColor Green "- Starting HAProxy OVF Configuration ($HAProxyHostname / $HAProxyManagementIPAddress)"

        $vappProperties = $vm.ExtensionData.Config.VAppConfig.Property
        $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
        $spec.vAppConfig = New-Object VMware.Vim.VmConfigSpec

        $ovfChanges = @{

        ### Message
		Write-Host -ForegroundColor Green "- Finished HAProxy OVF Configuration ($HAProxyHostname / $HAProxyManagementIPAddress)"
	catch {
		Write-Host -ForegroundColor Red "- HAProxy OVF Configuration failed ($HAProxyHostname / $HAProxyManagementIPAddress)"

    try {
        # Message
		Write-Host -ForegroundColor Green "- Starting HAProxy Update Specification ($HAProxyHostname / $HAProxyManagementIPAddress)"

        # Retrieve existing OVF properties from VM
        $vappProperties = $VM.ExtensionData.Config.VAppConfig.Property

        # Create a new Update spec based on the # of OVF properties to update
        $spec = New-Object VMware.Vim.VirtualMachineConfigSpec
        $spec.vAppConfig = New-Object VMware.Vim.VmConfigSpec
        $propertySpec = New-Object VMware.Vim.VAppPropertySpec[]($ovfChanges.count)

        # Find OVF property Id and update the Update Spec
        foreach ($vappProperty in $vappProperties) {
            if($ovfChanges.ContainsKey($vappProperty.Id)) {
                $tmp = New-Object VMware.Vim.VAppPropertySpec
                $tmp.Operation = "edit"
                $tmp.Info = New-Object VMware.Vim.VAppPropertyInfo
                $tmp.Info.Key = $vappProperty.Key
                $tmp.Info.value = $ovfChanges[$vappProperty.Id]
        $spec.VAppConfig.Property = $propertySpec

        # Message
		Write-Host -ForegroundColor Green "- Finished HAProxy Update Specification ($HAProxyHostname / $HAProxyManagementIPAddress)"

    catch {
        # Message
        Write-Host -ForegroundColor Red "- HAProxy Update Specification failed ($HAProxyHostname / $HAProxyManagementIPAddress)"

    # HAProxy reconfigure task for virtual machine
    try {
        # Message
        Write-Host -ForegroundColor Green "- Start Reconfigure VM task ($HAProxyHostname / $HAProxyManagementIPAddress)"
        $task = $vm.ExtensionData.ReconfigVM_Task($spec)
        $task1 = Get-Task -Id ("Task-$($task.value)")
        $task1 | Wait-Task | Out-Null
    catch {
        Write-Host -ForegroundColor Red "- Reconfigure VM task failed ($HAProxyHostname / $HAProxyManagementIPAddress)"

    # Message
    Write-Host -ForegroundColor Green "- Completed the reconfigure VM task ($HAProxyHostname / $HAProxyManagementIPAddress)"

function boot {
    # Banner
    Write-Host "`nBoot:"

	# Upgrade Virtual Hardware
	Try {
		Write-Host -ForegroundColor Green "- Upgrade Virtual Hardware ($HAProxyHostname / $HAProxyManagementIPAddress)";
		Get-VM -Name $vm | Set-VM -Version $Hardware -Confirm:$false | Out-Null
	Catch {
		Write-Host -ForegroundColor Red "- Upgrade Virtual Hardware failed ($HAProxyHostname / $HAProxyManagementIPAddress)";

	# Power-On Virtual Machine
	Try {
		Write-Host -ForegroundColor Green "- Power-on HAProxy started ($HAProxyHostname / $HAProxyManagementIPAddress)"
		Get-VM $vm | Start-VM | Out-Null
	Catch {
		Write-Host -ForegroundColor Red "- Starting HAProxy failed ($HAProxyHostname / $HAProxyManagementIPAddress)"

    Write-Host -ForegroundColor Green "- Power-on HAProxy completed ($HAProxyHostname / $HAProxyManagementIPAddress)"

function check {
    # Banner
    Write-Host "`nCheck:"

    # Set total of retries
    $TOTAL = "10"

    # Host retry interval (seconds)
	$HOST_WAIT = "10";

    # Start loop
    For ($i=0; $i -le $TOTAL; $i++) {

        # Number conversion to 2 digit:
        $NUMBER = [INT]$i + 1
        $NUMBER = "{0:D2}" -f $NUMBER

        # Check Host
        $Host_check_query = Test-Connection -computername $HAProxyHostname -count 1 -quiet -ErrorAction SilentlyContinue

        # Validate, else retry after a wait
        if ($Host_check_query -eq $false) {
            Write-Host -Foregroundcolor green "- [$NUMBER/$TOTAL] Checking HAProxy availability ($HAProxyHostname)"
            Start-Sleep $HOST_WAIT
        else {
            Write-Host -Foregroundcolor green "- [$NUMBER/$TOTAL] Checking HAProxy availability ($HAProxyHostname)"
            Write-Host -Foregroundcolor green "- [Ready] HAProxy is available ($HAProxyHostname)"

function retrieve_certificate {
    # Banner
    Write-Host "`nRetrieve certificate:"

    # Build URL
    $script:url = "https://${HAProxyHostname}:${HAProxyPort}/v2/info"

    # Configure local system
    try {
        # Message
        Write-Host -ForegroundColor Green "- Disable certificate checking on local system"

        # Disable certificate check
        [System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}
	catch {
		Write-Host -ForegroundColor Red "- Could not disable certificate checking on local system"

    # Download certificate
    try {
        # Message
        Write-Host -ForegroundColor Green "- Get HAProxy certificate ($url)"

        $req = [Net.HttpWebRequest]::Create($url)
        $req.ServicePoint | Out-Null

        # Authentication
        $req.Credentials = New-Object Net.NetworkCredential($HAProxyUsername, $HAProxyPassword);
	catch {
		Write-Host -ForegroundColor Red "- Could not get HAProxy Certificate ($url)"

    # Store error messages in variable to not crash a try and catch statement.
    $GetResponseResult = $req.GetResponse()

    # Store certificate as X.509 file
    try {
        # Message
        Write-Host -ForegroundColor Green "- Store HAProxy certificate as X.509 ($url)"

        $cert = $req.ServicePoint.Certificate
        $bytes = $cert.Export([Security.Cryptography.X509Certificates.X509ContentType]::Cert)
        set-content -value $bytes -encoding byte -path "$pwd\$HAProxyHostname.cer"
    catch {
        Write-Host -ForegroundColor Red "- HAProxy X.509 certificate could not be saved ($url)"
        Write-Host -ForegroundColor Red "- Result from GetResponse: ($GetResponseResult)";

    # Convert certificate to Base-64 file
    try {
        # Message
        Write-Host -ForegroundColor Green "- Store HAProxy certificate as Base-64 ($url)"

        $oMyCert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2($sMyCert)
        $oPem = New-Object System.Text.StringBuilder
        $oPem.AppendLine("-----BEGIN CERTIFICATE-----") | Out-Null
        $oPem.AppendLine([System.Convert]::ToBase64String($oMyCert.RawData,$InsertLineBreaks)) | Out-Null
        $oPem.AppendLine("-----END CERTIFICATE-----") | Out-Null
        $oPem.ToString() | out-file "$pwd\$HAProxyHostname.pem"
    catch {
        Write-Host -ForegroundColor Red "- HAProxy Base-64 certificate could not be saved ($url)"

function complete_banner {
    # Message
    Write-Host -ForegroundColor Green "- HAProxy deployment completed successfully! ($HAProxyHostname / $HAProxyManagementIPAddress)"

##### Main


I hope this blog about HAProxy unattended deployment was useful for some people or that the PowerShell code inspires people to deploy other types of OVA appliances.

If you got any improvements please commit them to the GIT repository and if you got any questions please respond below. Thank you for reading my blog post and see you next time!

Windows Server 2022 Core DC Boot Loop

On 11 January 2022 Microsoft released an update that puts some Microsoft Domain Controllers in a boot loop. This is related to Windows Server 2016, Windows Server 2019, and Windows Server 2022. When my lab environment was running into issues I noticed that I could not find the updates listed online. After some searching, I realized that Windows Server 2022 Core is different and has different updates installed.

In my environment, I am running two Windows Server 2022 Core edition domain controllers. On 19 January both domain controllers came into a boot loop at night after installing the updates automatically with my automated patching tools.

Server with Desktop Experience

As listed on the websites the following updates are the ones you need to delete from the system:

# Remove the updates
wusa /uninstall /kb:5009595 /quiet 
wusa /uninstall /kb:5009624 /quiet

When running the command the Windows Server 2022 domain controller told me that the updates were not installed on this system… So that was quite weird… after running the following command and checking the KB article description on the Microsoft website I found the correct one (the articles I am referring to are listed below).

# List installed updates on the machine
wmic qfe list

Server Core

I have created a simple procedure and added some screenshots. Here is the procedure that I performed on each domain controller in my environment:

  1. Disconnect the virtual network card.
  2. Connect with the virtual machine console through the hypervisor.
  3. Log in with the administrator account.
  4. Open the PowerShell prompt.
  5. Remove the update, see the command below.
  6. Reboot the system.
  7. Connect the virtual network card.
  8. Log in with the administrator account.
  9. Install update KB5010796.
  10. Reboot the system.
  11. Check the available updates to make sure it is installed.
  12. Everything should be working fine again.
# Remove the update (the quiet option was not working and the removal required mouse interaction to work)
wusa /uninstall /kb:5009555


I kind of forgot to screenshot everything because of the time between the boot loops but the end result is captured on the screenshot(s). Also added the screenshots in VMware vCenter for disabling and enabling the network card connectivity.


A couple of things I have learned so far, Windows Updates KB numbers are not identical between core and desktop experience (full) versions. It was strange that the domain controllers still listed the faulty update as the first update to install and would go into the same state back…

This wraps up the blog article hopefully it is useful for somebody, please respond below if you have any comments or additional information!


VMware Tanzu HAProxy Troubleshooting

This blog post is dedicated to HAProxy Troubleshooting for vSphere with Tanzu or also known as TKGs. Based on your configuration and deployment and the various items you need to configure you can make mistakes or items are not correctly configured. In my case, there were multiple problems at different deployments with parameters and reachability related to the network. In the end, after all the hours of troubleshooting, I ended up with a list of commands that might help others out. So that is the topic of this blog post.

HAProxy Background

First an introduction about the product HAProxy. HAProxy is a load balancer that is used by vSphere with Tanzu. This is not mandatory but is a product to choose from. The main reason for HAProxy compared to the others is that it is completed free/open-source. The HAProxy OVA is packaged and delivered by VMware and can be found in the following repository. All commands below have been tested against the HAProxy v0.2.0 version (haproxy-v0.2.0.ova) that is at the moment of writing the most recent version available.

Appliance access (SSH)

After a successful deployment, you can access the HAProxy appliance with an SSH session. This session can be established with a tool like PuTTY. The user account that should be used in the root account.

Keep in mind: Do not change configuration unless you absolutely know what you are doing. Almost all the issues I ran into were related to entering incorrect information into the deployment wizard or firewall issues.

Troubleshooting Services

One of the first things to check at first is that all services are running on the HAProxy appliance. When services are not started this is mostly caused by an invalid/incomplete configuration that is filled by the deployment wizard of the OVA.

### Check failed services
systemctl list-units --state=failed

### Check primary services for HAProxy and Tanzu Integration
systemctl status anyip-routes.service
systemctl status haproxy.service

### Restart services
systemctl restart haproxy

Troubleshooting Configuration Files

There are multiple configuration files in use by HAProxy here are the most important ones. Also, keep in mind what I already said before… do not change anything unless…

### Anyip-routes configuration file
cat /etc/vmware/anyip-routes.cfg

### HAProxy configuration file
cat /etc/haproxy/haproxy.cfg

### HAProxy dataplane api configuration file
cat /etc/haproxy/dataplaneapi.cfg

### Validation of configuration file
haproxy -c -f /etc/haproxy/haproxy.cfg

Troubleshooting HAProxy process output

Sometimes it is good to check the latest messages generated by the HAProxy process. There will be information about the startup of the process and the pool members.

### Show logging
journalctl -u haproxy.service --since today --no-pager

Troubleshooting IP Settings

By entering wrong IP information in the deployment wizard the configuration files surrounding the IP address settings, gateway, etc can be configured incorrectly. What I noticed is there is not really a check inside the deployment that verifies if the address that is entered is valid in any sort of way.

### List IP Settings

### Config files (incase of three NIC configuration)
cat /etc/systemd/network/
cat /etc/systemd/network/
cat /etc/systemd/network/

### Routing check
ip route

Troubleshooting Certificates

Certificates files used by the HAProxy application are inside the HAProxy directory on the local system. The certificates are BASE-64 encoded!

### Certificate authority file:
cat /etc/haproxy/ca.crt

### Certificate server file:
cat /etc/haproxy/server.crt

### Certificate URL by default:

Troubleshooting NTP

One of the all-time favorites that are notorious for disrupting IT systems is off course NTP. Here are some commands for troubleshooting on Photon OS.

### Check service status
systemctl status systemd-timesyncd

### Show NTP peers
ntpq -p

### Restart service
systemctl restart systemd-timesyncd

### Configuration file
cat /etc/systemd/timesyncd.conf

Troubleshooting the HAProxy API

The HAProxy API is used by Tanzu to configure HAProxy for the management and workload components. Authentication is set up when deploying the OVA and the credentials are entered in the wizard. With the second URL you can verify those credentials:

### Info page

### Authentication should work with the HAProxy user account (specified in the deployment wizard)


Thank you for reading this blog post about HAProxy troubleshooting for vSphere with Tanzu or in short TKGs. I hope it was useful to you! If you got something to add? Have additional tips or remarks please respond in the comment section below.

Have a nice day and see you next time.