Category: VMware

VMware is a virtualization and cloud computing software vendor based in Palo Alto, California. Founded in 1998, VMware is now a subsidiary of Dell Technologies. VMware bases its virtualization technologies on its bare-metal hypervisor ESX/ESXi in x86 architecture.

vRealize Automation SQL Database Installation

vRealize Automation SQL Database Installation

When deploying VMware vRealize Automation (vRA) you required to deploy a Windows Server that is responsible for the SQL Server Database. It does not matter if it is a small, medium or large deployment. All deployments require a Microsoft SQL Server. The Microsoft SQL Server is the database for the vRealize Automation IaaS components and is responsible for maintaining information about the machines it manages and its own elements and policies.

In this article, I’m going to describe the vRealize Automation SQL Database Installation with an automated/unattended PowerShell script. This article is mainly focused on the IaaS database part, so no additional information surrounding the installation of vRealize Automation components is descriped.

Requirements

The following items need to be performed before running the script:

  • Download the latest Microsoft SQL 2016 Server media from the Microsoft website (SQL 2016 – link).
  • Download the latest Microsoft SQL Server Management Studio from the Microsoft website (SSMS – link).
  • Create a virtual machine with Windows Server 2016 installed.
  • Install the latest Windows Updates.
  • Configure a static IP address.
  • Join the server to your company domain.

Environment

In my lab environment, I was deploying a minimal vRealize Automation installation with an external database. This means the following virtual machines are deployed for the vRealize Automation environment:

  • LAB-vRA-APP.Lab.local – vRealize Automation Appliance
  • LAB-vRA-IAAS.Lab.local – Running Windows Server 2016 responsible for vRealize Automation IaaS Components
  • LAB-vRA-SQL.Lab.local – Running Windows Server 2016 responsible for the vRealize Automation IaaS SQL Database
vRealize Automation - Minimal Deployment
vRealize Automation – Minimal Deployment

Note

Keep in mind the following items:

  • The installation code/script for Microsoft SQL Server is tested with the following versions (2016 Non-SP / 2016 SP1 / 2016 SP2). No doubt the code will work with newer releases but make sure to verify all your settings after installation.
  • I do not take any responsibility and I’m are not liable for any damage caused by this code.
  • I did not create a script that fully automates everything. It is based on one-liners with a complete configuration. The reason behind this is flexibility. Every environment is different and every customer is different. This way small tweaks can be made to parts of the code or you just use sections of the code.


Video

To display the full functionality of the code I recorded a video and uploaded it to YouTube. You can see the end result below. The video shows you the execution of the one-liners and what the server is configuring. The video starts after the requirements phase is completed.

Unattended Installation Files

So let’s list all the steps that are executed on the new Windows Server to make it ready to store the vRealize Automation IaaS Database:

  1. Format the D drive and make sure it is using a 64k allocation unit size for storing the Database
  2. Format the E drive and make sure it is using a 64k allocation unit size for storing the TempDB
  3. Format the F drive and make use of a default allocation unit size for storing the log files
  4. Assign both domain service accounts local administrator access on the server (svc-iaas / svc-vrasql)
  5. Create a Windows Firewall rule to allow SQL traffic on the network
  6. Enable multiple Windows Firewall rules to allow the Microsoft Distributed Transaction Coordinator (DTC)
  7. Install Microsoft SQL Server Management Studio
  8. Install Microsoft SQL Server 2016
  9. Reboot the system
  10. Configure the Microsoft Distributed Transaction Coordinator (DTC).
  11. Reboot the system
  12. Verify the system and functioning of the SQL Server

Optional:

  • In some cases, an additional G drive is required to store the SQL Backup.

PowerShell One-Liners

This file is saved as “Microsoft SQL Server 2016 – Commands.ps1” in the “C:\Temp” directory. The SQL Server media is mounted in the B drive (CD-ROM). The SQL Management Studio installation file is located in the “C:\Temp” directory.

##### Files required
# - Microsoft SQL Server 2016 Installation File
# - Microsoft SQL Management Studio 2016 Installation File

##### Volume 1 - DB 64k
$Disk = Get-Disk -Number 1
Set-Disk -InputObject $Disk -IsOffline $false
Initialize-Disk -InputObject $Disk
New-Partition $Disk.Number -UseMaximumSize -DriveLetter D
Format-Volume -DriveLetter D -FileSystem NTFS -AllocationUnitSize 65536 -NewFileSystemLabel "DB" -Confirm:$false

##### Volume 2 - TEMP DB 64k
$Disk = Get-Disk -Number 2
Set-Disk -InputObject $Disk -IsOffline $false
Initialize-Disk -InputObject $Disk
New-Partition $Disk.Number -UseMaximumSize -DriveLetter E
Format-Volume -DriveLetter E -FileSystem NTFS -AllocationUnitSize 65536 -NewFileSystemLabel "TEMP DB" -Confirm:$false

##### Volume 3 - Logs 4k
$Disk = Get-Disk -Number 3
Set-Disk -InputObject $Disk -IsOffline $false
Initialize-Disk -InputObject $Disk
New-Partition $Disk.Number -UseMaximumSize -DriveLetter F
Format-Volume -DriveLetter F -FileSystem NTFS -NewFileSystemLabel "Logs" -Confirm:$false

##### Make the service account member of the local administrators group
Add-LocalGroupMember -Group "Administrators" -Member "svc-vra-iaas"
Add-LocalGroupMember -Group "Administrators" -Member "svc-vra-sql"

##### Windows Firewall

    ### Windows Firewall - Allow Microsoft SQL Port 1433 TCP
    New-NetFirewallRule -DisplayName "Microsoft SQL Server 2016 - SQL - TCP" -Direction Inbound -LocalPort 1433 -Protocol TCP -Action Allow

    ### Windows Firewall - Enable rule for RPC for DTC
    Enable-NetFirewallRule -DisplayName "Distributed Transaction Coordinator (RPC-EPMAP)"

    ### Windows Firewall - Enable rule for Incoming DTC
    Enable-NetFirewallRule -DisplayName "Distributed Transaction Coordinator (TCP-In)"

    ### Windows Firewall - Enable rule for Outgoing DTC
    Enable-NetFirewallRule -DisplayName "Distributed Transaction Coordinator (TCP-Out)"

##### Microsoft SQL Management Studio
C:\Temp\SSMS-Setup-ENU.exe /install /passive /norestart

##### Microsoft SQL Server 2016
B:\Setup.exe /ConfigurationFile="C:\Temp\Microsoft SQL Server 2016 - Configuration.ini"

##### Reboot the server
shutdown -r -t 0

##### Configure the Microsoft Distributed Transaction Coordinator (DTC)
Set-DtcNetworkSetting -DtcName "Local" -RemoteClientAccessEnabled:$true -RemoteAdministrationAccessEnabled:$false -AuthenticationLevel "Mutual" -InboundTransactionsEnabled:$true -OutboundTransactionsEnabled:$true -XATransactionsEnabled:$false -LUTransactionsEnabled:$true -Confirm:$false

##### Reboot the server
shutdown -r -t 0

GIT Repository: Microsoft SQL Server 2016 – Commands.ps1

SQL Configuration File

This file is saved as “Microsoft SQL Server 2016 – Configuration.ini” in the “C:/Temp” directory.

; ************************************************************************************************************************
; Author:       M. Buijs
; Version:      1.0
; Date:         2018-09-26
;
; Information: 
; This configuration file is created to install the Microsoft SQL Server 2016 software for VMware vRealize Automation.
;
; VMware Requirements:
; - Enable TCP/IP protocol for SQL Server. 
; - The Microsoft Distributed Transaction Coordinator Service (MS DTC) is enabled on all SQL nodes in the system. MS DTC is required to support database transactions and actions such as workflow creation.
; - If you have a firewall running between the database server, Web servers or IaaS servers, deactivate the firewall or open the ports needed (MS SQL server uses port 1433 and MS DTC requires the use of port 135 over TCP and a random port between 1024 and 65535).
; - The database service account needs to be local administrator on the server (svc-vra-sql).
; - The IaaS service account needs to be local administrator on the server (svc-vra-iaas).
; - Set up Microsoft SQL server with separate OS volumes for SQL Data, Transaction Logs, TempDB, and Backup. 
; - The NTFS drive where SQL Server is installed has a file allocation unit (cluster) size of 64 KB.
; ************************************************************************************************************************
 
[OPTIONS]

; SQL Server - Workflow
    
    ; Setup Workflow (Install/Uninstall/Upgrade) 
    ACTION="Install"

    ; SQL Server - Accept License Agreement (True = Automatically accepted / False = Users needs to accept)
    IACCEPTSQLSERVERLICENSETERMS="True"

    ; Specify that SQL Server feature usage data can be collected and sent to Microsoft.  
    SQMREPORTING="False"
    ERRORREPORTING="False"

; SQL Server - Unatteded

    ; Parameter that controls the user interface behavior. 
    ; UIMODE="Normal"
    
    ; Setup will not display any user interface. 
    ; QUIET="True"
    
    ; Setup will display progress only, without any user interaction. 
    QUIETSIMPLE="True"

    ; Specifies that the detailed Setup log should be piped to the console. 
    INDICATEPROGRESS="False"

    ; Specify if errors can be reported to Microsoft to improve future SQL Server releases.  
    ERRORREPORTING="True"

    ; Displays the command line parameters usage 
    HELP="False"

    ; RSInputSettings_RSInstallMode_Description 
    RSINSTALLMODE="DefaultNativeMode"

; SQL Server - Installation Features

    ; Specifies features to install, uninstall, or upgrade. 
    FEATURES=SQLENGINE

    ; Language Pack (English / United States)
    ENU="True"

; SQL Server - Updates

    ; Specify whether SQL Server Setup should discover and include product updates. 
    UpdateEnabled="True"

    ; Specify the location where SQL Server Setup will obtain product updates (MU = Microsoft Update).
    UpdateSource="MU"

; SQL Server - Installation Directory

    ; Specify the root installation directory for shared components.  
    INSTALLSHAREDDIR="D:\Program Files\Microsoft SQL Server"
    
    ; Specify the root installation directory for the WOW64 shared components. 
    INSTALLSHAREDWOWDIR="D:\Program Files (x86)\Microsoft SQL Server"

    ; Default directory for the Database Engine backup files. 
    ; SQLBACKUPDIR="<InstallSQLDataDir>\<SQLInstanceID>\MSSQL\Backup"
    
    ; Default directory for the Database Engine user databases. 
    SQLUSERDBDIR="D:\Database"
    
    ; Directory for Database Engine TempDB files. 
    SQLTEMPDBDIR="E:\TempDB"

    ; Default directory for the Database Engine user database logs. 
    SQLUSERDBLOGDIR="F:\Logs"
    
; SQL Server - Instance

    ; Specify a default or named instance. MSSQLSERVER is the default instance for non-Express editions and SQLExpress for Express editions. 
    INSTANCENAME="MSSQLSERVER"
    
    ; Specify the Instance ID for the SQL Server features you have specified. 
    INSTANCEID="MSSQLSERVER"

    ; Specify the installation directory. 
    INSTANCEDIR="D:\Program Files\Microsoft SQL Server"

    ; Specifies a Windows collation or an SQL collation to use for the Database Engine. 
    SQLCOLLATION="SQL_Latin1_General_CP1_CI_AS"

    ; Level to enable FILESTREAM feature at (0, 1, 2 or 3). 
    FILESTREAMLEVEL="0"

; SQL Server - SA Account

    ; Specifies the password for the SQL Server SA account.
    SAPWD="VMware1!"

    ; Specifies the security mode for SQL Server.
    SECURITYMODE=SQL

; SQL Server - Service Access

    ; Windows account(s) to provision as SQL Server system administrators. 
    SQLSYSADMINACCOUNTS="Lab.local\Administrator" "BUILTIN\Administrators"
    
    ; Provision current user as a Database Engine system administrator for SQL Server 2012 Express. 
    ADDCURRENTUSERASSQLADMIN="False"
 
; SQL Server - Connection Support

    ; Specify 0 to disable or 1 to enable the TCP/IP protocol. 
    TCPENABLED="1"
    
    ; Specify 0 to disable or 1 to enable the Named Pipes protocol. 
    NPENABLED="0"

    ; CM brick TCP communication port 
    COMMFABRICPORT="0"
    
    ; How matrix will use private networks 
    COMMFABRICNETWORKLEVEL="0"
    
    ; How inter brick communication will be protected 
    COMMFABRICENCRYPTION="0"

    ; TCP port used by the CM brick 
    MATRIXCMBRICKCOMMPORT="0"
    
; SQL Server - Agent Service and Service Account

    ; Auto-start service after installation.  
    AGTSVCSTARTUPTYPE="Manual"

    ; Specifies the account for the SQL Server Agent service.
    AGTSVCACCOUNT="NT AUTHORITY\SYSTEM"

    ; Required Specifies the password for SQL Server Agent service account. 
    ;AGTSVCPASSWORD=""

; SQL Server - Browser Service and Service Account

    ; Startup type for Browser Service. 
    BROWSERSVCSTARTUPTYPE="Disabled"
 
; SQL Server - Server Service Account

    ; Startup type for the SQL Server service. 
    SQLSVCSTARTUPTYPE="Automatic"

    ; Account for SQL Server service: Domain\User or system account. 
    SQLSVCACCOUNT="Lab.local\svc-vra-sql"

    ; Specifies the password for SQLSVCACCOUNT.
    SQLSVCPASSWORD="heT5S-AbEsagu7ad"

; SQL Server - Report Server Service and Service Account

    ; Specifies how the startup mode of the report server NT service. 
    RSSVCSTARTUPTYPE="Manual"

    ; Specifies which account the report server NT service should execute under.  
    RSSVCACCOUNT="NT AUTHORITY\SYSTEM"

    ; Specifies the password for the startup account for the Reporting Services service.
    ;RSSVCPASSWORD=""

GIT Repository: Microsoft SQL Server 2016 – Configuration.ini



Final Word

This blog post helps you to set up a Microsoft SQL Server 2016 for vRealize Automation 7.X. The code can be used in lab and production environments but make sure to verify the configuration. I think there might be more improvements possible related to the one-lines but maybe also additional SQL configuration improvements. Do you have feedback please respond below or send me an email.

Sources

To create this article and scripts I used the following resources:


  • Update 2018-10-04 – Article was missing the Microsoft Distributed Transaction Coordinator (DTC) information and code (video recording is still missing that part).

ITQ Lightboard Session about vRealize Orchestrator (vRO)

ITQ-Logo

I’m happy to announce that as of today my lightboard session about VMware vRealize Orchestrator (vRO) is online. The guys at ITQ (the company I work for) build a lightboard to record lightboard sessions. So I decided to record a session to help people getting started with automation and introduced to the vRealize Orchestrator.

The session is called “Mischa Buijs explains the benefits of vRealize Orchestrator“. In the session, I explain a basic use case to automate virtual machine provisioning and integrate with external systems like Microsoft Active Directory and a storage array with an SSH interface.

The Storyline Lightboard Session:

Here is an overview of the storyline that I am presenting in the video:

  • Step 01: We are using the REST API integration in vRO for communicating with VMware NSX-V to provision a new virtual network (Logical Switch).
  • Step 02: We need storage to provide space for a new virtual machine. So we are leveraging the SSH integration in vRO to create a new volume on the storage array. This can be for example a NetApp FAS array or a FreeNAS virtual appliance.
  • Step 03: Let’s use vCenter to format the new datastore with VMFS, so it can be used by the ESXi Hosts. 
  • Step 04: Than again we are using the vCenter integration to mount the new volume on an entire vSphere Cluster so let’s say, ten ESXi hosts.
  • Step 05: We also use the vCenter integration to build a new virtual machine.
  • Step 06: Now it is time to leverage PowerShell to create a new computer account in Active Directory.
  • Step 07: The last step is with PowerShell, now we are creating forward and reverse DNS records on the Windows DNS Server.

ITQ YouTube Channel

At ITQ we created a lot of recordings the last couple of months. For all the videos go to the ITQ YouTube channel.

VMworld 2018 US Announcements

VMworld 2018 US - Featured Images

This blog post is dedicated to the VMworld 2018 US announcements. In the post, you will find the articles, links and highlights. VMworld 2018 US is an event that is organized by VMware. The US version is a five-day event that is held in Las Vegas. It takes place from 26 August to 30 August 2018. This page will be updated multiple times to coming days to add additional information and the latest announcements.

Please reply underneath this blog post if you have some additional information. This can also be additional links or blogs posts.

VMworld 2018 US – Product Announcements

In this chapter are all the product announcements. I can tell you there are a lot of announcements made at VMworld 2018.

vRealize Automation (vRA) 7.5

One the first day of VMworld 2018 US vRealize Automation 7.5 was announced.

The key highlights are:

  • Easy to Operate
    • Modernized vRealize Automation/vRealize Orchestrator(vRO) UI
    • Closed Loop Optimization with vRealize Operations
    • Enterprise-ready ServiceNow plugin
    • NSX-T On-premises Support
  • Built to Support Developers
    • Configuration Management with Ansible Tower
    • Kubernetes Cluster Management with VMware Pivotal Container Services (PKS)
  • Helps Embrace Multi-Cloud
    • AWS Enhancements
    • Azure Enhancements
    • Google Cloud Enhancements

Links:



vRealize Operations Manager (vROPS) 7.0

On the first day of VMworld 2018 US vRealize Operations Manager 7.0 was announced.

The key highlight are:

  • Enhanced User Interface
  • Automating Performance Based on Business and Operational Intent
  • Automated Host Based Placement, Driven by Business Intent
  • Capacity Analytics Enhanced with Exponential Decay and Calendar Awareness
  • Plan Capacity across Private Cloud and VMware Cloud on AWS
  • Simplified Dashboard Creation and Sharing
  • vRealize Operations AWS Management Pack Update
  • Other Miscellaneous Enhancements
    • Workload Right-sizing to avoid performance bottlenecks and reclaim over-allocated resources
    • Built-in vSphere config & compliance: PCI, HIPAA, DISA, FISMA, ISO, CIS
    • Ability to extend to the entire data center and cloud with updated management packs for Storage, vRO, Kubernetes, Federation etc.
    • vSAN performance, capacity, and troubleshooting including support for stretched clusters and through vRealize Operations plug-in in vCenter
    • Wavefront integration for application operations

Links:


vRealize Log Insight (vRLI) 4.7

On the first day of VMworld 2018 US vRealize Log Insight 4.7 was announced.

The key highlights are:

  • Security Improvements
    • Detailed verification of certificate when adding it to vRLI both from UI and REST API
    • Ability to retrieve current certificate details both from UI and REST API
    • LIAGENT_SSL_CA_PATH environment variable
  • Usability Enhancements
    • vCenter Integration now relies on user-provided hostname instead of trying to resolve it
    • Ability to choose the content packs, the fields of which should be dynamically extracted in the query result in Interactive Analytics
    • Improvements in the REST API
    • UI/UX improvements and bug fixes

Links:


vRealize Network Insight (vRNI) 3.9

On the first day of VMworld 2018 US vRealize Network Insight 3.9 was announced.

The key highlights are:

  • Plan micro-segmentation and get visibility for NSX Data Center, including NSX-T
  • Enhanced security for service access with Multi-factor Authentication
  • Custom dashboard support for Cisco ASA firewall and enhancements to Checkpoint Firewall support
  • Many customers are now using VMware Network Insight as a service, and customers based in Europe now also have the option of using the VMware Network Insight service hosted out of London, UK.

Links:


vCloud Director (vCD) 9.5

Just a couple of days before VMworld 2018 US vCloud Director 9.5 was announced:

The key highlights are:

  • Cross-site networking improvements powered by deeper integration with NSX
  • Initial integration with NSX-T
  • Full transition to an HTML5 UI for the cloud consumer
  • Improvements to role-based access control
  • Natively integrated data protection capabilities, powered by Dell-EMC Avamar
  • vCD virtual appliance deployment model

Links:


vSphere Platinum

On the first day of VMworld 2018 US, a new vSphere edition was announced. The product is called vSphere Platinum and it has a tight integration with VMware AppDefense.

The key highlights are:

  • Benefits for vSphere Admins
    • Gain visibility into the intent of each virtual machine, and a detailed inventory of application assets and context.
    • Understand how applications behave and be alerted to potential issues and deviations.
    • Shrink the attack surface and reduce the risk of security compromise.
      Establish a simple and powerful way to collaborate with security, compliance and application teams.
    • Get better visibility and protection with a simple, light-weight and scalable security solution, with no agents to manage, and minimal overhead.
    • Use what you already own, understand, and run in your data center – vSphere – with its unique visibility, automation and isolation qualities.
    • Play a larger and critical role in the security of your entire IT environment – Be the Security Hero!
  • Benefits for Security Teams
    • Better visibility and situational awareness of application behaviours, and virtual machine purpose.
    • Faster detection, analysis, and time to response – quickly understand attacks and make fast decisions using application context and scope.
    • Enhance existing security tools and support compliance efforts through contextual visibility and alerts into application communications and deviations.
    • Lower false positives – integrated behavioural analytics and machine learning offer a more precise method to identify and respond to threats.
    • Big data correlation for better identification and context using cloud SaaS model.
    • Security as an agile business enabler – support DevOps environment through continuous learning and protection.
    • Easily Coordinate with vSphere Admins and Application teams for better security while respecting existing workflows & maintaining separation of duties.
  • Secure Applications
    • VMware AppDefense – Protects the integrity of applications running on vSphere, using machine learning to monitor against threats and automate responses. AppDefense locks down the guest operating system for all applications, the VMware application stack and third-party applications.  To accomplish this, AppDefense gathers inventory data on virtual machines and applications from vCenter Server, development tools, and automation frameworks and applies machine learning to discover the intended state and establish the known good behaviours for the application and machine.  Any deviations from this state are detected and prevented, securing the integrity of the applications, infrastructure, and guest operating system. AppDefense provides detailed visibility for better change management and compliance reporting and also provides a rich set of automated or orchestrated incident response mechanisms to address attacks. Moreover, it leverages machine learning for a simple and automated way to conduct audits and reviews for applications.
  • Secure Data
    • FIPS 140-2 Validated VM Encryption, and cross-vCenter Encrypted vMotion – Secure against unauthorized data access both at rest and in motion, across the hybrid cloud.
      Secure Infrastructure
    • Secure Boot for ESXi – Allows only VMware and Partner signed code to run in your hypervisor.
      Secure Boot for Virtual Machines – Helps prevent images from being tampered with and prevents the loading of unauthorized components.
    • Support for TPM 2.0 for ESXi – Enables hypervisor integrity by validating the Secure Boot for ESXi process and enables remote host attestation.
    • Virtual TPM 2.0 – Provides the necessary support for guest operating system security features while retaining operational features such as vMotion and disaster recovery.
    • Support for Microsoft Virtualization Based Security – Supports Windows 10 and Windows 2016 security features, like Credential Guard, on vSphere.
  • Secure Access
    • Audit Quality Logging – Enables authorized administration and control by providing high fidelity visibility in vSphere operations.

Links:


vSphere 6.7 Update 1

On the first day of VMworld 2018 US vSphere 6.7 Update 1 was announced.

The key highlights are:

  • Fully Featured HTML5-based vSphere Client
  • Enhanced support for NVIDIA Quadro vDWS powered VMs; and Support for Intel FPGA
  • New vCenter Server Convergence Tool
  • Enhancements for HCI and vSAN
  • Enhanced vSphere Content Library

Links:


vSAN 6.7 Update 1

On the first day of VMworld 2018 US vSAN 6.7 Update 1 was announced.

The key highlights are:

  • Simplified Operations
    • Cluster Quickstart
    • Driver & Firmware Updates using Update Manager
    • Decommissioning and Maintenance Mode Safeguards in vSAN 6.7 U1
    • More vRealize Operations Intelligence
    • Improved Capacity Reporting
  • Efficient Infrastructure
    • TRIM/UNMAP Support
    • Mixed MTU Support for 2 Node and Stretched Clusters
    • Updated Sizing Tools
  • Rapid Support Resolution
    • Improved Health Check Guidance
    • Enhanced Support Diagnostics

Links:


VMware Validated Design 4.3

Also, the VMware Validated Design (VVD) received some new features and changes to the documentation. Personally, the greatest value in this release is the Visio stencils that are available for everyone.

The key highlights are:

  • Official NSX-T Support
  • Documentation Updates
    • IT Automating IT Scenarios
    • Intelligent Operations Scenarios
    • Introduction to Security and Compliance
    • Operational Verification
    • Certificate Replacement for 2-pod
    • Certificate Replacement for 1-pod
  • Architecture and Design of VMware PKS for Workload Domains
  • Design and Deployment of VMware Skyline
  • Architecture and Design Guidance for NIST 800-53
  • VVD Diagrams and Stencils

Links:


VMworld 2018 US – Technical Previews & Projects

There were also a lot of announcements surrounding some new developments/projects.

Project list:

  • Project Concord – Project Concord uses Byzantine fault-tolerant consensus protocols to deliver a functioning distributed trust system: one that is both “safe” and “alive.” Concord is a generic state machine replication library that can handle malicious (Byzantine) replicas.
  • Project Dimension – Project Dimension will extend VMware Cloud to deliver SDDC infrastructure and hardware as-a-service to on-premises locations.
  • Project Magna – Project Magna will make possible a self-driving data center based on machine learning.
  • RDS on VMware – VMware demonstrated how Amazon Web Service’s RDS service will run on VMware in a private data center, thus offering developers a familiar RDS Functionality available on VMware in a private data center or at the Edge.
  • Virtualization on 64-bit ARM for Edge – VMware demonstrated ESXi on 64-bit ARM running on a windmill farm at the Edge.

Links:


VMworld 2018 US – ITQ Blogs 

Here is a list of ITQ blogs with additional VMworld 2018 US content:


VMworld 2018 US – Keynotes

There are already some recordings available of the keynotes. These can be found on YouTube with the following links:


VMworld 2018 US – Recordings

Just like every year, William Lam from the website virtuallyGhetto creates a GIT repository with all the VMworld sessions. For each session, a recording and presentation are provided. It will probably be a couple of days till weeks until all sessions become available.

VyOS Configuration Restore

In my home lab environment, I wanted to rebuild my VyOS virtual router/firewall. So I exported the configuration from the old appliance and I tried to perform a restore on the new virtual appliance. The question that arose was: how do you perform a VyOS configuration restore?

Somehow on the internet, I could not find any tutorial or manual that explained to me how this action could be performed. There are enough write-ups and articles surrounding the TFTP, FTP and SCP restore procedure but the VyOS appliance is empty… with a default configuration. I just wanted to restore the configuration without setting up all kinds of services and configuring by hand my interfaces on the VyOS appliance.

So it was time to examine the VyOS appliance and figure out what was going on under the covers.

Why do you need VyOS?

Before diving any further let‘s talk about VyOS! I use VyOS for my Lab environment because it is easily configurable and has an entire feature set of enterprise-grade network technology onboard by default. Like the routing protocols BGP and OSPF and high availability option VRRP.

So why do you need OSPF and BGP at home? I’m a VMware Consultant that is responsible for SDDC / SDN / NSX designs and implementations. I regularly need to perform tests in my Lab environment. VMware NSX likes to have a dynamic routing protocol to connect the virtual overlay network to the physical world. Both routing protocols can be used to perform this. An article about a detailed configuration can be found here at Jeffrey Kusters his blog (my ITQ colleague). I not going into further detail on VMware NSX, this blog post is focused on VyOS.

VyOS Virtual Hardware

My VyOS appliance is deployed on a VMware vSphere 6.5 infrastructure. I used the OVA file that is available on the VyOS website (vyos-1.1.8-amd64.ova). The virtual machine is called the “LAB-FW01” this hostname will appear in the video record. The YouTube video is listed below.

The Virtual Machine hardware is configured as default. I only assigned the virtual network cards to the right networks. An overview is listed here:

  • Public – Network adapter 1 – Connected to a WAN interface
  • Private – Network adapter 2 – Connected to a VLAN trunk

VyOS Configuration Restore

Now it is time for restoring the VyOS configuration file on a newly deployed VyOS appliance.

Preparation:

  1. Deploy a new VyOS appliance and make sure that the virtual networks are connected to the correct adapter.
  2. Verify and/or change the MAC addresses were needed. The MAC addresses should align between the configuration file and the new virtual appliance:
    • Option 01: Change the virtual network card MAC address to the ones that were used on the old appliance.
    • Option 02: Change the MAC addresses in the configuration file that is used for the restore. The MAC address should align with the newly deployed VyOS appliance.
  3. Create an ISO file with your latest configuration on it. I used the following opensource tooling as displayed below. Link to IsoCreator.
Creating an ISO file with VyOS Configuration
Creating an ISO file with VyOS Configuration

Procedure:

  1. Open your vSphere Infrastructure and navigate to the Virtual Machine. This would be in my case “LAB-FW01“.
  2. Assign the newly created ISO file to the Virtual Machine. You connect the ISO file to the CD-ROM drive.
  3. Power-on the Virtual Machine.
  4. Make sure you select in the GRUB bootloader the following mode to startup from: “VyOS 1.1.8 linux (USB console)”.
  5. When VyOS is booted, log in with the following default credentials:
    • Username: vyos
    • Password: vyos
    • You are now logged in into the Linux Shell.
  6. Now it’s time to mount the connected CD-ROM media:
    • sudo mount /dev/cdrom /mnt
  7. To make sure my configuration is available. I list the directory content with the following command:
    • ls -l /mnt
  8. Now it is time to copy my old configuration to the startup configuration location of VyOS. Use the following command to perform this action (keep in mind: My configuration is called “2018-06-05-vyos.config.boot”): 
    • cp /mnt/2018-06-05-vyos.config.boot /config/config.boot
  9. To verify the copy action, I run the following command to display my hostname that is  listed in the configuration file:
    • cat /config/config.boot | grep LAB-FW01
  10. Now it is time to reboot the VyOS appliance. At the next boot, the old configuration will be loaded and everything should be restored. The following commands are required for rebooting VyOS:
    • reboot
    • yes
  11. After the reboot is completed you should log in with your old credentials that belong to the restored configuration.
  12. To verify that the configuration is loaded correctly I run the following command to display all my interfaces and sub-interfaces:
    • show interfaces
  13. From this point, everything should be working.

Tips:

I have listed all the VyOS configuration locations that are important to this article. 

In case you messed up your VyOS configuration you can always restore the default out-of-the-box configuration with the procedure described above. You only need to change the copy action in step eight to the following: (cp /opt/vyatta/etc/config.boot.default /config/config.boot).

  • Startup configuration: /config/config.boot
  • Default configuration: /opt/vyatta/etc/config.boot.default

YouTube Recording:

Because there are a couple of steps involved I decided to record a video of me performing the procedure. Keep in mind: my VMware Remote Console is in dutch :).

Final word

About six months back I switched from pfSense to VyOS. The main reason was the BGP support and stability of the BGP routing process. I am happy I did. The VyOS appliance is just amazing and extremely reliable and robust.

If you are familiar with the Cisco CLI than you will be flying through the VyOS CLI in no-time.

vRealize Automation enabling the VMware Identity Manager GUI

In this blog post, I am going to show you how to enable the VMware Identity Manager GUI when it is located inside vRealize Automation. I am going to explain why you sometimes should and how the procedure is performed.

For some who don’t know, VMware vRealize Automation (vRA) is using under the covers VMware Identity Manager (vIDM). By default is the VMware Identity Manager GUI disabled after the vRealize Automation installation process. If you want to configure parts of vIDM you are configuring it through the vRealize Automation GUI.

Let’s get everybody on the same page: So you are talking about two GUIs the vRealize Automation GUI and the VMware Identity Manager GUI? Who is who, explain it to me!

vRealize Automation GUI - Displaying Active Directory Users
Here is the vRealize Automation (vRA) GUI – Displaying multiple Active Directory users

VMware Identity Manager GUI - Displaying Active Directory User
Here is the VMware Identity Manager (vIDM) GUI – Displaying an Active Directory user (by the way: the name Dick is a common name in The Netherlands)

Keep in mind: I’m not completely sure what will happen if you would configure items with the vIDM GUI. Because you are bypassing the default vRealize Automation GUI. I have done some tests and no problems were caused in my Lab environment but it might cause issues later on… So why do you enable it then? Because of the large amount of information provided throughout the vIDM GUI. The vRealize Automation GUI is only showing a small peace and not the big picture.

Product support: I have tested this procedure on vRealize Automation version 7.3 and vRealize Automation 7.3.1. I would expect it to work on newer and older vRealize Automation 7.X releases.

Enable the vIDM Interface

This procedure is for enabling the interface:

  • Step 01: Connect with an SSH session to the vRealize Automation Appliance. Use for example Putty.
  • Step 02: Login with the root credentials.
  • Step 03: Run the following command to start the VMware Identity Manager Interface: (vcacvami horizon ui enable).
  • Step 04: Open a web browser and navigate to the following URL:(https://%FQDN%/SAAS/admin/).
VMware Identity Manager (vIDM) interface enable
VMware Identity Manager (vIDM) interface enable

Disable the vIDM Interface

This procedure is for disabling the interface:

  • Step 01: Connect with an SSH session to the vRealize Automation Appliance. Use for example Putty.
  • Step 02: Login with the root credentials.
  • Step 03: Run the following command to stop the VMware Identity Manager Interface: (vcacvami horizon ui disable).
  • Step 04: When you navigate to the following URL no page should appear: (https://%FQDN%/SAAS/admin/).
VMware Identity Manager (vIDM) Interface/GUI disable
VMware Identity Manager (vIDM) interface disable

Status of the vIDM Interface

This procedure is for viewing the vIDM Interface status:

  • Step 01: Connect with an SSH session to the vRealize Automation Appliance. Use for example Putty.
  • Step 02: Login with the root credentials.
  • Step 03: Run the following command to view the current status of the VMware Identity Manager Interface: (vcacvami horizon ui status).
  • Step 04: When you navigate to the following URL no page should appear: (https://%FQDN%/SAAS/admin/).
VMware Identity Manager (vIDM) interface status
VMware Identity Manager (vIDM) interface status

VMware Identity Manager URLs

The following URLs are available when the GUI is enabled (there are probably more URLs available than listed below):

  • Main page: https://%vRA-Appliance-FQDN%:8443
  • Tenant-specific page (Tenant vSphere.local): https://%vRA-Appliance-FQDN%/SAAS/t/vsphere.local
  • Tenant-specific page (Tenant Production): https://%vRA-Appliance-FQDN%/SAAS/t/production
  • Tenant-specific page (Tenant Development): https://%vRA-Appliance-FQDN%/SAAS/t/development

vRealize Automation Internal Proxy Explained

To explain what happens under the covers is the following: When you run the command to start or stop vIDM interface the reverse proxy configuration located in vRealize Automation Appliance (vRA) is changed. The command adds or removes some configuration files. Then the proxy daemon is reloaded to pick up the changes. At that point, the vIDM webpage becomes available or unavailable depending on the given command. Under the covers, vRealize Automation uses HAProxy for IT guys working in container or web hosting environments that might be a very familiar product. Here is a quote from the HAProxy website to explain what HAProxy is capable of:

HAProxy is a free, very fast and reliable solution offering high availability, load balancing, and proxying for TCP and HTTP-based applications. It is particularly suited for very high traffic websites and powers quite a number of the world’s most visited ones. Over the years it has become the de-facto standard opensource load balancer, is now shipped with most mainstream Linux distributions, and is often deployed by default in cloud platforms. Since it does not advertise itself, we only know it’s used when the admins report it.

https://www.haproxy.org/#desc

VMware Identity Manager GUI Advantages

The main advantages of enabling the GUI:

  • You can view the currently logged in users in vRealize Automation.
  • When logging into the default tenant in the vIDM GUI you can view the total amount of users and groups that are identified by vIDM (counters are from all tenants combined).
  • You can view the health status of VMware Identity Manager (vIDM).
  • You can view the user login history based on the last days and you have reporting functionality.

Final word

In this blog post, I explained how to enable and disable the VMware Identity Manager interface on a vRealize Automation Appliance. As noted before be careful! I personally only use this method for troubleshooting Identity Manager related problems with authentication and viewing user activity.

Content Update

The following items have been verified:

  • 2020-09-24: This is still working on vRealize Automation 7.6.

VMware vExpert NSX 2018

VMware vExpert NSX 2018 - Badge

I am happy to announce that as of today I entered the VMware vExpert NSX sub-program. Thanks, VMware for the award and congratulations to everyone else who also made it! Also an extraordinary achievement for our team at ITQ, we managed to get seven consultants in the vExpert NSX 2018 sub-program! This year VMware awarded 169 people worldwide for the vExpert NSX sub-program.

vExpert NSX:

  • Since when does the vExpert NSX sub-program exist?
    • The VMware vExpert NSX sub-program was introduced in the year 2016 and announcements are made every year.
  • So what is the difference between the vExpert NSX sub-program and vExpert program?
    • The program is built upon the vExpert program. To apply you must be a current vExpert as well as evangelizing VMware NSX.
  • What is VMware expecting of a vExpert NSX?
    • Each of these vExperts has demonstrated significant contributions to the community and a willingness to share their expertise with others. Contributing is not always blogging or Twitter as there are many public speakers, book authors, scriptwriters, VMUG leaders, VMTN community moderators and internal champions among this group.

Related links

vRealize Orchestrator Client and High-Resolution Screens

VMware VCAP6.5-DCV Design Exam Experience

VMware VCIX6.5-DCV Badge At VMware EMPOWER 2018 in Vienna, I passed the VMware Certified Advanced professional 6.5 Data center Virtualization Design exam or in sort VCAP6.5-DCV. This exam was already for a very long on my certification list, to be more precise for about four years. Because of the large number of new products VMware released the last couple of years, my priorities shifted many times because Data center Virtualization was not really too interesting to study anymore… (sorry Data center Virtualization for calling you not interesting). My experience with VMware vSphere started more than ten years ago and over the years I have been responsible for designing multiple infrastructures for customers.

The VMware way of describing the VMware Certified Advanced Professional 6.5 – Data Center Virtualization Design certification:

“The VCAP6.5-DCV Design certification validates advanced knowledge of simplifying data center operations through virtualization with vSphere 6.5 and its related components, and is able to recommend and design VMware solutions to meet specific goals and requirements. This industry-recognized certification improves your credibility among your peers and proves that you have a deep understanding of data center virtualization design principles and methodologies.”

 

So where is the exam about: The exam is about the VMware Design Methodology used for designing a VMware vSphere infrastructure the right way. To pass the exam you need to know everything about the following items and make sure you can apply them in real-life examples/use cases:
– AMPRS (Availability, Manageability, Performance, Recoverability and Security).
– RCAR (Requirement, Constraint, Assumption and Risk)
– Non-functional and functional requirements
– RPO (Recovery Point Objective)
– RTO (Recovery Time Objective)
– Conceptual, Logical and Physical Designs
– vSphere 6.5 features, what did the release of vSphere 6.5 provide to each feature and what are the requirements and constraints for implementation.
– vSAN 6.5 features, design and architecture
– Site Recovery Manager features, design and architecture
– vSphere Replication features, design and architecture

Preparation

For me, the preparation of the exam started with a couple of sessions about the vSphere Design Methodology. Jeffrey Kusters was so kind to host a couple of sessions in his spare time about the subject for all the ITQ colleges. Jeffrey Kusters is one of our VMware VCDXs that works at our firm and has a lot of years of experience designing vSphere Infrastructure as an IT architect.

Study Materials

I have used many different material/sources to prepare myself for the exam but watch out. There is so much information available that you can study until 2020 and then you are still not able to read all the information. Focus on the items listed above and you will have a good chance of passing. Important note, sometimes information on a blog or book is a couple of years old and you might expect it to be not relevant. This is not the case the VMware Design Mythology has not changed over the years.

Material list:
Blog – Jeffrey Kusters – Breaking down the conceptual design, RCARs and AMPRS …. VCDX style
Blog – Jeffrey Kusters – Passed VCAP6.5-DCV Design … Yes! Finally VCDX6
Course – VMware – VMware vSphere: Design Workshop [V6.5]
Documentation – VMware – vSphere 6.5 Availability Guide
Documentation – VMware – vSphere 6.5 Installation Setup Guide
Documentation – VMware – vSphere 6.5 Platform Services Controller Administration Guide
Documentation – VMware – vSphere 6.5 Security Guide
Documentation – VMware – vSphere 6.5 Upgrade Guide
– Ebook – VMware vSphere Design Second Edition by Scott Lowe, Forbes Guthrie and Kendrick Coleman
– Ebook – VMware vSphere 6.X Datacenter Design Cookbook SE by Hersey Cartwright
– Ebook – vSphere Design Pocketbook 2.0 Blog Edition
– Exam Guide – VMware – Link
Video – Pluralsight – What’s New in vSphere 6.5
Video – Youtube – vBrownbag – VCAP-DCV Design
White Paper – VMware – Storage Protocol Comparison

Conclusion

I passed the exam on the first attempt with a good score. The questions are decent around 12 drag en drop and 48 multiple choices. The questions are decent and require you to have deep knowledge of the products. So basically you know the answer or you just don’t. For me passing the VCAP6.5-DCV Design made me a VCIX6.5-DCV, so I was extra pleased with the result.

Special thanks to Jeffrey Kusters! You have been an absolute help!

vRealize Suite Lifecycle Manager Internet Proxy URLs

Recently I was deploying the vRealize Suite Lifecycle Manager (vRSLCM) at a customer. One of their requirement was that the appliance only could reach the internet with their internet proxy configured. The deployment was based on the latest version of the vRSLCM appliance version 1.2. The main use case for deploying this appliance at the customer was to reduce the time spend on their Lifecycle Management proces. The vRLCM appliance is able to maintain vRealize Automation (vRA), vRealize Business (vRB), vRealize Log Insight (vRLI), and vRealize Operations Manager (vROPS).

So for the proxy configuration, we need to identify which proxy URLs are required. Luckily in the VMware Documentation, there is a list of URLs and ports which the vRealize Suite Lifecycle Manager uses. So I configured the URLs but it appeared to be not working. The vRSLCM was complaining about the URL (https://my.vmware.com). So after analyzing the proxy logging, I could conclude that the URL list was not complete. Below this section, I display the difference between the official statement and what was required to get it working.

Proxy URLs – Official List:

This is the official URL list from the vRealize Suite Lifecycle Manager (vRSLCM) v1.2 documentation:

PortsPort numberBase URLs
My VMware443https://apigw.vmware.com
Solutions Exchange443https://marketplace.vmware.com
Updates443https://vapp-updates.vmware.com
Compatibility443https://vapp-updates.vmware.com

Proxy URLs – Unofficial List:

This is the list that was required to get vRSLCM working through the proxy, keep in mind there are a lot of different functionalities in the vRSLCM appliance and I was not using the entire set of functionalities. There might be even more URLs required…

PortsPort numberBase URLs
My VMware443https://apigw.vmware.com
Solutions Exchange443https://marketplace.vmware.com
Updates443https://vapp-updates.vmware.com
Compatibility443https://simservice.vmware.com
My VMware443https://my.vmware.com
vConnect443https://vconnect.vmware.com
Download location443https://download2.vmware.com
Download location443https://download3.vmware.com
Icons for marketplace443https://marketplace-download.vmware.com

Proxy Configuration:

In the steps below I explain the proxy configuration in the vRealize Suite Lifecycle Manager web interface:

  1. Open the web browser.
  2. Navigate to the vRSLCM URL (https://%FQDN%).
  3. Log in with an administrator account (admin@localhost).
  4. Navigate to the following page (Settings > My VMware).
  5. Toggle the Configure Proxy to enabled.
  6. Enter the following proxy information:
    1. Proxy Server: %FQDN% or %IP%
    2. Proxy Port: %Port Number%
    3. Proxy Username: %Username%
    4. Proxy Password: %Password%
  7. Click on the Submit button.
  8. Verify if the My VMware communication is working and the appliance is able to download the media.

Note:

  • vRealize Suite Lifecycle Manager (vRSLCM) requires ICMP communication with the internet proxy. When configuring a proxy in vRSLCM version 1.3 the configuration validation fails without allowing ICMP.

Documentation:

Article Update:

Over time the article has been updated a couple of times:

  • 2018-07-16:
    • Added additional proxy URLs for the vRealize Suite Lifecycle Manager version 1.3.
    • Added link for the vRealize Suite Lifecycle Manager manual version 1.3.
    • Added additional note about ICMP.
  • 2021-03-05:
    • Fixed the proxy URL table layout.

Cannot Remove Content Library in VCSA 6.5 Update 1