Tag: ITQ

Cannot Remove Content Library in VCSA 6.5 Update 1

0

Problem

At a customer I came across the following problem, the customer was not able to remove a Content Library from vCenter Server. They just created a Content Library and after that, they wanted to remove the item. When they tried to remove the content library it failed. We started troubleshooting the log files and tried to remove the Content Library in different ways with the vSphere Web Client, PowerShell and REST API but all ended with the same error. The error messages are listed below.

To add some more background information: the customer was running the environment with an external platform services controller and a vCenter Server (VCSA). The version that was being used was VCSA 6.5 Update 1e.

Content Library – Error messages

Cannot Remove Content Libary – 01

Cannot Remove Content Libary – 02

Cannot Remove Content Libary – 03

Solution

We ended up calling VMware Global Support Services (GSS) to resolve the issues. They were very helpful and fixed it within a couple of minutes. The knowledge base article listed below is only available for internal VMware personal.

The internal knowledge base article related to the issue:
– https://ikb.vmware.com/s/article/50121825 – Unable to delete the stale entry for the content library from the web client

Opening vSphere Web Client (Flash) on Windows Server 2016

1

Recently somebody asked me a question about VMware vCenter running on a Windows Server. The Windows Server was running VMware vCenter 6.5 and in case of a datacenter related problem, they wanted to get access to the vSphere Web Client (Flash) on the system locally.

It sounds easy right…? Just open the browser on the Windows Server and navigate to the vSphere Web Client page but that didn’t appear to be the case, because the system is missing the browser plugins required to open the vSphere Web Client.

So let’s dive into the problem.

Microsoft Browsers:
They are running Windows Server 2016 and you might expect it to have two browsers: Internet Explorer 11 and Microsoft Edge. That does not seem to be the case. Windows Server 2016 is only shipped with Internet Explorer 11. Why? Windows Server 2016 is marked as an LTSB (Long Time Service Branch) so this means no Microsoft Edge and it is also not available for manual installation.

Microsoft: “The Long-Term Servicing Branch (LTSB) versions of Windows, including Windows Server 2016, don’t include Microsoft Edge or many other Universal Windows Platform (UWP) apps. These apps and their services are frequently updated with new functionality, and can’t be supported on systems running the LTSB operating systems.”

Third-party browsers:
The company who was asking had a security policy that does not allow an installation of third-party browsers like Mozilla Firefox or Google Chrome. Alright, so this is not an option. Don’t have to look at that further.

Adobe Flash:
So let’s try Internet Explorer 11. It appears to be missing Adobe Flash and you can not download and install it from Adobe Website.

At this point, I was stuck and there did not seem to be a simple solution.

Screenshots

vSphere Web Client missing Adobe Flash

Adobe Flash not available for installation on Windows Server 2016

Solution

After searching for a solution for about an hour. I came across a Microsoft Blog article listed below. This article is talking about installing Adobe Flash on Windows Server 2016. It appears that all the software is already on the system but just needs to be installed.

Procedure:
– Step 01: Close all browsers
– Step 02: Start a PowerShell session with elevated rights.
– Step 03: Run the following command:
dism /online /add-package /packagepath:"C:\Windows\servicing\Packages\Adobe-Flash-For-Windows-Package~31bf3856ad364e35~amd64~~10.0.14393.0.mum"
– Step 04: Wait for the installation to complete.
– Step 05: Open a browser and navigate to the vSphere Web Client.
– Step 06: Everything should be working now.

Note: In the Microsoft Blog article they are talking about a reboot required in my case it was not required. Just a browser restart was enough.

Screenshots

Installing Adobe Flash on Windows Server 2016

vSphere Web Client with Adobe Flash

It sounded like an easy problem at first but it took some more time than I expected. The problem is solved with a simple one-liner and the customer is happy. I personally think that there might be other solution to the problem. If you know them please add a comment below.

Source(s):
– Blog.technet.microsoft.com – How to add the Adobe Flash Player / Shockwave Plugin for IE on Windows Server 2016
– Docs.microsoft.com – Microsoft Edge requirements and language support

No Workflow Output in vRealize Orchestrator (vRO) 7.4

2

Overview

Today I was checking my standalone vRealize Orchestator (vRO) appliance in my 24×7 Lab environment for scheduled task activities but something didn’t seem to be right. All scheduled tasks appeared to be running without problems, but there was no workflow output.

So what kind of schedule tasks is this Orchestrator running you might ask? This Orchestrator is responsible for maintaining my environment. Think about tasks like cleaning up snapshots, removing mounted CD-ROM media and adding new Virtual Machines to my backup schedule.

Let’s analyze the problem:

To check what was going-on I created a Workflow with a Scriptable Task that outputs text (the Workflow is displayed in the screenshot below). After realizing that vRO was not able to output any information to the screen it was time to start troubleshooting.

To start troubleshooting I connected with SSH to the vRealize Orchestrator appliance and started looking at the log files. Because vRO consists out of a lot of smaller sub systems it’s important to find the right log file.

  1. First I ran a command to show all log files with the latest change date. It appeared that the “catalina.out” was very busy. Every time I ran the command the file had a new timestamp.
    ls -ltur /var/log/vmware/vco/app-server/
  2. So lets look in the log file to figure out who is logging to this file:
    tail -f /var/log/vmware/vco/app-server/catalina.out
  3. It appeared that Apache Lucene was generating a continuous stream of messages/errors.
    Caused by: java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene6 0' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current cl asspath supports the following names: [Lucene62]
    at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116)
    at org.apache.lucene.codecs.Codec.forName(Codec.java:116)
    at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:424)
    ... 103 more
    19:30:49,588 WARN StubbornRetrier:38 - Retry 1/6 failed because of: Could not load codec 'Lucene60'. Did you forget to add lucene-backward-codecs.jar?
    19:30:49,609 WARN StubbornRetrier:38 - Retry 5/6 failed because of: Could not load codec 'Lucene60'. Did you forget to add lucene-backward-codecs.jar?
    19:30:49,611 WARN StubbornRetrier:38 - Retry 3/6 failed because of: Could not load codec 'Lucene60'. Did you forget to add lucene-backward-codecs.jar?

As I explained before vRO consists out of multiple smaller systems and Apache Lucene is one of them. So what is Apache Lucene exactly?

Apache LuceneTM is a high-performance, full-featured text search engine library written entirely in Java. It is a technology suitable for nearly any application that requires full-text search, especially cross-platform.

vRealize Orchestrator 7.4 – Version

vRealize Orchestrator 7.4 – Scriptable Task Output Logging

vRealize Orchestrator 7.4 – No log output

vRealize Orchestrator 7.4 – Catalina Log File

Procedure:

When searching on the log file error I directly found a VMware KB to resolve this issue.

  1. Start a SSH session with the vRO Appliance.
  2. Login with the root account.
  3. Stop the vRO server service.
    service vco-server stop
  4. Run the following command to remove the old lucene logs, this might be a single file or multiple files.
    rm -rf /var/log/vco/app-server/scripting.log_lucene*
  5. Start the vRO server service.
    service vco-server start

vRealize Orchestrator 7.4 – Login

vRealize Orchestrator 7.4 – Applying Fix

Test:

To confirm the procedure/fix I ran the test workflow again. The screenshot is listed below and shows output to the screen. This is the expected behavior so we can conclude that the fix solved the problem.

Based on the information already available on the internet it seems that the upgrade from vRealize Orchestrator 7.3.1 to 7.4 is the root cause of the problem. The positive side of the problem is that the vRealize Orchestrator engine is just doing its work. In my lab environment I discovered the problem three weeks after the upgrade. To clarify: all my workflows that were running in these three weeks were not experiencing any problems.

vRealize Orchestrator 7.4 – Output Working

Source:

Removing a Virtual Machine from vRealize Automation with the vRealize CloudClient

0

Overview

Today I was dealing with an issue in vRealize Automation (vRA). The customer had deleted some virtual machines in the vCenter inventory that were managed by vRealize Automation (vRA). So this resulted in registered virtual machines in vRealize Automation that were reported as missing.

For this blog post I recreated the problem in my Lab environment to have the ability to share the screenshots.

vRealize Automation Missing Virtual Machines

vRealize Automation 7.3 – Infrastructure – Managed Machines

Environment

Like explained in the beginning of this blog post. I recreated the problem in my home lab environment. To give you some background information about this environment. The Lab environment is running the following products and versions:
– VMware vCenter 6.5
– VMware ESXi 6.5
– VMware vRealize Automation 7.3.1
– VMware vRealize Orchestrator 7.3.1
– VMware NSX-V 6.4.0

The clusters are split into a “Mgmt & Edge” Cluster and a “Payload” Cluster. The virtual machines that were managed by vRealize Automation were located on the “Payload” cluster. So to recreate the problem I removed the virtual machines in the “Payload” cluster.

Incase you are interested in more information about my Home Lab, you can go to the following page.

Procedure:

  1. Download the latest version of the vRealize ClientCloud from the VMware website (the link can be found below).
  2. When the download is done, unzip the zip file (VMware_vRealize_CloudClient-4.5.0-8227624.zip).
  3. Navigate to the following directory (VMware_vRealize_CloudClient-4.5.0-8227624bin).
  4. Start the CloudClient (Windows: cloudclient.bat / Linux: cloudclient.sh).
  5. Log in to the vRA Appliance:
    Syntax:
    vra login userpass --user admin ID --tenant Tenant Name --server vRA FQDN --password Password

    Example:
    vra login userpass --user "M. Buijs" --password VMware1! --tenant Production --server https://cmp.lab.local

  6. Log in to the IaaS Machine:
    Syntax:
    vra login iaas --user service account --domain the domain --server web server --password Password

    Example:
    vra login iaas --user svc-vra-iaas --password VMware1! --domain lab.local --server iaasweb.lab.local

  7. Remove the virtual machine:
    Syntax:
    vra machines forceunregister --name %VMname%

    Example:
    vra machines forceunregister --name Prod-IT-00021

Keep in mind: It may take 5 to 10 minutes to remove the virtual machine from the interface.

vRealize CloudClient

vRealize CloudClient 4.5.0 – Started Application

vRealize CloudClient 4.5.0 – Authentication and removal

Links:

Here are some useful links related to the problem:
– VMware KB2144269 – Removing a virtual machine from vRealize Automation 7.x using CloudClient
– VMware vRealize CloudClient 4.5.0 – Download Page

The VMware vSAN 2017 Specialist Exam

0
VMware vSAN 2017 Specialist - Badge
VMware vSAN 2017 Specialist – Badge

This afternoon I took the VMware vSAN 2017 Specialist and I passed with a score of 456 points. The exam is about… guess what Virtual SAN (vSAN)! To be more precise the latest version of vSAN version 6.6. The exam is a mix of deployment & design questions. In my opinion it is a smaller version of the separated Deploy & Design exams as in the major exam tracks (DCV, NV, CMA, DTM). The questions were really good and realistic, some examples: like about space efficiency, cluster sizing and some customer use cases. The exam contains 60 questions and you have 105 minutes to complete the exam.

Over the years I have done a lot of vSAN deployments, starting in the vSAN 5.5 days. So personally it was not the most difficult exam to pass but a good one to add to my résumé.

Exam description

The official exam description from VMware: “The VMware vSAN 2017 Specialist badge holder is a technical professional who understands the vSAN 6.6 architecture and its complete feature set, knows how to conduct a vSAN design and deployment exercise, can implement a live vSAN hyper-converged infrastructure environment based on certified hardware/software components and best practices, and can administer/operate a vSAN cluster properly.”

The Specialist Exams

At VMworld 2017 three new specialist exams were announced by VMware Education:

  • VMware vRealize Operations 2017 Specialist
  • VMware vSAN 2017 Specialist Exam
  • VMware Validated Designs for Software-Defined Data Center 2017

Currently the remaining two are on my “To Do” list because they all cover my area of expertise. The vSAN Specialist exam is the only one of the three that is done at a test center, the other two are online exams.

Study Material:

There is a lot of free vSAN 6.6 content available, especially the VMworld 2017 sessions are full of useful information. Take a look Duncan Epping his YouTube Channel (the URL listed below).

PowerCLI 10.0.0 – Error Invalid Server Certificate

0

Overview

On the 28 of February VMware PowerCLI 10.0.0 was released. The biggest change in this release is the multi-platform support which includes the support for Mac OS and Linux. Oh and they kind of increased the version number a bit… from 6.5.X to 10.0.0 ;).

So I decided to upgrade PowerCLI in my Lab environment. In my Lab environment, I have a Windows 10 virtual machine that runs as an Administrator Workstation. The upgrade was very smooth and it took about one minute but after the upgrade, no connections were possible with my vCenter server.

The following error was displayed in the PowerShell prompt, screenshots are displayed below:
Error: Invalid server certificate. Use Set-PowerCLIConfiguration to set the value for the InvalidCertificateAction option to Prompt exception for this server.

It appears that they have changed the default PowerCLI behaviour regarding certificates.

Solution

Note: The solution I offer below is going ignore the SSL certificate checking, make sure this is something you want. A better way to validate this is to sign the certification of the vCenter Server with for example an Enterprise Certificate Authority. A write-up can be found here.

Procedure:

  1. After the PowerCLI 10.0.0 upgrade on your system, you try to connect to your vCenter Server, but you receive the following error (image 01).
  2. The error is pointing us to the following PowerCLI command-let “Get-PowerCLIConfiguration” (image 02). It appears that based on the output of the command the “InvalidCertificateAction” is configured as “Unset“.
  3. Let’s look at the possible settings available for “InvalidCertificateAction” (image 03). In my case “Ignore” is the best option because I’m running a test environment.
  4. Let’s change the setting to “Ignore” to disable the certificate check (Image 04).
  5. Now it’s time to connect with your vCenter Server again (Image 05).
  6. Everything should be working again without certificate warning messages.

Image(s)

PowerCLI 10.0.0 – 01 – Certificate Error
PowerCLI 10.0.0 – 02 – Show current SSL settings
PowerCLI 10.0.0 – 03 – List options
PowerCLI 10.0.0 – 04 – Configure settings for users
PowerCLI 10.0.0 – 05 – Connections are possible again

NLVMUG 2018 Event

0

This week I attended the NLVMUG 2018 on the 20th of march. The NLVMUG is the largest VMUG (VMware User Group) in the world and is held in the Netherlands. More than a thousand people attend the NLVMUG. The keynote was given than by no other than Pat Gelsinger the Chief Executive Officer (CEO) of VMware. The afternoon keynote was given by Joshua McKenty the Vice President from Pivatal.

The NLVMUG 2018 is a wrap but a great success!! Thanks to the VMUG leaders: Joep Piscaer en Dennis Hoegen Dijkhof.

NLVMUG 2018 – Overview

VMware Certified Implementation Expert – Network Virtualization (VCIX-NV)

0

I’m very happy to announce that I passed the VCAP6-NV Deploy exam and unlocked the VCIX-NV accreditation!

About the VMware VCIX-NV:VMware VCIX-NV

The VCIX-NV exam consists of approximately 23 live lab activities and the passing score for this exam is 300 (scale is from 100 to 500). The total time for this exam is 210 minutes, but candidates who take the VCIX-NV Exam and have a home address in a country where English is not a primary language will have an additional 30 minutes added to the exam time.

VMware - Network Virtualization Roadmap
VMware – Network Virtualization (NV) Roadmap

 

Study material:

For my study I used the following list of website’s, HOL Labs and Blogs. This helped me to pass the exam:

VMware Hands-on Labs:

VMware Sources:

Blogs:

My best advice is: Build a Home Lab and deploy VMware NSX-V. After the deployment you start using all the features that NSX-V has to offer (yeah, I know that is a lot). Get familiar and deploy and design like you would in a production environment. This will help you to get the best understanding possible for the exam.

Selected as a VMware vExpert 2018

1

Announcements

VMware vExpert 2018 - Notification

Last Friday VMware announced the vExperts for 2018. I’m pleased to announce that I have been rewarded with a vExpert 2018 award. This means I have been selected for a double in a row! I would like to congratulate everyone that has been awarded by VMware and VMware for supporting the community.

I’m also pleased to tell that my employer ITQ has fifteen vExperts for 2018. Congrats guys! For those interested ITQ is a VMware Premier Partner (the highest level in the VMware Partner Professional Services Program) and we are an independent VMware knowledge partner.

The VMware vExpert award?

The VMware vExpert program is VMware’s global evangelism and advocacy program. The program is designed to put VMware’s marketing resources towards your advocacy efforts. Promotion of your articles, exposure at our global events, co-op advertising, traffic analysis, and early access to beta programs and VMware’s roadmap.

Each year, we bring together in the vExpert Program the people who have made some of the most important contributions to the VMware community. These are the bloggers, book authors, VMUG leaders, speakers, tool builders, community leaders and general enthusiasts. They work as IT admins and architects for VMware customers, they act as trusted advisors and implementors for VMware partners or as independent consultants, and some work for VMware itself. All of them have the passion and enthusiasm for technology and applying technology to solve problems. They have contributed to the success of us all by sharing their knowledge and expertise over their days, nights, and weekends.

Website(s)

Lab Environment – New HP ProLiant DL360e G8

2

Today a blog about my Home Lab. At the end of 2017, it was time to replace the old Dell PowerEdge R710 servers with something new. Currently, I was running two R710 servers for my lab environment.
These servers are ‘powered-on for a couple of hours a week to test new products and learn for certifications. My other environment described on this page is running 24/7 is providing a full set of infrastructure services.

Because of the price and I’m very familiar with the DL360 G8 it was a no brainer. Over the last couple of years, all my virtualization projects were based for 75% on the DL360 Gen8… so we have a lot of history together ;).

Technical specifications – HP DL360e Gen8:

  • Chassis: Small Form Factor (8-bays)
  • CPU1: Intel® Xeon® Processor E5-2430 v2
  • CPU2: Intel® Xeon® Processor E5-2430 v2
  • Memory: 128 GB (8x 16GB DDR3 1600 MHz)
  • Disks HDD: 2x Seagate Constellation SAS 1TB
  • Disks SSD: 4x Samsung EVO 850 250 GB
  • Storage controller: HP SmartArray P420 with 1 GB FBWC
  • NIC: 4 port 1 Gbit
  • Rack mounting kit, cable arm, and security front bezel

The spinning drives provide “safe” storage because they are configured as a mirrored volume. The SSD drives are configured as JBOD drives for performance without data protection (if I want protection I just create a virtual machine back-up to one of my storage arrays).

HPE ProLiant DL360e G8 - SFF
HPE ProLiant DL360e G8 – SFF
HPE ProLiant DL360e G8 - SFF with Bezel
HPE ProLiant DL360e G8 – SFF with Bezel

Update 2018:

In 2018, I did a full write-up of this server on this page: link