Tag: Cisco

VMware vRealize Log Insight content pack for Cisco ASA

0

In this blog, we are going to set up the VMware vRealize Log Insight content pack for a Cisco ASA device for capturing syslog information. With setting up this pack we are able to provide a central location for storing the logging information and a way to maintain the data for longer periods of time.

Almost a year ago I moved from pfSense to a physical Cisco ASA firewall and it was time to improve the visibility into the firewall rules that were blocking and allowing traffic in my network. This was a nice opportunity to configure VMware vRealize Log Insight with an additional content pack.

Environment

When I was writing this blog post I was using the following software releases:

In essence, the procedure is the same for older and newer versions of Log Insight and a Cisco ASA.

Log Insight Content Pack

Let’s start by installing the content pack on vRealize Log Insight. Make sure you install the Cisco ASA content pack for vRealize Log Insight. This can be found in the VMware marketplace that is available in the central VMware vRealize Log Insight interface.

Here is a screenshot with the location of were you can find the content pack:

Cisco ASA Configuration

Login into your Cisco ASA firewall with a console or SSH session and configure the syslog settings as displayed below. Keep in mind this is an example configuration, change the config based on your needs!

Here is an basic configuration example:

config t
  logging enable
  logging timestamp
  logging trap debugging
  logging host %interface% %ip-address_syslog_facility%
exit

To verify the status of the configuration run the following commands


### Show configuration and logging forwarding status
show logging

### View configuration
show run | grep logging

Here is an example output of my Cisco ASA:

Viewing information

After everything has been set up the dashboards will be populated with information received from the Cisco ASA.

Here are some screenshots from my environment:

Here are some useful examples of with kind of information you can expect from the Cisco ASA Content Pack for vRealize Log Insight. I personally think it is one of the best free content packs because the dashboards are really good at providing a lot of information with good solid diagrams.

pfSense: arpresolve: can’t allocate llinfo for X.X.X.X on emX

29

After some network maintenance, my virtual pfSense firewall started to cause some major issues and parts of the network started to stop functioning. In the end, it came down to the following error message (arpresolve: can’t allocate llinfo for X.X.X.X on emX). Because this wasn’t the first time… it was time to do a proper blog post about this issue.

In this post, I’m going to describe the issues and solution to get your network back up and running.

Environment

Here is an overview of my current environment where the problem occurred. I created a basic image of the setup:

Basic Network Overview
Basic Network Overview

Not very spectacular, just two routers/firewalls connected with a interconnect network and they use the BGP Routing Protocol. On both sides, you have a set of VLANs connected to the router with a trunk. A so-called router on a stick topology. The reason I use the BGP routing protocol is related to my daily job. The BGP Routing Protocol is kind of the preferred one for VMware NSX.

Problem

The problem started when… I upgraded my physical Cisco Firewall with new firmware and IOS. The interconnect network between pfSense and the Cisco stopped working completely. At first, I wasted about 1.5 hours on the physical Cisco firewall instead of the pfSense appliance. Because it looked to me that the IOS update was causing issues.

I was completely wrong… the Cisco Firewall was running without any issues… but pfSense had developed a new feature…

When I looked in the pfSense interface and went to the “System Logs > System > General” there was a serious error message. The pfSense kernel reported the following issue (arpresolve: can’t allocate llinfo for 192.168.80.253 on em8). Here is a screen capture of the message:

pfSense kernel message: (arpresolve: can’t allocate llinfo for X.X.X.X on emX)

It was unable to learn any new ARP entries in the interconnect network. This was causing the Cisco ASA & pfSense appliance to not form a BGP relationship.

In the case of sending a simple ping, it was not possible. Here is a basic diagram of the issue. All networks on both sides are functioning but the two routers are not able to talk and exchange BGP routes.

Basic Network Failure
Basic Network Failure

Solution

In the end, there are two solutions available for solving the problem:

  • Option 01: Restart the entire pfSense appliance > Problem solved!
  • Option 02: Deactivate interface and activate interface:
    • Connect with Putty to the pfSense appliance.
    • Activate the shell.
    • Deactivate interface (ifconfig em8 down)
    • Activate interface (ifconfig em8 up)
    • Problem solved!

The root cause is not completely clear to me… In total, I encountered this issue for about ten times. This is what I figured out so far:

  • The problem occurred for the first time after installing the package OpenBGPd on pfSense.
  • The ARP issue is only triggered when BGP neighbour states change, not always but sometimes.
  • The issue only occurs on the interconnect network… all other networks just work.

Article Update Juli 2019

After some frustration, I finally found a temporary workaround. After restarting the interface I have a couple of seconds to enter a static ARP entry before it stops working or does not allow me to execute the command:

  • Create two SSH sessions (one for restarting the interface and one for creating the static ARP entry).
    • Session 01:
      • Deactivate interface (ifconfig em8 down)
      • Activate interface (ifconfig em8 up)
    • Session 02:
      • Create static arp entry (arp -s hs-fw01.home-server.local 28:6f:7f:02:45:15)
  • This should fix the problem, in my case, it is working now for about 1 month.

Anybody experiencing the same problems? Anybody who has a definitive solution? Please comment below :)!