Home » pfSense: arpresolve: can’t allocate llinfo for X.X.X.X on emX

pfSense: arpresolve: can’t allocate llinfo for X.X.X.X on emX

After some network maintenance, my virtual pfSense firewall started to cause some major issues and parts of the network started to stop functioning. In the end, it came down to the following error message (arpresolve: can’t allocate llinfo for X.X.X.X on emX). Because this wasn’t the first time… it was time to do a proper blog post about this issue.

In this post, I’m going to describe the issues and solution to get your network back up and running.

Environment

Here is an overview of my current environment where the problem occurred. I created a basic image of the setup:

Basic Network Overview
Basic Network Overview

Not very spectacular, just two routers/firewalls connected with a interconnect network and they use the BGP Routing Protocol. On both sides, you have a set of VLANs connected to the router with a trunk. A so-called router on a stick topology. The reason I use the BGP routing protocol is related to my daily job. The BGP Routing Protocol is kind of the preferred one for VMware NSX.



Problem

The problem started when… I upgraded my physical Cisco Firewall with new firmware and IOS. The interconnect network between pfSense and the Cisco stopped working completely. At first, I wasted about 1.5 hours on the physical Cisco firewall instead of the pfSense appliance. Because it looked to me that the IOS update was causing issues.

I was completely wrong… the Cisco Firewall was running without any issues… but pfSense had developed a new feature…

When I looked in the pfSense interface and went to the “System Logs > System > General” there was a serious error message. The pfSense kernel reported the following issue (arpresolve: can’t allocate llinfo for 192.168.80.253 on em8). Here is a screen capture of the message:

pfSense kernel message: (arpresolve: can’t allocate llinfo for X.X.X.X on emX)

It was unable to learn any new ARP entries in the interconnect network. This was causing the Cisco ASA & pfSense appliance to not form a BGP relationship.

In the case of sending a simple ping, it was not possible. Here is a basic diagram of the issue. All networks on both sides are functioning but the two routers are not able to talk and exchange BGP routes.

Basic Network Failure
Basic Network Failure


Solution

In the end, there are two solutions available for solving the problem:

  • Option 01: Restart the entire pfSense appliance > Problem solved!
  • Option 02: Deactivate interface and activate interface:
    • Connect with Putty to the pfSense appliance.
    • Activate the shell.
    • Deactivate interface (ifconfig em8 down)
    • Activate interface (ifconfig em8 up)
    • Problem solved!

The root cause is not completely clear to me… In total, I encountered this issue for about ten times. This is what I figured out so far:

  • The problem occurred for the first time after installing the package OpenBGPd on pfSense.
  • The ARP issue is only triggered when BGP neighbour states change, not always but sometimes.
  • The issue only occurs on the interconnect network… all other networks just work.

Article Update Juli 2019

After some frustration, I finally found a temporary workaround. After restarting the interface I have a couple of seconds to enter a static ARP entry before it stops working or does not allow me to execute the command:

  • Create two SSH sessions (one for restarting the interface and one for creating the static ARP entry).
    • Session 01:
      • Deactivate interface (ifconfig em8 down)
      • Activate interface (ifconfig em8 up)
    • Session 02:
      • Create static arp entry (arp -s hs-fw01.home-server.local 28:6f:7f:02:45:15)
  • This should fix the problem, in my case, it is working now for about 1 month.

Anybody experiencing the same problems? Anybody who has a definitive solution? Please comment below :)!

Share this:

5 comments

  1. I see this message when the dynamic IP changes for my residential Verizon FIOS service. It takes quite a while to clear this message and actually get a new IP & update my dynamic DNS entries. Sorry, I can’t speak more to your experience here, as this is quite a different setup.

  2. Andrew says:

    I’m currently experiencing this issue, unrelated to BGP.

    Network is essentially, (ISP/WAN) edge router < 192.168.20.1 > 192.168.20.x < 192.168.20.250 > pfsense firewall < 192.168.1.1 > 192.168.1.x & (via VPN) 192.168.100.x

    Periodically I get the same system -> log arpresolve can’t allocate llinfo for 192.168.20.1 on mvnetaX.

    My setup was working without any incident, then I changed the edge router & things started to freak out.

    • Andrew says:

      Small update:

      I was able to resolve this issue by change my 192.168.20.250 interface from DHCP to static.

      The exact nature of the problem still escapes me though.

      The 192.168.20.250 interface could have it’s IP set via DHCP (which infers a link/route exists). But then once it’s IP was/is set, it would say that it had to access to the 192.168.20.x network. The reasons for the link being down post DHCP is bizarre. Also the DHCP server could see that the device was there, but the PFsense interface couldn’t see the DHCP server/router.

      There was some chatter that I came across but can’t find it again of the issue potentially boiling down to invalid MTU sizes for certain hardware manufacturers. Given I don’t need DHCP & I could take the static approach, I stopped digging into what was causing this issue. But I thought I’d come back here & report my findings.

      • Andrew says:

        edits:
        I was able to resolve this issue by CHANGING my…”
        “…it would say that it had NO access to the 192.168.20.x…”

        • Hello Andrew,

          Thanks for replying on the blog post, I just added a new section to the blog (Article Update Juli 2019).
          This update explains how I finally got it working without issues…. but it is still a workaround :(.

          Keep in mind it is still a workaround and not a fix :(.
          In some cases, a static ARP entry is not ideal.

          Best regards,

          Mischa Buijs

Leave a Reply