Performance issues with Intel E810-XXV on pfSense 2.7.2 – Single RX queue and dropped packets

[ellison]

Hi everyone,

We're experiencing performance issues with a new server running pfSense 2.7.2 (FreeBSD 14.0-CURRENT), specifically when using the Intel E810-XXV 25GbE NIC (driver: ice v1.37.11-k).

When the network traffic approaches 3 Gbps, the system starts dropping packets, even though the NICs are 25G capable. This happens consistently and doesn't appear to be related to the switch (confirmed by the datacenter team).

Upon inspection with dmesg and sysctl, we found the following:

• Only 1 RX and TX queue is being used:

  ice0: Using 1 Tx and Rx queues
  

• The DDP module fails to load:

  ice0: The DDP package module (ice_ddp) failed to load or could not be found. Entering Safe Mode.
  

• Attempting to force DDP loading via loader.conf using ice_ddp_load="YES" had no effect.

This limitation seems to be causing poor CPU distribution, as we’re not seeing traffic balanced across cores as expected.

By contrast, in an older server (Dell R730) using an Intel X520 (ix driver), we get:

ix0: Using 8 RX queues 8 TX queues

This older setup handles the same or higher throughput without issues.

We suspect the problem lies with the current state of the ice driver or lack of proper support for the Intel E810 DDP packages on pfSense.

Hardware info:

• CPU: Intel Xeon Gold 5318Y (96 cores detected)
• NICs: Intel E810-XXV (4 ports)
• pfSense version: 2.7.2 CE (FreeBSD 14.0-CURRENT)

Questions:

1. Is there any known limitation of the ice driver in pfSense 2.7.2 regarding E810 cards?
2. Is there any way to enable multiple RX/TX queues or properly load the DDP package on pfSense?
3. Would pfSense Plus offer better support for E810 or ice driver features?

Thanks in advance!

[stephenw10]

Do you see the ice_ddp module loaded after boot in kldstat?

Did you import the module since it's not present in 2.7.2 by default? The module is present in the 2.7.2 iso.

[ellison]

Hello @stephenw10

# kldstat
Id Refs Address                Size Name
 1   30 0xffffffff80200000  339ce08 kernel
 2    1 0xffffffff8359d000     76f8 cryptodev.ko
 3    1 0xffffffff835a6000   5d7790 zfs.ko
 4    1 0xffffffff83b7e000    1e2b0 opensolaris.ko
 5    1 0xffffffff84711000     2220 cpuctl.ko
 6    1 0xffffffff84714000     22a4 if_urndis.ko
 7    1 0xffffffff84717000     9288 aesni.ko
 8    1 0xffffffff84721000     3158 amdtemp.ko
 9    1 0xffffffff84725000     2130 amdsmn.ko

Is there any way to enable this module?

[stephenw10]

Yes, like I said you need to extract it from the ISO image and upload to /boot/modules if you have not done so already.

[ellison]

Hi @stephenw10,

Thank you so much for your help — it worked perfectly!

For future reference, here's a summary of the steps I followed to manually load the ice_ddp module on pfSense CE 2.7.2 with an Intel E810-XXV NIC:

Goal:

Enable full DDP (Dynamic Device Personalization) support for Intel E810 in pfSense.

Steps:

1. Mount the pfSense ISO image (version 2.7.2) on a separate machine.

2. Locate the module: Inside the ISO, I found the ICE_DDP.KO file under:

   /boot/kernel/ICE_DDP.KO
   

3. Copy to pfSense:

   • Renamed it to ice_ddp.ko

   • Uploaded it to the firewall at:

     /boot/modules/ice_ddp.ko
     

4. Configured it to load at boot:
   Created /boot/loader.conf.local with the following line:

   ice_ddp_load="YES"
   

5. Rebooted the system.

After reboot:

• dmesg confirms:

  The DDP package was successfully loaded: ICE OS Default Package version 1.3.30.0
  

• kldstat shows the module as loaded.

• NIC now supports 24 Tx/Rx queues and MSI-X interrupts — huge improvement for 25G performance.

Thanks again

[stephenw10]

Nice. What sort of throughput are you actually seeing?

[ellison]

Hi! At the moment, the average traffic is around 4 Gbps