Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    [BUG?] New PPPoE module (if_pppoe) causes high "Errors Out" on WAN (Vivo Fibra)

    Scheduled Pinned Locked Moved Routing and Multi WAN
    16 Posts 6 Posters 296 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      dorabiatto
      last edited by

      [EN_US]

      Hi everyone,

      I'm testing the new pfSense feature that allows using the kernel-based PPPoE driver (if_pppoe) instead of the traditional mpd5, by enabling the following option:

      System > Advanced > Networking

      93fe8c34-aed1-4d53-a896-91e6c3f14d4f-image.png

      In my setup, enabling this option successfully establishes the PPPoE connection, but I immediately start seeing a large number of "Errors Out" on the WAN interface connected to Vivo Fibra (PPPoE ISP).

      290ec121-510a-43e7-9d89-83b6b85af84c-image.png

      Additional details:
      Affected interface: WAN - VIVO_FIBRA

      NIC: Ethernet Controller I226-V
      ONU/Modem: MitraStar GPT-2742GX4X5

      Hardware checksum, TCP segmentation offload, and large receive offload are all disabled (all boxes checked under System > Advanced > Networking)

      MTU is set to 1492

      As soon as I revert back to the traditional mpd5 PPPoE client, the errors stop entirely.

      Question:
      Has anyone else experienced this issue with the new if_pppoe module?
      Could this be a bug or a driver compatibility issue?

      I'm happy to provide logs or run further tests if needed.

      Thanks in advance!


      [PT_BR]

      Olá a todos,

      Estou testando a nova opção do pfSense que permite usar o driver PPPoE via kernel (if_pppoe) em vez do tradicional mpd5, ativando a opção em:

      System > Advanced > Networking

      93fe8c34-aed1-4d53-a896-91e6c3f14d4f-image.png

      No meu caso, ao ativar essa opção, a conexão funciona normalmente, porém percebo um aumento imediato e constante de "Errors Out" na interface WAN que está conectada à Vivo Fibra (PPPoE).

      290ec121-510a-43e7-9d89-83b6b85af84c-image.png

      Informações adicionais:
      Interface afetada: WAN - VIVO_FIBRA

      NIC: Ethernet Controller I226-V
      ONU/Modem: MitraStar GPT-2742GX4X5

      Hardware checksum, TSO e LRO estão desativados (todas as opções marcadas em System > Advanced > Networking)

      MTU da interface está em 1492

      Quando retorno para o driver tradicional (mpd5), os erros desaparecem completamente.

      Pergunta:
      Alguém mais percebeu esse comportamento com o novo if_pppoe?
      Pode ser um bug no módulo ou alguma incompatibilidade com o driver da NIC?

      Agradeço qualquer insight. Estou disposto a fazer testes e enviar logs se necessário.

      J 1 Reply Last reply Reply Quote 0
      • J
        jonathan.young @dorabiatto
        last edited by

        I am experiencing the same problem I think with my Intel n100 nuc which has 2x Intel i226-v nics.

        As soon as I turn on "Use if_pppoe kernel module for PPPoE client" and reboot, the web interface is inoperable and syslog is flooded with "check_reload_status[518]: Could not connect to /var/run/php-fpm.socket"
        messages. I can ssh into the machine and the wan and lan interfaces do botth seem connected. The only solution I have found is to reboot again with the wan interface disconnected and disable "Use if_pppoe kernel module for PPPoE client" option and reboot again.

        I would like to be able to use this functionality if I can but at the moment it seems to be broken with the igc driver.

        1 Reply Last reply Reply Quote 0
        • patient0P
          patient0
          last edited by

          Maybe it is the same issue as Netgate forum thread: WAN out errors since 2.8 upgrade?

          brookheatherB 1 Reply Last reply Reply Quote 0
          • brookheatherB
            brookheather @patient0
            last edited by

            @patient0 Yes I am having the same problem - strange that there aren't more reports of this issue with enabling if_pppoe.

            S 1 Reply Last reply Reply Quote 0
            • S
              slu @brookheather
              last edited by

              @brookheather said in [BUG?] New PPPoE module (if_pppoe) causes high "Errors Out" on WAN (Vivo Fibra):

              strange that there aren't more reports of this issue with enabling if_pppoe.

              see also some errors, but without this thread I wouldn't have noticed it at all:

              In/out packets    [..] (144.98 GiB/8.38 GiB) 
              In/out packets (pass)    [..] (144.98 GiB/8.38 GiB) 
              In/out packets (block)    [..] (2.21 MiB/60 KiB) 
              In/out errors    0/226 
              Collisions    0
              

              pfSense Gold subscription

              brookheatherB 1 Reply Last reply Reply Quote 0
              • brookheatherB
                brookheather @slu
                last edited by

                @slu thanks for confirming - @stephenw10 please note these additional reports of WAN out errors with if_pppoe.

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  OK if you can try running this at the actual CLI (not the gui command prompt!) and wait for some errors to occur:
                  dtrace -n 'fbt::if_inc_counter:entry / arg1 == 3 / { stack(); }'

                  I managed to trigger some errors in my local test connection by restarting the server but you should see some pretty quickly with the rate you're seeing.

                  It should look something like this:

                  2.8.0-RELEASE][admin@m370.stevew.lan]/root: dtrace -n 'fbt::if_inc_counter:entry / arg1 == 3 / { stack(); }'
                  dtrace: description 'fbt::if_inc_counter:entry ' matched 1 probe
                  
                  CPU     ID                    FUNCTION:NAME
                    0  43915             if_inc_counter:entry 
                                if_pppoe.ko`sppp_output+0x15e
                                kernel`pf_route+0x8e5
                                kernel`pf_test+0x1093
                                kernel`pf_check_out+0x2e
                                kernel`pfil_mbuf_out+0x38
                                kernel`ip_output+0xbea
                                kernel`rip_send+0x3f4
                                kernel`sosend_generic_locked+0x60f
                                kernel`sosend_generic+0x61
                                kernel`sousrsend+0x5f
                                kernel`kern_sendit+0x144
                                kernel`sendit+0x1a8
                                kernel`sys_sendto+0x4d
                                kernel`amd64_syscall+0x115
                                kernel`0xffffffff812c145b
                  

                  Use ctl+c to esacpe once you have some traces.

                  brookheatherB 1 Reply Last reply Reply Quote 0
                  • brookheatherB
                    brookheather @stephenw10
                    last edited by

                    @stephenw10 here you go:

                    [2.8.0-RELEASE][admin@pfSense.localdomain]/root: dtrace -n 'fbt::if_inc_counter:entry / arg1 == 3 / { stack(); }'
                    dtrace: description 'fbt::if_inc_counter:entry ' matched 1 probe
                    CPU     ID                    FUNCTION:NAME
                      3  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      3  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      3  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      3  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      3  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      3  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      3  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      3  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      3  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      3  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      3  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      3  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      3  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      1  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      0  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`netisr_dispatch_src+0x1fa
                                  kernel`ether_demux+0x194
                                  kernel`ether_nh_input+0x32c
                                  kernel`netisr_dispatch_src+0x9f
                                  kernel`ether_input+0x56
                                  kernel`iflib_rxeof+0xbde
                                  kernel`_task_fn_rx+0x72
                                  kernel`gtaskqueue_run_locked+0x14e
                                  kernel`gtaskqueue_thread_loop+0xc2
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      1  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      1  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`swi_net+0x128
                                  kernel`ithread_loop+0x239
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      0  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`netisr_dispatch_src+0x1fa
                                  kernel`ether_demux+0x194
                                  kernel`ether_nh_input+0x32c
                                  kernel`netisr_dispatch_src+0x9f
                                  kernel`ether_input+0x56
                                  kernel`iflib_rxeof+0xbde
                                  kernel`_task_fn_rx+0x72
                                  kernel`gtaskqueue_run_locked+0x14e
                                  kernel`gtaskqueue_thread_loop+0xc2
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      0  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`netisr_dispatch_src+0x1fa
                                  kernel`ether_demux+0x194
                                  kernel`ether_nh_input+0x32c
                                  kernel`netisr_dispatch_src+0x9f
                                  kernel`ether_input+0x56
                                  kernel`iflib_rxeof+0xbde
                                  kernel`_task_fn_rx+0x72
                                  kernel`gtaskqueue_run_locked+0x14e
                                  kernel`gtaskqueue_thread_loop+0xc2
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      0  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`netisr_dispatch_src+0x1fa
                                  kernel`ether_demux+0x194
                                  kernel`ether_nh_input+0x32c
                                  kernel`netisr_dispatch_src+0x9f
                                  kernel`ether_input+0x56
                                  kernel`iflib_rxeof+0xbde
                                  kernel`_task_fn_rx+0x72
                                  kernel`gtaskqueue_run_locked+0x14e
                                  kernel`gtaskqueue_thread_loop+0xc2
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      0  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`netisr_dispatch_src+0x1fa
                                  kernel`ether_demux+0x194
                                  kernel`ether_nh_input+0x32c
                                  kernel`netisr_dispatch_src+0x9f
                                  kernel`ether_input+0x56
                                  kernel`iflib_rxeof+0xbde
                                  kernel`_task_fn_rx+0x72
                                  kernel`gtaskqueue_run_locked+0x14e
                                  kernel`gtaskqueue_thread_loop+0xc2
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      0  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`netisr_dispatch_src+0x1fa
                                  kernel`ether_demux+0x194
                                  kernel`ether_nh_input+0x32c
                                  kernel`netisr_dispatch_src+0x9f
                                  kernel`ether_input+0x56
                                  kernel`iflib_rxeof+0xbde
                                  kernel`_task_fn_rx+0x72
                                  kernel`gtaskqueue_run_locked+0x14e
                                  kernel`gtaskqueue_thread_loop+0xc2
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                      0  43915             if_inc_counter:entry 
                                  if_pppoe.ko`sppp_output+0x1ec
                                  kernel`pf_route+0x8e5
                                  kernel`pf_test+0x1093
                                  kernel`pf_check_out+0x2e
                                  kernel`pfil_mbuf_fwd+0x38
                                  kernel`ip_tryforward+0x288
                                  kernel`ip_input+0x34a
                                  kernel`netisr_dispatch_src+0x1fa
                                  kernel`ether_demux+0x194
                                  kernel`ether_nh_input+0x32c
                                  kernel`netisr_dispatch_src+0x9f
                                  kernel`ether_input+0x56
                                  kernel`iflib_rxeof+0xbde
                                  kernel`_task_fn_rx+0x72
                                  kernel`gtaskqueue_run_locked+0x14e
                                  kernel`gtaskqueue_thread_loop+0xc2
                                  kernel`fork_exit+0x7b
                                  kernel`0xffffffff812c1bae
                    
                    stephenw10S 1 Reply Last reply Reply Quote 1
                    • D
                      dorabiatto
                      last edited by

                      [EN_US]

                      Just wanted to share my experience that might help others dealing with the same issue described in this thread.

                      I was getting a lot of “Errors Out” on the WAN interface when using pfSense with PPPoE on Vivo Fibra, specifically with the Mitrastar ONU provided by the ISP. I tried changing MTU, tuning system settings, updating pfSense — nothing helped. The errors kept increasing, affecting connection stability and performance.

                      I decided to replace the Mitrastar ONU with a VSOL V2802RH ONU set to bridge mode — and that completely stopped the errors. Since the switch, pfSense shows at most 1 single error out right after boot, and then nothing more.

                      Summary:

                      Before (Mitrastar): constant Errors Out, especially under load.

                      After (VSOL): only 1 error after boot, then stable with no further errors.

                      So it seems the issue is likely related to some kind of compatibility quirk between the Mitrastar ONU and the new if_pppoe module. Switching to a different ONU (VSOL) significantly reduces or eliminates the issue.

                      [PT-BR]

                      Fica aqui meu relato que pode ajudar outros usuários enfrentando o mesmo problema descrito neste tópico.

                      Estava tendo altos “Errors Out” na interface WAN ao usar pfSense com PPPoE da Vivo Fibra, especialmente com a ONU Mitrastar fornecida pela operadora. Nada resolvia: MTU ajustado, tunable, atualizações — e os erros persistiam, afetando a estabilidade e performance.

                      Resolvi trocar a ONU por uma VSOL XPON configurada em modo bridge — e os erros pararam completamente. Desde a troca, o pfSense mostra no máximo 1 erro Out isolado após boot, e depois disso a conexão segue estável, sem erros acumulando.

                      Resumo:

                      Antes (Mitrastar): errors out constantes, especialmente com alto tráfego.

                      Depois (VSOL): apenas 1 erro após boot, e mais nada.

                      Ou seja, o problema parece estar relacionado à compatibilidade (ou comportamento estranho) da Mitrastar com o módulo PPPoE novo (if_pppoe), e a troca de ONU resolve ou mitiga o problema drasticamente.

                      8ad45427-1bee-49c2-8fdd-678055d828b0-image.png

                      1 Reply Last reply Reply Quote 1
                      • stephenw10S
                        stephenw10 Netgate Administrator @brookheather
                        last edited by

                        @brookheather Did the PPPoE remain up during that time?

                        brookheatherB 1 Reply Last reply Reply Quote 0
                        • brookheatherB
                          brookheather @stephenw10
                          last edited by

                          @stephenw10 Yes those errors are beig produced while the PPPoE connection is up and running.

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            OK try:
                            dtrace -n 'fbt::if_sendq_enqueue:return / arg1 != 0 / { stack(); printf("=> %d", arg1); }'

                            If that shows ENOBUFS (55) then it looks like it's exhausting the available buffers on your system for some reason. We can try just increasing it to allow for larger bursts.

                            brookheatherB 1 Reply Last reply Reply Quote 0
                            • brookheatherB
                              brookheather @stephenw10
                              last edited by

                              @stephenw10 here is the output:

                              [2.8.0-RELEASE][admin@pfSense.localdomain]/root: dtrace -n 'fbt::if_sendq_enqueue:return / arg1 != 0 / { stack(); printf("=> %d", arg1); }'
                              dtrace: description 'fbt::if_sendq_enqueue:return ' matched 1 probe
                              CPU     ID                    FUNCTION:NAME
                                1  65524          if_sendq_enqueue:return 
                                            if_pppoe.ko`sppp_output+0x1d0
                                            kernel`pf_route+0x8e5
                                            kernel`pf_test+0x1093
                                            kernel`pf_check_out+0x2e
                                            kernel`pfil_mbuf_fwd+0x38
                                            kernel`ip_tryforward+0x288
                                            kernel`ip_input+0x34a
                                            kernel`swi_net+0x128
                                            kernel`ithread_loop+0x239
                                            kernel`fork_exit+0x7b
                                            kernel`0xffffffff812c1bae
                              => 55
                                1  65524          if_sendq_enqueue:return 
                                            if_pppoe.ko`sppp_output+0x1d0
                                            kernel`pf_route+0x8e5
                                            kernel`pf_test+0x1093
                                            kernel`pf_check_out+0x2e
                                            kernel`pfil_mbuf_fwd+0x38
                                            kernel`ip_tryforward+0x288
                                            kernel`ip_input+0x34a
                                            kernel`swi_net+0x128
                                            kernel`ithread_loop+0x239
                                            kernel`fork_exit+0x7b
                                            kernel`0xffffffff812c1bae
                              => 55
                                1  65524          if_sendq_enqueue:return 
                                            if_pppoe.ko`sppp_output+0x1d0
                                            kernel`pf_route+0x8e5
                                            kernel`pf_test+0x1093
                                            kernel`pf_check_out+0x2e
                                            kernel`pfil_mbuf_fwd+0x38
                                            kernel`ip_tryforward+0x288
                                            kernel`ip_input+0x34a
                                            kernel`swi_net+0x128
                                            kernel`ithread_loop+0x239
                                            kernel`fork_exit+0x7b
                                            kernel`0xffffffff812c1bae
                              => 55
                                1  65524          if_sendq_enqueue:return 
                                            if_pppoe.ko`sppp_output+0x1d0
                                            kernel`pf_route+0x8e5
                                            kernel`pf_test+0x1093
                                            kernel`pf_check_out+0x2e
                                            kernel`pfil_mbuf_fwd+0x38
                                            kernel`ip_tryforward+0x288
                                            kernel`ip_input+0x34a
                                            kernel`swi_net+0x128
                                            kernel`ithread_loop+0x239
                                            kernel`fork_exit+0x7b
                                            kernel`0xffffffff812c1bae
                              => 55
                                1  65524          if_sendq_enqueue:return 
                                            if_pppoe.ko`sppp_output+0x1d0
                                            kernel`pf_route+0x8e5
                                            kernel`pf_test+0x1093
                                            kernel`pf_check_out+0x2e
                                            kernel`pfil_mbuf_fwd+0x38
                                            kernel`ip_tryforward+0x288
                                            kernel`ip_input+0x34a
                                            kernel`swi_net+0x128
                                            kernel`ithread_loop+0x239
                                            kernel`fork_exit+0x7b
                                            kernel`0xffffffff812c1bae
                              => 55
                                1  65524          if_sendq_enqueue:return 
                                            if_pppoe.ko`sppp_output+0x1d0
                                            kernel`pf_route+0x8e5
                                            kernel`pf_test+0x1093
                                            kernel`pf_check_out+0x2e
                                            kernel`pfil_mbuf_fwd+0x38
                                            kernel`ip_tryforward+0x288
                                            kernel`ip_input+0x34a
                                            kernel`swi_net+0x128
                                            kernel`ithread_loop+0x239
                                            kernel`fork_exit+0x7b
                                            kernel`0xffffffff812c1bae
                              => 55
                                1  65524          if_sendq_enqueue:return 
                                            if_pppoe.ko`sppp_output+0x1d0
                                            kernel`pf_route+0x8e5
                                            kernel`pf_test+0x1093
                                            kernel`pf_check_out+0x2e
                                            kernel`pfil_mbuf_fwd+0x38
                                            kernel`ip_tryforward+0x288
                                            kernel`ip_input+0x34a
                                            kernel`swi_net+0x128
                                            kernel`ithread_loop+0x239
                                            kernel`fork_exit+0x7b
                                            kernel`0xffffffff812c1bae
                              => 55
                                1  65524          if_sendq_enqueue:return 
                                            if_pppoe.ko`sppp_output+0x1d0
                                            kernel`pf_route+0x8e5
                                            kernel`pf_test+0x1093
                                            kernel`pf_check_out+0x2e
                                            kernel`pfil_mbuf_fwd+0x38
                                            kernel`ip_tryforward+0x288
                                            kernel`ip_input+0x34a
                                            kernel`swi_net+0x128
                                            kernel`ithread_loop+0x239
                                            kernel`fork_exit+0x7b
                                            kernel`0xffffffff812c1bae
                              => 55
                                1  65524          if_sendq_enqueue:return 
                                            if_pppoe.ko`sppp_output+0x1d0
                                            kernel`pf_route+0x8e5
                                            kernel`pf_test+0x1093
                                            kernel`pf_check_out+0x2e
                                            kernel`pfil_mbuf_fwd+0x38
                                            kernel`ip_tryforward+0x288
                                            kernel`ip_input+0x34a
                                            kernel`swi_net+0x128
                                            kernel`ithread_loop+0x239
                                            kernel`fork_exit+0x7b
                                            kernel`0xffffffff812c1bae
                              => 55
                                1  65524          if_sendq_enqueue:return 
                                            if_pppoe.ko`sppp_output+0x1d0
                                            kernel`pf_route+0x8e5
                                            kernel`pf_test+0x1093
                                            kernel`pf_check_out+0x2e
                                            kernel`pfil_mbuf_fwd+0x38
                                            kernel`ip_tryforward+0x288
                                            kernel`ip_input+0x34a
                                            kernel`swi_net+0x128
                                            kernel`ithread_loop+0x239
                                            kernel`fork_exit+0x7b
                                            kernel`0xffffffff812c1bae
                              => 55
                                1  65524          if_sendq_enqueue:return 
                                            if_pppoe.ko`sppp_output+0x1d0
                                            kernel`pf_route+0x8e5
                                            kernel`pf_test+0x1093
                                            kernel`pf_check_out+0x2e
                                            kernel`pfil_mbuf_fwd+0x38
                                            kernel`ip_tryforward+0x288
                                            kernel`ip_input+0x34a
                                            kernel`swi_net+0x128
                                            kernel`ithread_loop+0x239
                                            kernel`fork_exit+0x7b
                                            kernel`0xffffffff812c1bae
                              => 55
                                1  65524          if_sendq_enqueue:return 
                                            if_pppoe.ko`sppp_output+0x1d0
                                            kernel`pf_route+0x8e5
                                            kernel`pf_test+0x1093
                                            kernel`pf_check_out+0x2e
                                            kernel`pfil_mbuf_fwd+0x38
                                            kernel`ip_tryforward+0x288
                                            kernel`ip_input+0x34a
                                            kernel`swi_net+0x128
                                            kernel`ithread_loop+0x239
                                            kernel`fork_exit+0x7b
                                            kernel`0xffffffff812c1bae
                              => 55
                              
                              
                              1 Reply Last reply Reply Quote 0
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by stephenw10

                                Ok great so it is exhausting the interface buffer.

                                Check the current buffer size: sysctl net.link.ifqmaxlen

                                Try increasing it by 50%. It's a loader value so create the file /boot/loader.conf.local and add the line:
                                net.link.ifqmaxlen=192

                                Then reboot. And see if that makes any difference.

                                brookheatherB 1 Reply Last reply Reply Quote 0
                                • brookheatherB
                                  brookheather @stephenw10
                                  last edited by brookheather

                                  @stephenw10 the current size is 128 - I will apply the change and reboot - might be tomorrow now. Why would this only be affecting the new if_pppoe code - how does it differ from the old code? Is it really an error if the buffer is full - doesn't it just wait until it can write the packet?

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    The if_pppoe code is very different so there could be any number of reasons.

                                    Another interesting question might be why it's only affecting a small number of users. Something unique about your connection maybe.

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post
                                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.