[Freeswitch-users] too many stacked extensions

Mon Mar 11 12:59:19 UTC 2019

Hello,

We've recently upgraded some of our servers to 1.8.5 and are now occasionally experiencing the error "too many stacked extensions" for some calls.
According to Jira this has been patched since 1.8.1, but perhaps this is due to some other scenario being triggered.

We're using ESL (bgapi command) to spawn a python script which in turn does an originate.

See below a snippet of the cli output.
As far as I can tell, this doesn't have any real consequences resource consumption wise (CPU and memory), only the disk will get hammered in the case when logging to syslog.
Also, I haven't been able to reproduce this myself, but I'm still in the process of doing more testing.
It seems to be happening before the call is connected to the destination.

Any ideas how to properly debug this and find out what's triggering this?
I've tried the following:

  *   Monitoring all ESL events
  *   fsctl debug_level 10
  *   sofia loglevel all 9

They all output a lot of debugging information, which does get overwhelming.

In a nutshell, this is the flow:
ESL:
bgapi pyrun app originateCommand

Python:
session = freeswitch.Session(originateCommand)
session.execute("wait_for_answer")
session.streamFile(file)

2019-03-11 12:58:08.133457 [ALERT] switch_core_session.c:1047 Send signal sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 [BREAK]
2019-03-11 12:58:08.133457 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.133457 [ALERT] switch_core_state_machine.c:705 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 session thread wake state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.133457 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.133457 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.133457 [ALERT] switch_core_state_machine.c:701 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 session thread sleep state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.153454 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.173495 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.193453 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.213458 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.233455 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.253489 [ALERT] switch_core_session.c:1047 Send signal sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 [BREAK]
2019-03-11 12:58:08.253489 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.253489 [ALERT] switch_core_state_machine.c:705 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 session thread wake state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.253489 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.253489 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.253489 [ALERT] switch_core_state_machine.c:701 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 session thread sleep state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.273448 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.293454 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.313459 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.333443 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.353456 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.373450 [ALERT] switch_core_session.c:1047 Send signal sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 [BREAK]
2019-03-11 12:58:08.373450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.373450 [ALERT] switch_core_state_machine.c:705 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 session thread wake state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.373450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.373450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.373450 [ALERT] switch_core_state_machine.c:701 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 session thread sleep state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.393440 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.413450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.433452 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.453461 [ERR] switch_ivr.c:904 Error sofia/local_eth0/00393421615027 at 10.14.26.10:5060 too many stacked extensions
2019-03-11 12:58:08.473474 [ALERT] switch_core_session.c:1047 Send signal sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 [BREAK]
2019-03-11 12:58:08.473474 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.473474 [ALERT] switch_core_state_machine.c:705 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 session thread wake state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.473474 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.473474 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.473474 [ALERT] switch_core_state_machine.c:701 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 session thread sleep state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.493452 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.513447 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio timeout
2019-03-11 12:58:08.513447 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio stat 99.00 301/302 flaws: 1 mos: 4.49 v: 0.11 0.11/1.78
2019-03-11 12:58:08.513447 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.533439 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio timeout
2019-03-11 12:58:08.533439 [ALERT] switch_rtp.c:1565 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio 2 consecutive flaws, adding 2 flaw penalty
2019-03-11 12:58:08.533439 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio stat 98.00 298/302 flaws: 4 mos: 4.48 v: 0.11 0.11/1.78
2019-03-11 12:58:08.533439 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.553450 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio timeout
2019-03-11 12:58:08.553450 [ALERT] switch_rtp.c:1565 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio 3 consecutive flaws, adding 2 flaw penalty
2019-03-11 12:58:08.553450 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio stat 97.00 295/302 flaws: 7 mos: 4.47 v: 0.11 0.11/1.78
2019-03-11 12:58:08.553450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.573441 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio timeout
2019-03-11 12:58:08.573441 [ALERT] switch_rtp.c:1565 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio 4 consecutive flaws, adding 2 flaw penalty
2019-03-11 12:58:08.573441 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio stat 96.00 292/302 flaws: 10 mos: 4.46 v: 0.11 0.11/1.78
2019-03-11 12:58:08.573441 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 too many stacked extensions
2019-03-11 12:58:08.593450 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio timeout
2019-03-11 12:58:08.593450 [ALERT] switch_rtp.c:1565 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio 5 consecutive flaws, adding 2 flaw penalty
2019-03-11 12:58:08.593450 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060 audio stat 95.00 289/302 flaws: 13 mos: 4.44 v: 0.11 0.11/1.78

Regards,

Grant
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20190311/c2e14e32/attachment.html>