[Freeswitch-users] too many stacked extensions

Grant Bagdasarian gb at cm.nl
Mon Mar 11 20:31:41 UTC 2019 

Hi Brian,

The following string (some fields changed for security reasons) is passed to the freeswitch.Session function for originating the call.
{originate_timeout='60',ignore_early_media='false',origination_uuid='b4cd3cec-ba43-4e52-82eb-6f8ac11f6850',origination_caller_id_name='XXXXXXXXXXXXX',origination_caller_id_number='XXXXXXXXXXXXX',return_ring_ready='true',sip_cid_type='pid',origination_privacy='hide_name:screen',sip_h_X-Custom-Header1 ='1111',sip_h_X-Custom-Header2='222',sip_h_X-Custom-Header3 ='abcde',sip_h_X-Custom-Header4='1'}sofia/local_eth0/XXXXXXXXX at X.X.X.X:5060"

We actually pass a JSON object to the pyrun application as arguments which contains more data we need during the call, like which files to play, etc.
The Originate String above is contained inside this JSON. Please let me know to which originateCommand you were referring.

Are the origination variables treated as extensions? The X-Customer-Headers may vary in amount passed (Usually 4 or 5), the rest of the variables are always there, sometimes with different values.

Thanks and regards,

Grant

From: FreeSWITCH-users <freeswitch-users-bounces at lists.freeswitch.org> On Behalf Of Brian West
Sent: maandag 11 maart 2019 18:27
To: FreeSWITCH Users Help <freeswitch-users at lists.freeswitch.org>
Subject: Re: [Freeswitch-users] too many stacked extensions

What are you passing in originateCommand?

/b


On Mon, Mar 11, 2019 at 8:00 AM Grant Bagdasarian <gb at cm.nl<mailto:gb at cm.nl>> wrote:
Hello,

We’ve recently upgraded some of our servers to 1.8.5 and are now occasionally experiencing the error “too many stacked extensions” for some calls.
According to Jira this has been patched since 1.8.1, but perhaps this is due to some other scenario being triggered.

We’re using ESL (bgapi command) to spawn a python script which in turn does an originate.

See below a snippet of the cli output.
As far as I can tell, this doesn’t have any real consequences resource consumption wise (CPU and memory), only the disk will get hammered in the case when logging to syslog.
Also, I haven’t been able to reproduce this myself, but I’m still in the process of doing more testing.
It seems to be happening before the call is connected to the destination.

Any ideas how to properly debug this and find out what’s triggering this?
I’ve tried the following:

  *   Monitoring all ESL events
  *   fsctl debug_level 10
  *   sofia loglevel all 9

They all output a lot of debugging information, which does get overwhelming.

In a nutshell, this is the flow:
ESL:
bgapi pyrun app originateCommand

Python:
session = freeswitch.Session(originateCommand)
session.execute("wait_for_answer")
session.streamFile(file)

2019-03-11 12:58:08.133457 [ALERT] switch_core_session.c:1047 Send signal sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> [BREAK]
2019-03-11 12:58:08.133457 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.133457 [ALERT] switch_core_state_machine.c:705 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> session thread wake state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.133457 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.133457 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.133457 [ALERT] switch_core_state_machine.c:701 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> session thread sleep state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.153454 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.173495 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.193453 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.213458 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.233455 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.253489 [ALERT] switch_core_session.c:1047 Send signal sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> [BREAK]
2019-03-11 12:58:08.253489 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.253489 [ALERT] switch_core_state_machine.c:705 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> session thread wake state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.253489 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.253489 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.253489 [ALERT] switch_core_state_machine.c:701 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> session thread sleep state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.273448 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.293454 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.313459 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.333443 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.353456 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.373450 [ALERT] switch_core_session.c:1047 Send signal sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> [BREAK]
2019-03-11 12:58:08.373450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.373450 [ALERT] switch_core_state_machine.c:705 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> session thread wake state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.373450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.373450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.373450 [ALERT] switch_core_state_machine.c:701 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> session thread sleep state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.393440 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.413450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.433452 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.453461 [ERR] switch_ivr.c:904 Error sofia/local_eth0/00393421615027 at 10.14.26.10:5060<http://00393421615027@10.14.26.10:5060> too many stacked extensions
2019-03-11 12:58:08.473474 [ALERT] switch_core_session.c:1047 Send signal sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> [BREAK]
2019-03-11 12:58:08.473474 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.473474 [ALERT] switch_core_state_machine.c:705 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> session thread wake state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.473474 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.473474 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.473474 [ALERT] switch_core_state_machine.c:701 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> session thread sleep state: CS_SOFT_EXECUTE!
2019-03-11 12:58:08.493452 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.513447 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio timeout
2019-03-11 12:58:08.513447 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio stat 99.00 301/302 flaws: 1 mos: 4.49 v: 0.11 0.11/1.78
2019-03-11 12:58:08.513447 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.533439 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio timeout
2019-03-11 12:58:08.533439 [ALERT] switch_rtp.c:1565 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio 2 consecutive flaws, adding 2 flaw penalty
2019-03-11 12:58:08.533439 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio stat 98.00 298/302 flaws: 4 mos: 4.48 v: 0.11 0.11/1.78
2019-03-11 12:58:08.533439 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.553450 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio timeout
2019-03-11 12:58:08.553450 [ALERT] switch_rtp.c:1565 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio 3 consecutive flaws, adding 2 flaw penalty
2019-03-11 12:58:08.553450 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio stat 97.00 295/302 flaws: 7 mos: 4.47 v: 0.11 0.11/1.78
2019-03-11 12:58:08.553450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.573441 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio timeout
2019-03-11 12:58:08.573441 [ALERT] switch_rtp.c:1565 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio 4 consecutive flaws, adding 2 flaw penalty
2019-03-11 12:58:08.573441 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio stat 96.00 292/302 flaws: 10 mos: 4.46 v: 0.11 0.11/1.78
2019-03-11 12:58:08.573441 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> too many stacked extensions
2019-03-11 12:58:08.593450 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio timeout
2019-03-11 12:58:08.593450 [ALERT] switch_rtp.c:1565 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio 5 consecutive flaws, adding 2 flaw penalty
2019-03-11 12:58:08.593450 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX at X.X.X.X:5060> audio stat 95.00 289/302 flaws: 13 mos: 4.44 v: 0.11 0.11/1.78

Regards,

Grant
_________________________________________________________________________
Professional FreeSWITCH Services
sales at freeswitch.com<mailto:sales at freeswitch.com>
https://freeswitch.com

Official FreeSWITCH Sites
https://freeswitch.com/oss
https://freeswitch.org/confluence
https://cluecon.com

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org<mailto:FreeSWITCH-users at lists.freeswitch.org>
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
https://freeswitch.com


--


Brian West | Co-founder and Developer

Need Commercial support? email sales at freeswitch.com<mailto:sales at freeswitch.com>

FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045<https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g>

Email: brian at freeswitch.com<mailto:brian at freeswitch.com>

Mobile: 918-424-9378

Website: https://www.FreeSWITCH.com<https://www.freeswitch.com/>

[https://www.facebook.com/signalwireinc?src=email]<https://www.facebook.com/freeswitch>[https://twitter.com/freeswitch]<https://twitter.com/freeswitch>

________________________________
Please be cautious
This email was sent outside of your organisation
________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20190311/9f678a9f/attachment-0001.html>

More information about the FreeSWITCH-users mailing list