Hi Brian, The following string (some fields changed for security reasons) is passed to the freeswitch.Session function for originating the call. {originate_timeout='60',ignore_early_media='false',origination_uuid='b4cd3cec-ba43-4e52-82eb-6f8ac11f6850',origination_caller_id_name='XXXXXXXXXXXXX',origination_caller_id_number='XXXXXXXXXXXXX',return_ring_ready='true',sip_cid_type='pid',origination_privacy='hide_name:screen',sip_h_X-Custom-Header1 ='1111',sip_h_X-Custom-Header2='222',sip_h_X-Custom-Header3 ='abcde',sip_h_X-Custom-Header4='1'}sofia/local_eth0/XXXXXXXXX@X.X.X.X:5060" We actually pass a JSON object to the pyrun application as arguments which contains more data we need during the call, like which files to play, etc. The Originate String above is contained inside this JSON. Please let me know to which originateCommand you were referring. Are the origination variables treated as extensions? The X-Customer-Headers may vary in amount passed (Usually 4 or 5), the rest of the variables are always there, sometimes with different values. Thanks and regards, Grant From: FreeSWITCH-users <freeswitch-users-bounces@lists.freeswitch.org> On Behalf Of Brian West Sent: maandag 11 maart 2019 18:27 To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] too many stacked extensions What are you passing in originateCommand? /b On Mon, Mar 11, 2019 at 8:00 AM Grant Bagdasarian <gb@cm.nl<mailto:gb@cm.nl>> wrote: Hello, We’ve recently upgraded some of our servers to 1.8.5 and are now occasionally experiencing the error “too many stacked extensions” for some calls. According to Jira this has been patched since 1.8.1, but perhaps this is due to some other scenario being triggered. We’re using ESL (bgapi command) to spawn a python script which in turn does an originate. See below a snippet of the cli output. As far as I can tell, this doesn’t have any real consequences resource consumption wise (CPU and memory), only the disk will get hammered in the case when logging to syslog. Also, I haven’t been able to reproduce this myself, but I’m still in the process of doing more testing. It seems to be happening before the call is connected to the destination. Any ideas how to properly debug this and find out what’s triggering this? I’ve tried the following: * Monitoring all ESL events * fsctl debug_level 10 * sofia loglevel all 9 They all output a lot of debugging information, which does get overwhelming. In a nutshell, this is the flow: ESL: bgapi pyrun app originateCommand Python: session = freeswitch.Session(originateCommand) session.execute("wait_for_answer") session.streamFile(file) 2019-03-11 12:58:08.133457 [ALERT] switch_core_session.c:1047 Send signal sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> [BREAK] 2019-03-11 12:58:08.133457 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.133457 [ALERT] switch_core_state_machine.c:705 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> session thread wake state: CS_SOFT_EXECUTE! 2019-03-11 12:58:08.133457 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.133457 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.133457 [ALERT] switch_core_state_machine.c:701 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> session thread sleep state: CS_SOFT_EXECUTE! 2019-03-11 12:58:08.153454 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.173495 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.193453 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.213458 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.233455 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.253489 [ALERT] switch_core_session.c:1047 Send signal sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> [BREAK] 2019-03-11 12:58:08.253489 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.253489 [ALERT] switch_core_state_machine.c:705 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> session thread wake state: CS_SOFT_EXECUTE! 2019-03-11 12:58:08.253489 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.253489 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.253489 [ALERT] switch_core_state_machine.c:701 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> session thread sleep state: CS_SOFT_EXECUTE! 2019-03-11 12:58:08.273448 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.293454 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.313459 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.333443 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.353456 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.373450 [ALERT] switch_core_session.c:1047 Send signal sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> [BREAK] 2019-03-11 12:58:08.373450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.373450 [ALERT] switch_core_state_machine.c:705 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> session thread wake state: CS_SOFT_EXECUTE! 2019-03-11 12:58:08.373450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.373450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.373450 [ALERT] switch_core_state_machine.c:701 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> session thread sleep state: CS_SOFT_EXECUTE! 2019-03-11 12:58:08.393440 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.413450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.433452 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.453461 [ERR] switch_ivr.c:904 Error sofia/local_eth0/00393421615027@10.14.26.10:5060<http://00393421615027@10.14.26.10:5060> too many stacked extensions 2019-03-11 12:58:08.473474 [ALERT] switch_core_session.c:1047 Send signal sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> [BREAK] 2019-03-11 12:58:08.473474 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.473474 [ALERT] switch_core_state_machine.c:705 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> session thread wake state: CS_SOFT_EXECUTE! 2019-03-11 12:58:08.473474 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.473474 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.473474 [ALERT] switch_core_state_machine.c:701 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> session thread sleep state: CS_SOFT_EXECUTE! 2019-03-11 12:58:08.493452 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.513447 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio timeout 2019-03-11 12:58:08.513447 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio stat 99.00 301/302 flaws: 1 mos: 4.49 v: 0.11 0.11/1.78 2019-03-11 12:58:08.513447 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.533439 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio timeout 2019-03-11 12:58:08.533439 [ALERT] switch_rtp.c:1565 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio 2 consecutive flaws, adding 2 flaw penalty 2019-03-11 12:58:08.533439 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio stat 98.00 298/302 flaws: 4 mos: 4.48 v: 0.11 0.11/1.78 2019-03-11 12:58:08.533439 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.553450 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio timeout 2019-03-11 12:58:08.553450 [ALERT] switch_rtp.c:1565 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio 3 consecutive flaws, adding 2 flaw penalty 2019-03-11 12:58:08.553450 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio stat 97.00 295/302 flaws: 7 mos: 4.47 v: 0.11 0.11/1.78 2019-03-11 12:58:08.553450 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.573441 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio timeout 2019-03-11 12:58:08.573441 [ALERT] switch_rtp.c:1565 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio 4 consecutive flaws, adding 2 flaw penalty 2019-03-11 12:58:08.573441 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio stat 96.00 292/302 flaws: 10 mos: 4.46 v: 0.11 0.11/1.78 2019-03-11 12:58:08.573441 [ERR] switch_ivr.c:904 Error sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> too many stacked extensions 2019-03-11 12:58:08.593450 [ALERT] switch_rtp.c:7695 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio timeout 2019-03-11 12:58:08.593450 [ALERT] switch_rtp.c:1565 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio 5 consecutive flaws, adding 2 flaw penalty 2019-03-11 12:58:08.593450 [ALERT] switch_rtp.c:1584 sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060<mailto:sofia/local_eth0/XXXXXXXXXX@X.X.X.X:5060> audio stat 95.00 289/302 flaws: 13 mos: 4.44 v: 0.11 0.11/1.78 Regards, Grant _________________________________________________________________________ Professional FreeSWITCH Services sales@freeswitch.com<mailto:sales@freeswitch.com> https://freeswitch.com Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org<mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com -- Brian West | Co-founder and Developer Need Commercial support? email sales@freeswitch.com<mailto:sales@freeswitch.com> FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045<https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g> Email: brian@freeswitch.com<mailto:brian@freeswitch.com> Mobile: 918-424-9378 Website: https://www.FreeSWITCH.com<https://www.freeswitch.com/> [https://www.facebook.com/signalwireinc?src=email]<https://www.facebook.com/freeswitch>[https://twitter.com/freeswitch]<https://twitter.com/freeswitch> ________________________________ Please be cautious This email was sent outside of your organisation ________________________________