[Freeswitch-users] FreeSWITCH Stops Responding

Brian West brian at freeswitch.com
Thu Sep 2 19:24:42 UTC 2021


And these exhibit the issue on command?

On Wed, Sep 1, 2021 at 5:06 PM Cary Shouse <cary at ytel.com> wrote:

> Here are two originate samples.
>
> https://pastebin.com/YKRKrJLu
>
> https://pastebin.com/TEX3yfX6
>
>
>
> On Wed, Sep 1, 2021 at 8:00 PM Brian West <brian at freeswitch.com> wrote:
>
>> Can you try to create one where this happens without all this mess?
>>
>> On Wed, Sep 1, 2021 at 2:41 PM Cary Shouse <cary at ytel.com> wrote:
>>
>>> Here is the LUA code regarding creating sessions and bridging. It is
>>> broken up into 5 sections.
>>> The top part is the common functions, and the 4 sections below are
>>> different call scenarios.
>>> https://pastebin.com/qqGw3tms
>>>
>>>
>>> On Wed, Sep 1, 2021 at 3:53 PM Brian West <brian at freeswitch.com> wrote:
>>>
>>>> I you do sessions from your lua, I need to see how you do that.
>>>>
>>>> On Tue, Aug 31, 2021 at 4:20 PM Cary Shouse <cary at ytel.com> wrote:
>>>>
>>>>> Yes, it bridges the inbound call to an outbound one.
>>>>>
>>>>> (ProxyOutbound.luac)https://pastebin.com/RDUMy473
>>>>>
>>>>>
>>>>> On Tue, Aug 31, 2021 at 8:39 PM Brian West <brian at freeswitch.com>
>>>>> wrote:
>>>>>
>>>>>> Guessing you create sessions in lua? ProxyOutbound.luac?
>>>>>>
>>>>>> On Tue, Aug 31, 2021 at 3:33 PM Brian West <brian at freeswitch.com>
>>>>>> wrote:
>>>>>>
>>>>>>> and to make sure when you uuid kill one it says it didn't exist but
>>>>>>> show channels had it still listed correct?
>>>>>>>
>>>>>>> On Tue, Aug 31, 2021 at 2:58 PM Cary Shouse <cary at ytel.com> wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> We have got the logs for a few calls that were stuck.
>>>>>>>> (Show channels fe8f206c-2bd9-4af2-b329-35f53228a38a)
>>>>>>>> https://pastebin.com/NTnte7J7
>>>>>>>>
>>>>>>>> (Extracted from logs fe8f206c-2bd9-4af2-b329-35f53228a38a)
>>>>>>>> https://pastebin.com/sAhFiGjY
>>>>>>>>
>>>>>>>> (Show channels 46c6e04b-a58e-4b82-9968-fb0e9c2c0fb3)
>>>>>>>> https://pastebin.com/bmcdjzAr
>>>>>>>>
>>>>>>>> (Extracted from logs 46c6e04b-a58e-4b82-9968-fb0e9c2c0fb3)
>>>>>>>> https://pastebin.com/aGyY3m9F
>>>>>>>>
>>>>>>>> (Show channels 624cac90-0a7b-11ec-b7d0-6f543a588268)
>>>>>>>> https://pastebin.com/0cnQaRhC
>>>>>>>>
>>>>>>>> (Extracted from logs 624cac90-0a7b-11ec-b7d0-6f543a588268)
>>>>>>>> https://pastebin.com/atX0VXdZ
>>>>>>>>
>>>>>>>> Please let us know if there is anything else we can provide that
>>>>>>>> would be useful. Thank you.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Aug 30, 2021 at 8:20 PM Brian West <brian at freeswitch.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Check the UUID of the session in question, and extract its logs,
>>>>>>>>> and let's diagnose it
>>>>>>>>>
>>>>>>>>> On Mon, Aug 30, 2021 at 12:39 PM Cary Shouse <cary at ytel.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>  Yes, that is what we are seeing when running uuid_kill.  We
>>>>>>>>>> cannot "replicate" the issue, but it happens multiple times a day for us
>>>>>>>>>> across our servers.   We have looked through logs across
>>>>>>>>>> multiple systems, taken SIP traces to see if that showed anything.
>>>>>>>>>> Last we got a Core dump of the running process after the failure.
>>>>>>>>>> Nothing seems to stand out as the cause.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  Are there any other debug steps you can think of or what other
>>>>>>>>>> information to look at?  We could capture all traffic on the server, or
>>>>>>>>>> recompile with special debug flags whatever it is that might help.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Steps we have taken:
>>>>>>>>>>
>>>>>>>>>> sofia loglevel all 9
>>>>>>>>>>
>>>>>>>>>> sofia global siptrace on
>>>>>>>>>>
>>>>>>>>>> fsctl loglevel 7
>>>>>>>>>>
>>>>>>>>>> # Set environment variable
>>>>>>>>>> TPORT_LOG=1
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sat, Aug 28, 2021 at 12:54 AM Brian West <brian at freeswitch.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> It's an issue we've been searching for, if you uuid_kill it, and
>>>>>>>>>>> it says no session, it only means the record was NOT removed from the
>>>>>>>>>>> sqlite database.
>>>>>>>>>>>
>>>>>>>>>>> Nobody can seem to replicate the steps required to get into that
>>>>>>>>>>> condition, maybe you could look thru the logs and see if you can replicate
>>>>>>>>>>> what ever happened on that lingering UUID
>>>>>>>>>>>
>>>>>>>>>>> /b
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Aug 27, 2021 at 5:40 PM Cary Shouse <cary at ytel.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>  We recently started having a problem with our FreeSWITCH
>>>>>>>>>>>> servers where calls are working well, with about 200 calls operating
>>>>>>>>>>>> sometimes for a couple of days, and then something happens and all call
>>>>>>>>>>>> progress stops. Then any new inbound calls that come into the server get
>>>>>>>>>>>> stuck in initial status, new inbound get stuck at RINGING. This happens
>>>>>>>>>>>> until the max sessions are hit and of course nothing else works from there.
>>>>>>>>>>>> Our thought is something is creating a lock and stopping anything else from
>>>>>>>>>>>> progressing.
>>>>>>>>>>>>  We are seeing the stuck sessions using "show channels". Doing
>>>>>>>>>>>> a uuid_kill on the calls seems to make no difference to the output of "show
>>>>>>>>>>>> channels", but we have noticed in the logs that some events do happen to
>>>>>>>>>>>> some of the calls.
>>>>>>>>>>>>  We had ran into a similar issue when writing a custom C module
>>>>>>>>>>>> and made some mistakes with session_rwlock etc. But in this case, all of
>>>>>>>>>>>> this is using LUA.
>>>>>>>>>>>> Some general information:
>>>>>>>>>>>> FreeSWITCH Versions 1.10.6 and 1.10.1
>>>>>>>>>>>> Modules used:
>>>>>>>>>>>> LUA, xml_curl(dialplan), event_socket,
>>>>>>>>>>>>
>>>>>>>>>>>>  Most of this is just application playing audio files and or
>>>>>>>>>>>> creating a new call to bridge to.
>>>>>>>>>>>>
>>>>>>>>>>>>  Really looking for some help on what debugging we can enable
>>>>>>>>>>>> to help identify what is getting stuck to identify where a bug might be on
>>>>>>>>>>>> our usage or possible issue with FreeSWITCH.
>>>>>>>>>>>>
>>>>>>>>>>>>  We have captured a core dump of the running process when it
>>>>>>>>>>>> got into the bad state as well, if that is helpful, we can send it in.
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you for your help.
>>>>>>>>>>>>
>>>>>>>>>>>> _________________________________________________________________________
>>>>>>>>>>>>
>>>>>>>>>>>> The FreeSWITCH project is sponsored by SignalWire
>>>>>>>>>>>> https://signalwire.com
>>>>>>>>>>>> Enhance your FreeSWITCH install with disruptive priced SMS and
>>>>>>>>>>>> PSTN services.
>>>>>>>>>>>> Build your next product on our scalable cloud platform.
>>>>>>>>>>>>
>>>>>>>>>>>> Join our online community to chat in real time
>>>>>>>>>>>> https://signalwire.community
>>>>>>>>>>>>
>>>>>>>>>>>> Professional FreeSWITCH Services
>>>>>>>>>>>> sales at freeswitch.com
>>>>>>>>>>>> https://freeswitch.com
>>>>>>>>>>>>
>>>>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>>>>> https://freeswitch.com/oss
>>>>>>>>>>>> https://freeswitch.org/confluence
>>>>>>>>>>>> https://cluecon.com
>>>>>>>>>>>>
>>>>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>>>>> https://freeswitch.com
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>>
>>>>>>>>>>> Brian West | Co-founder and Developer
>>>>>>>>>>>
>>>>>>>>>>> Need Commercial support? email sales at freeswitch.com
>>>>>>>>>>>
>>>>>>>>>>> FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI
>>>>>>>>>>> 53045
>>>>>>>>>>> <https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g>
>>>>>>>>>>>
>>>>>>>>>>> Email: brian at freeswitch.com
>>>>>>>>>>>
>>>>>>>>>>> Mobile: 918-424-9378
>>>>>>>>>>>
>>>>>>>>>>> Website: https://www.FreeSWITCH.com
>>>>>>>>>>> <https://www.freeswitch.com/>
>>>>>>>>>>>
>>>>>>>>>>> [image: https://www.facebook.com/signalwireinc?src=email]
>>>>>>>>>>> <https://www.facebook.com/freeswitch> [image:
>>>>>>>>>>> https://twitter.com/freeswitch] <https://twitter.com/freeswitch>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> Brian West | Co-founder and Developer
>>>>>>>>>
>>>>>>>>> Need Commercial support? email sales at freeswitch.com
>>>>>>>>>
>>>>>>>>> FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI
>>>>>>>>> 53045
>>>>>>>>> <https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g>
>>>>>>>>>
>>>>>>>>> Email: brian at freeswitch.com
>>>>>>>>>
>>>>>>>>> Mobile: 918-424-9378
>>>>>>>>>
>>>>>>>>> Website: https://www.FreeSWITCH.com <https://www.freeswitch.com/>
>>>>>>>>>
>>>>>>>>> [image: https://www.facebook.com/signalwireinc?src=email]
>>>>>>>>> <https://www.facebook.com/freeswitch> [image:
>>>>>>>>> https://twitter.com/freeswitch] <https://twitter.com/freeswitch>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Brian West | Co-founder and Developer
>>>>>>>
>>>>>>> Need Commercial support? email sales at freeswitch.com
>>>>>>>
>>>>>>> FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045
>>>>>>> <https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g>
>>>>>>>
>>>>>>> Email: brian at freeswitch.com
>>>>>>>
>>>>>>> Mobile: 918-424-9378
>>>>>>>
>>>>>>> Website: https://www.FreeSWITCH.com <https://www.freeswitch.com/>
>>>>>>>
>>>>>>> [image: https://www.facebook.com/signalwireinc?src=email]
>>>>>>> <https://www.facebook.com/freeswitch> [image:
>>>>>>> https://twitter.com/freeswitch] <https://twitter.com/freeswitch>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Brian West | Co-founder and Developer
>>>>>>
>>>>>> Need Commercial support? email sales at freeswitch.com
>>>>>>
>>>>>> FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045
>>>>>> <https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g>
>>>>>>
>>>>>> Email: brian at freeswitch.com
>>>>>>
>>>>>> Mobile: 918-424-9378
>>>>>>
>>>>>> Website: https://www.FreeSWITCH.com <https://www.freeswitch.com/>
>>>>>>
>>>>>> [image: https://www.facebook.com/signalwireinc?src=email]
>>>>>> <https://www.facebook.com/freeswitch> [image:
>>>>>> https://twitter.com/freeswitch] <https://twitter.com/freeswitch>
>>>>>>
>>>>>
>>>>
>>>> --
>>>>
>>>> Brian West | Co-founder and Developer
>>>>
>>>> Need Commercial support? email sales at freeswitch.com
>>>>
>>>> FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045
>>>> <https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g>
>>>>
>>>> Email: brian at freeswitch.com
>>>>
>>>> Mobile: 918-424-9378
>>>>
>>>> Website: https://www.FreeSWITCH.com <https://www.freeswitch.com/>
>>>>
>>>> [image: https://www.facebook.com/signalwireinc?src=email]
>>>> <https://www.facebook.com/freeswitch> [image:
>>>> https://twitter.com/freeswitch] <https://twitter.com/freeswitch>
>>>>
>>>
>>
>> --
>>
>> Brian West | Co-founder and Developer
>>
>> Need Commercial support? email sales at freeswitch.com
>>
>> FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045
>> <https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g>
>>
>> Email: brian at freeswitch.com
>>
>> Mobile: 918-424-9378
>>
>> Website: https://www.FreeSWITCH.com <https://www.freeswitch.com/>
>>
>> [image: https://www.facebook.com/signalwireinc?src=email]
>> <https://www.facebook.com/freeswitch> [image:
>> https://twitter.com/freeswitch] <https://twitter.com/freeswitch>
>>
>

-- 

Brian West | Co-founder and Developer

Need Commercial support? email sales at freeswitch.com

FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045
<https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g>

Email: brian at freeswitch.com

Mobile: 918-424-9378

Website: https://www.FreeSWITCH.com <https://www.freeswitch.com/>

[image: https://www.facebook.com/signalwireinc?src=email]
<https://www.facebook.com/freeswitch> [image:
https://twitter.com/freeswitch] <https://twitter.com/freeswitch>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20210902/69e969ff/attachment-0001.html>


More information about the FreeSWITCH-users mailing list