[Freeswitch-users] FreeSWITCH Stops Responding

Tue Aug 31 19:58:01 UTC 2021

Hello,

We have got the logs for a few calls that were stuck.
(Show channels fe8f206c-2bd9-4af2-b329-35f53228a38a)
https://pastebin.com/NTnte7J7

(Extracted from logs fe8f206c-2bd9-4af2-b329-35f53228a38a)
https://pastebin.com/sAhFiGjY

(Show channels 46c6e04b-a58e-4b82-9968-fb0e9c2c0fb3)
https://pastebin.com/bmcdjzAr

(Extracted from logs 46c6e04b-a58e-4b82-9968-fb0e9c2c0fb3)
https://pastebin.com/aGyY3m9F

(Show channels 624cac90-0a7b-11ec-b7d0-6f543a588268)
https://pastebin.com/0cnQaRhC

(Extracted from logs 624cac90-0a7b-11ec-b7d0-6f543a588268)
https://pastebin.com/atX0VXdZ

Please let us know if there is anything else we can provide that would be
useful. Thank you.

On Mon, Aug 30, 2021 at 8:20 PM Brian West <brian at freeswitch.com> wrote:

> Check the UUID of the session in question, and extract its logs, and let's
> diagnose it
>
> On Mon, Aug 30, 2021 at 12:39 PM Cary Shouse <cary at ytel.com> wrote:
>
>>  Yes, that is what we are seeing when running uuid_kill.  We cannot
>> "replicate" the issue, but it happens multiple times a day for us across
>> our servers.   We have looked through logs across multiple systems,
>> taken SIP traces to see if that showed anything.  Last we got a Core
>> dump of the running process after the failure. Nothing seems to stand out
>> as the cause.
>>
>>
>>
>>  Are there any other debug steps you can think of or what other
>> information to look at?  We could capture all traffic on the server, or
>> recompile with special debug flags whatever it is that might help.
>>
>>
>>
>> Steps we have taken:
>>
>> sofia loglevel all 9
>>
>> sofia global siptrace on
>>
>> fsctl loglevel 7
>>
>> # Set environment variable
>> TPORT_LOG=1
>>
>>
>> On Sat, Aug 28, 2021 at 12:54 AM Brian West <brian at freeswitch.com> wrote:
>>
>>> It's an issue we've been searching for, if you uuid_kill it, and it
>>> says no session, it only means the record was NOT removed from the sqlite
>>> database.
>>>
>>> Nobody can seem to replicate the steps required to get into that
>>> condition, maybe you could look thru the logs and see if you can replicate
>>> what ever happened on that lingering UUID
>>>
>>> /b
>>>
>>>
>>> On Fri, Aug 27, 2021 at 5:40 PM Cary Shouse <cary at ytel.com> wrote:
>>>
>>>>  We recently started having a problem with our FreeSWITCH servers where
>>>> calls are working well, with about 200 calls operating sometimes for a
>>>> couple of days, and then something happens and all call progress stops.
>>>> Then any new inbound calls that come into the server get stuck in initial
>>>> status, new inbound get stuck at RINGING. This happens until the max
>>>> sessions are hit and of course nothing else works from there. Our thought
>>>> is something is creating a lock and stopping anything else from progressing.
>>>>  We are seeing the stuck sessions using "show channels". Doing a
>>>> uuid_kill on the calls seems to make no difference to the output of "show
>>>> channels", but we have noticed in the logs that some events do happen to
>>>> some of the calls.
>>>>  We had ran into a similar issue when writing a custom C module and
>>>> made some mistakes with session_rwlock etc. But in this case, all of this
>>>> is using LUA.
>>>> Some general information:
>>>> FreeSWITCH Versions 1.10.6 and 1.10.1
>>>> Modules used:
>>>> LUA, xml_curl(dialplan), event_socket,
>>>>
>>>>  Most of this is just application playing audio files and or creating a
>>>> new call to bridge to.
>>>>
>>>>  Really looking for some help on what debugging we can enable to help
>>>> identify what is getting stuck to identify where a bug might be on our
>>>> usage or possible issue with FreeSWITCH.
>>>>
>>>>  We have captured a core dump of the running process when it got into
>>>> the bad state as well, if that is helpful, we can send it in.
>>>>
>>>> Thank you for your help.
>>>>
>>>> _________________________________________________________________________
>>>>
>>>> The FreeSWITCH project is sponsored by SignalWire
>>>> https://signalwire.com
>>>> Enhance your FreeSWITCH install with disruptive priced SMS and PSTN
>>>> services.
>>>> Build your next product on our scalable cloud platform.
>>>>
>>>> Join our online community to chat in real time
>>>> https://signalwire.community
>>>>
>>>> Professional FreeSWITCH Services
>>>> sales at freeswitch.com
>>>> https://freeswitch.com
>>>>
>>>> Official FreeSWITCH Sites
>>>> https://freeswitch.com/oss
>>>> https://freeswitch.org/confluence
>>>> https://cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> https://freeswitch.com
>>>
>>>
>>>
>>> --
>>>
>>> Brian West | Co-founder and Developer
>>>
>>> Need Commercial support? email sales at freeswitch.com
>>>
>>> FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045
>>> <https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g>
>>>
>>> Email: brian at freeswitch.com
>>>
>>> Mobile: 918-424-9378
>>>
>>> Website: https://www.FreeSWITCH.com <https://www.freeswitch.com/>
>>>
>>> [image: https://www.facebook.com/signalwireinc?src=email]
>>> <https://www.facebook.com/freeswitch> [image:
>>> https://twitter.com/freeswitch] <https://twitter.com/freeswitch>
>>>
>>
>
> --
>
> Brian West | Co-founder and Developer
>
> Need Commercial support? email sales at freeswitch.com
>
> FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045
> <https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g>
>
> Email: brian at freeswitch.com
>
> Mobile: 918-424-9378
>
> Website: https://www.FreeSWITCH.com <https://www.freeswitch.com/>
>
> [image: https://www.facebook.com/signalwireinc?src=email]
> <https://www.facebook.com/freeswitch> [image:
> https://twitter.com/freeswitch] <https://twitter.com/freeswitch>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20210831/777cb5d4/attachment.html>