[Freeswitch-users] Leg stuck in a conference

Anthony Minessale anthony.minessale at gmail.com
Thu Sep 25 00:33:56 MSD 2014 

Do you know what is on the other side of these clients?
I get a bad vibe when i see cent6 in reports of odd behaviors as you know
;)

So if you uuid_kill it, then it exits?  The only logical explanation would
be that its stuck doing recvfrom.  But in that case the uuid_kill shouldn't
work either.

do these 2 commands first then a few kick commands then the uuid_kill  to
compare

fsctl loglevel debug
console loglevel debug




On Wed, Sep 24, 2014 at 2:08 PM, Oleg Stolyar <olegstolyar at gmail.com> wrote:

> Hi Anthony,
>
> One more piece of information.
>
> If I do "kick all" it returns "OK kicked 21698"
>
> But if I do "kick 21698" it returns "Non-Existant ID 21698" and there is no BREAK entry in the log.
>
>
> On Tue, Sep 23, 2014 at 12:03 PM, Oleg Stolyar <olegstolyar at gmail.com>
> wrote:
>
>> Yes, exactly
>>
>> On Tue, Sep 23, 2014 at 11:37 AM, Anthony Minessale <
>> anthony.minessale at gmail.com> wrote:
>>
>>> Doing more kick commands results in more break messages but it never
>>> works even with subsequent calls to kick?
>>>
>>>
>>>
>>> On Tue, Sep 23, 2014 at 1:23 PM, Oleg Stolyar <olegstolyar at gmail.com>
>>> wrote:
>>>
>>>> Version is: FreeSWITCH Version
>>>> 1.5.13b+git~20140529T194631Z~0d7b1ff42a~64bit (git 0d7b1ff 2014-05-29
>>>> 19:46:31Z 64bit)
>>>>
>>>> Since I cannot reproduce this at will, I would need to upgrade my
>>>> production to the latest master to try to reproduce it there.
>>>>
>>>> The client is WebRTC with JsSip on Chrome 34.
>>>>
>>>> I do not enable proxy_media and in fact there is no reference to
>>>> proxy_media anywhere in my config.
>>>> Similarly, I did not change anything related to timers but if you;d
>>>> like me to check a specific setting, I'll be happy to.
>>>>
>>>> I still have that FS instance running, so if there is anything else I
>>>> can do to help track this down, please let me know.
>>>>
>>>>
>>>> On Tue, Sep 23, 2014 at 11:06 AM, Anthony Minessale <
>>>> anthony.minessale at gmail.com> wrote:
>>>>
>>>>> It looks like its in recvfrom.  What version of FS is this and what
>>>>> type of call client etc is this?
>>>>> Do you have proxy_media on or timers disabled?
>>>>>
>>>>> It doesn't seem like this should be a likely scenario based on the
>>>>> description and the code I'm looking at.  Have you tried to reproduce it
>>>>> with master?
>>>>> The break signal should cause the read to come up to the conference
>>>>> app where it will detect that its done.
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Sep 23, 2014 at 12:49 PM, Oleg Stolyar <olegstolyar at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I'll do you one better :-)
>>>>>>
>>>>>> Here is the complete output file of "thread apply all bt"
>>>>>>
>>>>>> https://www.dropbox.com/s/gslm6l13jrxkr6s/gdb.txt
>>>>>>
>>>>>>
>>>>>> On Tue, Sep 23, 2014 at 10:40 AM, Anthony Minessale <
>>>>>> anthony.minessale at gmail.com> wrote:
>>>>>>
>>>>>>> Are there conference_loop_input threads to go with those?
>>>>>>> search for other threads with one of the sessions in it e.g.
>>>>>>> 0x7f3f503a8b48
>>>>>>>
>>>>>>> On Tue, Sep 23, 2014 at 12:33 PM, Oleg Stolyar <
>>>>>>> olegstolyar at gmail.com> wrote:
>>>>>>>
>>>>>>>> Thanks Anthony!
>>>>>>>> I got two such stuck legs right now.  Here are their threads.
>>>>>>>>
>>>>>>>> Thread 45 (Thread 0x7f3f574fb940 (LWP 25467)):
>>>>>>>> #0  0x0000003d8d207c65 in pthread_join () from
>>>>>>>> /lib64/libpthread.so.0
>>>>>>>> #1  0x00007f3f712b9db9 in apr_thread_join () from
>>>>>>>> /usr/local/freeswitch/lib/libfreeswitch.so.1
>>>>>>>> #2  0x00007f3f71186d2c in switch_thread_join
>>>>>>>> (retval=0x7f3f574fa0fc, thd=0x7f3f683f8cb8) at src/switch_apr.c:1171
>>>>>>>> #3  0x00007f3f667b233c in conference_loop_output
>>>>>>>> (member=0x7f3f574fa540) at mod_conference.c:4481
>>>>>>>> #4  0x00007f3f667c1a4c in conference_function
>>>>>>>> (session=0x7f3f503a8b48, data=0x7f3f681cbd10
>>>>>>>> "5d7b86e0-3ea5-11e4-b722-1231393fb01c at default+flags{join-only}")
>>>>>>>> at mod_conference.c:8695
>>>>>>>> #5  0x00007f3f711c23e7 in switch_core_session_exec
>>>>>>>> (session=0x7f3f503a8b48, application_interface=0x7f3f600e1920,
>>>>>>>> arg=0x7f3f681cbd10 "5d7b86e0-3ea5-11e4-b722-1231393fb01c at default
>>>>>>>> +flags{join-only}")
>>>>>>>>     at src/switch_core_session.c:2862
>>>>>>>> #6  0x00007f3f711c1c85 in
>>>>>>>> switch_core_session_execute_application_get_flags (session=0x7f3f503a8b48,
>>>>>>>> app=0x7f3f681cbd00 "conference",
>>>>>>>>     arg=0x7f3f681cbd10 "5d7b86e0-3ea5-11e4-b722-1231393fb01c at default+flags{join-only}",
>>>>>>>> flags=0x0) at src/switch_core_session.c:2737
>>>>>>>> #7  0x00007f3f711c3c3c in switch_core_standard_on_execute
>>>>>>>> (session=0x7f3f503a8b48) at src/switch_core_state_machine.c:281
>>>>>>>> #8  0x00007f3f711c4de0 in switch_core_session_run
>>>>>>>> (session=0x7f3f503a8b48) at src/switch_core_state_machine.c:530
>>>>>>>> #9  0x00007f3f711be45c in switch_core_session_thread
>>>>>>>> (thread=0x7f3f50b9f990, obj=0x7f3f503a8b48) at
>>>>>>>> src/switch_core_session.c:1597
>>>>>>>> #10 0x00007f3f711be913 in switch_core_session_thread_pool_worker
>>>>>>>> (thread=0x7f3f50b9f990, obj=0x7f3f50b9f820) at
>>>>>>>> src/switch_core_session.c:1689
>>>>>>>> #11 0x00007f3f712b9b7d in dummy_worker () from
>>>>>>>> /usr/local/freeswitch/lib/libfreeswitch.so.1
>>>>>>>> #12 0x0000003d8d20683d in start_thread () from
>>>>>>>> /lib64/libpthread.so.0
>>>>>>>> #13 0x0000003d8cad4f8d in clone () from /lib64/libc.so.6
>>>>>>>>
>>>>>>>>
>>>>>>>> Thread 60 (Thread 0x7f3f575af940 (LWP 28590)):
>>>>>>>> #0  0x0000003d8d207c65 in pthread_join () from
>>>>>>>> /lib64/libpthread.so.0
>>>>>>>> #1  0x00007f3f712b9db9 in apr_thread_join () from
>>>>>>>> /usr/local/freeswitch/lib/libfreeswitch.so.1
>>>>>>>> #2  0x00007f3f71186d2c in switch_thread_join
>>>>>>>> (retval=0x7f3f575ae0fc, thd=0x215a358) at src/switch_apr.c:1171
>>>>>>>> #3  0x00007f3f667b233c in conference_loop_output
>>>>>>>> (member=0x7f3f575ae540) at mod_conference.c:4481
>>>>>>>> #4  0x00007f3f667c1a4c in conference_function
>>>>>>>> (session=0x7f3f50087998, data=0x2152a38
>>>>>>>> "2e42aba0-430b-11e4-bc8b-22000a1cca7e at default+flags{join-only}")
>>>>>>>> at mod_conference.c:8695
>>>>>>>> #5  0x00007f3f711c23e7 in switch_core_session_exec
>>>>>>>> (session=0x7f3f50087998, application_interface=0x7f3f600e1920,
>>>>>>>> arg=0x2152a38 "2e42aba0-430b-11e4-bc8b-22000a1cca7e at default
>>>>>>>> +flags{join-only}")
>>>>>>>>     at src/switch_core_session.c:2862
>>>>>>>> #6  0x00007f3f711c1c85 in
>>>>>>>> switch_core_session_execute_application_get_flags (session=0x7f3f50087998,
>>>>>>>> app=0x2152a28 "conference", arg=0x2152a38
>>>>>>>> "2e42aba0-430b-11e4-bc8b-22000a1cca7e at default+flags{join-only}",
>>>>>>>>     flags=0x0) at src/switch_core_session.c:2737
>>>>>>>> #7  0x00007f3f711c3c3c in switch_core_standard_on_execute
>>>>>>>> (session=0x7f3f50087998) at src/switch_core_state_machine.c:281
>>>>>>>> #8  0x00007f3f711c4de0 in switch_core_session_run
>>>>>>>> (session=0x7f3f50087998) at src/switch_core_state_machine.c:530
>>>>>>>> #9  0x00007f3f711be45c in switch_core_session_thread
>>>>>>>> (thread=0x214d4b0, obj=0x7f3f50087998) at src/switch_core_session.c:1597
>>>>>>>> #10 0x00007f3f711be913 in switch_core_session_thread_pool_worker
>>>>>>>> (thread=0x214d4b0, obj=0x214d340) at src/switch_core_session.c:1689
>>>>>>>> #11 0x00007f3f712b9b7d in dummy_worker () from
>>>>>>>> /usr/local/freeswitch/lib/libfreeswitch.so.1
>>>>>>>> #12 0x0000003d8d20683d in start_thread () from
>>>>>>>> /lib64/libpthread.so.0
>>>>>>>> #13 0x0000003d8cad4f8d in clone () from /lib64/libc.so.6
>>>>>>>>
>>>>>>>> On Tue, Sep 23, 2014 at 9:16 AM, Anthony Minessale <
>>>>>>>> anthony.minessale at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> It may be trapped inside some blocking call from within the
>>>>>>>>> conference.
>>>>>>>>> Your best bet would be to catch it in that state, do the kick,
>>>>>>>>> when it doesn't work gcore the process and open it in gdb and get "thread
>>>>>>>>> apply all bt" then you can see the stack trace for that session and see
>>>>>>>>> which code its blocking on.
>>>>>>>>>
>>>>>>>>> The BREAK signal you show there is designed to pull it out of most
>>>>>>>>> blocking situations so it must be something particular.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Sep 23, 2014 at 10:53 AM, Oleg Stolyar <
>>>>>>>>> olegstolyar at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Guys,
>>>>>>>>>>
>>>>>>>>>> I wonder if anyone else ran into this situation:  Very rarely I
>>>>>>>>>> have a conference participant stuck in a conference.  When I try to run
>>>>>>>>>> kick command on that conference, it returns "OK kicked <member ID>" but
>>>>>>>>>> nothing happens.  In the logs there is only this:
>>>>>>>>>>
>>>>>>>>>> db1790fe-fa75-42a6-9054-75c600b9539c 2014-09-23 15:43:07.066947 [DEBUG] mod_conference.c:5619 Send signal sofia/<profile>/anonymous at anonymous.invalid [BREAK]
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> As an additional piece of information if I use uuid_kill on the leg, it does disconnect the leg and ends the conference.
>>>>>>>>>>
>>>>>>>>>> Any suggestions?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _________________________________________________________________________
>>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>>> consulting at freeswitch.org
>>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>>
>>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>> http://confluence.freeswitch.org
>>>>>>>>>> http://www.cluecon.com
>>>>>>>>>>
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>
>>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Anthony Minessale II       ♬ @anthmfs  ♬ @FreeSWITCH  ♬
>>>>>>>>>
>>>>>>>>> ☞ http://freeswitch.org/  ☞ http://cluecon.com/  ☞
>>>>>>>>> http://twitter.com/FreeSWITCH
>>>>>>>>> ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+
>>>>>>>>> <http://freeswitch.org/g+>*
>>>>>>>>>
>>>>>>>>> ClueCon Weekly Development Call
>>>>>>>>> ☎ sip:888 at conference.freeswitch.org  ☎ +19193869900
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _________________________________________________________________________
>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>> consulting at freeswitch.org
>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>
>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>> http://www.freeswitch.org
>>>>>>>>> http://confluence.freeswitch.org
>>>>>>>>> http://www.cluecon.com
>>>>>>>>>
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>
>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _________________________________________________________________________
>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>> consulting at freeswitch.org
>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>
>>>>>>>> Official FreeSWITCH Sites
>>>>>>>> http://www.freeswitch.org
>>>>>>>> http://confluence.freeswitch.org
>>>>>>>> http://www.cluecon.com
>>>>>>>>
>>>>>>>> 
>>>>>>>> 
>>>>>>>>
>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>> UNSUBSCRIBE:
>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>> http://www.freeswitch.org
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Anthony Minessale II       ♬ @anthmfs  ♬ @FreeSWITCH  ♬
>>>>>>>
>>>>>>> ☞ http://freeswitch.org/  ☞ http://cluecon.com/  ☞
>>>>>>> http://twitter.com/FreeSWITCH
>>>>>>> ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+
>>>>>>> <http://freeswitch.org/g+>*
>>>>>>>
>>>>>>> ClueCon Weekly Development Call
>>>>>>> ☎ sip:888 at conference.freeswitch.org  ☎ +19193869900
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _________________________________________________________________________
>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>> consulting at freeswitch.org
>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>
>>>>>>> Official FreeSWITCH Sites
>>>>>>> http://www.freeswitch.org
>>>>>>> http://confluence.freeswitch.org
>>>>>>> http://www.cluecon.com
>>>>>>>
>>>>>>> 
>>>>>>> 
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>> UNSUBSCRIBE:
>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _________________________________________________________________________
>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>> consulting at freeswitch.org
>>>>>> http://www.freeswitchsolutions.com
>>>>>>
>>>>>> Official FreeSWITCH Sites
>>>>>> http://www.freeswitch.org
>>>>>> http://confluence.freeswitch.org
>>>>>> http://www.cluecon.com
>>>>>>
>>>>>> 
>>>>>> 
>>>>>>
>>>>>> FreeSWITCH-users mailing list
>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>> http://www.freeswitch.org
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Anthony Minessale II       ♬ @anthmfs  ♬ @FreeSWITCH  ♬
>>>>>
>>>>> ☞ http://freeswitch.org/  ☞ http://cluecon.com/  ☞
>>>>> http://twitter.com/FreeSWITCH
>>>>> ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+
>>>>> <http://freeswitch.org/g+>*
>>>>>
>>>>> ClueCon Weekly Development Call
>>>>> ☎ sip:888 at conference.freeswitch.org  ☎ +19193869900
>>>>>
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://confluence.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> 
>>>>> 
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://confluence.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> 
>>>> 
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>
>>>
>>>
>>> --
>>> Anthony Minessale II       ♬ @anthmfs  ♬ @FreeSWITCH  ♬
>>>
>>> ☞ http://freeswitch.org/  ☞ http://cluecon.com/  ☞
>>> http://twitter.com/FreeSWITCH
>>> ☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+
>>> <http://freeswitch.org/g+>*
>>>
>>> ClueCon Weekly Development Call
>>> ☎ sip:888 at conference.freeswitch.org  ☎ +19193869900
>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://confluence.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> 
>>> 
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>
>>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://confluence.freeswitch.org
> http://www.cluecon.com
>
> 
> 
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>



-- 
Anthony Minessale II       ♬ @anthmfs  ♬ @FreeSWITCH  ♬

☞ http://freeswitch.org/  ☞ http://cluecon.com/  ☞
http://twitter.com/FreeSWITCH
☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+
<http://freeswitch.org/g+>*

ClueCon Weekly Development Call
☎ sip:888 at conference.freeswitch.org  ☎ +19193869900
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20140924/2da691f4/attachment-0001.html

Join us at ClueCon 2016 Aug 8-12, 2016
More information about the FreeSWITCH-users mailing list