[Freeswitch-dev] Strange bug - 2

Thu Sep 16 09:13:02 PDT 2010

The shutdown issue was just a forgotten call in part of a new patch
it's already fixed in tree.

mod_dingaling does not work on every system, it uses system libs which
can be unstable on some platforms.  This is why re recommend 64 bit
centos5 as a target platform for beginners.

2010/9/16 Oleg Khovayko <khovayko at gmail.com>:
> Rupa Schomaker wrote:
>
> I should note that you are looking for identical stack traces (up to a
> certain depth) that are not blocked (eg on a mutex).  If there is a runaway
> thread (100% cpu implies this) then it should be visible in the stack
> traces.
>
> Excuse me, I don't know, how to see stack trace in the running process,
> without stopping it.
>
> I assume you tried running with loglevel 7 and there is nothing useful in
> the logs?
>
> This time - no. But, when analogous situation happens ~1 month ago, I tried
> to use loglevel 7, ans siptrace on - did not found any suspicious message.
> Prev time, it was happeining, also keep for ~2 days, and wanished. Reboot,
> reinstall binaries, was not help.
> Looks like deadlock with some files access.
>
>
> This time, I have fixed (or, maybe problem vanished itself again) by
> following:
> I disabled load modules mod_portaudio and mod_dingaling.
> Thereafter, program eats ~4%CPU (not 60%), memory does not leaks, and FS
> works.
> But, prev. time, program consumed ~2%. I think, problem persist, but just
> hidden.
>
> Also, when I run fs manually, and use "shutdown", then shutdown log freeze
> on something like "Wait for pending SQL transaction".
> I tried to clear ./db directory, and restart FS - does not help...
>
>
> On Thu, Sep 16, 2010 at 6:39 AM, Rupa Schomaker <rupa at rupa.com> wrote:
>>
>> That core isn't going to help much, it is probably due to running out of
>> memory due to whatever is the "real" problem.
>> 1) Review the "Reporting Bugs" page on the wiki.
>
> OK, I will see, thank you.
>>
>> 2) After getting FS started and exhibiting the behavior, get a few core
>> dumps separated by time (10s apart?).  The procedure should be on the wiki.
>>  Use gcore.
>
> I will do, when problem will raised again.
>>
>> 3) What platform is this?  32b or 64b?
>
> Intel-32, celeron-300.
>>
>>
>> On Wed, Sep 15, 2010 at 7:05 PM, Oleg Khovayko <khovayko at gmail.com> wrote:
>>>
>>> As I wrote before, my FS has strange behaviour:
>>> When it starts, it creates all threads, and go to command prompt.
>>> But, CPU usage so high, and program eat more and more memory.
>>>
>>> When I invoke "shutdown", ito goes to shutdown, and after destroy
>>> gateway, freeze for ~2 minutes. Thereafter, crashes to core.
>>>
>>> GDB output following.
>>>
>>> PS: This is today's GIT build:
>>>
>>>
>>> 2010-09-15 19:57:45.229882 [NOTICE] sofia_glue.c:4350 deleted gateway
>>> example.com
>>> 2010-09-15 19:57:45.229882 [NOTICE] sofia_glue.c:4350 deleted gateway
>>> voipdiscount.com
>>> 2010-09-15 19:57:45.229882 [NOTICE] sofia_glue.c:4350 deleted gateway
>>> sipsorcery.com
>>> 2010-09-15 19:57:45.229882 [NOTICE] sofia_glue.c:4350 deleted gateway
>>> gizmo5.com
>>> 2010-09-15 19:59:54.935939 [NOTICE] switch_loadable_module.c:572
>>> Deleting Application 'hash'
>>> 2010-09-15 19:59:54.944150 [NOTICE] switch_loadable_module.c:599
>>> Deleting API Function 'hash'
>>> 2010-09-15 19:59:54.948691 [NOTICE] switch_loadable_module.c:599
>>> Deleting API Function 'hash_dump'
>>> 2010-09-15 19:59:54.951357 [NOTICE] switch_loadable_module.c:599
>>> Deleting API Function 'hash_remote'
>>> 2010-09-15 19:59:54.954504 [NOTICE] switch_loadable_module.c:803
>>> Deleting Limit interface 'hash'
>>> 2010-09-15 19:59:54.958190 [CONSOLE] switch_loadable_module.c:1396
>>> Stopping: mod_hash
>>> 2010-09-15 19:59:54.960612 [NOTICE] switch_loadable_module.c:557
>>> Deleting Timer 'soft'
>>> 2010-09-15 19:59:54.960612 [CONSOLE] switch_loadable_module.c:1396
>>> Stopping: CORE_SOFTTIMER_MODULE
>>> 2010-09-15 19:59:54.965641 [CONSOLE] switch_time.c:852 Soft timer thread
>>> exiting.
>>> 2010-09-15 19:59:54.966260 [NOTICE] switch_loadable_module.c:99 Thread
>>> ended for CORE_SOFTTIMER_MODULE
>>> 2010-09-15 19:59:54.977525 [NOTICE] switch_event.c:1500 Event Binding
>>> deleted for CORE_SOFTTIMER_MODULE:RELOADXML
>>> 2010-09-15 19:59:54.980836 [NOTICE] switch_loadable_module.c:491
>>> Deleting Endpoint 'dingaling'
>>> 2010-09-15 19:59:54.983836 [NOTICE] switch_loadable_module.c:599
>>> Deleting API Function 'dl_debug'
>>> [New Thread 0x28802030 (LWP 100077)]
>>>
>>> Program received signal SIGSEGV, Segmentation fault.
>>> [Switching to Thread 0x28802030 (LWP 100077)]
>>> sqlite3HashInsert (pH=0x0, pKey=0x2959cd78, nKey=15, data=0x28c41080) at
>>> ./src/hash.c:347
>>> 347       xHash = hashFunction(pH->keyClass);
>>> (gdb) where
>>> #0  sqlite3HashInsert (pH=0x0, pKey=0x2959cd78, nKey=15,
>>> data=0x28c41080) at ./src/hash.c:347
>>> #1  0x280e972f in switch_core_hash_insert (hash=0x0, key=0x2959cd78
>>> "173.79.240.220", data=0x28c41080) at src/switch_core_hash.c:80
>>> #2  0x28eb0969 in sofia_glue_add_profile (key=0x2959cd78
>>> "173.79.240.220", profile=0x28c41080) at sofia_glue.c:4246
>>> #3  0x28ed4d50 in sofia_reg_handle_sip_i_register (nua=0x2882a240,
>>> profile=0x28c41080, nh=0x2b5f2de0, sofia_private=0x0, sip=0x2b5df89c,
>>>     tags=0x2b6033bc) at sofia_reg.c:1442
>>> #4  0x28eaa5a5 in sofia_event_callback (event=nua_i_register,
>>> status=100, phrase=0x2b6033c4 "Trying", nua=0x2882a240,
>>> profile=0x28c41080,
>>>     nh=0x2b5f2de0, sofia_private=0x0, sip=0x2b5df89c, tags=0x2b6033bc)
>>> at sofia.c:912
>>> #5  0x28f1e866 in nua_application_event (dummy=0x0, sumsg=0xbf354cd8,
>>> ee=0x2b6033a4) at nua_stack.c:393
>>> #6  0x28f6ec3c in su_base_port_execute_msgs (queue=0x0) at
>>> su_base_port.c:280
>>> #7  0x28f6f4a8 in su_base_port_run (self=0x28806c50) at
>>> su_base_port.c:335
>>> #8  0x28f6a6a1 in su_root_run (self=0x28c46200) at su_port.h:326
>>> #9  0x28ea1a59 in sofia_profile_thread_run (thread=0x28c41d68,
>>> obj=0x28c41080) at sofia.c:1571
>>> #10 0x2817b436 in dummy_worker (opaque=0x28c41d68) at
>>> threadproc/unix/thread.c:138
>>> #11 0x282766ff in pthread_getprio () from /lib/libthr.so.3
>>> #12 0x00000000 in ?? ()
>>> (gdb)
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> FreeSWITCH-dev mailing list
>>> FreeSWITCH-dev at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
>>> http://www.freeswitch.org
>>
>>
>>
>> --
>> -Rupa
>
>
>
> --
> -Rupa
>
> _______________________________________________
> FreeSWITCH-dev mailing list
> FreeSWITCH-dev at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
> http://www.freeswitch.org
>
>
> _______________________________________________
> FreeSWITCH-dev mailing list
> FreeSWITCH-dev at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
> http://www.freeswitch.org
>
>

-- 
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org
googletalk:conf+888 at conference.freeswitch.org
pstn:+19193869900