Rupa Schomaker wrote:
I should note that you are looking for identical stack traces (up to a certain depth) that are not blocked (eg on a mutex). If there is a runaway thread (100% cpu implies this) then it should be visible in the stack traces.
Excuse me, I don't know, how to see stack trace in the running process, without stopping it.
I assume you tried running with loglevel 7 and there is nothing useful in the logs? This time - no. But, when analogous situation happens ~1 month ago, I tried to use loglevel 7, ans siptrace on - did not found any suspicious message. Prev time, it was happeining, also keep for ~2 days, and wanished. Reboot, reinstall binaries, was not help. Looks like deadlock with some files access.
This time, I have fixed (or, maybe problem vanished itself again) by following: I disabled load modules mod_portaudio and mod_dingaling. Thereafter, program eats ~4%CPU (not 60%), memory does not leaks, and FS works. But, prev. time, program consumed ~2%. I think, problem persist, but just hidden. Also, when I run fs manually, and use "shutdown", then shutdown log freeze on something like "Wait for pending SQL transaction". I tried to clear ./db directory, and restart FS - does not help...
On Thu, Sep 16, 2010 at 6:39 AM, Rupa Schomaker <rupa@rupa.com <mailto:rupa@rupa.com>> wrote:
That core isn't going to help much, it is probably due to running out of memory due to whatever is the "real" problem.
1) Review the "Reporting Bugs" page on the wiki.
OK, I will see, thank you.
2) After getting FS started and exhibiting the behavior, get a few core dumps separated by time (10s apart?). The procedure should be on the wiki. Use gcore.
I will do, when problem will raised again.
3) What platform is this? 32b or 64b?
Intel-32, celeron-300.
On Wed, Sep 15, 2010 at 7:05 PM, Oleg Khovayko <khovayko@gmail.com <mailto:khovayko@gmail.com>> wrote:
As I wrote before, my FS has strange behaviour: When it starts, it creates all threads, and go to command prompt. But, CPU usage so high, and program eat more and more memory.
When I invoke "shutdown", ito goes to shutdown, and after destroy gateway, freeze for ~2 minutes. Thereafter, crashes to core.
GDB output following.
PS: This is today's GIT build:
2010-09-15 19:57:45.229882 [NOTICE] sofia_glue.c:4350 deleted gateway example.com <http://example.com> 2010-09-15 19:57:45.229882 [NOTICE] sofia_glue.c:4350 deleted gateway voipdiscount.com <http://voipdiscount.com> 2010-09-15 19:57:45.229882 [NOTICE] sofia_glue.c:4350 deleted gateway sipsorcery.com <http://sipsorcery.com> 2010-09-15 19:57:45.229882 [NOTICE] sofia_glue.c:4350 deleted gateway gizmo5.com <http://gizmo5.com> 2010-09-15 19:59:54.935939 [NOTICE] switch_loadable_module.c:572 Deleting Application 'hash' 2010-09-15 19:59:54.944150 [NOTICE] switch_loadable_module.c:599 Deleting API Function 'hash' 2010-09-15 19:59:54.948691 [NOTICE] switch_loadable_module.c:599 Deleting API Function 'hash_dump' 2010-09-15 19:59:54.951357 [NOTICE] switch_loadable_module.c:599 Deleting API Function 'hash_remote' 2010-09-15 19:59:54.954504 [NOTICE] switch_loadable_module.c:803 Deleting Limit interface 'hash' 2010-09-15 19:59:54.958190 [CONSOLE] switch_loadable_module.c:1396 Stopping: mod_hash 2010-09-15 19:59:54.960612 [NOTICE] switch_loadable_module.c:557 Deleting Timer 'soft' 2010-09-15 19:59:54.960612 [CONSOLE] switch_loadable_module.c:1396 Stopping: CORE_SOFTTIMER_MODULE 2010-09-15 19:59:54.965641 [CONSOLE] switch_time.c:852 Soft timer thread exiting. 2010-09-15 19:59:54.966260 [NOTICE] switch_loadable_module.c:99 Thread ended for CORE_SOFTTIMER_MODULE 2010-09-15 19:59:54.977525 [NOTICE] switch_event.c:1500 Event Binding deleted for CORE_SOFTTIMER_MODULE:RELOADXML 2010-09-15 19:59:54.980836 [NOTICE] switch_loadable_module.c:491 Deleting Endpoint 'dingaling' 2010-09-15 19:59:54.983836 [NOTICE] switch_loadable_module.c:599 Deleting API Function 'dl_debug' [New Thread 0x28802030 (LWP 100077)]
Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x28802030 (LWP 100077)] sqlite3HashInsert (pH=0x0, pKey=0x2959cd78, nKey=15, data=0x28c41080) at ./src/hash.c:347 347 xHash = hashFunction(pH->keyClass); (gdb) where #0 sqlite3HashInsert (pH=0x0, pKey=0x2959cd78, nKey=15, data=0x28c41080) at ./src/hash.c:347 #1 0x280e972f in switch_core_hash_insert (hash=0x0, key=0x2959cd78 "173.79.240.220", data=0x28c41080) at src/switch_core_hash.c:80 #2 0x28eb0969 in sofia_glue_add_profile (key=0x2959cd78 "173.79.240.220", profile=0x28c41080) at sofia_glue.c:4246 #3 0x28ed4d50 in sofia_reg_handle_sip_i_register (nua=0x2882a240, profile=0x28c41080, nh=0x2b5f2de0, sofia_private=0x0, sip=0x2b5df89c, tags=0x2b6033bc) at sofia_reg.c:1442 #4 0x28eaa5a5 in sofia_event_callback (event=nua_i_register, status=100, phrase=0x2b6033c4 "Trying", nua=0x2882a240, profile=0x28c41080, nh=0x2b5f2de0, sofia_private=0x0, sip=0x2b5df89c, tags=0x2b6033bc) at sofia.c:912 #5 0x28f1e866 in nua_application_event (dummy=0x0, sumsg=0xbf354cd8, ee=0x2b6033a4) at nua_stack.c:393 #6 0x28f6ec3c in su_base_port_execute_msgs (queue=0x0) at su_base_port.c:280 #7 0x28f6f4a8 in su_base_port_run (self=0x28806c50) at su_base_port.c:335 #8 0x28f6a6a1 in su_root_run (self=0x28c46200) at su_port.h:326 #9 0x28ea1a59 in sofia_profile_thread_run (thread=0x28c41d68, obj=0x28c41080) at sofia.c:1571 #10 0x2817b436 in dummy_worker (opaque=0x28c41d68) at threadproc/unix/thread.c:138 #11 0x282766ff in pthread_getprio () from /lib/libthr.so.3 #12 0x00000000 in ?? () (gdb)
_______________________________________________ FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org <mailto:FreeSWITCH-dev@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
-- -Rupa
-- -Rupa
_______________________________________________ FreeSWITCH-dev mailing list FreeSWITCH-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org