<html><head><style type="text/css"><!-- DIV {margin:0px;} --></style></head><body><div style="font-family:courier,monaco,monospace,sans-serif;font-size:12pt">If the problem persists can you open a bug on jira.<br><br><span><a target="_blank" href="http://jira.freeswitch.org">http://jira.freeswitch.org</a> </span><br><br>That way we can track it better and you can attach traces etc.<br><br>That crash appears to come from the sofia sip library.<br><br>there have been a few updates to that library that may not be applied to your tree can you try the following commands from your unix shell in the main root of the freeswitch src?<br><br>svn update<br>cd libs/sofia-sip<br>make clean<br>make<br>cd ../..<br>make clean modwipe<br>make install<br><br><br>what does your dialplan look like that you are testing calls to?<br><br><br><br><br><div> </div><div>Anthony Minessale II<br><br><span>FreeSWITCH <a target="_blank"
href="http://www.freeswitch.org/">http://www.freeswitch.org/</a></span><br><span>ClueCon <a target="_blank" href="http://www.cluecon.com/">http://www.cluecon.com/</a></span><br><br>AIM: anthm<br>MSN:anthony_minessale@hotmail.com<br>GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<br>IRC: irc.freenode.net #freeswitch</div><div><br>FreeSWITCH Developer Conference<br>sip:888@conference.freeswitch.org<br>iax:guest@conference.freeswitch.org/888<br>googletalk:conf+888@conference.freeswitch.org<br>pstn:213-799-1400</div><div style="font-family: courier,monaco,monospace,sans-serif; font-size: 12pt;"><br><br><div style="font-family: times new roman,new york,times,serif; font-size: 12pt;">----- Original Message ----<br>From: Freeswitch Freeswitch <freeswitch@2ride.com><br>To: freeswitch-users@lists.freeswitch.org<br>Sent: Saturday, November 24, 2007 10:48:28 AM<br>Subject: Re: [Freeswitch-users] mem leak and core dump<br><br>
Here are my ulimit settings before I adjusted to meet your recommendations Anthony.<br>I had increased file descriptors and decreased stack size.<br><br>t: cpu time (seconds) unlimited<br>-f: file size (blocks) unlimited
<br>-d: data seg size (kbytes) unlimited<br>-s: stack size (kbytes) 1024<br>-c: core file size (blocks) 0<br>-m: resident set size (kbytes) unlimited<br>-u: processes 16381<br>-n: file descriptors 100000
<br>-l: locked-in-memory size (kb) 32<br>-v: address space (kb) unlimited<br>-x: file locks unlimited<br>-i: pending signals 16381<br>-q: bytes in POSIX msg queues 819200<br>-N 13: 0
<br>-N 14: 0<br><br>Here are the settings I am running with now.<br><br>-t: cpu time (seconds) unlimited<br>-f: file size (blocks) unlimited<br>-d: data seg size (kbytes) unlimited
<br>-s: stack size (kbytes) unlimited<br>-c: core file size (blocks) unlimited<br>-m: resident set size (kbytes) unlimited<br>-u: processes 16381<br>-n: file descriptors 999999<br>-l: locked-in-memory size (kb) 32
<br>-v: address space (kb) unlimited<br>-x: file locks unlimited<br>-i: pending signals 16381<br>-q: bytes in POSIX msg queues 819200<br>-N 13: 0<br>-N 14: 0
<br><br><br>Here is the stack from my lastest crash.<br>I noticed some of the libraries did not seem to be built with debug so I cleaned and rebuilt and reinstall everything.<br>Currently running for 15 minutes, over 5000 total sessions and 300 concurrent.
<br>Finger crossed .......<br><br>GNU gdb Red Hat Linux (6.6-8.fc7rh)<br>Copyright (C) 2006 Free Software Foundation, Inc.<br>GDB is free software, covered by the GNU General Public License, and you are<br>welcome to change it and/or distribute copies of it under certain conditions.
<br>Type "show copying" to see the conditions.<br>There is absolutely no warranty for GDB. Type "show warranty" for details.<br>This GDB was configured as "x86_64-redhat-linux-gnu"...<br>Using host libthread_db library "/lib64/libthread_db.so.1".
<br>Reading symbols from /lib64/libm.so.6...done.<br>Loaded symbols for /lib64/libm.so.6<br>Reading symbols from /usr/local/freeswitch1121/lib/libfreeswitch.so.1...done.<br>Loaded symbols for /usr/local/freeswitch1121/lib/libfreeswitch.so.1
<br>Reading symbols from /usr/lib64/libsqlite3.so.0...done.<br>Loaded symbols for /usr/lib64/libsqlite3.so.0<br>Reading symbols from /lib64/libuuid.so.1...done.<br>Loaded symbols for /lib64/libuuid.so.1<br>Reading symbols from /lib64/librt.so.1...done.
<br>Loaded symbols for /lib64/librt.so.1<br>Reading symbols from /lib64/libdl.so.2...done.<br>Loaded symbols for /lib64/libdl.so.2<br>Reading symbols from /lib64/libcrypt.so.1...done.<br>Loaded symbols for /lib64/libcrypt.so.1
<br>Reading symbols from /lib64/libpthread.so.0...done.<br>Loaded symbols for /lib64/libpthread.so.0<br>Reading symbols from /lib64/libc.so.6...done.<br>Loaded symbols for /lib64/libc.so.6<br>Reading symbols from /usr/lib64/libstdc++.so.6...done.
<br>Loaded symbols for /usr/lib64/libstdc++.so.6<br>Reading symbols from /lib64/libgcc_s.so.1...done.<br>Loaded symbols for /lib64/libgcc_s.so.1<br>Reading symbols from /lib64/ld-linux-x86-64.so.2...done.<br>Loaded symbols for /lib64/ld-
<a target="_blank" href="http://linux-x86-64.so">linux-x86-64.so</a>.2<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_console.so...done.<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_console.so<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_syslog.so...done.
<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_syslog.so<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_xml_rpc.so...done.<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_xml_rpc.so<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_sofia.so...done.
<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_sofia.so<br>Reading symbols from /lib64/libssl.so.6...done.<br>Loaded symbols for /lib64/libssl.so.6<br>Reading symbols from /lib64/libcrypto.so.6...done.<br>Loaded symbols for /lib64/libcrypto.so.6
<br>Reading symbols from /lib64/libz.so.1...done.<br>Loaded symbols for /lib64/libz.so.1<br>Reading symbols from /usr/lib64/libgssapi_krb5.so.2...done.<br>Loaded symbols for /usr/lib64/libgssapi_krb5.so.2<br>Reading symbols from /usr/lib64/libkrb5.so.3...done.
<br>Loaded symbols for /usr/lib64/libkrb5.so.3<br>Reading symbols from /lib64/libcom_err.so.2...done.<br>Loaded symbols for /lib64/libcom_err.so.2<br>Reading symbols from /usr/lib64/libk5crypto.so.3...done.<br>Loaded symbols for /usr/lib64/libk5crypto.so.3
<br>Reading symbols from /usr/lib64/libkrb5support.so.0...done.<br>Loaded symbols for /usr/lib64/libkrb5support.so.0<br>Reading symbols from /lib64/libkeyutils.so.1...done.<br>Loaded symbols for /lib64/libkeyutils.so.1<br>
Reading symbols from /lib64/libresolv.so.2...done.<br>Loaded symbols for /lib64/libresolv.so.2<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_commands.so...done.<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_commands.so
<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_conference.so...done.<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_conference.so<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_dptools.so...done.
<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_dptools.so<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_expr.so...Error while reading shared library symbols:<br>DW_FORM_strp pointing outside of .debug_str section [in module /usr/local/freeswitch1121/mod/mod_expr.so]
<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_fifo.so...done.<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_fifo.so<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_voicemail.so...done.<br>
Loaded symbols for /usr/local/freeswitch1121/mod/mod_voicemail.so<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_limit.so...done.<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_limit.so<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_dialplan_xml.so...done.
<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_dialplan_xml.so<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_g711.so...done.<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_g711.so<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_g729.so...done.
<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_g729.so<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_l16.so...done.<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_l16.so<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_say_en.so...done.
<br>Loaded symbols for /usr/local/freeswitch1121/mod/mod_say_en.so<br>Reading symbols from /usr/local/freeswitch1121/mod/mod_expr.so...Error while reading shared library symbols:<br>DW_FORM_strp pointing outside of .debug_str section [in module /usr/local/freeswitch1121/mod/mod_expr.so]
<br>Core was generated by `./freeswitch'.<br>Program terminated with signal 11, Segmentation fault.<br>#0 nua_stack_timer (nua=<value optimized out>, t=<value optimized out>, a=0x0) at nua_stack.c:513<br>
513 if (du->du_refresh && du->du_refresh < next)<br>(gdb) where<br>#0 nua_stack_timer (nua=<value optimized out>, t=<value optimized out>, a=0x0) at nua_stack.c:513<br>#1 0x00002aaaabdb1ae8 in su_timer_expire (timers=<value optimized out>, timeout=0x402b40d8, now={tv_sec = 3404833764, tv_usec = 661017}) at su_timer.c:533
<br>#2 0x00002aaaabdb71a6 in su_base_port_run (self=0x6901f0) at su_base_port.c:328<br>#3 0x00002aaaabda7e7b in su_pthread_port_clone_main (varg=<value optimized out>) at su_pthread_port.c:321<br>#4 0x00002aaaaae051c5 in start_thread () from /lib64/libpthread.so.0
<br>#5 0x0000003baaed062d in clone () from /lib64/libc.so.6<br>(gdb)<br><br>After this crash I did a clean rebuild and reinstall - since then I have had no subsequent crashes.<br><br><br><br><div><span class="gmail_quote">
On 11/23/07, <b class="gmail_sendername">Anthony Minessale</b> <<a rel="nofollow" ymailto="mailto:anthmct@yahoo.com" target="_blank" href="mailto:anthmct@yahoo.com">anthmct@yahoo.com</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><div style="font-family: courier,monaco,monospace,sans-serif; font-size: 12pt;">please disable the crash-protection option in switch.conf.xml<br>and execute the command 'ulimit -c unlimited' from your unix shell before you start freeswitch..
<br><br>here are some other ulimits you may want to turn up for high-volume calls:<br><br>vi u<br>also you can use the unix command "script" to log all the console output to a file so you can capture all of the output for us to examine.
<br><br><br><br><div> </div><div>Anthony Minessale II<br><br><span>FreeSWITCH <a rel="nofollow" target="_blank" href="http://www.freeswitch.org/">http://www.freeswitch.org/</a></span>
<br><span>ClueCon
<a rel="nofollow" target="_blank" href="http://www.cluecon.com/">http://www.cluecon.com/</a></span><br><br>AIM: anthm<br><a rel="nofollow" ymailto="mailto:MSN:anthony_minessale@hotmail.com" target="_blank" href="mailto:MSN:anthony_minessale@hotmail.com">
MSN:anthony_minessale@hotmail.com</a><br>GTALK/JABBER/PAYPAL:<a rel="nofollow" ymailto="mailto:anthony.minessale@gmail.com" target="_blank" href="mailto:anthony.minessale@gmail.com">anthony.minessale@gmail.com</a><br>IRC: <a rel="nofollow" target="_blank" href="http://irc.freenode.net">
irc.freenode.net</a> #freeswitch</div><div><br>FreeSWITCH Developer Conference<br><a rel="nofollow" ymailto="mailto:sip:888@conference.freeswitch.org" target="_blank" href="mailto:sip:888@conference.freeswitch.org">sip:888@conference.freeswitch.org
</a><br><a rel="nofollow" target="_blank" href="http://iax:guest@conference.freeswitch.org/888">iax:guest@conference.freeswitch.org/888</a><br><a rel="nofollow" ymailto="mailto:googletalk:conf+888@conference.freeswitch.org" target="_blank" href="mailto:googletalk:conf+888@conference.freeswitch.org">
googletalk:conf+888@conference.freeswitch.org</a><br>pstn:213-799-1400</div><div><span class="e" id="q_1166d8f5bca18719_1"><div style="font-family: courier,monaco,monospace,sans-serif; font-size: 12pt;"><br><br><div style="font-family: times new roman,new york,times,serif; font-size: 12pt;">
----- Original Message ----<br>From: Freeswitch Freeswitch <<a rel="nofollow" ymailto="mailto:freeswitch@2ride.com" target="_blank" href="mailto:freeswitch@2ride.com">freeswitch@2ride.com</a>><br>To: <a rel="nofollow" ymailto="mailto:freeswitch-users@lists.freeswitch.org" target="_blank" href="mailto:freeswitch-users@lists.freeswitch.org">
freeswitch-users@lists.freeswitch.org</a><br>Sent: Friday, November 23, 2007 10:47:53 AM<br>Subject: Re: [Freeswitch-users] mem leak and core dump<br><br>
Caught signal 11 for unmapped thread!<br><br><div><span class="gmail_quote">On 11/23/07, <b class="gmail_sendername">Freeswitch Freeswitch</b> <<a rel="nofollow" ymailto="mailto:freeswitch@2ride.com" target="_blank" href="mailto:freeswitch@2ride.com">
freeswitch@2ride.com</a>> wrote:</span>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Hello,<br><br>Now I have updated to the latest trunk as of Nov 21 and moved to a fedora 7 system.
<br><br>Still chasing random crashes of FS. First time this happened last night there were around 1000 sessions up.<br>Crit error was logged referring to a signal caught on an unmapped thread. I wasn't able to catch the specific signal in the console before it scrolled off. This morning I had another crash. This time around 200 sessions running. There was no logging reported during the second crash.
<br><br>Calling scenario is about as simple as it gets.<br><br>gw1 -> freeswitch -> gw2 all sip<br>codecs, 729 and 711 <br>transcoding off<br><br>I have been playing with the various rtp knobs in the sofia conf. Sometimes running with rewriting rtp timerstamps, sometimes not. Also tried with soft timer and off. ( can someone point me to an explanation of what the soft timer does? ).
<br><br>Also notable that I compiled the code with debug symbols.<br><br>No core has been generated during crashes. Is there some way to enable core dumps?<br><br>Overall I am very impressed with FS - would love to get the cause of these crashes nailed so I can press forward.
<br><br>Any help is very much appreciated.<br><br>Best regards,<br><br>Tom<br><br><br><br><br><br><div><div><span><span class="gmail_quote">On 11/19/07, <b class="gmail_sendername">Michael Jerris
</b> <<a rel="nofollow" ymailto="mailto:mike@jerris.com" target="_blank" href="mailto:mike@jerris.com">
mike@jerris.com</a>> wrote:</span></span></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><span>
<span>On 11/19/07, <b class="gmail_sendername">
Freeswitch Freeswitch</b> <<a rel="nofollow" ymailto="mailto:freeswitch@2ride.com" target="_blank" href="mailto:freeswitch@2ride.com">freeswitch@2ride.com</a>> wrote:</span><div><span><span class="gmail_quote">
</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hello Freeswitch'rs,<br><br>I recently began using the latest freeswitch beta code as a pure B2BUA for the purpose of topology hiding of IP - PSTN calls. <br></blockquote><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
running env:<br>Linux b2bua 2.6.18 #6 SMP Thu Nov 16 10:35:45 EST 2006 i686 Intel(R) Xeon(R) CPU 5140 @
2.33GHz GNU/Linux<br>Freeswitch version: FreeSwitch Version 1.0.pre1 (6220M)<br><br>The modules I am using are 711 and 729 codecs, sip, directory, xml_rpc, cdr, xml_cdr, console and syslog.<br><br>I am seeing what appears to be a memory leak. Consumed memory continues to drop linearly based on number of calls.
<br>Also had one core dump.</blockquote></span><div><br>Can you provide details of the call scenarios you are testing, call rates, number of concurrent calls and any other details?<br> </div><span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
What can I do to help troubleshoot these issues?<br><br>Today I plan to stop using the CDR, rpc and logging modules to see if it makes a difference.</blockquote></span><div><br>We have tested quite heavily with mod_console running, that should not be an issue.
<br></div><span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Also considering running on one of my fedora core 7 boxes instead of gentoo, again to see if anything changes.
</blockquote></span><div><br>We have had repeated issues with gentoo due to patches made either to gcc or to libc or the kernel. It is the only os/distro that I highly suggest never using.<br></div><br>Mike<br></div><br>
<br></span></div><span>_______________________________________________<br>Freeswitch-users mailing list<br><a rel="nofollow" ymailto="mailto:Freeswitch-users@lists.freeswitch.org" target="_blank" href="mailto:Freeswitch-users@lists.freeswitch.org">
Freeswitch-users@lists.freeswitch.org
</a><br><a rel="nofollow" target="_blank" href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
</a><br>
UNSUBSCRIBE:
<a rel="nofollow" target="_blank" href="http://lists.freeswitch.org/mailman/options/freeswitch-users">http://lists.freeswitch.org/mailman/options/freeswitch-users</a>
<br><a rel="nofollow" target="_blank" href="http://www.freeswitch.org">
http://www.freeswitch.org</a><br><br></span></blockquote></div><br>
</blockquote></div><br>
</div><br></div></span></div></div><span class="ad"><br>
<hr size="1">Get easy, one-click access to your favorites.
<a rel="nofollow" target="_blank" href="http://us.rd.yahoo.com/evt=51443/*http://www.yahoo.com/r/hs"> Make Yahoo! your homepage.</a>
</span></div><br>_______________________________________________<br>Freeswitch-users mailing list<br><a rel="nofollow" ymailto="mailto:Freeswitch-users@lists.freeswitch.org" target="_blank" href="mailto:Freeswitch-users@lists.freeswitch.org">Freeswitch-users@lists.freeswitch.org
</a><br><a rel="nofollow" target="_blank" href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>UNSUBSCRIBE:
<a rel="nofollow" target="_blank" href="http://lists.freeswitch.org/mailman/options/freeswitch-users">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br><a rel="nofollow" target="_blank" href="http://www.freeswitch.org">
http://www.freeswitch.org</a><br><br></blockquote></div><br>
</div><br></div></div><br>
<hr size=1>Be a better sports nut! Let your teams follow you
with Yahoo Mobile. <a href="http://us.rd.yahoo.com/evt=51731/*http://mobile.yahoo.com/sports;_ylt=At9_qDKvtAbMuh1G1SQtBI7ntAcJ
">Try it now.</a></body></html>