If you do get a coredump from gcore, see <a href="http://wiki.freeswitch.org/wiki/Reporting_Bugs#Creating_A_Backtrace_With_gdb_.28Linux.2FUnix.29">http://wiki.freeswitch.org/wiki/Reporting_Bugs#Creating_A_Backtrace_With_gdb_.28Linux.2FUnix.29</a><div>
<br></div><div>You will see in the backtrace what each thread is doing, which may highlight what's causing the non-response.</div><div><br></div><div>I highly suggest as your very first step you upgrade to 1.2.5.2. This on its own may stop your issue if it's down to a bug that was fixed, and if you're still having the issue will put you on a version that can be supported.</div>
<div class="gmail_extra"><br><br><div class="gmail_quote">On 30 November 2012 18:20, Steven Ayre <span dir="ltr"><<a href="mailto:steveayre@gmail.com" target="_blank">steveayre@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><font face="arial, sans-serif">User-Agent: FreeSWITCH-mod_sofia/1.0.head-git-46f097c 2011-08-07 01-33-26 -0400</font></blockquote>
<div><br></div><div>That's very old, you should update. There'll be a lot of bugfixes since then, and the age will make it harder to support.</div><div class="im"><div><font face="arial, sans-serif"><br></font></div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<font face="arial, sans-serif">1)</font><span style="font-family:'Times New Roman';font-size:7pt"> </span><u style="font-family:arial,sans-serif;font-size:12.800000190734863px"></u><span style="font-family:arial,sans-serif;font-size:12.800000190734863px">We have a lot of monitoring in place for all our applications, and I could probably alerted our Operations team if FreeSWITCH crashed or some error was logged in the FS log. However I was unable to found anything meaningful in the logs that indicates any sort of error. It just suddenly stopped creating new channels, and we get the alert/complain from the client – ouch!</span></blockquote>
<div><br></div></div><div>You could try sending a SIP OPTIONS packet from a monitoring system and check for a reply. Consecutive requests with no reply (timeout) could indicate a problem. One isn't enough in case the packet gets dropped.</div>
<div class="im">
<div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span style="font-family:arial,sans-serif;font-size:12.800000190734863px">2)<span style="font-size:7pt;font-family:'Times New Roman'"> </span></span><u style="font-family:arial,sans-serif;font-size:12.800000190734863px"></u><span style="font-family:arial,sans-serif;font-size:12.800000190734863px">I was also not sure what other information I should collect when something like this happens. As the application didn’t crash it didn’t create any coredump files. I know that restarting it fixes the problem, but that is not what the client wants to hear.</span></blockquote>
<div><br></div></div><div>Check out gcore - it can create a coredump of a running process. Very useful for debugging.</div><div><br></div><div>If you're compiling from git, you'll need to keep the source you compiled from (gdb needs the source to interpret the coredump).</div>
<div><br></div><div>-Steve</div><div><br></div><div><br></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div class="h5">On 30 November 2012 17:06, Hector Geraldino <span dir="ltr"><<a href="mailto:Hector.Geraldino@ipsoft.com" target="_blank">Hector.Geraldino@ipsoft.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal">Hello everyone,<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I have a FreeSWITCH instance that has been running for months without an issue, until last Monday. The setup is for a hotline where we receive incoming calls transferred from the Client Cisco UCM to our FreeSWITCH instance, and the calls
are handled by a custom Java application via ESL.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">As I said it was working flawlessly for weeks, but last Monday FreeSWITCH just stopped accepting new calls. I was able to see the SIP messages flowing back and forth from FS<->UCM, but looks like FreeSWITCH was unable to create new channels.
Once I restarted FreeSWITCH everything was back to normal. Here are links to the SIP/console logs I was able to capture which covers both scenarios (non-working FS and working FS):<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Non-working: <a href="http://pastebin.freeswitch.org/20275" target="_blank">
http://pastebin.freeswitch.org/20275</a><u></u><u></u></p>
<p class="MsoNormal">Working: <a href="http://pastebin.freeswitch.org/20274" target="_blank">http://pastebin.freeswitch.org/20274</a><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">As you can see, in the non-working scenario no channels were created on FreeSWITCH, so no 200 ACK message was sent back to UCM after the INVITE, which caused the UCM to send a CANCEL message after a few seconds and a 487 Request terminated
from FS -> UCM. Restarting the application fixed the issue, but I have a couple of questions regarding this:<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p><u></u><span>1)<span style="font:7.0pt "Times New Roman"">
</span></span><u></u>We have a lot of monitoring in place for all our applications, and I could probably alerted our Operations team if FreeSWITCH crashed or some error was logged in the FS log. However I was unable to found anything meaningful in the logs
that indicates any sort of error. It just suddenly stopped creating new channels, and we get the alert/complain from the client – ouch!<u></u><u></u></p>
<p><u></u><span>2)<span style="font:7.0pt "Times New Roman"">
</span></span><u></u>I was also not sure what other information I should collect when something like this happens. As the application didn’t crash it didn’t create any coredump files. I know that restarting it fixes the problem, but that is not what the
client wants to hear.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Any recommendations on what to do/look at to proactively detect this kind of issues and/or how to collect the appropriate info to get the root cause will be greatly appreciated.
<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Thank you all for your time,<u></u><u></u></p>
<p class="MsoNormal">Hector<u></u><u></u></p>
</div>
</div>
<br></div></div>_________________________________________________________________________<br>
Professional FreeSWITCH Consulting Services:<br>
<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" target="_blank">http://www.freeswitchsolutions.com</a><br>
<br>
FreeSWITCH-powered IP PBX: The CudaTel Communication Server<br>
<a href="http://www.cudatel.com" target="_blank">http://www.cudatel.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://wiki.freeswitch.org" target="_blank">http://wiki.freeswitch.org</a><br>
<a href="http://www.cluecon.com" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div>