Hello everyone,

 

I have a FreeSWITCH instance that has been running for months without an issue, until last Monday. The setup is for a hotline where we receive incoming calls transferred from the Client Cisco UCM to our FreeSWITCH instance, and the calls are handled by a custom Java application via ESL.

 

As I said it was working flawlessly for weeks, but last Monday FreeSWITCH just stopped accepting new calls. I was able to see the SIP messages flowing back and forth from FS<->UCM, but looks like FreeSWITCH was unable to create new channels. Once I restarted FreeSWITCH everything was back to normal. Here are links to the SIP/console logs I was able to capture which covers both scenarios (non-working FS and working FS):

 

Non-working: http://pastebin.freeswitch.org/20275

Working: http://pastebin.freeswitch.org/20274

 

As you can see, in the non-working scenario no channels were created on FreeSWITCH, so no 200 ACK message was sent back to UCM after the INVITE, which caused the UCM to send a CANCEL message after a few seconds and a 487 Request terminated from FS -> UCM. Restarting the application fixed the issue, but I have a couple of questions regarding this:

 

1)      We have a lot of monitoring in place for all our applications, and I could probably alerted our Operations team if FreeSWITCH crashed or some error was logged in the FS log. However I was unable to found anything meaningful in the logs that indicates any sort of error. It just suddenly stopped creating new channels, and we get the alert/complain from the client – ouch!

2)      I was also not sure what other information I should collect when something like this happens. As the application didn’t crash it didn’t create any coredump files. I know that restarting it fixes the problem, but that is not what the client wants to hear.

 

Any recommendations on what to do/look at to proactively detect this kind of issues and/or how to collect the appropriate info to get the root cause will be greatly appreciated.

 

Thank you all for your time,

Hector