<div dir="ltr">Thanks Alexey for your help in diagnosing the problem.<br>I dug further into the TCP stack and found out we are having 2 separate restart problems.<br><br>The first problem is not that the socket is failing to be released, but apparently Sofia is trying to open the port without the SO_REUSEADDR option, which means FS must wait for all connection to close first.<br>So as long as there are any TCP connections to those ports in the TIME_WAIT state, FS will not start Sofia.<br>I tested with netcat and that program can bind to the port regardless of the state of old connections, so I suspect this is a compile error.<br><br>The second problem is that the TLS and TCP registrations in sofia_reg_internal.db cause FS to throw the assertion error “Assertion failed: orq->orq_queue, file nta.c, line 8101”.<br>UDP registrations are fine.<br>Does FS try to continue these registrations assuming that the connection is still up or reopen a connection to the same port?<br>The assert line is found in nta.c’s outgoing_create function. It seems that the outgoing message queue is uninitialized, so the assertion error is likely more a symptom of some other problem, instead of being the cause of the problem.<br>How should I go about debugging the outgoing message queue? <br><br><br>Thanks in advance,<br>Mikko<br><br>---------- Edelleenlähetetty viesti ----------<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">From: Alexey Sibyakin <<a href="mailto:alex@freeswitch.com">alex@freeswitch.com</a>><br>To: FreeSWITCH Users Help <<a href="mailto:freeswitch-users@lists.freeswitch.org">freeswitch-users@lists.freeswitch.org</a>><br>Cc: <br>Bcc: <br>Date: Tue, 13 Feb 2018 17:23:06 +0900<br>Subject: Re: [Freeswitch-users] FS fails to restart when TLS endpoints are registered (Solaris 11.3)<br><div dir="ltr">It
looks like Sofia is not releasing ip on Solaris. If so it's a bug and
it should be reported to Sofia's bug tracker or Solaris or both. If you
can reproduce this problem on supported OS (Debian 8 at this moment) you
can open an issue on FreeSWITCH Jira.<div><br></div><div>Consider to read this first: <a href="https://freeswitch.org/confluence/display/FREESWITCH/Reporting+Bugs+to+JIRA" target="_blank">https://freeswitch.org/<wbr>confluence/display/FREESWITCH/<wbr>Reporting+Bugs+to+JIRA</a></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Feb 13, 2018 at 2:30 AM, GBC Networks Oy <span dir="ltr"><<a href="mailto:info@gbc.fi" target="_blank">info@gbc.fi</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">We're
running FS on Solaris 11.3 and it is working perfectly, except that
we're unable to restart FS without deleting the TLS registrations from
sofia_reg_internal.db.<br>Our FS version is FreeSWITCH Version 1.9.0+git~20171010T231940Z~594<wbr>0b04d4e~64bit (git 5940b04 2017-10-10 23:19:40Z 64bit)<br><br>Here are the symptoms:<br>1. Service stops FS with fs_cli -x "fsctl shutdown now"<br>2. FS stops and the service is restarted<br>3. FS starts up and fs_cli is working, but ” sofia status” is empty<br>The console shows:<br>freeswitch@fs> 2018-02-12 17:06:23.875991 [ERR] sofia.c:3221 Error Creating SIP UA for profile: internal (<a href="mailto:sip%3Amod_sofia@123.123.123." target="_blank">sip:mod_sofia@123.123.123.</a>123<wbr>:5060;maddr=123.123.123.123;<wbr>transport=udp,tcp) ATTEMPT 3 (RETRY IN 5 SEC) <wbr> <wbr> <wbr> <br>2018-02-12 17:06:23.875991 [ERR] sofia.c:3231 Error Creating SIP UA for profile: internal (<a href="mailto:sip%3Amod_sofia@123.123.123." target="_blank">sip:mod_sofia@123.123.123.</a>123<wbr>:5060;maddr=123.123.123.123;<wbr>transport=udp,tcp) <wbr> <wbr> <wbr> <wbr> <wbr> <br>The likely causes for this are: <wbr> <wbr> <wbr> <wbr> <br>1) Another application is already listening on the specified address. <wbr> <wbr> <wbr> <br>2) The IP the profile is attempting to bind to is not local to this system.<br><br>Note: the IP is correct and nothing is listening on the port.<br><br>4. Next we stop FS and restart<br>This time the console shows:<br>2018-02-12 13:33:51.132676 [INFO] switch_core_sqldb.c:1693 sofia:internal Starting SQL thread.<br>2018-02-12 13:33:51.132770 [DEBUG] sofia.c:3373 Starting thread for internal<br>2018-02-12 13:33:51.132777 [DEBUG] sofia.c:3020 Launching worker thread for internal<br>2018-02-12 13:33:51.138255 [NOTICE] sofia_reg.c:3398 Added gateway '5e4ac47f-6ce4-462d-9247-56571<wbr>1fe3d4e' to profile 'internal'<br>2018-02-12 13:33:51.138405 [NOTICE] sofia_reg.c:3398 Added gateway 'b2563fad-0051-4067-abc3-6a431<wbr>e05481e' to profile 'internal'<br>2018-02-12 13:33:51.247731 [DEBUG] freeswitch_lua.cpp:372 DBH handle 125b300 Connected.<br>2018-02-12 13:33:51.254128 [DEBUG] freeswitch_lua.cpp:401 DBH handle 125b300 released.<br>Assertion failed: orq->orq_queue, file nta.c, line 8101<br><br>5. sqlite3 sofia_reg_internal.db "DELETE from sip_registrations WHERE status LIKE '%TLS%'"<br>6. FS starts normally<br><br><br>Here is a registration (non-TLS) that doesn’t break sofia:<br># sqlite3 -header sofia_reg_internal.db "SELECT * from sip_registrations"<br>call_id|sip_user|sip_host|pres<wbr>ence_hosts|contact|status|ping<wbr>_status|ping_count|ping_time|<wbr>force_ping|rpid|expires|ping_<wbr>expires|user_agent|server_<wbr>user|server_host|profile_name|<wbr>hostname|network_ip|network_<wbr>port|sip_username|sip_realm|<wbr>mwi_user|mwi_host|orig_server_<wbr>host|orig_hostname|sub_host<br>313531383139323735393532323132<wbr>-m5mp9qlsdu27|USERNAME|<a href="http://fs.domain.com" target="_blank">fs.doma<wbr>in.com</a>||"FS" <sip:USERNAME@REMOTE-IP:2054>|<wbr>Registered(UDP)|Reachable|3|39<wbr>824|1|unknown|1518455533|15184<wbr>53577|snom320/<a href="http://8.7.5.35" target="_blank">8.7.5.35</a>|USERNAM<wbr>E||internal|fs|REMOTE-IP|2054|<wbr>USERNAME|<a href="http://fs.domain.com" target="_blank">fs.domain.com</a>|<wbr>USERNAME|<a href="http://fs.domain.com" target="_blank">fs.domain.com</a>||fs|<a href="http://fs.domain.com" target="_blank">fs.<wbr>domain.com</a><br><br>Here is a registration (TLS) that breaks the restart:<br># sqlite3 -header sofia_reg_internal.db "SELECT * from sip_registrations" <wbr> <br>call_id|sip_user|sip_host|pres<wbr>ence_hosts|contact|status|ping<wbr>_status|ping_count|ping_time|<wbr>force_ping|rpid|expires|ping_<wbr>expires|user_agent|server_<wbr>user|server_host|profile_name|<wbr>hostname|network_ip|network_<wbr>port|sip_username|sip_realm|<wbr>mwi_user|mwi_host|orig_server_<wbr>host|orig_hostname|sub_host<br>313531383139323735393532323132<wbr>-m5mp9qlsdu27|USERNAME|<a href="http://fs.domain.com" target="_blank">fs.doma<wbr>in.com</a>||"FS" <<a href="mailto:sip%3AUSERNAME@192.168.6.236" target="_blank">sip:USERNAME@192.168.6.236</a>:21<wbr>70;transport=tls;fs_nat=yes;fs<wbr>_path=sip%3AUSERNAME%40REMOTE-<wbr>IP%3A2170%3Btransport%3Dtls>|<wbr>Registered(TLS-NAT)|Reachable|<wbr>0||1|unknown|1518457302|<wbr>1518453664|snom320/<a href="http://8.7.5.35" target="_blank">8.7.5.35</a>|<wbr>USERNAME||internal|fs|REMOTE-<wbr>IP|2170|USERNAME|<a href="http://fs.domain.com" target="_blank">fs.domain.com</a><wbr>|USERNAME|<a href="http://fs.domain.com" target="_blank">fs.domain.com</a>||fs|<a href="http://fs.domain.com" target="_blank">fs<wbr>.domain.com</a><br><br><br>Best regards,<br>Mikko<br></div>
<br>______________________________<wbr>______________________________<wbr>_____________<br>
Professional FreeSWITCH Consulting Services:<br>
<a href="mailto:consulting@freeswitch.org" target="_blank">consulting@freeswitch.org</a><br>
<a href="http://www.freeswitchsolutions.com" rel="noreferrer" target="_blank">http://www.freeswitchsolutions<wbr>.com</a><br>
<br>
Official FreeSWITCH Sites<br>
<a href="http://www.freeswitch.org" rel="noreferrer" target="_blank">http://www.freeswitch.org</a><br>
<a href="http://confluence.freeswitch.org" rel="noreferrer" target="_blank">http://confluence.freeswitch.o<wbr>rg</a><br>
<a href="http://www.cluecon.com" rel="noreferrer" target="_blank">http://www.cluecon.com</a><br>
<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org" target="_blank">FreeSWITCH-users@lists.freeswi<wbr>tch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" rel="noreferrer" target="_blank">http://lists.freeswitch.org/ma<wbr>ilman/listinfo/freeswitch-user<wbr>s</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" rel="noreferrer" target="_blank">http://lists.frees<wbr>witch.org/mailman/options/<wbr>freeswitch-users</a><br>
<a href="http://www.freeswitch.org" rel="noreferrer" target="_blank">http://www.freeswitch.org</a><br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail-m_8842679982169464259gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div style="font-family:HelveticaNeue,"Helvetica Neue",Helvetica,Arial,"Lucida Grande",sans-serif"><div dir="ltr" style="color:rgb(0,0,0);font-size:12.8px;line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:4.5pt"><img src="https://docs.google.com/uc?export=download&id=1xswZRZyVDo0WQhaemK47pU266yzDRmi0&revid=0B2xnT7i45ngrMTVKM1dpSHZIN28zU0QzbW9xeVF6RXFyRHhBPQ"><br></div><div dir="ltr" style="color:rgb(0,0,0);font-size:12.8px;line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:4.5pt"><span style="background-color:transparent;vertical-align:baseline"><font style="font-family:Arial;font-size:8pt;white-space:pre-wrap" color="#000000">Alex Sibyakin | Support Engineer</font></span></div><div dir="ltr" style="color:rgb(0,0,0);font-size:12.8px;line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:4.5pt"><span style="font-size:8pt;font-family:Arial;background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><font color="#000000">FreeSWITCH Solutions | <a rel="nofollow" shape="rect" href="https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g" style="margin:0px;padding:0px;background:transparent none repeat scroll 0% 0%;color:rgb(25,106,212);outline:medium none" target="_blank">17345 Civic Drive #2531 Brookfield, WI 53045</a></font></span></div><div dir="ltr" style="font-size:12.8px;line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:4.5pt"><font><span style="font-size:8pt;font-family:Arial;background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><font color="#000000">Email: </font><a href="mailto:alex@freeswitch.com" target="_blank"><font color="#3d85c6">alex@freeswitch.com</font></a></span></font></div><div dir="ltr" style="color:rgb(0,0,0);font-size:12.8px;line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:4.5pt"><font color="#000000"><span style="font-size:8pt;font-family:Arial;background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Website: </span><span style="margin:0px;padding:0px;background:transparent none repeat scroll 0% 0%;color:rgb(25,106,212);outline:medium none;font-size:8pt;font-family:Arial;vertical-align:baseline;white-space:pre-wrap"><a rel="nofollow" shape="rect" href="https://www.freeswitch.com/" style="margin:0px;padding:0px;background:transparent none repeat scroll 0% 0%;color:rgb(25,106,212);outline:medium none" target="_blank">https://www.FreeSWITCH.com</a></span></font></div><div dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;margin-left:4.5pt"><font size="1"><span style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><font color="#000000">Need commercial support? Contact</font></span><span style="color:rgb(51,51,51);font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"> </span><a rel="noopener noreferrer" href="mailto:sales@freeswitch.com" style="color:rgb(53,114,176);font-family:"Helvetica Neue",Helvetica,Arial,sans-serif" target="_blank">sales@freeswitch.com</a><span style="color:rgb(51,51,51);font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"> </span><span style="font-family:"Helvetica Neue",Helvetica,Arial,sans-serif"><font color="#000000">f<wbr>or details.</font></span></font><font style="color:rgb(0,0,0);font-size:12.8px" color="#000000"><br></font></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div>
<br>______________________________<wbr>_________________<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org">FreeSWITCH-users@lists.<wbr>freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" rel="noreferrer" target="_blank">http://lists.freeswitch.org/<wbr>mailman/listinfo/freeswitch-<wbr>users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" rel="noreferrer" target="_blank">http://lists.<wbr>freeswitch.org/mailman/<wbr>options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" rel="noreferrer" target="_blank">http://www.freeswitch.org</a><br>
<br></blockquote></div><br></div></div>