Thanks Alexey for your help in diagnosing the problem. I dug further into the TCP stack and found out we are having 2 separate restart problems. The first problem is not that the socket is failing to be released, but apparently Sofia is trying to open the port without the SO_REUSEADDR option, which means FS must wait for all connection to close first. So as long as there are any TCP connections to those ports in the TIME_WAIT state, FS will not start Sofia. I tested with netcat and that program can bind to the port regardless of the state of old connections, so I suspect this is a compile error. The second problem is that the TLS and TCP registrations in sofia_reg_internal.db cause FS to throw the assertion error “Assertion failed: orq->orq_queue, file nta.c, line 8101”. UDP registrations are fine. Does FS try to continue these registrations assuming that the connection is still up or reopen a connection to the same port? The assert line is found in nta.c’s outgoing_create function. It seems that the outgoing message queue is uninitialized, so the assertion error is likely more a symptom of some other problem, instead of being the cause of the problem. How should I go about debugging the outgoing message queue? Thanks in advance, Mikko ---------- Edelleenlähetetty viesti ----------

From: Alexey Sibyakin <alex@freeswitch.com> To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> Cc: Bcc: Date: Tue, 13 Feb 2018 17:23:06 +0900 Subject: Re: [Freeswitch-users] FS fails to restart when TLS endpoints are registered (Solaris 11.3) It looks like Sofia is not releasing ip on Solaris. If so it's a bug and it should be reported to Sofia's bug tracker or Solaris or both. If you can reproduce this problem on supported OS (Debian 8 at this moment) you can open an issue on FreeSWITCH Jira.

Consider to read this first: https://freeswitch.org/ confluence/display/FREESWITCH/Reporting+Bugs+to+JIRA

On Tue, Feb 13, 2018 at 2:30 AM, GBC Networks Oy <info@gbc.fi> wrote:

...

We're running FS on Solaris 11.3 and it is working perfectly, except that we're unable to restart FS without deleting the TLS registrations from sofia_reg_internal.db. Our FS version is FreeSWITCH Version 1.9.0+git~20171010T231940Z~5940b04d4e~64bit (git 5940b04 2017-10-10 23:19:40Z 64bit)

Here are the symptoms: 1. Service stops FS with fs_cli -x "fsctl shutdown now" 2. FS stops and the service is restarted 3. FS starts up and fs_cli is working, but ” sofia status” is empty The console shows: freeswitch@fs> 2018-02-12 17:06:23.875991 [ERR] sofia.c:3221 Error Creating SIP UA for profile: internal (sip:mod_sofia@123.123.123.123 :5060;maddr=123.123.123.123;transport=udp,tcp) ATTEMPT 3 (RETRY IN 5 SEC)

2018-02-12 17:06:23.875991 [ERR] sofia.c:3231 Error Creating SIP UA for profile: internal (sip:mod_sofia@123.123.123.123 :5060;maddr=123.123.123.123;transport=udp,tcp)

The likely causes for this are:

1) Another application is already listening on the specified address.

2) The IP the profile is attempting to bind to is not local to this system.

Note: the IP is correct and nothing is listening on the port.

4. Next we stop FS and restart This time the console shows: 2018-02-12 13:33:51.132676 [INFO] switch_core_sqldb.c:1693 sofia:internal Starting SQL thread. 2018-02-12 13:33:51.132770 [DEBUG] sofia.c:3373 Starting thread for internal 2018-02-12 13:33:51.132777 [DEBUG] sofia.c:3020 Launching worker thread for internal 2018-02-12 13:33:51.138255 [NOTICE] sofia_reg.c:3398 Added gateway '5e4ac47f-6ce4-462d-9247-565711fe3d4e' to profile 'internal' 2018-02-12 13:33:51.138405 [NOTICE] sofia_reg.c:3398 Added gateway 'b2563fad-0051-4067-abc3-6a431e05481e' to profile 'internal' 2018-02-12 13:33:51.247731 [DEBUG] freeswitch_lua.cpp:372 DBH handle 125b300 Connected. 2018-02-12 13:33:51.254128 [DEBUG] freeswitch_lua.cpp:401 DBH handle 125b300 released. Assertion failed: orq->orq_queue, file nta.c, line 8101

5. sqlite3 sofia_reg_internal.db "DELETE from sip_registrations WHERE status LIKE '%TLS%'" 6. FS starts normally

Here is a registration (non-TLS) that doesn’t break sofia: # sqlite3 -header sofia_reg_internal.db "SELECT * from sip_registrations" call_id|sip_user|sip_host|presence_hosts|contact|status|ping _status|ping_count|ping_time|force_ping|rpid|expires|ping_ expires|user_agent|server_user|server_host|profile_name| hostname|network_ip|network_port|sip_username|sip_realm| mwi_user|mwi_host|orig_server_host|orig_hostname|sub_host 313531383139323735393532323132-m5mp9qlsdu27|USERNAME|fs.domain.com||"FS" <sip:USERNAME@REMOTE-IP:2054>|Registered(UDP)|Reachable|3|39 824|1|unknown|1518455533|1518453577|snom320/8.7.5.35|USERNAM E||internal|fs|REMOTE-IP|2054|USERNAME|fs.domain.com|USERNAME| fs.domain.com||fs|fs.domain.com

Here is a registration (TLS) that breaks the restart: # sqlite3 -header sofia_reg_internal.db "SELECT * from sip_registrations" call_id|sip_user|sip_host|presence_hosts|contact|status|ping _status|ping_count|ping_time|force_ping|rpid|expires|ping_ expires|user_agent|server_user|server_host|profile_name| hostname|network_ip|network_port|sip_username|sip_realm| mwi_user|mwi_host|orig_server_host|orig_hostname|sub_host 313531383139323735393532323132-m5mp9qlsdu27|USERNAME|fs.domain.com||"FS" <sip:USERNAME@192.168.6.236:2170;transport=tls;fs_nat=yes;fs _path=sip%3AUSERNAME%40REMOTE-IP%3A2170%3Btransport%3Dtls>| Registered(TLS-NAT)|Reachable|0||1|unknown|1518457302|1518453664|snom320/ 8.7.5.35|USERNAME||internal|fs|REMOTE-IP|2170|USERNAME|fs.domain.com |USERNAME|fs.domain.com||fs|fs.domain.com

Best regards, Mikko

_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com

Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com

FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

--

Alex Sibyakin | Support Engineer FreeSWITCH Solutions | 17345 Civic Drive #2531 Brookfield, WI 53045 <https://maps.google.com/?q=17345+Civic+Drive+%232531+Brookfield,+WI+53045&entry=gmail&source=g> Email: alex@freeswitch.com Website: https://www.FreeSWITCH.com <https://www.freeswitch.com/> Need commercial support? Contact sales@freeswitch.com for details.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org