Delay in Options Packets
Hello, Every now and then we see long delay in response for options. Example in the trace below free-switch took 44 seconds before it send 200 OK. What will be best way to troubleshoot this? U 2019/11/25 10:49:27.185325 192.168.1.1:49020 -> 192.168.1.5:5060 OPTIONS sip:100000@192.168.1.5 SIP/2.0. Via: SIP/2.0/UDP 192.168.2.1:49020;branch=z9hG4bKhjhs8ass877. Max-Forwards: 70. To: sip:100000@192.168.1.5. From: sip:1000@192.168.2.1:49020;tag=cbzxq0. Call-ID: hcrtfi@192.168.2.1. CSeq: 1 OPTIONS. Accept: application/sdp. Content-Length: 0. . U 2019/11/25 10:49:51.819959 192.168.1.5:5060 -> 192.168.1.1:49020 SIP/2.0 200 OK. Via: SIP/2.0/UDP 192.168.2.1:49020 ;branch=z9hG4bKhjhs8ass877;received=192.168.1.1. From: <sip:1000@192.168.2.1:49020>;tag=cbzxq0. To: <sip:100000@192.168.1.5>;tag=aUUQj55rcetma. Call-ID: hcrtfi@192.168.2.1. CSeq: 1 OPTIONS. Contact: <sip:192.168.1.5:5660>. User-Agent: Monitor. Accept: application/sdp. Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, MESSAGE, INFO, UPDATE, REGISTER, REFER, NOTIFY, PUBLISH, SUBSCRIBE. Supported: path, replaces. Allow-Events: talk, hold, conference, presence, as-feature-event, dialog, line-seize, call-info, sla, include-session-description, presence.winfo, message-summary, refer. Content-Length: 0.
Our code to handle options couldn’t be much simpler. If you are seeing massive delays the only thing I would guess would be that there is some sort of massive ddos type attack causing significant queuing of inbound requests or a register storm and your registrations handlers are getting behind. Take a look at logging and sip traffic around the same time for clues on what else might be happening on this sip profile at the same time.
On Nov 25, 2019, at 12:22 PM, Jai Rangi <jprangi@gmail.com> wrote:
Hello,
Every now and then we see long delay in response for options. Example in the trace below free-switch took 44 seconds before it send 200 OK.
What will be best way to troubleshoot this?
U 2019/11/25 10:49:27.185325 192.168.1.1:49020 <http://192.168.1.1:49020/> -> 192.168.1.5:5060 <http://192.168.1.5:5060/> OPTIONS sip:100000@192.168.1.5 <mailto:sip%3A100000@192.168.1.5> SIP/2.0. Via: SIP/2.0/UDP 192.168.2.1:49020;branch=z9hG4bKhjhs8ass877. Max-Forwards: 70. To: sip:100000@192.168.1.5 <mailto:sip%3A100000@192.168.1.5>. From: sip:1000@192.168.2.1:49020;tag=cbzxq0. Call-ID: hcrtfi@192.168.2.1 <mailto:hcrtfi@192.168.2.1>. CSeq: 1 OPTIONS. Accept: application/sdp. Content-Length: 0. .
U 2019/11/25 10:49:51.819959 192.168.1.5:5060 <http://192.168.1.5:5060/> -> 192.168.1.1:49020 <http://192.168.1.1:49020/> SIP/2.0 200 OK. Via: SIP/2.0/UDP 192.168.2.1:49020;branch=z9hG4bKhjhs8ass877;received=192.168.1.1. From: <sip:1000@192.168.2.1:49020 <http://sip:1000@192.168.2.1:49020/>>;tag=cbzxq0. To: <sip:100000@192.168.1.5 <mailto:sip%3A100000@192.168.1.5>>;tag=aUUQj55rcetma. Call-ID: hcrtfi@192.168.2.1 <mailto:hcrtfi@192.168.2.1>. CSeq: 1 OPTIONS. Contact: <sip:192.168.1.5:5660 <http://192.168.1.5:5660/>>. User-Agent: Monitor. Accept: application/sdp. Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, MESSAGE, INFO, UPDATE, REGISTER, REFER, NOTIFY, PUBLISH, SUBSCRIBE. Supported: path, replaces. Allow-Events: talk, hold, conference, presence, as-feature-event, dialog, line-seize, call-info, sla, include-session-description, presence.winfo, message-summary, refer. Content-Length: 0.
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
That's what I thought initially. There are around 1000 registrations (mostly TCP) (LUA + Mysql) with timeout of 15 minutes. There are lots of BLFs, subscribes and shared lines, and multiple registrations (average of 3.5, so total of 3500 registrations points). We do ping for keepalive every 3 minutes. I guess increasing ping time and registration timeout to 30 minutes will help. Thoughts? -Jai On Mon, Nov 25, 2019 at 4:33 PM Mike Jerris <mike@freeswitch.org> wrote:
Our code to handle options couldn’t be much simpler. If you are seeing massive delays the only thing I would guess would be that there is some sort of massive ddos type attack causing significant queuing of inbound requests or a register storm and your registrations handlers are getting behind. Take a look at logging and sip traffic around the same time for clues on what else might be happening on this sip profile at the same time.
On Nov 25, 2019, at 12:22 PM, Jai Rangi <jprangi@gmail.com> wrote:
Hello,
Every now and then we see long delay in response for options. Example in the trace below free-switch took 44 seconds before it send 200 OK.
What will be best way to troubleshoot this?
U 2019/11/25 10:49:27.185325 192.168.1.1:49020 -> 192.168.1.5:5060 OPTIONS sip:100000@192.168.1.5 SIP/2.0. Via: SIP/2.0/UDP 192.168.2.1:49020;branch=z9hG4bKhjhs8ass877. Max-Forwards: 70. To: sip:100000@192.168.1.5. From: sip:1000@192.168.2.1:49020;tag=cbzxq0. Call-ID: hcrtfi@192.168.2.1. CSeq: 1 OPTIONS. Accept: application/sdp. Content-Length: 0. .
U 2019/11/25 10:49:51.819959 192.168.1.5:5060 -> 192.168.1.1:49020 SIP/2.0 200 OK. Via: SIP/2.0/UDP 192.168.2.1:49020 ;branch=z9hG4bKhjhs8ass877;received=192.168.1.1. From: <sip:1000@192.168.2.1:49020>;tag=cbzxq0. To: <sip:100000@192.168.1.5>;tag=aUUQj55rcetma. Call-ID: hcrtfi@192.168.2.1. CSeq: 1 OPTIONS. Contact: <sip:192.168.1.5:5660>. User-Agent: Monitor. Accept: application/sdp. Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, MESSAGE, INFO, UPDATE, REGISTER, REFER, NOTIFY, PUBLISH, SUBSCRIBE. Supported: path, replaces. Allow-Events: talk, hold, conference, presence, as-feature-event, dialog, line-seize, call-info, sla, include-session-description, presence.winfo, message-summary, refer. Content-Length: 0.
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
On Tue, Nov 26, 2019 at 7:53 PM Jai Rangi <jprangi@gmail.com> wrote:
That's what I thought initially. There are around 1000 registrations (mostly TCP) (LUA + Mysql) with timeout of 15 minutes. There are lots of BLFs, subscribes and shared lines, and multiple registrations (average of 3.5, so total of 3500 registrations points). We do ping for keepalive every 3 minutes. I guess increasing ping time and registration timeout to 30 minutes will help. Thoughts?
You may want to uncomment (or add if not there) the second line: <!-- launch a new thread to process each new inbound register when using heavier backends --> <!-- <param name="inbound-reg-in-new-thread" value="true"/> --> in sip_profiles/internal.xml -giovanni
-Jai
On Mon, Nov 25, 2019 at 4:33 PM Mike Jerris <mike@freeswitch.org> wrote:
Our code to handle options couldn’t be much simpler. If you are seeing massive delays the only thing I would guess would be that there is some sort of massive ddos type attack causing significant queuing of inbound requests or a register storm and your registrations handlers are getting behind. Take a look at logging and sip traffic around the same time for clues on what else might be happening on this sip profile at the same time.
On Nov 25, 2019, at 12:22 PM, Jai Rangi <jprangi@gmail.com> wrote:
Hello,
Every now and then we see long delay in response for options. Example in the trace below free-switch took 44 seconds before it send 200 OK.
What will be best way to troubleshoot this?
U 2019/11/25 10:49:27.185325 192.168.1.1:49020 -> 192.168.1.5:5060 OPTIONS sip:100000@192.168.1.5 SIP/2.0. Via: SIP/2.0/UDP 192.168.2.1:49020;branch=z9hG4bKhjhs8ass877. Max-Forwards: 70. To: sip:100000@192.168.1.5. From: sip:1000@192.168.2.1:49020;tag=cbzxq0. Call-ID: hcrtfi@192.168.2.1. CSeq: 1 OPTIONS. Accept: application/sdp. Content-Length: 0. .
U 2019/11/25 10:49:51.819959 192.168.1.5:5060 -> 192.168.1.1:49020 SIP/2.0 200 OK. Via: SIP/2.0/UDP 192.168.2.1:49020 ;branch=z9hG4bKhjhs8ass877;received=192.168.1.1. From: <sip:1000@192.168.2.1:49020>;tag=cbzxq0. To: <sip:100000@192.168.1.5>;tag=aUUQj55rcetma. Call-ID: hcrtfi@192.168.2.1. CSeq: 1 OPTIONS. Contact: <sip:192.168.1.5:5660>. User-Agent: Monitor. Accept: application/sdp. Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, MESSAGE, INFO, UPDATE, REGISTER, REFER, NOTIFY, PUBLISH, SUBSCRIBE. Supported: path, replaces. Allow-Events: talk, hold, conference, presence, as-feature-event, dialog, line-seize, call-info, sla, include-session-description, presence.winfo, message-summary, refer. Content-Length: 0.
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
-- Sincerely, Giovanni Maruzzelli OpenTelecom.IT cell: +39 347 266 56 18
participants (3)
-
Giovanni Maruzzelli
-
Jai Rangi
-
Mike Jerris