[Freeswitch-users] Problem with rtp streams and stair-stepped (or ramped) increased lag

Wed Jun 26 04:36:22 MSD 2013

Both... Graphs are from voipmonitor, but the wide shifts in delay are very visible when processing the stream in wireshark.

-- Nathan

On 06/25/2013 07:35 PM, William King wrote:
> What are you using to analyze the rtp streams? Wireshark? or voipmonitor?
>
> William King
> Senior Engineer
> Quentus Technologies, INC
> 1037 NE 65th St Suite 273
> Seattle, WA 98115
> Main:   (877) 211-9337
> Office: (206) 388-4772
> Cell:   (253) 686-5518
> william.king at quentustech.com
>
> On 06/25/2013 05:19 PM, Nathan Neulinger wrote:
>> Very interesting... look forward to seeing the comparison of those two
>> captures... I'm in the process of trying to rule out vmware as a
>> contributing factor by moving the FS servers to dedicated hardware, but
>> your description sounds like there may be some other issue in addition.
>>
>> In my case, it's all confined within local network, so there shouldn't
>> be much of any latency spikes, at least not significant ones.
>>
>> Looking at the graphs of a few calls today, on at least one of the
>> not-so-bad ones, I saw a very periodic blip on the stairstep every 60
>> seconds. That doesn't cover the large drift ones in the graphs I posted
>> though.
>>
>> -- Nathan
>>
>> On 06/25/2013 06:48 PM, William King wrote:
>>> Nathan,
>>>
>>> I had a chance to diagnose the issue live on a system today. While I'm
>>> not yet sure we are hitting the same issue, I'm now confident the
>>> problem I'm hitting is not FS related, but looks to be kernel related. I
>>> was able to get multiple calls up all with large 5+ second delays.
>>> Restarting the calls did nothing to clear the problem. Restarting
>>> Freeswitch repeatedly also had no effect. Upon restarting the box, the
>>> problem was gone.
>>>
>>> Also in my diagnostics I've found that it isn't all UDP packets that are
>>> effected. I now have a switch mirroring the port and a pcap running
>>> locally on the machine. I'll see tomorrow if it's just a matter of an
>>> inbound or outbound network stack buffer(or network stack priority)
>>> issue.
>>>
>>> William King
>>> Senior Engineer
>>> Quentus Technologies, INC
>>> 1037 NE 65th St Suite 273
>>> Seattle, WA 98115
>>> Main:   (877) 211-9337
>>> Office: (206) 388-4772
>>> Cell:   (253) 686-5518
>>> william.king at quentustech.com
>>>
>>> On 06/25/2013 05:13 AM, Nathan Neulinger wrote:
>>>> I guess the big question is this:
>>>>
>>>>      Is it _supposed_ to recover?
>>>>
>>>>    From Tony's notes, it's sounding like it's very critically
>>>> dependent on the timer in the switch, which very well be the
>>>> current cause of my issues.
>>>>
>>>> What I probably need to do is have a pair of phones automatically
>>>> calling each other, and just leaving them up - just to
>>>> generate a continuous independent background stream. That'll tell me
>>>> if any glitches are specific to a particular
>>>> channel, or if they are common across multiple calls - which I'd
>>>> expect if the timer was swinging.
>>>>
>>>> I don't know where my latency is being introduced unfortunately...
>>>>
>>>> -- Nathan
>>>>
>>>> On 06/25/2013 01:59 AM, Avi Marcus wrote:
>>>>> It sounds like I saw  this on a home internet connection to a public
>>>>> server when the connection had varying latency.
>>>>> Once the call jittered, it never recovered.
>>>>> My stop-gap was to add a small jitter buffer
>>>>> <http://wiki.freeswitch.org/wiki/Jitterbuffer>, e.g. <action
>>>>> application="jitterbuffer" data="100:200:20"/> to the user dialplan.
>>>>> This /seemed /to make FS handle rewriting the timestamps for drift
>>>>> rather than relying on the Linksys ATA to do that.
>>>>>
>>>>> Probably the wrong solution, but all I had time/understanding for.
>>>>> -Avi
>>>>>
>>>>>
>>>>> On Tue, Jun 25, 2013 at 4:42 AM, Nathan Neulinger <nneul at mst.edu
>>>>> <mailto:nneul at mst.edu>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>       On 06/24/2013 08:24 PM, Anthony Minessale wrote:
>>>>>        > fsctl debug_level 10
>>>>>        > Also make sure ve host has high res timing.
>>>>>
>>>>>       Turned up logging, will wait for next occurrence.
>>>>>
>>>>>       Not sure on the latter - it's a vmware ESXi 5.1 box running FC
>>>>> 17 x86_64 as the guest.
>>>>>
>>>>>       /dev/rtc /dev/rtc0 exist
>>>>>       posix and soft timers are loaded in fs
>>>>>
>>>>>       I've got a bunch of unused Dell 1950's on hand, so could use a
>>>>> few of those machines if needed to get vmware out of the
>>>>>       picture.
>>>>>
>>>>>       -- Nathan
>>>>>
>>>>>       ------------------------------------------------------------
>>>>>       Nathan Neulinger nneul at mst.edu <mailto:nneul at mst.edu>
>>>>>       Missouri S&T Information Technology (573) 612-1412
>>>>> <tel:%28573%29%20612-1412>
>>>>>       System Administrator - Architect
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>>
>>>>>       Professional FreeSWITCH Consulting Services:
>>>>>       consulting at freeswitch.org <mailto:consulting at freeswitch.org>
>>>>>       http://www.freeswitchsolutions.com
>>>>>
>>>>>       
>>>>>       
>>>>>
>>>>>       Official FreeSWITCH Sites
>>>>>       http://www.freeswitch.org
>>>>>       http://wiki.freeswitch.org
>>>>>       http://www.cluecon.com
>>>>>
>>>>>       FreeSWITCH-users mailing list
>>>>>       FreeSWITCH-users at lists.freeswitch.org
>>>>> <mailto:FreeSWITCH-users at lists.freeswitch.org>
>>>>>       http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>
>>>>>       http://www.freeswitch.org
>>>>>
>>>>>
>>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> 
>>> 
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>

-- 
------------------------------------------------------------
Nathan Neulinger                       nneul at mst.edu
Missouri S&T Information Technology    (573) 612-1412
System Administrator - Architect