[Freeswitch-users] Choppy audio when conferencing 4+ participants

Tue Apr 18 20:56:28 MSD 2017

In general when a thread about performance or vm turns to lots of
theories.  This is a science so we need facts to diagnose and its often not
possible once we get this far down the rabbit hole.  To the contrary,
asking about timer test was a good start!

On Tue, Apr 18, 2017 at 11:41 AM Michael Jerris <mike at jerris.com> wrote:

> Superstition comment was in regards to tc malloc, not you.
>
> On Apr 18, 2017, at 7:35 AM, David Ponzone <david.ponzone at gmail.com>
> wrote:
>
> Anthony,
>
> Could you elaborate a little bit more on why I am being superstitious ?
> Is timer_test command obsolete ?
> Or is it a way to emphase the fact that you won’t support FS on VM, anyway
> ?
>
> Le 17 avr. 2017 à 23:27, Anthony Minessale <anthony.minessale at gmail.com>
> a écrit :
>
> I believe we may be stumbling into superstition at this point.
>
> On Fri, Apr 14, 2017 at 6:03 PM, David Ponzone <david.ponzone at gmail.com>
> wrote:
>
> Did you run the timer_test command ?
>>
> Le 14 avr. 2017 à 23:04, Bilal Dar <bilal at rgate-systems.com> a écrit :
>>
>> The issue triggered when I ran out of inodes on the server, even after
>> freeing inodes things never went back to normal. I was running earlier
>>  m3.large and now moved to m3.2xlarge servers, CPU/memory utilization is
>> negligible.
>>
>> Model   vCPU    Mem (GiB)       SSD Storage (GB)
>> m3.large        2       7.5     1 x 32
>> m3.2xlarge      8       30      2 x 80
>>
>>
>>
>> On Fri, Apr 14, 2017 at 1:53 PM, Colin Morelli <colin.morelli at gmail.com>
>> wrote:
>>
>>> Robert,
>>>
>>> While I'd love to see VMs provide more stable ground for FS, it's simply
>>> not the best task for a VM. Virtual machines scale well specifically
>>> because you can overprovision them. It would not be nearly as
>>> cost-effective to run VMs if each instance had a guaranteed dedicated slice
>>> of hardware to operate on. While hypervisors are *very* good at task
>>> prioritization, they're not perfect. If the hypervisor can't schedule
>>> processor time when FS needs it because the CPU cores are momentarily taken
>>> on other tasks, there's not a whole lot FS can do. This is not an issue
>>> with just FS, but with all real-time applications. In most apps, even large
>>> clock skews and bad hypervisors schedulers can go completely unnoticed. If
>>> there's consistent 5-10ms every time you click to load a web page, you'd
>>> probably have no idea. If there's 5-10ms jitter every time you try to read
>>> 20ms of audio, you have *really bad* audio. Granted most skews are not
>>> that bad, but the effects are pronounced when you're dealing with data
>>> that's real-time in nature.
>>>
>>> Bilal,
>>>
>>> I have no idea what AMI you're running, but a very rough "ear test" has
>>> made me fairly confident that I can get better performance running
>>> AmazonLinux AMIs over Ubuntu (and probably many others). It wouldn't
>>> surprise me if AmazonLinux builds a custom kernel that has been tuned to
>>> run better on AWS hardware. I'd say it's at least worth a quick experiment.
>>>
>>> Best,
>>> Colin
>>>
>>> On Fri, Apr 14, 2017 at 4:34 PM, Mundkowsky, Robert <rmundkowsky at ets.org
>>> > wrote:
>>>
>>>> Michael
>>>>
>>>>
>>>>
>>>> Just curious, why so many problems with VMs?
>>>>
>>>>
>>>>
>>>> I would think most applications need real time clocks that provide
>>>> consistent valid data?
>>>>
>>>>
>>>>
>>>> Bilal,
>>>>
>>>>
>>>>
>>>> you might try a larger AWS instance to make sure your are getting 100%
>>>> of the box; might help some.
>>>>
>>>>
>>>>
>>>> Robert
>>>>
>>>>
>>>>
>>>> *From:* freeswitch-users-bounces at lists.freeswitch.org [mailto:
>>>> freeswitch-users-bounces at lists.freeswitch.org] *On Behalf Of *Michael
>>>> Jerris
>>>> *Sent:* Friday, April 14, 2017 4:25 PM
>>>> *To:* FreeSWITCH Users Help <freeswitch-users at lists.freeswitch.org>
>>>> *Subject:* Re: [Freeswitch-users] Choppy audio when conferencing 4+
>>>> participants
>>>>
>>>>
>>>>
>>>> can you reproduce the same issue on real hardware?  We’ve seen all
>>>> kinds of weird timing issues that could account for this running on aws.
>>>>
>>>>
>>>>
>>>> On Apr 14, 2017, at 4:16 PM, Bilal Dar <bilal at rgate-systems.com> wrote:
>>>>
>>>>
>>>>
>>>> Its an AWS m3.2xlarge instance.
>>>>
>>>>
>>>>
>>>> On Fri, Apr 14, 2017 at 9:57 AM, Michael Jerris <mike at jerris.com>
>>>> wrote:
>>>>
>>>> Real hardware or VM?
>>>>
>>>> > On Apr 14, 2017, at 12:48 PM, Bilal Dar <bilal at rgate-systems.com>
>>>> wrote:
>>>> >
>>>> > I have been struggling with an issue for almost 2 weeks.
>>>> >
>>>> > Our regular calls have no quality issue and looking RTCP statistics
>>>> network conditions are perfect. We have normally on peak hr 60 calls and
>>>> around 10 conferences.
>>>> >
>>>> > We have noticed that when we have 2 conferences of 4 or 5
>>>> participants, audio starts breaking for the users who are on conference.
>>>> Regular calls do not experience any quality degradation.
>>>> >
>>>> > I upgraded the server to specs of 30Gig memory and 8 vCPU but still
>>>> the issue exists. Common thing I have noticed even during off-peak hrs is
>>>> that two 4+ participant call can cause the issue.
>>>> >
>>>> > I have ruled out network & hardware. Last change I made was moved all
>>>> users to G.711 from G.722. Now I am not sure what other steps I can take.
>>>> Appreciate any suggestions.
>>>>
>>>>
>>>>
>>> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://confluence.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org

-- 
Anthony Minessale II       ♬ @anthmfs  ♬ @FreeSWITCH  ♬

☞ http://freeswitch.org/  ☞ http://cluecon.com/  ☞
http://twitter.com/FreeSWITCH
☞ irc.freenode.net #freeswitch ☞ *http://freeswitch.org/g+
<http://freeswitch.org/g+>*

ClueCon Weekly Development Call
☎ sip:888 at conference.freeswitch.org  ☎ +19193869900

https://www.youtube.com/watch?v=9XXgW34t40s
https://www.youtube.com/watch?v=NLaDpGQuZDA
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20170418/991c239b/attachment-0001.html 