[Freeswitch-users] Timing issues in AWS?

Thu Nov 17 16:49:01 MSK 2011

I'm running freeswitch on about 40  different m1.small c1.medium's in AWS
regions, us-east, us-west, eu-west, and asia..  They are used in videoconf
component of BigBlueButton.org .   For most part they work great. There are
occasional issues with voip quality but the app is 100% voip, with the BBB
client all browser based.  So the conference is subject to every ones local
network connections and most issues "blamed" on the internet instead of
freeswitch J  We also tie in skype and DID direct to improve latency for
some clients.  But expectations are set so we meet them.

You probably have tougher business conferencing clients that want perfect
audio.  But these are production deployed and generate revenue.      All
these are on Ubuntu 10.04 official amis with no mods to kernels.  

I'm sure there are tweaks that can be made, and bare metal solutions that
would work a little better.  Does anyone have any ideas how to optimize a
Ubuntu instance.  I would engage in a few hours of consulting is so.

The tests below are interesting but above my paygrade to understand what
they mean? 

I'm running them but don't have a clue how to interpret.  If you want a
simple of real running data, would be glad to run sample tests on these
distributed servers and provide back for analysis.

Regards,

Stephen

From: freeswitch-users-bounces at lists.freeswitch.org
[mailto:freeswitch-users-bounces at lists.freeswitch.org] On Behalf Of Michael
Picher
Sent: Thursday, November 17, 2011 8:03 AM
To: FreeSWITCH Users Help
Subject: Re: [Freeswitch-users] Timing issues in AWS?

Not having anywhere near the same trouble in a Xen Server in my lab...

Although test_time is skewed 180 degrees from what it was in AWS....

freeswitch at internal> timer_test 120 10

Avg: 119.945ms Total Time: 1199.727ms

2011-11-17 07:57:22.721105 [CONSOLE] mod_commands.c:461 Timer Test:
samplecount after init: 1
2011-11-17 07:57:22.751026 [CONSOLE] mod_commands.c:466 Timer Test:
samplecount after first step: 2
2011-11-17 07:57:22.872047 [CONSOLE] mod_commands.c:475 Timer Test: 1 sleep
120 120993
2011-11-17 07:57:22.990983 [CONSOLE] mod_commands.c:475 Timer Test: 2 sleep
120 118914
2011-11-17 07:57:23.110953 [CONSOLE] mod_commands.c:475 Timer Test: 3 sleep
120 119919
2011-11-17 07:57:23.230928 [CONSOLE] mod_commands.c:475 Timer Test: 4 sleep
120 120032
2011-11-17 07:57:23.350831 [CONSOLE] mod_commands.c:475 Timer Test: 5 sleep
120 119802
2011-11-17 07:57:23.470886 [CONSOLE] mod_commands.c:475 Timer Test: 6 sleep
120 119999
2011-11-17 07:57:23.590873 [CONSOLE] mod_commands.c:475 Timer Test: 7 sleep
120 119975
2011-11-17 07:57:23.710893 [CONSOLE] mod_commands.c:475 Timer Test: 8 sleep
120 120003
2011-11-17 07:57:23.830774 [CONSOLE] mod_commands.c:475 Timer Test: 9 sleep
120 119849
2011-11-17 07:57:23.950808 [CONSOLE] mod_commands.c:475 Timer Test: 10 sleep
120 119971

freeswitch at internal> time_test 600 10

test 1 sleep 600 1
test 2 sleep 600 1
test 3 sleep 600 0
test 4 sleep 600 1
test 5 sleep 600 0
test 6 sleep 600 1
test 7 sleep 600 1
test 8 sleep 600 0
test 9 sleep 600 0
test 10 sleep 600 1
avg 0

[root at openuc bin]# uname -r

2.6.18-274.7.1.el5

[root at openuc bin]# grep CONFIG_HZ /boot/config-*

/boot/config-2.6.18-238.12.1.el5:# CONFIG_HZ_100 is not set
/boot/config-2.6.18-238.12.1.el5:# CONFIG_HZ_250 is not set
/boot/config-2.6.18-238.12.1.el5:CONFIG_HZ_1000=y
/boot/config-2.6.18-238.12.1.el5:CONFIG_HZ=1000
/boot/config-2.6.18-238.19.1.el5:# CONFIG_HZ_100 is not set
/boot/config-2.6.18-238.19.1.el5:# CONFIG_HZ_250 is not set
/boot/config-2.6.18-238.19.1.el5:CONFIG_HZ_1000=y
/boot/config-2.6.18-238.19.1.el5:CONFIG_HZ=1000
/boot/config-2.6.18-238.9.1.el5:# CONFIG_HZ_100 is not set
/boot/config-2.6.18-238.9.1.el5:# CONFIG_HZ_250 is not set
/boot/config-2.6.18-238.9.1.el5:CONFIG_HZ_1000=y
/boot/config-2.6.18-238.9.1.el5:CONFIG_HZ=1000
/boot/config-2.6.18-274.3.1.el5:# CONFIG_HZ_100 is not set
/boot/config-2.6.18-274.3.1.el5:# CONFIG_HZ_250 is not set
/boot/config-2.6.18-274.3.1.el5:CONFIG_HZ_1000=y
/boot/config-2.6.18-274.3.1.el5:CONFIG_HZ=1000
/boot/config-2.6.18-274.7.1.el5:# CONFIG_HZ_100 is not set
/boot/config-2.6.18-274.7.1.el5:# CONFIG_HZ_250 is not set
/boot/config-2.6.18-274.7.1.el5:CONFIG_HZ_1000=y
/boot/config-2.6.18-274.7.1.el5:CONFIG_HZ=1000

Thoughts as to why AWS results are so different from XenServer?  Other than
not knowing who else is on the AWS box?

Thanks,
  Mike

On Wed, Nov 16, 2011 at 11:56 AM, Michael Picher <mpicher at gmail.com> wrote:

m1.large

I have a c1.xlarge queued up and ready to test...

On Wed, Nov 16, 2011 at 11:52 AM, Chris Chen <chris.chen2004 at gmail.com>
wrote:

Just a simple question, what kind of AWS instance are you running your
FreeSWITCH?

It makes huge difference.

Thanks,

Chris

On Wed, Nov 16, 2011 at 10:43 AM, Michael Picher <mpicher at gmail.com> wrote:

Hi guys,

Trying to get to the bottom of some conference bridge issues I'm having with
running the system in AWS.

We're hearing a bunch of snap-crackle-pops in conference bridges and when I
tcpdum on the server itself I see them in the RTP and see RTP timestamp
problems.

I've run the following:

timer_test

freeswitch at 127.0.0.1@internal> timer_test 120 10

Avg: 120.004ms Total Time: 1200.315ms

2011-11-16 10:25:56.121131 [CONSOLE] mod_commands.c:461 Timer Test:
samplecount after init: 1
freeswitch at 127.0.0.1@internal> 2011-11-16 10:25:56.219163 [CONSOLE]
mod_commands.c:466 Timer Test: samplecount after first step: 2
2011-11-16 10:25:56.339195 [CONSOLE] mod_commands.c:475 Timer Test: 1 sleep
120 120006
2011-11-16 10:25:56.459259 [CONSOLE] mod_commands.c:475 Timer Test: 2 sleep
120 120040
2011-11-16 10:25:56.579259 [CONSOLE] mod_commands.c:475 Timer Test: 3 sleep
120 119976
2011-11-16 10:25:56.699291 [CONSOLE] mod_commands.c:475 Timer Test: 4 sleep
120 120007
2011-11-16 10:25:56.819318 [CONSOLE] mod_commands.c:475 Timer Test: 5 sleep
120 120001
2011-11-16 10:25:56.939375 [CONSOLE] mod_commands.c:475 Timer Test: 6 sleep
120 120031
2011-11-16 10:25:57.059397 [CONSOLE] mod_commands.c:475 Timer Test: 7 sleep
120 119996
2011-11-16 10:25:57.179422 [CONSOLE] mod_commands.c:475 Timer Test: 8 sleep
120 119994
2011-11-16 10:25:57.299461 [CONSOLE] mod_commands.c:475 Timer Test: 9 sleep
120 120005
2011-11-16 10:25:57.419478 [CONSOLE] mod_commands.c:475 Timer Test: 10 sleep
120 119991

test_time

freeswitch at 127.0.0.1@internal> time_test 600 10

test 1 sleep 600 1592
test 2 sleep 600 986
test 3 sleep 600 1018
test 4 sleep 600 980
test 5 sleep 600 1005
test 6 sleep 600 1000
test 7 sleep 600 972
test 8 sleep 600 990
test 9 sleep 600 1006
test 10 sleep 600 994
avg 1054

For kernel:

[root at openuc bin]# uname -r

2.6.21.7-2.fc8xen

CONFIG_HZ:

[root at openuc bin]# grep CONFIG_HZ /boot/config-*

/boot/config-2.6.16.33-xenU-x86_64:CONFIG_HZ_100=y
/boot/config-2.6.16.33-xenU-x86_64:# CONFIG_HZ_250 is not set
/boot/config-2.6.16.33-xenU-x86_64:# CONFIG_HZ_1000 is not set
/boot/config-2.6.16.33-xenU-x86_64:CONFIG_HZ=100
/boot/config-2.6.18-164.15.1.el5.centos.plus:# CONFIG_HZ_100 is not set
/boot/config-2.6.18-164.15.1.el5.centos.plus:# CONFIG_HZ_250 is not set
/boot/config-2.6.18-164.15.1.el5.centos.plus:CONFIG_HZ_1000=y
/boot/config-2.6.18-164.15.1.el5.centos.plus:CONFIG_HZ=1000
/boot/config-2.6.21-2952.fc8xen:# CONFIG_HZ_100 is not set
/boot/config-2.6.21-2952.fc8xen:# CONFIG_HZ_250 is not set
/boot/config-2.6.21-2952.fc8xen:# CONFIG_HZ_300 is not set
/boot/config-2.6.21-2952.fc8xen:CONFIG_HZ_1000=y
/boot/config-2.6.21-2952.fc8xen:CONFIG_HZ=1000

are the xenU kernel settings screwing me here?

Thanks,
  Mike

-- 
There are 10 kinds of people in this world, those who understand binary and
those who don't.

mpicher at gmail.com
blog: http://www.sipxecs.info
call: sip:mpicher at sipxecs.info <mailto:sip%3Ampicher at sipxecs.info> 

_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting at freeswitch.org
http://www.freeswitchsolutions.com

Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org

_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting at freeswitch.org
http://www.freeswitchsolutions.com

Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com

FreeSWITCH-users mailing list
FreeSWITCH-users at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
http://www.freeswitch.org

-- 
There are 10 kinds of people in this world, those who understand binary and
those who don't.

mpicher at gmail.com
blog: http://www.sipxecs.info
call: sip:mpicher at sipxecs.info <mailto:sip%3Ampicher at sipxecs.info> 

-- 
There are 10 kinds of people in this world, those who understand binary and
those who don't.

mpicher at gmail.com
blog: http://www.sipxecs.info
call: sip:mpicher at sipxecs.info <mailto:sip%3Ampicher at sipxecs.info> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20111117/426aefed/attachment-0001.html 