[Freeswitch-users] Strange Performance when using as SBC
rod
kawarod at laposte.net
Mon Feb 2 02:00:12 PST 2009
Hi Ken, Jay,
thanks for pointing to proxy media, I will test.
Ken, you are right, I was brain damaged (a stupid mistake) when setting
INFO cause this kind of level could be very verbose. I'm switching to
CRIT or ERR.
Thanks guys,
rod.
thanks for
Ken Rice wrote:
> If you don't have to transcode, using proxy media mode will still save you
> some CPU time. This is 1/2 way between bypass media and the default media
> interactive mode. The other draw back to this mode is if you are using FS to
> clean up RTP and DTMF you loose those functions but they are not needed in
> most use cases.
>
> As far as the log level goes, I found that once I had things stable setting
> the loglevel to helped a good deal... Info is probably a bit too high of a
> loglevel I would probably go for CRIT or ERR (2 or 1 respectively) if you
> insist on leaving logging turned on... On a busy system these can and will
> generate a good deal of activity (and disk IO if using mod_logfile)
>
> Ken
>
>
>
>> From: rod <kawarod at laposte.net>
>> Reply-To: <freeswitch-users at lists.freeswitch.org>
>> Date: Mon, 02 Feb 2009 11:36:35 +0400
>> To: <freeswitch-users at lists.freeswitch.org>
>> Subject: Re: [Freeswitch-users] Strange Performance when using as SBC
>>
>> Hi Ken,
>>
>> 1) I'd like to use FS to hide topology, so bypass media is not possible
>> 2) done
>> 3) done
>> 4) not used
>> 5) i'm using this ins switch.xml -> <param name="loglevel"
>> value="info"/>, if you think an other log level is more suitable.
>>
>> Regarding logging, I can see in console and in the freeswitch.log that
>> there is still a lot of NOTICE logging, see below:
>> 2009-02-02 08:33:56 [NOTICE] switch_core_session.c:960
>> switch_core_session_thread() Session 8721
>> (sofia/internal/sipp at 10.10.10.1:5060) Ended
>> 2009-02-02 08:33:56 [NOTICE] switch_core_session.c:962
>> switch_core_session_thread() Close Channel
>> sofia/internal/sipp at 10.10.10.1:5060 [CS_HANGUP]
>> 2009-02-02 08:33:56 [NOTICE] switch_core_session.c:960
>> switch_core_session_thread() Session 8722
>> (sofia/external/9998 at 10.10.20.100) Ended
>> 2009-02-02 08:33:56 [NOTICE] switch_core_session.c:962
>> switch_core_session_thread() Close Channel
>> sofia/external/9998 at 10.10.20.100 [CS_HANGUP]
>> 2009-02-02 08:33:56 [NOTICE] sofia.c:3164 sofia_handle_sip_i_state()
>> Channel [sofia/external/9998 at 10.10.20.100] has been answered
>> 2009-02-02 08:33:56 [WARNING] mod_sofia.c:740 sofia_read_frame()
>> Changing codec ptime to 30. I bet you have a linksys/sipura =D
>>
>> Do you have any idea where I can switch off this kind of logging. I
>> thought it should be in /dialplan/internal.xml, but I see that in
>> internal.xml -> <param name="debug" value="0"/>
>>
>> thanks a lot for your suggestion.
>>
>> regards,
>> rod
>>
>> Ken Rice wrote:
>>
>>> Dont forget there are several things you can do to increase performance...
>>>
>>> 1) where possible use bypass media or media proxy modes
>>> 2) mount freeswitch/db as a ram drive (if you are using voicemail with
>>> the internal FS DBs you'll need a way to make this persistant across
>>> reboots)
>>> 3) see the wiki for setting reasonable ulimits
>>> 4) (this is my oppinion others may vary) dont use mod_cdr_csv
>>> 5) turn off (or reduce logging) in switch.conf.xml
>>>
>>> all of these thing can greatly improve performance.
>>>
>>> On Mon, Feb 2, 2009 at 1:04 AM, rod <kawarod at laposte.net
>>> <mailto:kawarod at laposte.net>> wrote:
>>>
>>> Thanks Anthony,
>>>
>>> the setup is like this:
>>>
>>> sipp server ---- FS 1 ---- FS2
>>>
>>> FS1 is the AMD CPU that has only one extension in dialplan that
>>> bridges
>>> 9999 to FS2. 9999 is the first extension in FS2 dialplan that
>>> plays moh,
>>> FS2 has no CPU pbm.
>>>
>>> FS1 is maxing out at 60 bridged calls without your option -hp.
>>>
>>> Using -hp, I'm now able to bridge 200 concurrent calls (a great
>>> improvement) and the system is still reactive. CPU load is high
>>> but not
>>> 100% and as the system responds well, I think that doesn't matter. The
>>> 2GB of memory are completely consumed (top command shows 700MB for FS
>>> process).
>>>
>>> I understand that FS1 server is not the best hardware platform,
>>> and I'm
>>> waiting for new 4 cores server for testing.
>>> I will update those numbers when testing with the new hardware.
>>>
>>> regards,
>>> rod.
>>>
>>> Anthony Minessale wrote:
>>>
>>>> Which of the 2 machines has the load issue? You said it was one box
>>>> calling the other.
>>>>
>>>> You have 2 major things against you, single CPU and AMD, but you
>>>> should at least be able to get in the vicinity of 800-1000 calls
>>>>
>>> on a
>>>
>>>> box like that.
>>>>
>>>> Are you calling the default 9999? It's not really an appropriate
>>>> extension for load testing.
>>>> On the terminating box you should set up a manual extension that is
>>>> the first one in the dial plan
>>>> to play a wav file from preferably a ram disk or /tmp
>>>>
>>>> If you do plan on using this in production accept nothing less
>>>>
>>> than a
>>>
>>>> multi-core intel machine with at least 4 cores, the more cores the
>>>> better because that parallel processing is where FS gets it's
>>>>
>>> atvantage.
>>>
>>>>
>>>> On Fri, Jan 30, 2009 at 5:56 AM, rod <kawarod at laposte.net
>>>>
>>> <mailto:kawarod at laposte.net>
>>>
>>>> <mailto:kawarod at laposte.net <mailto:kawarod at laposte.net>>> wrote:
>>>>
>>>> Dear list,
>>>>
>>>> I've been playing with freeswitch for some time (2 months)
>>>>
>>> and the
>>>
>>>> fact
>>>> is that I'm very pleased with the functionnalities of this
>>>>
>>> software.
>>>
>>>> I'd like to use FS as a SBC handling media and I'm doing some
>>>> tests with
>>>> sipp to load the machine but I'm unable to bridge more than
>>>>
>>> 60 calls
>>>
>>>> without seeing the CPU being loaded at 100%. I'm sure
>>>>
>>> something is
>>>
>>>> going
>>>> wrong with my setup but I'm unable to see what.
>>>>
>>>> The test machine has the following specs:
>>>> Athlon XP 3500+ with 2GB of memory (I know this is not a
>>>>
>>> high end
>>>
>>>> machine :p)
>>>>
>>>> Freeswitch:/opt/freeswitch/log# cat /proc/cpuinfo
>>>> processor : 0
>>>> vendor_id : AuthenticAMD
>>>> cpu family : 15
>>>> model : 95
>>>> model name : AMD Athlon(tm) 64 Processor 3500+
>>>> stepping : 2
>>>> cpu MHz : 2199.973
>>>> cache size : 512 KB
>>>> fpu : yes
>>>> fpu_exception : yes
>>>> cpuid level : 1
>>>> wp : yes
>>>> flags : fpu vme de pse tsc msr pae mce cx8 apic
>>>>
>>> sep mtrr pge
>>>
>>>> mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext
>>>> fxsr_opt
>>>> rdtscp lm 3dnowext 3dnow up rep_good pni cx16 lahf_lm svm
>>>>
>>> extapic
>>>
>>>> cr8_legacy
>>>> bogomips : 4402.97
>>>> TLB size : 1024 4K pages
>>>> clflush size : 64
>>>> cache_alignment : 64
>>>> address sizes : 40 bits physical, 48 bits virtual
>>>> power management: ts fid vid ttp tm stc
>>>>
>>>> I installed FS on a fresh debian 64:
>>>> Linux Freeswitch 2.6.26-1-amd64 #1 SMP Sat Jan 10 17:57:00
>>>>
>>> UTC 2009
>>>
>>>> x86_64 GNU/Linux
>>>>
>>>> I set the ulimit parameters like those on the website:
>>>> freeswitch at internal> ...
>>>> Freeswitch:/opt/free-svn/bin# ulimit -a
>>>> core file size (blocks, -c) unlimited
>>>> data seg size (kbytes, -d) unlimited
>>>> scheduling priority (-e) 0
>>>> file size (blocks, -f) unlimited
>>>> pending signals (-i) unlimited
>>>> max locked memory (kbytes, -l) unlimited
>>>> max memory size (kbytes, -m) unlimited
>>>> open files (-n) 999999
>>>> pipe size (512 bytes, -p) 8
>>>> POSIX message queues (bytes, -q) unlimited
>>>> real-time priority (-r) 0
>>>> stack size (kbytes, -s) 244
>>>> cpu time (seconds, -t) unlimited
>>>> max user processes (-u) unlimited
>>>> virtual memory (kbytes, -v) unlimited
>>>> file locks (-x) unlimited
>>>>
>>>>
>>>> My network setup is the following:
>>>>
>>>> SIPP machine (10.10.10.1/24)----------------vlan
>>>>
>>> <http://10.10.10.1/24%29----------------vlan>
>>>
>>>> <http://10.10.10.1/24%29----------------vlan> 55
>>>> ----------(10.10.10.254/24 <http://10.10.10.254/24>
>>>>
>>> <http://10.10.10.254/24>) FS
>>>
>>>> (10.10.20.254/24)--------------
>>>>
>>> <http://10.10.20.254/24%29-------------->
>>>
>>>> <http://10.10.20.254/24%29--------------> vlan56
>>>> -------------------(10.10.20.100/24 <http://10.10.20.100/24>
>>>>
>>> <http://10.10.20.100/24>)
>>>
>>>> OTHER STOCK FS
>>>>
>>>>
>>>> I launched sipp with:
>>>> sipp -sn uac_pcap -s 9999 -r 10 -l 80 -d 60000 -mi
>>>>
>>> 10.10.10.1 -i
>>>
>>>> 10.10.10.1 -mp 25000 10.10.10.254:5060
>>>>
>>> <http://10.10.10.254:5060> <http://10.10.10.254:5060>
>>>
>>>> The dialplan on FS is very simple:
>>>> <?xml version="1.0" encoding="utf-8"?>
>>>> <!-- http://wiki.freeswitch.org/wiki/Dialplan_XML -->
>>>> <include>
>>>> <context name="default">
>>>>
>>>> <extension name="hold_music_bridge">
>>>> <condition field="destination_number" expression="^9999$">
>>>> <action application="answer"/>
>>>> <action application="bridge"
>>>> data="sofia/external/9999 at 10.10.20.100
>>>>
>>> <mailto:9999 at 10.10.20.100> <mailto:9999 at 10.10.20.100
>>> <mailto:9999 at 10.10.20.100>>"/>
>>>
>>>> </condition>
>>>> </extension>
>>>> </context>
>>>>
>>>> </include>
>>>>
>>>> FreeSWITCH Version 1.0.trunk (11560M) Started.
>>>> Crash Protection [Disabled]
>>>> Max Sessions[1000]
>>>> Session Rate[100]
>>>> SQL [Enabled]
>>>>
>>>>
>>>> The test is very simple: sipp dial 9999 that matches in my
>>>>
>>> FS dialplan
>>>
>>>> and this is bridged to an other FS machine playing music on
>>>>
>>> hold.
>>>
>>>> When I launch "top" I see after 30 to 40 s that FS consumes all
>>>> the CPU
>>>> ressources (with a mean of 50-60 % before), with 80 calls.
>>>> When I set 70 calls, I have to wait 70-80 s before seeing
>>>>
>>> the same
>>>
>>>> issue.
>>>>
>>>> Presence is set to false on the 2 profile.
>>>>
>>>> I have the same issue with FS 1.0.2 that' s why I tried FS
>>>>
>>> 11560.
>>>
>>>> When I use the FS machine as a router to test the packet per
>>>>
>>> second
>>>
>>>> performance, I'm reaching 100Mbps with 8000pps in each
>>>>
>>> direction (from
>>>
>>>> vlan 55 to vlan56) with less than 12% CPU. So that I don't think
>>>> there's
>>>> an issue with the network.
>>>>
>>>> Here is an "mpstat -P ALL 1" to show you what's happening
>>>>
>>> suddenly
>>>
>>>> with
>>>> 70 bridge calls:
>>>> 12:31:26 CPU %user %nice %sys %iowait %irq %soft
>>>> %steal %idle intr/s
>>>> 12:31:27 all 3,00 0,00 3,00 0,00 1,00 4,00
>>>> 0,00 89,00 6241,00
>>>> 12:31:27 0 3,00 0,00 3,00 0,00 1,00 4,00
>>>> 0,00 89,00 6241,00
>>>>
>>>> 12:31:27 CPU %user %nice %sys %iowait %irq %soft
>>>> %steal %idle intr/s
>>>> 12:31:28 all 14,14 0,00 56,57 0,00 2,02 5,05
>>>> 0,00 22,22 6035,35
>>>> 12:31:28 0 14,14 0,00 56,57 0,00 2,02 5,05
>>>> 0,00 22,22 6035,35
>>>>
>>>> 12:31:28 CPU %user %nice %sys %iowait %irq %soft
>>>> %steal %idle intr/s
>>>> 12:31:29 all 24,75 0,00 67,33 0,00 0,99 6,93
>>>> 0,00 0,00 5483,17
>>>> 12:31:29 0 24,75 0,00 67,33 0,00 0,99 6,93
>>>> 0,00 0,00 5483,17
>>>>
>>>>
>>>> The CPU is going from 89% idle to 0% in less than 2 seconds.
>>>>
>>>> I know that I don't have to expect too much from this kind of
>>>> hardware,
>>>> but it seems strange that the CPU power vanished so suddenly.
>>>>
>>>> Thanks a lot for the guys that have read this long mail :p
>>>>
>>>> kind regards,
>>>> rod
>>>>
>>>>
>>>> _______________________________________________
>>>> Freeswitch-users mailing list
>>>> Freeswitch-users at lists.freeswitch.org
>>>>
>>> <mailto:Freeswitch-users at lists.freeswitch.org>
>>>
>>>> <mailto:Freeswitch-users at lists.freeswitch.org
>>>>
>>> <mailto:Freeswitch-users at lists.freeswitch.org>>
>>>
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>
>>>>
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>
>>>> http://www.freeswitch.org
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Anthony Minessale II
>>>>
>>>> FreeSWITCH http://www.freeswitch.org/
>>>> ClueCon http://www.cluecon.com/
>>>>
>>>> AIM: anthm
>>>> MSN:anthony_minessale at hotmail.com
>>>>
>>> <mailto:MSN%3Aanthony_minessale at hotmail.com>
>>>
>>>> <mailto:MSN%3Aanthony_minessale at hotmail.com
>>>>
>>> <mailto:MSN%253Aanthony_minessale at hotmail.com>>
>>>
>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>
>>> <mailto:PAYPAL%3Aanthony.minessale at gmail.com>
>>>
>>>> <mailto:PAYPAL%3Aanthony.minessale at gmail.com
>>>>
>>> <mailto:PAYPAL%253Aanthony.minessale at gmail.com>>
>>>
>>>> IRC: irc.freenode.net <http://irc.freenode.net>
>>>>
>>> <http://irc.freenode.net> #freeswitch
>>>
>>>> FreeSWITCH Developer Conference
>>>> sip:888 at conference.freeswitch.org
>>>>
>>> <mailto:sip%3A888 at conference.freeswitch.org>
>>>
>>>> <mailto:sip%3A888 at conference.freeswitch.org
>>>>
>>> <mailto:sip%253A888 at conference.freeswitch.org>>
>>>
>>>> iax:guest at conference.freeswitch.org/888
>>>>
>>> <http://iax:guest@conference.freeswitch.org/888>
>>>
>>>> <http://iax:guest@conference.freeswitch.org/888>
>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>
>>> <mailto:googletalk%3Aconf%2B888 at conference.freeswitch.org>
>>>
>>>> <mailto:googletalk%3Aconf%2B888 at conference.freeswitch.org
>>>>
>>> <mailto:googletalk%253Aconf%252B888 at conference.freeswitch.org>>
>>>
>>>> pstn:213-799-1400
>>>>
>>>>
>>> ------------------------------------------------------------------------
>>>
>>>> _______________________________________________
>>>> Freeswitch-users mailing list
>>>> Freeswitch-users at lists.freeswitch.org
>>>>
>>> <mailto:Freeswitch-users at lists.freeswitch.org>
>>>
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>
>>>>
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>
>>>> http://www.freeswitch.org
>>>>
>>>>
>>> _______________________________________________
>>> Freeswitch-users mailing list
>>> Freeswitch-users at lists.freeswitch.org
>>> <mailto:Freeswitch-users at lists.freeswitch.org>
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Freeswitch-users mailing list
>>> Freeswitch-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>> _______________________________________________
>> Freeswitch-users mailing list
>> Freeswitch-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>
>
>
> _______________________________________________
> Freeswitch-users mailing list
> Freeswitch-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
>
More information about the Freeswitch-users
mailing list