[Freeswitch-users] Strange Performance when using as SBC
rod
kawarod at laposte.net
Mon Feb 2 04:36:44 PST 2009
Some update:
- I switched log level to "err"
- I'm now using proxy-media
- and I erased the directive answer in the dialplan (useless and
seems that it consumes lots of CPU, don't know why)
the dialplan now looks like this:
<extension name="hold_music_bridge">
<condition field="destination_number" expression="^9999$">
<action application="set" data="proxy_media=true"/>
<action application="bridge"
data="sofia/external/9999 at 10.10.20.100"/>
</condition>
</extension>
instead of
<extension name="hold_music_bridge">
<condition field="destination_number" expression="^9999$">
<action application="answer"/>
<action application="bridge"
data="sofia/external/9999 at 10.10.20.100"/>
</condition>
</extension>
The box is now able to bridge 300 calls with 20-30% of free CPU. I will
run a long term test.
I see this error in the log and don't understand exactly if somebody
could help (I'm running latest trunk 11592M):
2009-02-02 13:29:54 [ERR] switch_core_io.c:117
switch_core_session_read_frame() sofia/external/9998 at 10.10.20.100 has no
read codec.
regards,
rodrigue
rod wrote:
> Hi Ken, Jay,
>
> thanks for pointing to proxy media, I will test.
>
> Ken, you are right, I was brain damaged (a stupid mistake) when setting
> INFO cause this kind of level could be very verbose. I'm switching to
> CRIT or ERR.
>
> Thanks guys,
> rod.
>
> thanks for
>
> Ken Rice wrote:
>
>> If you don't have to transcode, using proxy media mode will still save you
>> some CPU time. This is 1/2 way between bypass media and the default media
>> interactive mode. The other draw back to this mode is if you are using FS to
>> clean up RTP and DTMF you loose those functions but they are not needed in
>> most use cases.
>>
>> As far as the log level goes, I found that once I had things stable setting
>> the loglevel to helped a good deal... Info is probably a bit too high of a
>> loglevel I would probably go for CRIT or ERR (2 or 1 respectively) if you
>> insist on leaving logging turned on... On a busy system these can and will
>> generate a good deal of activity (and disk IO if using mod_logfile)
>>
>> Ken
>>
>>
>>
>>
>>> From: rod <kawarod at laposte.net>
>>> Reply-To: <freeswitch-users at lists.freeswitch.org>
>>> Date: Mon, 02 Feb 2009 11:36:35 +0400
>>> To: <freeswitch-users at lists.freeswitch.org>
>>> Subject: Re: [Freeswitch-users] Strange Performance when using as SBC
>>>
>>> Hi Ken,
>>>
>>> 1) I'd like to use FS to hide topology, so bypass media is not possible
>>> 2) done
>>> 3) done
>>> 4) not used
>>> 5) i'm using this ins switch.xml -> <param name="loglevel"
>>> value="info"/>, if you think an other log level is more suitable.
>>>
>>> Regarding logging, I can see in console and in the freeswitch.log that
>>> there is still a lot of NOTICE logging, see below:
>>> 2009-02-02 08:33:56 [NOTICE] switch_core_session.c:960
>>> switch_core_session_thread() Session 8721
>>> (sofia/internal/sipp at 10.10.10.1:5060) Ended
>>> 2009-02-02 08:33:56 [NOTICE] switch_core_session.c:962
>>> switch_core_session_thread() Close Channel
>>> sofia/internal/sipp at 10.10.10.1:5060 [CS_HANGUP]
>>> 2009-02-02 08:33:56 [NOTICE] switch_core_session.c:960
>>> switch_core_session_thread() Session 8722
>>> (sofia/external/9998 at 10.10.20.100) Ended
>>> 2009-02-02 08:33:56 [NOTICE] switch_core_session.c:962
>>> switch_core_session_thread() Close Channel
>>> sofia/external/9998 at 10.10.20.100 [CS_HANGUP]
>>> 2009-02-02 08:33:56 [NOTICE] sofia.c:3164 sofia_handle_sip_i_state()
>>> Channel [sofia/external/9998 at 10.10.20.100] has been answered
>>> 2009-02-02 08:33:56 [WARNING] mod_sofia.c:740 sofia_read_frame()
>>> Changing codec ptime to 30. I bet you have a linksys/sipura =D
>>>
>>> Do you have any idea where I can switch off this kind of logging. I
>>> thought it should be in /dialplan/internal.xml, but I see that in
>>> internal.xml -> <param name="debug" value="0"/>
>>>
>>> thanks a lot for your suggestion.
>>>
>>> regards,
>>> rod
>>>
>>> Ken Rice wrote:
>>>
>>>
>>>> Dont forget there are several things you can do to increase performance...
>>>>
>>>> 1) where possible use bypass media or media proxy modes
>>>> 2) mount freeswitch/db as a ram drive (if you are using voicemail with
>>>> the internal FS DBs you'll need a way to make this persistant across
>>>> reboots)
>>>> 3) see the wiki for setting reasonable ulimits
>>>> 4) (this is my oppinion others may vary) dont use mod_cdr_csv
>>>> 5) turn off (or reduce logging) in switch.conf.xml
>>>>
>>>> all of these thing can greatly improve performance.
>>>>
>>>> On Mon, Feb 2, 2009 at 1:04 AM, rod <kawarod at laposte.net
>>>> <mailto:kawarod at laposte.net>> wrote:
>>>>
>>>> Thanks Anthony,
>>>>
>>>> the setup is like this:
>>>>
>>>> sipp server ---- FS 1 ---- FS2
>>>>
>>>> FS1 is the AMD CPU that has only one extension in dialplan that
>>>> bridges
>>>> 9999 to FS2. 9999 is the first extension in FS2 dialplan that
>>>> plays moh,
>>>> FS2 has no CPU pbm.
>>>>
>>>> FS1 is maxing out at 60 bridged calls without your option -hp.
>>>>
>>>> Using -hp, I'm now able to bridge 200 concurrent calls (a great
>>>> improvement) and the system is still reactive. CPU load is high
>>>> but not
>>>> 100% and as the system responds well, I think that doesn't matter. The
>>>> 2GB of memory are completely consumed (top command shows 700MB for FS
>>>> process).
>>>>
>>>> I understand that FS1 server is not the best hardware platform,
>>>> and I'm
>>>> waiting for new 4 cores server for testing.
>>>> I will update those numbers when testing with the new hardware.
>>>>
>>>> regards,
>>>> rod.
>>>>
>>>> Anthony Minessale wrote:
>>>>
>>>>
>>>>> Which of the 2 machines has the load issue? You said it was one box
>>>>> calling the other.
>>>>>
>>>>> You have 2 major things against you, single CPU and AMD, but you
>>>>> should at least be able to get in the vicinity of 800-1000 calls
>>>>>
>>>>>
>>>> on a
>>>>
>>>>
>>>>> box like that.
>>>>>
>>>>> Are you calling the default 9999? It's not really an appropriate
>>>>> extension for load testing.
>>>>> On the terminating box you should set up a manual extension that is
>>>>> the first one in the dial plan
>>>>> to play a wav file from preferably a ram disk or /tmp
>>>>>
>>>>> If you do plan on using this in production accept nothing less
>>>>>
>>>>>
>>>> than a
>>>>
>>>>
>>>>> multi-core intel machine with at least 4 cores, the more cores the
>>>>> better because that parallel processing is where FS gets it's
>>>>>
>>>>>
>>>> atvantage.
>>>>
>>>>
>>>>> On Fri, Jan 30, 2009 at 5:56 AM, rod <kawarod at laposte.net
>>>>>
>>>>>
>>>> <mailto:kawarod at laposte.net>
>>>>
>>>>
>>>>> <mailto:kawarod at laposte.net <mailto:kawarod at laposte.net>>> wrote:
>>>>>
>>>>> Dear list,
>>>>>
>>>>> I've been playing with freeswitch for some time (2 months)
>>>>>
>>>>>
>>>> and the
>>>>
>>>>
>>>>> fact
>>>>> is that I'm very pleased with the functionnalities of this
>>>>>
>>>>>
>>>> software.
>>>>
>>>>
>>>>> I'd like to use FS as a SBC handling media and I'm doing some
>>>>> tests with
>>>>> sipp to load the machine but I'm unable to bridge more than
>>>>>
>>>>>
>>>> 60 calls
>>>>
>>>>
>>>>> without seeing the CPU being loaded at 100%. I'm sure
>>>>>
>>>>>
>>>> something is
>>>>
>>>>
>>>>> going
>>>>> wrong with my setup but I'm unable to see what.
>>>>>
>>>>> The test machine has the following specs:
>>>>> Athlon XP 3500+ with 2GB of memory (I know this is not a
>>>>>
>>>>>
>>>> high end
>>>>
>>>>
>>>>> machine :p)
>>>>>
>>>>> Freeswitch:/opt/freeswitch/log# cat /proc/cpuinfo
>>>>> processor : 0
>>>>> vendor_id : AuthenticAMD
>>>>> cpu family : 15
>>>>> model : 95
>>>>> model name : AMD Athlon(tm) 64 Processor 3500+
>>>>> stepping : 2
>>>>> cpu MHz : 2199.973
>>>>> cache size : 512 KB
>>>>> fpu : yes
>>>>> fpu_exception : yes
>>>>> cpuid level : 1
>>>>> wp : yes
>>>>> flags : fpu vme de pse tsc msr pae mce cx8 apic
>>>>>
>>>>>
>>>> sep mtrr pge
>>>>
>>>>
>>>>> mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext
>>>>> fxsr_opt
>>>>> rdtscp lm 3dnowext 3dnow up rep_good pni cx16 lahf_lm svm
>>>>>
>>>>>
>>>> extapic
>>>>
>>>>
>>>>> cr8_legacy
>>>>> bogomips : 4402.97
>>>>> TLB size : 1024 4K pages
>>>>> clflush size : 64
>>>>> cache_alignment : 64
>>>>> address sizes : 40 bits physical, 48 bits virtual
>>>>> power management: ts fid vid ttp tm stc
>>>>>
>>>>> I installed FS on a fresh debian 64:
>>>>> Linux Freeswitch 2.6.26-1-amd64 #1 SMP Sat Jan 10 17:57:00
>>>>>
>>>>>
>>>> UTC 2009
>>>>
>>>>
>>>>> x86_64 GNU/Linux
>>>>>
>>>>> I set the ulimit parameters like those on the website:
>>>>> freeswitch at internal> ...
>>>>> Freeswitch:/opt/free-svn/bin# ulimit -a
>>>>> core file size (blocks, -c) unlimited
>>>>> data seg size (kbytes, -d) unlimited
>>>>> scheduling priority (-e) 0
>>>>> file size (blocks, -f) unlimited
>>>>> pending signals (-i) unlimited
>>>>> max locked memory (kbytes, -l) unlimited
>>>>> max memory size (kbytes, -m) unlimited
>>>>> open files (-n) 999999
>>>>> pipe size (512 bytes, -p) 8
>>>>> POSIX message queues (bytes, -q) unlimited
>>>>> real-time priority (-r) 0
>>>>> stack size (kbytes, -s) 244
>>>>> cpu time (seconds, -t) unlimited
>>>>> max user processes (-u) unlimited
>>>>> virtual memory (kbytes, -v) unlimited
>>>>> file locks (-x) unlimited
>>>>>
>>>>>
>>>>> My network setup is the following:
>>>>>
>>>>> SIPP machine (10.10.10.1/24)----------------vlan
>>>>>
>>>>>
>>>> <http://10.10.10.1/24%29----------------vlan>
>>>>
>>>>
>>>>> <http://10.10.10.1/24%29----------------vlan> 55
>>>>> ----------(10.10.10.254/24 <http://10.10.10.254/24>
>>>>>
>>>>>
>>>> <http://10.10.10.254/24>) FS
>>>>
>>>>
>>>>> (10.10.20.254/24)--------------
>>>>>
>>>>>
>>>> <http://10.10.20.254/24%29-------------->
>>>>
>>>>
>>>>> <http://10.10.20.254/24%29--------------> vlan56
>>>>> -------------------(10.10.20.100/24 <http://10.10.20.100/24>
>>>>>
>>>>>
>>>> <http://10.10.20.100/24>)
>>>>
>>>>
>>>>> OTHER STOCK FS
>>>>>
>>>>>
>>>>> I launched sipp with:
>>>>> sipp -sn uac_pcap -s 9999 -r 10 -l 80 -d 60000 -mi
>>>>>
>>>>>
>>>> 10.10.10.1 -i
>>>>
>>>>
>>>>> 10.10.10.1 -mp 25000 10.10.10.254:5060
>>>>>
>>>>>
>>>> <http://10.10.10.254:5060> <http://10.10.10.254:5060>
>>>>
>>>>
>>>>> The dialplan on FS is very simple:
>>>>> <?xml version="1.0" encoding="utf-8"?>
>>>>> <!-- http://wiki.freeswitch.org/wiki/Dialplan_XML -->
>>>>> <include>
>>>>> <context name="default">
>>>>>
>>>>> <extension name="hold_music_bridge">
>>>>> <condition field="destination_number" expression="^9999$">
>>>>> <action application="answer"/>
>>>>> <action application="bridge"
>>>>> data="sofia/external/9999 at 10.10.20.100
>>>>>
>>>>>
>>>> <mailto:9999 at 10.10.20.100> <mailto:9999 at 10.10.20.100
>>>> <mailto:9999 at 10.10.20.100>>"/>
>>>>
>>>>
>>>>> </condition>
>>>>> </extension>
>>>>> </context>
>>>>>
>>>>> </include>
>>>>>
>>>>> FreeSWITCH Version 1.0.trunk (11560M) Started.
>>>>> Crash Protection [Disabled]
>>>>> Max Sessions[1000]
>>>>> Session Rate[100]
>>>>> SQL [Enabled]
>>>>>
>>>>>
>>>>> The test is very simple: sipp dial 9999 that matches in my
>>>>>
>>>>>
>>>> FS dialplan
>>>>
>>>>
>>>>> and this is bridged to an other FS machine playing music on
>>>>>
>>>>>
>>>> hold.
>>>>
>>>>
>>>>> When I launch "top" I see after 30 to 40 s that FS consumes all
>>>>> the CPU
>>>>> ressources (with a mean of 50-60 % before), with 80 calls.
>>>>> When I set 70 calls, I have to wait 70-80 s before seeing
>>>>>
>>>>>
>>>> the same
>>>>
>>>>
>>>>> issue.
>>>>>
>>>>> Presence is set to false on the 2 profile.
>>>>>
>>>>> I have the same issue with FS 1.0.2 that' s why I tried FS
>>>>>
>>>>>
>>>> 11560.
>>>>
>>>>
>>>>> When I use the FS machine as a router to test the packet per
>>>>>
>>>>>
>>>> second
>>>>
>>>>
>>>>> performance, I'm reaching 100Mbps with 8000pps in each
>>>>>
>>>>>
>>>> direction (from
>>>>
>>>>
>>>>> vlan 55 to vlan56) with less than 12% CPU. So that I don't think
>>>>> there's
>>>>> an issue with the network.
>>>>>
>>>>> Here is an "mpstat -P ALL 1" to show you what's happening
>>>>>
>>>>>
>>>> suddenly
>>>>
>>>>
>>>>> with
>>>>> 70 bridge calls:
>>>>> 12:31:26 CPU %user %nice %sys %iowait %irq %soft
>>>>> %steal %idle intr/s
>>>>> 12:31:27 all 3,00 0,00 3,00 0,00 1,00 4,00
>>>>> 0,00 89,00 6241,00
>>>>> 12:31:27 0 3,00 0,00 3,00 0,00 1,00 4,00
>>>>> 0,00 89,00 6241,00
>>>>>
>>>>> 12:31:27 CPU %user %nice %sys %iowait %irq %soft
>>>>> %steal %idle intr/s
>>>>> 12:31:28 all 14,14 0,00 56,57 0,00 2,02 5,05
>>>>> 0,00 22,22 6035,35
>>>>> 12:31:28 0 14,14 0,00 56,57 0,00 2,02 5,05
>>>>> 0,00 22,22 6035,35
>>>>>
>>>>> 12:31:28 CPU %user %nice %sys %iowait %irq %soft
>>>>> %steal %idle intr/s
>>>>> 12:31:29 all 24,75 0,00 67,33 0,00 0,99 6,93
>>>>> 0,00 0,00 5483,17
>>>>> 12:31:29 0 24,75 0,00 67,33 0,00 0,99 6,93
>>>>> 0,00 0,00 5483,17
>>>>>
>>>>>
>>>>> The CPU is going from 89% idle to 0% in less than 2 seconds.
>>>>>
>>>>> I know that I don't have to expect too much from this kind of
>>>>> hardware,
>>>>> but it seems strange that the CPU power vanished so suddenly.
>>>>>
>>>>> Thanks a lot for the guys that have read this long mail :p
>>>>>
>>>>> kind regards,
>>>>> rod
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Freeswitch-users mailing list
>>>>> Freeswitch-users at lists.freeswitch.org
>>>>>
>>>>>
>>>> <mailto:Freeswitch-users at lists.freeswitch.org>
>>>>
>>>>
>>>>> <mailto:Freeswitch-users at lists.freeswitch.org
>>>>>
>>>>>
>>>> <mailto:Freeswitch-users at lists.freeswitch.org>>
>>>>
>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>>
>>>>>
>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>
>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Anthony Minessale II
>>>>>
>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>> ClueCon http://www.cluecon.com/
>>>>>
>>>>> AIM: anthm
>>>>> MSN:anthony_minessale at hotmail.com
>>>>>
>>>>>
>>>> <mailto:MSN%3Aanthony_minessale at hotmail.com>
>>>>
>>>>
>>>>> <mailto:MSN%3Aanthony_minessale at hotmail.com
>>>>>
>>>>>
>>>> <mailto:MSN%253Aanthony_minessale at hotmail.com>>
>>>>
>>>>
>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>
>>>>>
>>>> <mailto:PAYPAL%3Aanthony.minessale at gmail.com>
>>>>
>>>>
>>>>> <mailto:PAYPAL%3Aanthony.minessale at gmail.com
>>>>>
>>>>>
>>>> <mailto:PAYPAL%253Aanthony.minessale at gmail.com>>
>>>>
>>>>
>>>>> IRC: irc.freenode.net <http://irc.freenode.net>
>>>>>
>>>>>
>>>> <http://irc.freenode.net> #freeswitch
>>>>
>>>>
>>>>> FreeSWITCH Developer Conference
>>>>> sip:888 at conference.freeswitch.org
>>>>>
>>>>>
>>>> <mailto:sip%3A888 at conference.freeswitch.org>
>>>>
>>>>
>>>>> <mailto:sip%3A888 at conference.freeswitch.org
>>>>>
>>>>>
>>>> <mailto:sip%253A888 at conference.freeswitch.org>>
>>>>
>>>>
>>>>> iax:guest at conference.freeswitch.org/888
>>>>>
>>>>>
>>>> <http://iax:guest@conference.freeswitch.org/888>
>>>>
>>>>
>>>>> <http://iax:guest@conference.freeswitch.org/888>
>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>
>>>>>
>>>> <mailto:googletalk%3Aconf%2B888 at conference.freeswitch.org>
>>>>
>>>>
>>>>> <mailto:googletalk%3Aconf%2B888 at conference.freeswitch.org
>>>>>
>>>>>
>>>> <mailto:googletalk%253Aconf%252B888 at conference.freeswitch.org>>
>>>>
>>>>
>>>>> pstn:213-799-1400
>>>>>
>>>>>
>>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>>> _______________________________________________
>>>>> Freeswitch-users mailing list
>>>>> Freeswitch-users at lists.freeswitch.org
>>>>>
>>>>>
>>>> <mailto:Freeswitch-users at lists.freeswitch.org>
>>>>
>>>>
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>
>>>>>
>>>>>
>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>
>>>>
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Freeswitch-users mailing list
>>>> Freeswitch-users at lists.freeswitch.org
>>>> <mailto:Freeswitch-users at lists.freeswitch.org>
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Freeswitch-users mailing list
>>>> Freeswitch-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>>
>>>>
>>> _______________________________________________
>>> Freeswitch-users mailing list
>>> Freeswitch-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>> _______________________________________________
>> Freeswitch-users mailing list
>> Freeswitch-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>>
>>
>
> _______________________________________________
> Freeswitch-users mailing list
> Freeswitch-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
>
More information about the Freeswitch-users
mailing list