[Freeswitch-users] XML_CURL max connection limit

Cal Leeming [Simplicity Media Ltd] cal.leeming at simplicitymedialtd.co.uk
Wed Jun 5 22:38:28 MSD 2013 

Oh, sorry one last thing, if your bottleneck is DB, you can be super ghetto
and scale up now and worry about scaling out later.

Scaling up basically means throwing as much capacity as possible on a
single box, i.e. 800GB RAM, RAID with SSDs, dual quad core with HT etc etc.

Throw as much memory into the chassis as possible so that the entire innodb
buffer pool sits in memory, combined with SSDs, you should be able to get
away with that.

However as said before, YMMV, because you will be likely encounter other
problems along the way - and this is really only a quick fix, you'd need to
immediately work on a long term fix.

And also - nothing beats doing things properly.. all the above are options
to give you an immediate fix to keep production running.. you *must* still
take time to properly re-design the architecture to support high throughput.

Hope this helps

Cal

On Wed, Jun 5, 2013 at 5:40 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leeming at simplicitymedialtd.co.uk> wrote:

> Also, make sure you don't mix long and short running queries, else you
> will thrash the innodb thread queue.
>
> If you want complete HA, then go for 2x master<>master, and 4x slave (in
> HA pairs).
>
> If you need a ghetto solution, go for 1 master, 1 slave for short queries,
> 1 slave for long.
>
> If you need a really ghetto solution, throw some SSDs on the master and
> tweak the innodb conf, you *might* get away with it but YMMV.
>
> Cal
>
>
> On Wed, Jun 5, 2013 at 4:45 PM, Muhammad Shahzad <shaheryarkh at gmail.com>wrote:
>
>> Thanks Cal, let me discuss it with my team and see what can be
>> implemented quickly, since we are running short on project deadline.
>>
>> Thank you.
>>
>>
>>
>> On Wed, Jun 5, 2013 at 5:29 PM, Cal Leeming [Simplicity Media Ltd] <
>> cal.leeming at simplicitymedialtd.co.uk> wrote:
>>
>>> Hello,
>>>
>>> This isn't really on topic within FreeSWITCH and is more related to your
>>> web application scalability.
>>>
>>> However, semi high traffic web application platforms is one area I do
>>> have a decent amount of experience in, so let me offer some thoughts (I
>>> assume you are running PHP based on your log output).
>>>
>>> I would advise looking into the following;
>>>
>>> * PHP FPM with NGINX (you'll get better throughput with this than
>>> mod_php and apache - look on google for benchmarks)
>>> * APC or xcache opcode acceleration (out of the box, this should speed
>>> up execution)
>>> * Assuming you have a DB, implement a caching library to reduce DB calls
>>> (look at memcache, redis if you need atomicity or scaling, or even just
>>> xcache)
>>> * Implement haproxy/zxtm and balance across multiple web servers
>>> (assuming your code isn't vuln to race, and you are able to switch to a
>>> db/cache based session backend, rather than disk)
>>> * Get new relic on your boxes, identify the bottlenecks, and performance
>>> tweak where necessary
>>> * If you stay on apache, don't just increase maxclient's... you need to
>>> calculate how much resources the box needs to handle that many concurrent
>>> requests, and identify the max memory space needed for each request, cpu
>>> time etc.
>>> * Assuming you use MySQL, switch over to Percona, use InnoDB, and
>>> separate out read/writes from the same server (you can do this in
>>> master<>master or master<>slave, or any combination you like). This will
>>> reduce the innodb thread contention and give you much better throughput
>>>
>>> Hope this helps
>>>
>>> Cal
>>>
>>> On Wed, Jun 5, 2013 at 3:47 PM, Muhammad Shahzad <shaheryarkh at gmail.com>wrote:
>>>
>>>> I have already set MaxClients to 2048, but that limit is never reached.
>>>>
>>>> Thank you.
>>>>
>>>>
>>>> On Wed, Jun 5, 2013 at 4:17 PM, Samir Doshi <smrdoshi at gmail.com> wrote:
>>>>
>>>>>  On 06/05/2013 07:36 PM, Michael Jerris wrote:
>>>>>
>>>>> Is the request going to your server and getting no response?
>>>>>
>>>>>  On Jun 5, 2013, at 9:50 AM, Muhammad Shahzad <shaheryarkh at gmail.com>
>>>>> wrote:
>>>>>
>>>>>  Hi,
>>>>>
>>>>>  Is there any connection limit perhaps hard coded anywhere in
>>>>> xml_curl module? I am load testing FS with xml_curl to provide config,
>>>>> directory and dial-plan. I observed i start getting HTTP ERROR 0 when i
>>>>> have around 1600 concurrent calls on machine with media. Although i see no
>>>>> sign of system overload either on FS (load average < 1.8) or Apache (load
>>>>> average < 4,75). Both have plenty of RAM and bandwidth free.
>>>>>
>>>>>  Is there anyway to get rid of this HTTP ERROR 0? It must be
>>>>> generated by FS internally, since i don't see anything like this error in
>>>>> Apache logs.
>>>>>
>>>>>  Below 1,638 concurrent calls @ 50 CPS, i get success rate of 99.999%
>>>>> but as soon as it crosses this mark, success rate drastically drops to 60%
>>>>> and even below (when concurrent calls go beyond 2,000). Here is error
>>>>> console log.
>>>>>
>>>>>  ###################################################
>>>>>  2013-06-05 18:09:13.591229 [ERR] mod_xml_curl.c:310 Received HTTP
>>>>> error 0 trying to fetch http://172.16.171.61/vm-users.php
>>>>> data:
>>>>> [hostname=vms01&section=directory&tag_name=domain&key_name=name&key_value=10.1.3.2&Event-Name=GENERAL&Core-UUID=84c917dc-cdde-11e2-9bb2-8b779d6a4e97&FreeSWITCH-Hostname=vms01&FreeSWITCH-Switchname=vms01&FreeSWITCH-IPv4=10.1.3.2&FreeSWITCH-IPv6=%3A%3A1&Event-Date-Local=2013-06-05%2018%3A09%3A03&Event-Date-GMT=Wed,%2005%20Jun%202013%2013%3A09%3A03%20GMT&Event-Date-Timestamp=1370437743591210&Event-Calling-File=mod_voicemail.c&Event-Calling-Function=resolve_id&Event-Calling-Line-Number=1350&Event-Sequence=1776930&action=message-count&key=id&user=989000049598&domain=10.1.3.2]
>>>>>  ###################################################
>>>>>
>>>>>  Although this setup is more then sufficient for my requirement but i
>>>>> just want to know if i can extend it beyond, since i still have lot of
>>>>> resources free.
>>>>>
>>>>>  OS: Debian 6.0.7 x64
>>>>> Apache: 2.2.16
>>>>> FreeSWITCH: 1.5.1b+git~20130424T005233Z~66d1755b06
>>>>>
>>>>>  Both FS and Apache are running on separate dedicated machines on
>>>>> same LAN.
>>>>>
>>>>>  Thank you.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:consulting at freeswitch.orghttp://www.freeswitchsolutions.com
>>>>>
>>>>> FreeSWITCH-powered IP PBX: The CudaTel Communication Server
>>>>>
>>>>> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing listFreeSWITCH-users at lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
>>>>>
>>>>>
>>>>> I think you need to do something with Apache configuration. Maybe
>>>>> increase apache child instance/Threads will give you better performance.
>>>>>
>>>>> --
>>>>> Thanks,
>>>>> Sam
>>>>>
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> 
>>>>> 
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://wiki.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Mit freundlichen Grüßen
>>>> Muhammad Shahzad
>>>> -----------------------------------
>>>> CISCO Rich Media Communication Specialist (CRMCS)
>>>> CISCO Certified Network Associate (CCNA)
>>>> Cell: +49 176 99 83 10 85
>>>> MSN: shari_786pk at hotmail.com
>>>> Email: shaheryarkh at googlemail.com
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> 
>>>> 
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://wiki.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> 
>>> 
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>>
>>
>> --
>> Mit freundlichen Grüßen
>> Muhammad Shahzad
>> -----------------------------------
>> CISCO Rich Media Communication Specialist (CRMCS)
>> CISCO Certified Network Associate (CCNA)
>> Cell: +49 176 99 83 10 85
>> MSN: shari_786pk at hotmail.com
>> Email: shaheryarkh at googlemail.com
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20130605/04142bb5/attachment-0001.html

Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users mailing list