[Freeswitch-users] Apparent performance bottleneck and FS freeze

Mon Jul 22 23:40:43 MSD 2013

Did you turn of the presence in the sofia profile, that is enabled by
default and very costly.  comment out manage-presence

On Mon, Jul 22, 2013 at 2:32 PM, Victor Chukalovskiy <
victor.chukalovskiy at gmail.com> wrote:

>  Michael, Ken,
>
> Thank you for the response. Firstly, let me clarify setup for each of DBs
> in use:
>
> *core_db:* sqlite, /dev/shm/core.db (RAM) - using default 50 max handles
> *mod_sofia db**:* sqlite, /opt/fs/db (HDD)
> *mod_lcr db:* Postgres
> *mod_limit:* hash backend
>
> Note: I believe we rarely use mod_sofia DB's...this is because we don't
> use any registrations, acl only. When I list sofia per-profile db files on
> production box, most of them has not been edited for a couple days. Are my
> conclusions correct here, or do I miss something? Maybe they are accessed a
> lot but only for reading?
>
> I understand recommendations from Ken and from Michaels are along the
> lines of increasing core db and sofia db throughput. However here is what
> puzzles me:
> -the db used most extensively is core_db. It' already in the ram.
> -we don't have much load on sofia dbs (or am I mistaken in my logic?)
> -error log started with:
>
>
> [CRIT] switch_time.c:967 Virtual Migration Detected! Syncing Clock
> [WARNING] switch_scheduler.c:115 Task was executed late by 19 seconds 1
> heartbeat (core)
>
> Does it mean that there were abnormal events prior hitting 50 max handles
> limit?
>
> Thank you,
> -Victor
>
>
> On 13-07-22 02:52 PM, Anthony Minessale wrote:
>
> Check the odbcinst.ini settings from the wiki or use native pg mode.
> increase the max number of pg connections on your server and turn up FS to
> match in max-db-handles in switch.conf.xml
>
>  You are already beating it up pretty good esp for a fake server though.
>  2500 is a lot.
>
>
>
> On Mon, Jul 22, 2013 at 1:44 PM, Ken Rice <krice at freeswitch.org> wrote:
>
>> Mount the entire freeswitch/db dir as tmpfs... most of these errors are
>> for
>> the sofia DB getting backed up...
>>
>> The coredb is a completely different sqlite db from the sofia db
>>
>> On 7/22/13 12:54 PM, "Victor Chukalovskiy" <victor.chukalovskiy at gmail.com
>> >
>> wrote:
>>
>> > Hello,
>> >
>> > Need some help finding the bottleneck here. Running 1.2.stable on a
>> > decent multi-core vmware platform. No RTP, SIP signaling only. Around
>> > 2500 sessions and 30 average CPS load. Essentially, it's a multi-profile
>> > .xml config that uses mod_lcr extensively (postgres DB). Yesterday it
>> > stopped processing calls with the following log output:
>> >
>> > 2013-07-21 06:00:37.642678 [CRIT] switch_time.c:967 Virtual Migration
>> > Detected! Syncing Clock
>> > 2013-07-21 06:01:07.141734 [WARNING] switch_scheduler.c:115 Task was
>> > executed late by 19 seconds 1 heartbeat (core)
>> > 2013-07-21 06:02:32.402668 [CRIT] switch_time.c:967 Virtual Migration
>> > Detected! Syncing Clock
>> > 2013-07-21 06:04:42.499744 [WARNING] switch_scheduler.c:115 Task was
>> > executed late by 115 seconds 1 heartbeat (core)
>> > 2013-07-21 06:04:42.499744 [WARNING] switch_scheduler.c:115 Task was
>> > executed late by 74 seconds 2 check_ip (core)
>> > 2013-07-21 06:05:15.102683 [CRIT] switch_time.c:967 Virtual Migration
>> > Detected! Syncing Clock
>> > 2013-07-21 06:05:44.961004 [WARNING] switch_scheduler.c:115 Task was
>> > executed late by 22 seconds 1 heartbeat (core)
>> > 2013-07-21 06:05:44.961004 [WARNING] switch_scheduler.c:115 Task was
>> > executed late by 2 seconds 2 check_ip (core)
>> > 2013-07-21 06:05:49.322672 [WARNING] mod_lcr.c:448 Max handles 50
>> > exceeded, blocking....
>> > 2013-07-21 06:05:50.962987 [WARNING] sofia_glue.c:6605 Max handles 50
>> > exceeded, blocking....
>> > 2013-07-21 06:05:50.962987 [WARNING] sofia_glue.c:6605 Max handles 50
>> > exceeded, blocking....
>> > 2013-07-21 06:05:50.962987 [WARNING] sofia_glue.c:6605 Max handles 50
>> > exceeded, blocking....
>> > ................................
>> > 2013-07-21 06:05:53.962672 [WARNING] sofia_glue.c:6605 Max handles 50
>> > exceeded, blocking....
>> > 2013-07-21 06:05:54.422671 [ERR] mod_lcr.c:448 Error connecting
>> > 2013-07-21 06:05:54.422671 [WARNING] mod_lcr.c:1779 LCR lookup failed
>> > for 14389955082
>> > 2013-07-21 06:05:54.962671 [WARNING] sofia_glue.c:6605 Max handles 50
>> > exceeded, blocking....
>> > 2013-07-21 06:05:54.962671 [WARNING] sofia_glue.c:6605 Max handles 50
>> > exceeded, blocking....
>> > ................................
>> > 2013-07-21 06:05:55.962758 [WARNING] sofia_glue.c:6605 Max handles 50
>> > exceeded, blocking....
>> > 2013-07-21 06:05:56.062671 [ERR] sofia_glue.c:6605 Error connecting
>> > 2013-07-21 06:05:56.062671 [ERR] sofia_glue.c:6606 Error Opening DB
>> > 2013-07-21 06:05:56.062671 [WARNING] sofia_glue.c:6605 Max handles 50
>> > exceeded, blocking....
>> > 2013-07-21 06:05:56.062671 [ERR] sofia_glue.c:6605 Error connecting
>> > 2013-07-21 06:05:56.062671 [ERR] sofia_glue.c:6606 Error Opening DB
>> >
>> > FreeSWITCH stayed unusable since then until it was restarted.
>> >
>> > FYI core DB is already in the RAM:
>> > <param name="core-db-name" value="/dev/shm/core.db" />
>> >
>> >
>> > Thank you,
>> > Victor
>> >
>> >
>> _________________________________________________________________________
>> > Professional FreeSWITCH Consulting Services:
>> > consulting at freeswitch.org
>> > http://www.freeswitchsolutions.com
>> >
>> > 
>> > 
>> >
>> > Official FreeSWITCH Sites
>> > http://www.freeswitch.org
>> > http://wiki.freeswitch.org
>> > http://www.cluecon.com
>> >
>> > FreeSWITCH-users mailing list
>> > FreeSWITCH-users at lists.freeswitch.org
>> > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> > UNSUBSCRIBE:
>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>> > http://www.freeswitch.org
>>
>>  --
>> Ken
>> http://www.FreeSWITCH.org
>> http://www.ClueCon.com
>> http://www.OSTAG.org
>> irc.freenode.net #freeswitch
>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>
>
>
>  --
> Anthony Minessale II
>
> FreeSWITCH http://www.freeswitch.org/
> ClueCon http://www.cluecon.com/
> Twitter: http://twitter.com/FreeSWITCH_wire
>
> AIM: anthm
> MSN:anthony_minessale at hotmail.com
> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
> IRC: irc.freenode.net #freeswitch
>
> FreeSWITCH Developer Conference
> sip:888 at conference.freeswitch.org
> googletalk:conf+888 at conference.freeswitch.org
> pstn:+19193869900
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:consulting at freeswitch.orghttp://www.freeswitchsolutions.com
>
> FreeSWITCH-powered IP PBX: The CudaTel Communication Server
>
> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
>
> FreeSWITCH-users mailing listFreeSWITCH-users at lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
>
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>

-- 
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org
googletalk:conf+888 at conference.freeswitch.org
pstn:+19193869900
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20130722/a742a0ad/attachment.html 