Did you turn of the presence in the sofia profile, that is enabled by default and very costly. comment out manage-presence On Mon, Jul 22, 2013 at 2:32 PM, Victor Chukalovskiy < victor.chukalovskiy@gmail.com> wrote:
Michael, Ken,
Thank you for the response. Firstly, let me clarify setup for each of DBs in use:
*core_db:* sqlite, /dev/shm/core.db (RAM) - using default 50 max handles *mod_sofia db**:* sqlite, /opt/fs/db (HDD) *mod_lcr db:* Postgres *mod_limit:* hash backend
Note: I believe we rarely use mod_sofia DB's...this is because we don't use any registrations, acl only. When I list sofia per-profile db files on production box, most of them has not been edited for a couple days. Are my conclusions correct here, or do I miss something? Maybe they are accessed a lot but only for reading?
I understand recommendations from Ken and from Michaels are along the lines of increasing core db and sofia db throughput. However here is what puzzles me: -the db used most extensively is core_db. It' already in the ram. -we don't have much load on sofia dbs (or am I mistaken in my logic?) -error log started with:
[CRIT] switch_time.c:967 Virtual Migration Detected! Syncing Clock [WARNING] switch_scheduler.c:115 Task was executed late by 19 seconds 1 heartbeat (core)
Does it mean that there were abnormal events prior hitting 50 max handles limit?
Thank you, -Victor
On 13-07-22 02:52 PM, Anthony Minessale wrote:
Check the odbcinst.ini settings from the wiki or use native pg mode. increase the max number of pg connections on your server and turn up FS to match in max-db-handles in switch.conf.xml
You are already beating it up pretty good esp for a fake server though. 2500 is a lot.
On Mon, Jul 22, 2013 at 1:44 PM, Ken Rice <krice@freeswitch.org> wrote:
Mount the entire freeswitch/db dir as tmpfs... most of these errors are for the sofia DB getting backed up...
The coredb is a completely different sqlite db from the sofia db
On 7/22/13 12:54 PM, "Victor Chukalovskiy" <victor.chukalovskiy@gmail.com
wrote:
Hello,
Need some help finding the bottleneck here. Running 1.2.stable on a decent multi-core vmware platform. No RTP, SIP signaling only. Around 2500 sessions and 30 average CPS load. Essentially, it's a multi-profile .xml config that uses mod_lcr extensively (postgres DB). Yesterday it stopped processing calls with the following log output:
2013-07-21 06:00:37.642678 [CRIT] switch_time.c:967 Virtual Migration Detected! Syncing Clock 2013-07-21 06:01:07.141734 [WARNING] switch_scheduler.c:115 Task was executed late by 19 seconds 1 heartbeat (core) 2013-07-21 06:02:32.402668 [CRIT] switch_time.c:967 Virtual Migration Detected! Syncing Clock 2013-07-21 06:04:42.499744 [WARNING] switch_scheduler.c:115 Task was executed late by 115 seconds 1 heartbeat (core) 2013-07-21 06:04:42.499744 [WARNING] switch_scheduler.c:115 Task was executed late by 74 seconds 2 check_ip (core) 2013-07-21 06:05:15.102683 [CRIT] switch_time.c:967 Virtual Migration Detected! Syncing Clock 2013-07-21 06:05:44.961004 [WARNING] switch_scheduler.c:115 Task was executed late by 22 seconds 1 heartbeat (core) 2013-07-21 06:05:44.961004 [WARNING] switch_scheduler.c:115 Task was executed late by 2 seconds 2 check_ip (core) 2013-07-21 06:05:49.322672 [WARNING] mod_lcr.c:448 Max handles 50 exceeded, blocking.... 2013-07-21 06:05:50.962987 [WARNING] sofia_glue.c:6605 Max handles 50 exceeded, blocking.... 2013-07-21 06:05:50.962987 [WARNING] sofia_glue.c:6605 Max handles 50 exceeded, blocking.... 2013-07-21 06:05:50.962987 [WARNING] sofia_glue.c:6605 Max handles 50 exceeded, blocking.... ................................ 2013-07-21 06:05:53.962672 [WARNING] sofia_glue.c:6605 Max handles 50 exceeded, blocking.... 2013-07-21 06:05:54.422671 [ERR] mod_lcr.c:448 Error connecting 2013-07-21 06:05:54.422671 [WARNING] mod_lcr.c:1779 LCR lookup failed for 14389955082 2013-07-21 06:05:54.962671 [WARNING] sofia_glue.c:6605 Max handles 50 exceeded, blocking.... 2013-07-21 06:05:54.962671 [WARNING] sofia_glue.c:6605 Max handles 50 exceeded, blocking.... ................................ 2013-07-21 06:05:55.962758 [WARNING] sofia_glue.c:6605 Max handles 50 exceeded, blocking.... 2013-07-21 06:05:56.062671 [ERR] sofia_glue.c:6605 Error connecting 2013-07-21 06:05:56.062671 [ERR] sofia_glue.c:6606 Error Opening DB 2013-07-21 06:05:56.062671 [WARNING] sofia_glue.c:6605 Max handles 50 exceeded, blocking.... 2013-07-21 06:05:56.062671 [ERR] sofia_glue.c:6605 Error connecting 2013-07-21 06:05:56.062671 [ERR] sofia_glue.c:6606 Error Opening DB
FreeSWITCH stayed unusable since then until it was restarted.
FYI core DB is already in the RAM: <param name="core-db-name" value="/dev/shm/core.db" />
Thank you, Victor
_________________________________________________________________________
Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Ken http://www.FreeSWITCH.org http://www.ClueCon.com http://www.OSTAG.org irc.freenode.net #freeswitch
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire
AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
_________________________________________________________________________ Professional FreeSWITCH Consulting Services:consulting@freeswitch.orghttp://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Serverhttp://www.cudatel.com
Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
FreeSWITCH-users mailing listFreeSWITCH-users@lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.frees...
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900