[Freeswitch-users] High CPS performance meltdown

[Victor Chukalovskiy]

Good day,

I'm running a few stress-testing scenarios with SIPp. On FS side call is 
dipped for LRN via SIP redirect, LRN saved / checked with mod_memcache, 
and routing done by mod_lcr with odbc connection to pgsql.
Each call is ended by FS as soon as all steps above are complete, so no 
media is involved and calls are never answered.

What I observe is that it can run at a very high CPS initially for a 
minute or so (100-200 CPS range). After that, it will drop to 50 CPS and 
will stay there. Suppose I stop all test calls and resume in a few 
minutes. It will not go back to initial 100-200 CPS values, it will stay 
at the 50 CPS.

Whatever the bottleneck is, it clears with full FS restart. It does not 
clear with individual components restart.

So far I've tried VM and real hardware and don't see much change with 
the bottleneck above. The initial CPS may differ (real HW better than 
VM). But, the value of 45-50CPS where it get's stuck is still the same. 
Tried CentOS 6.5 vs Ubuntu 14.04 no difference. Tried disabling 
different dial plan steps one-by-one, same pattern.

A couple words on the testing scenario: say, sipp is limited to 200 CPS 
and 200 concurrent calls. So, until the bottle-neck condition occurs, it 
runs at full 200 CPS and some 10-20 concurrent calls. As soon as 
condition occurs, CPS drop to 40-50 and concurrent calls rise to max 
value of 200...which is pretty much a feeding buffer for FS going forth.

Would such bottle-neck be expected from FS and what would be the steps 
to isolate it?

Thank you,
-Victor