[Freeswitch-dev] Fwd: Call Center - Memory Leak

Mon Apr 25 21:35:49 MSD 2016

Hi Nicholas, thank you for your reply.

Regarding your questions:

1. Are you sure that channels are being released -> yes for sure. That is
the first thing i checked. Test environment which is created do not goes
over 65 channels as i mentioned within initial email.
channels count, status and all other cli comands confirms that.
Additionally on sipp side you are always able to check in live how much
calls you have connected currently.

2. Regarding the schedule hangup, there is no issues with it i suppose,
because i use schedule hangup to recreate simple call center with pure
freeswitch installation. Within our call center solution we are not hanging
up calls like that and it is encountering same issue as well.

3. Regarding the request for memory usage per application:
    That test i explained above is created using plain freeswitch
installation without any kind of additions. Basically during that test, on
server we had only freeswitch running and nothing else. As a confirmation
here is the screenshot http://prnt.sc/awpxr6

Additionally i need to mention that i ran the same test before maybe few
weeks ago with valgrind, and I was not able to find anything.

All the best,

Mersed

On Mon, Apr 25, 2016 at 7:18 PM, Nicholas Blasgen <nicholas at hellohunter.com>
wrote:

> NewRelic, which you seem to be using, also provides per-application memory
> usage.  Would be nice to see that after a day or partial day run.
>
> Freeswitch uses a good amount of memory per channel.  Are you sure the
> channels are being released?  fs_cli -x status >> log_file ... and maybe
> run it on a crontab just to make sure the channels are being released.
>
> Besides that, I personally don't have any ideas without recreating your
> test environment.  I guess you could see if FIFO or Scheduled Hangup is the
> issue by calling HANGUP instead of FIFO for extension 7011.  I've never
> personally used Schedule Hangup and as that's the method being used here to
> release the channels, it might be a concern.
>
>
> Nicholas Blasgen
> Developer
> Predictive Dialer Limited
> +1 (724) 252-7436 (cell)
> Skype: nblasgen
>
> 24/7 Support available:
> www.hellohunter.com | (800) 513-5555 | skype hello.hunter |
> support at hellohunter.com
>
> On Mon, Apr 25, 2016 at 6:11 AM, Mersed Kahrimanovic <
> kahrimanovic.mersed at gmail.com> wrote:
>
>> Hello,
>>
>> We have implemented call center using a freeswitch as switch for handling
>> calls. It is basically the logic where you have an agents within the queue
>> waiting for an contacts to come in, or contact within the queue waiting if
>> there is no any agents available.
>>
>> For this kind of implementation we used pure lua and hash tables (and not
>> mod_fifo) in order to have a better flexibility and possibility to manage
>> different contact/agent states.
>>
>> Everything was working fine on our old servers, now we are migrating to
>> the cloud and we are using aws for that purpose, of course trying to cut
>> some costs.
>>
>> ---------------
>>
>> After initial setup of our solution we found out that there is a memory
>> leak causing for freeswitch to consume 3 - 10 MB per minute during
>> production hours.  (we did not notice anything on old servers because we
>> had like 24 cores and 94GB of ram on those).
>> This is not too much of course but the bottom point is that increase is
>> linear and its constantly raising consuming more and more of RAM.
>>
>> In order to isolate an issue and to conclude that we don't have an issue
>> within our solution I installed 1.6.7 version of freeswitch and setup
>> really simple call center with mod_fifo without anything else (totally
>> independent from call center we are working on)
>>
>> For the reference:
>>
>> *Aws:* m4.large
>>          vCPU: 2
>>          RAM: 8GB
>>          Throughput (Mbps): 450
>>
>> *Freeswitch version:* 1.6.7
>> *OS:* Debian GNU/Linux 8 (jessie)
>> *Number of agents:* 30
>> *Number of calls per agent:* 1
>> *Total numbers of channels at time:* 60
>> *Agents simulated with:* pjsip(http://www.pjsip.org/)
>> *Contact simulated with:* sipp(http://sipp.sourceforge.net/)
>> *Configuration:* I used configuration which came with installation of
>> freeswitch. The only things i changed is:
>>      - default password
>>      - internal rtp timeout
>>      - rtpip and sip ip
>>
>> Test scenario:
>> 1. Start the freeswitch.
>> 2. Autoload lua script is going to ring all of the internal (agents) and
>> push them to the extension where we had mod_fifo
>> 3. After some time, same autoload script is going to start calling
>> contacts each 800 miliseconds and push them to the extension where we have
>> mod_fifo.
>> 4. mod_fifo will do the rest, bridge agent with the contact and handling
>> the queue properly.
>>
>> As a result we will have 30 agents constantly "talking", and when ever
>> call is dropped we will have agent connected with another. Parameters for
>> the test match that scenario, and we dont have more then 60 - 65 channels
>> open at the time (never).
>>
>> Dialplan for handling mod_fifo
>> http://pastie.org/10812261
>>
>> Autoload lua script
>> http://pastie.org/10812263
>>
>> Result
>> http://prnt.sc/awm4ao
>>
>> The results are for 24hrs period, and indeed 24hrs is a lot. We will
>> never have production running more then 12hrs (and daily restart will
>> occur). But this basically means that we are not able to host more then
>> 40-45 agents on servers with 8GB of ram.
>>
>> My question is:
>> Is this something which can be considered as memory leak, or this is
>> simply how freeswitch works, behaves and requirements it have?
>>
>> Thank you,
>>
>> Mersed
>>
>>
>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-dev mailing list
>> FreeSWITCH-dev at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
>> http://www.freeswitch.org
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-dev/attachments/20160425/2c345794/attachment-0001.html 