[Freeswitch-dev] transfer_history performance

[David Brazier]

François

Thanks for the suggestion, will try re-working with execute_extension.

David


From: François Delawarde
Sent: 17/01/2013 11:26
To: freeswitch-dev at lists.freeswitch.org
Subject: Re: [Freeswitch-dev] transfer_history performance
Hi David,

The "transfer" app is quite heavy to be chained 1000s of times as it
does many things to keep track of what happened (handle max-fw, dup
caller profile, history var, ...). You might end-up with other issues
trying to scale, maybe disk bottleneck writing huge xml-cdr files, ...

What about making your application use "execute_extension" instead? Note
that I still vote yes for your optimization ideas!

François.


On Thu, 2013-01-17 at 10:48 +0000, David Brazier wrote:
> Hi again
>
> A bit more information on reproducing it: each call using
> test_transfer will fully occupy one CPU, so on a 4 core machine you
> need to start 4 of these calls to tie it up, then audio on other calls
> is broken or completely stopped, CLI stops responding etc.  As a
> workaround I have commented out from switch_ivr.c line 1808 near end
> of switch_ivr_session_transfer:
>
> /*switch_channel_add_variable_var_check(channel,
> SWITCH_TRANSFER_HISTORY_VARIABLE, new_profile->transfer_source,
> SWITCH_FALSE, SWITCH_STACK_PUSH);*/
>
> Of course that means transfer_history is no longer populated, but it
> does remove the performance problem, the test_transfer calls now run
> through so fast I had to start the loop off at 9999 and although they
> do use a bit of CPU (it is a tight loop with no sleep/IO etc after
> all) 4 simultaneously do not tie up the server and other calls
> continue fine.  I can think of a few more complete solutions:
>      1. Improve the performance of appending to variables with big
>         values - I think this is ultimately done in
>         switch_event_base_add_header in switch_event.c, though that
>         looks like quite a complex function.  In the end, repeated
>         appending to long strings is never going to be efficient.
>      2. Maintain the transfer history as a stack/list/array structure
>         and either render it to the variable string whenever it is
>         changed (still expensive?) or generate the string only on
>         demand - though I'm not sure how that would work with info,
>         uuid_dump and verbose events in event sockets etc.
>      3. Allow channels to opt out of having transfer_history tracked,
>         perhaps by setting it to "suppress" or "false", this could be
>         checked before the line above.  Would put the onus on the
>         dialplan to know if it was going to do lots of transfers, but
>         it would be backward-compatible with anyone who needs this
>         variable.  Out of interest why was it added?
> Once I have managed to log in to Jira I will raise all this there.
>
> Cheers
>
> David
>
>
> On Thu, Jan 17, 2013 at 12:57 AM, David Brazier
> <davidjbrazier at gmail.com> wrote:
>         Hi
>
>         Our application makes use of long-running calls with a lot of
>         diaplan transfers to bridge them to other calls, put them on
>         hold etc.  We've noticed a slow down in performance (high CPU,
>         audio dropped, delayed) after several hours.  I can reproduce
>         it with some dialplan that does 1000 transfers:
>
>         <extension name="test_transfer">
>           <condition field="destination_number"
>         expression="^test_transfer$">
>             <action application="set" data="max_forwards=9999"/>
>             <action application="transfer" data="test_transfer_999"/>
>           </condition>
>         </extension>
>         <extension name="test_transfer_0" continue="false">
>           <condition field="destination_number"
>         expression="^test_transfer_0$">
>             <action application="hangup"/>
>           </condition>
>         </extension>
>         <extension name="test_transfer_n">
>           <condition field="destination_number"
>         expression="^test_transfer_(\d+)$">
>             <!--<action application="info"/>-->
>             <action application="transfer" data="test_transfer_${expr
>         $1-1}"/>
>           </condition>
>         </extension>
>
>         and then
>
>         originate user/1000 test_transfer XML blah
>
>         On a Windows laptop (git head) it slows down to more than 1
>         sec per iteration after it gets past 800, on a bigger Linux
>         server (FS 1.2.5.1) it gets a bit futher but not much.  On a
>         server with an old version (git-1086cba 2011-05-23 22-51-43
>         -0500) it zips through to the end with no slow down.  If you
>         put the info call in you can see that
>         variable_transfer_history is growing, and I think this is the
>         source of the problem - this variable is missing from the
>         older version but the newer ones are apparently appending to
>         it on each iteration which becomes a very expensive
>         operation.
>
>         I'll raise a Jira.  Any suggestion for a workaround? (Apart
>         from rewriting our dialplan!)
>
>         David
>
>
>
>
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-dev mailing list
> FreeSWITCH-dev at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
> http://www.freeswitch.org



_________________________________________________________________________
Professional FreeSWITCH Consulting Services:
consulting at freeswitch.org
http://www.freeswitchsolutions.com




Official FreeSWITCH Sites
http://www.freeswitch.org
http://wiki.freeswitch.org
http://www.cluecon.com

FreeSWITCH-dev mailing list
FreeSWITCH-dev at lists.freeswitch.org
http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev
UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev
http://www.freeswitch.org