[Freeswitch-users] ReloadXML takes very long time

Yiftach Golan yiftah at choochee.com
Fri Oct 26 04:17:15 MSD 2012 

Have you used it for directory as well ?
<variable name="outbound_caller_id_number" value="SELECT
outbound_caller_id_number FROM directory WHERE domain = ${domain} AND user
= ${user}"/>

On Thu, Oct 25, 2012 at 5:11 PM, Avi Marcus <avi at avimarcus.net> wrote:

> I do that via mod_odbc_query.
> http://wiki.freeswitch.org/wiki/Mod_odbc_query
>
> It does a query and loads the result (only supports 1 row, I think) as a
> chan variable, which you can then use.
>
> -Avi
>
>
> On Fri, Oct 26, 2012 at 2:04 AM, Yiftach Golan <yiftah at choochee.com>wrote:
>
>> OK I loaded the code to :
>> http://jira.freeswitch.org/browse/FS-4766
>> I have to say that this version is much less tested than the one we had
>> for FreeSWITCH1.0.7 but hopefully we can still gain from this improvement
>>
>> I have a different question on the same issue :
>> Even with this fix it is very hard to put huge XMLs, the memory
>> consumption is going crazy
>> Did you think about adding MySQL support ?
>> I thought on adding something like :
>> <action application="bridge" data="MYSQL(SELECT bridge_data FROM users
>> WHERE destination_number = ${destination_number})"/>
>> This will allow us to still use the flexibility of the XMLs and the
>> scalability of a database
>> Do you know if someone did already?
>>
>> Thanks,
>> Yiftach.
>>
>>
>>
>> On Thu, Oct 25, 2012 at 3:19 PM, Anthony Minessale <
>> anthony.minessale at gmail.com> wrote:
>>
>>> Can you git diff it and post it to JIRA so we can have some people look
>>> at it and test?
>>>
>>>
>>> On Thu, Oct 25, 2012 at 5:11 PM, Yiftach Golan <yiftah at choochee.com>wrote:
>>>
>>>> It may be a bit problematic since I will need 3 hooks :
>>>> 1. *switch_xml_parse_fd :* Create the initial pre allocated buffer
>>>> 2. *preprocess :* indexing the file names (adding a note before and
>>>> after each file )
>>>> 3. *switch_xml_parse_file :*  use my reload instead the regular one
>>>> for creating the big xml and then I just let the function continue as usual
>>>> through the "__switch_xml_open_root" function
>>>> I am reloading the file to the memory and then let the regular
>>>> "switch_xml.c" do its work, so using the existing functions in
>>>> "switch_xml.c" will not do the work
>>>> However, we do not have to add it I just thought it would be good
>>>> improvement for FreeSWITCH as it was very helpful for us
>>>>
>>>> On Thu, Oct 25, 2012 at 2:58 PM, Anthony Minessale <
>>>> anthony.minessale at gmail.com> wrote:
>>>>
>>>>> I thought you were going to move it to be a module?
>>>>>
>>>>> On Thu, Oct 25, 2012 at 4:50 PM, Yiftach Golan <yiftah at choochee.com>wrote:
>>>>>
>>>>>> I merged my code to the HEAD here are the results (I yet checked in
>>>>>> the code) :
>>>>>> # time /opt/freeswitch/bin/fs_cli -x "reloadxml"
>>>>>> +OK [Success]
>>>>>> real    *0m24.424s*
>>>>>> user    0m0.000s
>>>>>> sys    0m0.020s
>>>>>>
>>>>>> # time /opt/freeswitch/bin/fs_cli -x "reloadxml
>>>>>> /repository/generated_configs/tenant_dialplan/156792923823_tenant_dialplan.xml"
>>>>>> +OK [Success]
>>>>>> real    *0m0.315s*
>>>>>> user    0m0.000s
>>>>>> sys    0m0.010s
>>>>>>
>>>>>> Looks like there is major improvement in the loading time
>>>>>> How does it work, I just need to check in my code to the git? with my
>>>>>> subscribe user?
>>>>>> Thanks,
>>>>>> Yiftach.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Oct 23, 2012 at 5:42 PM, Yiftach Golan <yiftah at choochee.com>wrote:
>>>>>>
>>>>>>> OK I will do so
>>>>>>>
>>>>>>> On Tue, Oct 23, 2012 at 5:41 PM, Anthony Minessale <
>>>>>>> anthony.minessale at gmail.com> wrote:
>>>>>>>
>>>>>>>> Also, if you design it cleanly and generic enough you can check the
>>>>>>>> module itself into HEAD and maintain it yourself and others can choose to
>>>>>>>> use it.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Oct 23, 2012 at 5:54 PM, Yiftach Golan <yiftah at choochee.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Yes I agree, I can solve it this way for me, if anyone else in the
>>>>>>>>> community is looking for a very fast way of loading a specific file from
>>>>>>>>> the "reloadxml" command I can update it to the HEAD
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Oct 23, 2012 at 3:49 PM, Anthony Minessale <
>>>>>>>>> anthony.minessale at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> The man xml registry is already reloaded using the method
>>>>>>>>>> described.  In fact the xml root has reference counting and everyone who
>>>>>>>>>> was on the old one will be able to hold on to that root until they release
>>>>>>>>>> it even when a new one has been loaded into place.
>>>>>>>>>>
>>>>>>>>>> I think if you pursue the custom C module with the XML hook, that
>>>>>>>>>> you can put your same logic into a module and not bother the core as any
>>>>>>>>>> access to the core xml will be redirected to your module callback where you
>>>>>>>>>> can do whatever you want and not disturb the core of FS unless you have
>>>>>>>>>> some kind of functionality that is useful to everyone in general.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Oct 23, 2012 at 4:32 PM, Yiftach Golan <
>>>>>>>>>> yiftah at choochee.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> You may be able to develop it as well but it will not solve our
>>>>>>>>>>> problem of pushing configuration fast
>>>>>>>>>>> We have a front end that pushes the configuration changes and it
>>>>>>>>>>> needs to be fast, so even if we change it in the background it will be slow
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Oct 23, 2012 at 2:03 PM, Puskás Zsolt <errotan at elder.hu>wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I'm not an expert ( correct me if I'm wrong ) but I suppose the
>>>>>>>>>>>> xml config memory section is locked while the files are proccessed so other
>>>>>>>>>>>> modules are waiting for it. Instead of loading just one file you could
>>>>>>>>>>>> continue to load all the files into somekind of a new temporary config
>>>>>>>>>>>> section without locking the main. Once the files are loaded and processed
>>>>>>>>>>>> the temporary and main section could be swapped or the temporary is used
>>>>>>>>>>>> and the other is deleted from memory. No idea how many time this could take
>>>>>>>>>>>> to implement.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2012-10-23 02:56 keltezéssel, Yiftach Golan írta:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi FreeSWITCH community,
>>>>>>>>>>>> My name is Yiftah and I am working for ChooChee which is a
>>>>>>>>>>>> hosted PBX solution company
>>>>>>>>>>>> We have been using FreeSWITCH for almost 2 years and we added a
>>>>>>>>>>>> lot of code changes to the core and to the different modules
>>>>>>>>>>>> Lately we have made a decision to get more close to the
>>>>>>>>>>>> community so that we can enjoy the bug fixes and the new feature from the
>>>>>>>>>>>> community and we can contribute our knowledge and bug fixes
>>>>>>>>>>>> I would please like to start with a very basic question :
>>>>>>>>>>>> We are using dialplan xmls in order to load the configuration,
>>>>>>>>>>>> we have a lot of users on each freeswitch (+4k), it took a very long time
>>>>>>>>>>>> for the configuration to load (around 5 minutes) during this time no new
>>>>>>>>>>>> calls could be served by FreeSWITCH
>>>>>>>>>>>> In order to solve this problem we introduce reload xml that
>>>>>>>>>>>> accepts file names and has the ability to load just one file name into the
>>>>>>>>>>>> memory (we achieved 10ms loading time instead of 5 minutes)
>>>>>>>>>>>> The problem is that we had to introduce a lot of code in order
>>>>>>>>>>>> to solve this problem, has anyone else encounter this problem? we do not
>>>>>>>>>>>> want to use mod_xml_curl as we want to take care bringing the file to the
>>>>>>>>>>>> machine
>>>>>>>>>>>> Another approach that we thought is that a database might be
>>>>>>>>>>>> much better fit in here, because every xml update includes the entire
>>>>>>>>>>>> tenant (business) and has the potential in harming other users in the same
>>>>>>>>>>>> tenant, also DB has much higher capacities then XMLs, did anyone think of
>>>>>>>>>>>> enabling MySQL as a directory of parameters instead of xml?
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Yiftach.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _________________________________________________________________________
>>>>>>>>>>>> Professional FreeSWITCH Consulting Services:consulting at freeswitch.orghttp://www.freeswitchsolutions.com
>>>>>>>>>>>>
>>>>>>>>>>>> FreeSWITCH-powered IP PBX: The CudaTel Communication Server
>>>>>>>>>>>>
>>>>>>>>>>>> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
>>>>>>>>>>>>
>>>>>>>>>>>> FreeSWITCH-users mailing listFreeSWITCH-users at lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> _________________________________________________________________________
>>>>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>>>>> consulting at freeswitch.org
>>>>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>>>>
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>
>>>>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>>>> http://wiki.freeswitch.org
>>>>>>>>>>>> http://www.cluecon.com
>>>>>>>>>>>>
>>>>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _________________________________________________________________________
>>>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>>>> consulting at freeswitch.org
>>>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>>>
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>
>>>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>>> http://wiki.freeswitch.org
>>>>>>>>>>> http://www.cluecon.com
>>>>>>>>>>>
>>>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Anthony Minessale II
>>>>>>>>>>
>>>>>>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>>>>>> ClueCon http://www.cluecon.com/
>>>>>>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>>>>>>
>>>>>>>>>> AIM: anthm
>>>>>>>>>> MSN:anthony_minessale at hotmail.com
>>>>>>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>>>>>> IRC: irc.freenode.net #freeswitch
>>>>>>>>>>
>>>>>>>>>> FreeSWITCH Developer Conference
>>>>>>>>>> sip:888 at conference.freeswitch.org
>>>>>>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>>>>>> pstn:+19193869900
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _________________________________________________________________________
>>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>>> consulting at freeswitch.org
>>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>>
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>
>>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>> http://wiki.freeswitch.org
>>>>>>>>>> http://www.cluecon.com
>>>>>>>>>>
>>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _________________________________________________________________________
>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>> consulting at freeswitch.org
>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>
>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>> http://www.freeswitch.org
>>>>>>>>> http://wiki.freeswitch.org
>>>>>>>>> http://www.cluecon.com
>>>>>>>>>
>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Anthony Minessale II
>>>>>>>>
>>>>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>>>> ClueCon http://www.cluecon.com/
>>>>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>>>>
>>>>>>>> AIM: anthm
>>>>>>>> MSN:anthony_minessale at hotmail.com
>>>>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>>>> IRC: irc.freenode.net #freeswitch
>>>>>>>>
>>>>>>>> FreeSWITCH Developer Conference
>>>>>>>> sip:888 at conference.freeswitch.org
>>>>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>>>> pstn:+19193869900
>>>>>>>>
>>>>>>>>
>>>>>>>> _________________________________________________________________________
>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>> consulting at freeswitch.org
>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>
>>>>>>>> 
>>>>>>>> 
>>>>>>>>
>>>>>>>> Official FreeSWITCH Sites
>>>>>>>> http://www.freeswitch.org
>>>>>>>> http://wiki.freeswitch.org
>>>>>>>> http://www.cluecon.com
>>>>>>>>
>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>> UNSUBSCRIBE:
>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>> http://www.freeswitch.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> _________________________________________________________________________
>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>> consulting at freeswitch.org
>>>>>> http://www.freeswitchsolutions.com
>>>>>>
>>>>>> 
>>>>>> 
>>>>>>
>>>>>> Official FreeSWITCH Sites
>>>>>> http://www.freeswitch.org
>>>>>> http://wiki.freeswitch.org
>>>>>> http://www.cluecon.com
>>>>>>
>>>>>> FreeSWITCH-users mailing list
>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>> http://www.freeswitch.org
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Anthony Minessale II
>>>>>
>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>> ClueCon http://www.cluecon.com/
>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>
>>>>> AIM: anthm
>>>>> MSN:anthony_minessale at hotmail.com
>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>> IRC: irc.freenode.net #freeswitch
>>>>>
>>>>> FreeSWITCH Developer Conference
>>>>> sip:888 at conference.freeswitch.org
>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>> pstn:+19193869900
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> 
>>>>> 
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://wiki.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> 
>>>> 
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://wiki.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>>
>>>
>>>
>>> --
>>> Anthony Minessale II
>>>
>>> FreeSWITCH http://www.freeswitch.org/
>>> ClueCon http://www.cluecon.com/
>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>
>>> AIM: anthm
>>> MSN:anthony_minessale at hotmail.com
>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>> IRC: irc.freenode.net #freeswitch
>>>
>>> FreeSWITCH Developer Conference
>>> sip:888 at conference.freeswitch.org
>>> googletalk:conf+888 at conference.freeswitch.org
>>> pstn:+19193869900
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> 
>>> 
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20121025/c6edce86/attachment-0001.html

Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users mailing list