[Freeswitch-users] ReloadXML takes very long time

Fri Oct 26 04:11:11 MSD 2012

I do that via mod_odbc_query. http://wiki.freeswitch.org/wiki/Mod_odbc_query

It does a query and loads the result (only supports 1 row, I think) as a
chan variable, which you can then use.

-Avi

On Fri, Oct 26, 2012 at 2:04 AM, Yiftach Golan <yiftah at choochee.com> wrote:

> OK I loaded the code to :
> http://jira.freeswitch.org/browse/FS-4766
> I have to say that this version is much less tested than the one we had
> for FreeSWITCH1.0.7 but hopefully we can still gain from this improvement
>
> I have a different question on the same issue :
> Even with this fix it is very hard to put huge XMLs, the memory
> consumption is going crazy
> Did you think about adding MySQL support ?
> I thought on adding something like :
> <action application="bridge" data="MYSQL(SELECT bridge_data FROM users
> WHERE destination_number = ${destination_number})"/>
> This will allow us to still use the flexibility of the XMLs and the
> scalability of a database
> Do you know if someone did already?
>
> Thanks,
> Yiftach.
>
>
>
> On Thu, Oct 25, 2012 at 3:19 PM, Anthony Minessale <
> anthony.minessale at gmail.com> wrote:
>
>> Can you git diff it and post it to JIRA so we can have some people look
>> at it and test?
>>
>>
>> On Thu, Oct 25, 2012 at 5:11 PM, Yiftach Golan <yiftah at choochee.com>wrote:
>>
>>> It may be a bit problematic since I will need 3 hooks :
>>> 1. *switch_xml_parse_fd :* Create the initial pre allocated buffer
>>> 2. *preprocess :* indexing the file names (adding a note before and
>>> after each file )
>>> 3. *switch_xml_parse_file :*  use my reload instead the regular one for
>>> creating the big xml and then I just let the function continue as usual
>>> through the "__switch_xml_open_root" function
>>> I am reloading the file to the memory and then let the regular
>>> "switch_xml.c" do its work, so using the existing functions in
>>> "switch_xml.c" will not do the work
>>> However, we do not have to add it I just thought it would be good
>>> improvement for FreeSWITCH as it was very helpful for us
>>>
>>> On Thu, Oct 25, 2012 at 2:58 PM, Anthony Minessale <
>>> anthony.minessale at gmail.com> wrote:
>>>
>>>> I thought you were going to move it to be a module?
>>>>
>>>> On Thu, Oct 25, 2012 at 4:50 PM, Yiftach Golan <yiftah at choochee.com>wrote:
>>>>
>>>>> I merged my code to the HEAD here are the results (I yet checked in
>>>>> the code) :
>>>>> # time /opt/freeswitch/bin/fs_cli -x "reloadxml"
>>>>> +OK [Success]
>>>>> real    *0m24.424s*
>>>>> user    0m0.000s
>>>>> sys    0m0.020s
>>>>>
>>>>> # time /opt/freeswitch/bin/fs_cli -x "reloadxml
>>>>> /repository/generated_configs/tenant_dialplan/156792923823_tenant_dialplan.xml"
>>>>> +OK [Success]
>>>>> real    *0m0.315s*
>>>>> user    0m0.000s
>>>>> sys    0m0.010s
>>>>>
>>>>> Looks like there is major improvement in the loading time
>>>>> How does it work, I just need to check in my code to the git? with my
>>>>> subscribe user?
>>>>> Thanks,
>>>>> Yiftach.
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Oct 23, 2012 at 5:42 PM, Yiftach Golan <yiftah at choochee.com>wrote:
>>>>>
>>>>>> OK I will do so
>>>>>>
>>>>>> On Tue, Oct 23, 2012 at 5:41 PM, Anthony Minessale <
>>>>>> anthony.minessale at gmail.com> wrote:
>>>>>>
>>>>>>> Also, if you design it cleanly and generic enough you can check the
>>>>>>> module itself into HEAD and maintain it yourself and others can choose to
>>>>>>> use it.
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Oct 23, 2012 at 5:54 PM, Yiftach Golan <yiftah at choochee.com>wrote:
>>>>>>>
>>>>>>>> Yes I agree, I can solve it this way for me, if anyone else in the
>>>>>>>> community is looking for a very fast way of loading a specific file from
>>>>>>>> the "reloadxml" command I can update it to the HEAD
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Oct 23, 2012 at 3:49 PM, Anthony Minessale <
>>>>>>>> anthony.minessale at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> The man xml registry is already reloaded using the method
>>>>>>>>> described.  In fact the xml root has reference counting and everyone who
>>>>>>>>> was on the old one will be able to hold on to that root until they release
>>>>>>>>> it even when a new one has been loaded into place.
>>>>>>>>>
>>>>>>>>> I think if you pursue the custom C module with the XML hook, that
>>>>>>>>> you can put your same logic into a module and not bother the core as any
>>>>>>>>> access to the core xml will be redirected to your module callback where you
>>>>>>>>> can do whatever you want and not disturb the core of FS unless you have
>>>>>>>>> some kind of functionality that is useful to everyone in general.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Oct 23, 2012 at 4:32 PM, Yiftach Golan <
>>>>>>>>> yiftah at choochee.com> wrote:
>>>>>>>>>
>>>>>>>>>> You may be able to develop it as well but it will not solve our
>>>>>>>>>> problem of pushing configuration fast
>>>>>>>>>> We have a front end that pushes the configuration changes and it
>>>>>>>>>> needs to be fast, so even if we change it in the background it will be slow
>>>>>>>>>>
>>>>>>>>>> On Tue, Oct 23, 2012 at 2:03 PM, Puskás Zsolt <errotan at elder.hu>wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I'm not an expert ( correct me if I'm wrong ) but I suppose the
>>>>>>>>>>> xml config memory section is locked while the files are proccessed so other
>>>>>>>>>>> modules are waiting for it. Instead of loading just one file you could
>>>>>>>>>>> continue to load all the files into somekind of a new temporary config
>>>>>>>>>>> section without locking the main. Once the files are loaded and processed
>>>>>>>>>>> the temporary and main section could be swapped or the temporary is used
>>>>>>>>>>> and the other is deleted from memory. No idea how many time this could take
>>>>>>>>>>> to implement.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2012-10-23 02:56 keltezéssel, Yiftach Golan írta:
>>>>>>>>>>>
>>>>>>>>>>> Hi FreeSWITCH community,
>>>>>>>>>>> My name is Yiftah and I am working for ChooChee which is a
>>>>>>>>>>> hosted PBX solution company
>>>>>>>>>>> We have been using FreeSWITCH for almost 2 years and we added a
>>>>>>>>>>> lot of code changes to the core and to the different modules
>>>>>>>>>>> Lately we have made a decision to get more close to the
>>>>>>>>>>> community so that we can enjoy the bug fixes and the new feature from the
>>>>>>>>>>> community and we can contribute our knowledge and bug fixes
>>>>>>>>>>> I would please like to start with a very basic question :
>>>>>>>>>>> We are using dialplan xmls in order to load the configuration,
>>>>>>>>>>> we have a lot of users on each freeswitch (+4k), it took a very long time
>>>>>>>>>>> for the configuration to load (around 5 minutes) during this time no new
>>>>>>>>>>> calls could be served by FreeSWITCH
>>>>>>>>>>> In order to solve this problem we introduce reload xml that
>>>>>>>>>>> accepts file names and has the ability to load just one file name into the
>>>>>>>>>>> memory (we achieved 10ms loading time instead of 5 minutes)
>>>>>>>>>>> The problem is that we had to introduce a lot of code in order
>>>>>>>>>>> to solve this problem, has anyone else encounter this problem? we do not
>>>>>>>>>>> want to use mod_xml_curl as we want to take care bringing the file to the
>>>>>>>>>>> machine
>>>>>>>>>>> Another approach that we thought is that a database might be
>>>>>>>>>>> much better fit in here, because every xml update includes the entire
>>>>>>>>>>> tenant (business) and has the potential in harming other users in the same
>>>>>>>>>>> tenant, also DB has much higher capacities then XMLs, did anyone think of
>>>>>>>>>>> enabling MySQL as a directory of parameters instead of xml?
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Yiftach.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _________________________________________________________________________
>>>>>>>>>>> Professional FreeSWITCH Consulting Services:consulting at freeswitch.orghttp://www.freeswitchsolutions.com
>>>>>>>>>>>
>>>>>>>>>>> FreeSWITCH-powered IP PBX: The CudaTel Communication Server
>>>>>>>>>>>
>>>>>>>>>>> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
>>>>>>>>>>>
>>>>>>>>>>> FreeSWITCH-users mailing listFreeSWITCH-users at lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> _________________________________________________________________________
>>>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>>>> consulting at freeswitch.org
>>>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>>>
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>
>>>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>>> http://wiki.freeswitch.org
>>>>>>>>>>> http://www.cluecon.com
>>>>>>>>>>>
>>>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _________________________________________________________________________
>>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>>> consulting at freeswitch.org
>>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>>
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>
>>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>> http://wiki.freeswitch.org
>>>>>>>>>> http://www.cluecon.com
>>>>>>>>>>
>>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Anthony Minessale II
>>>>>>>>>
>>>>>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>>>>> ClueCon http://www.cluecon.com/
>>>>>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>>>>>
>>>>>>>>> AIM: anthm
>>>>>>>>> MSN:anthony_minessale at hotmail.com
>>>>>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>>>>> IRC: irc.freenode.net #freeswitch
>>>>>>>>>
>>>>>>>>> FreeSWITCH Developer Conference
>>>>>>>>> sip:888 at conference.freeswitch.org
>>>>>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>>>>> pstn:+19193869900
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _________________________________________________________________________
>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>> consulting at freeswitch.org
>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>
>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>> http://www.freeswitch.org
>>>>>>>>> http://wiki.freeswitch.org
>>>>>>>>> http://www.cluecon.com
>>>>>>>>>
>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _________________________________________________________________________
>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>> consulting at freeswitch.org
>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>
>>>>>>>> 
>>>>>>>> 
>>>>>>>>
>>>>>>>> Official FreeSWITCH Sites
>>>>>>>> http://www.freeswitch.org
>>>>>>>> http://wiki.freeswitch.org
>>>>>>>> http://www.cluecon.com
>>>>>>>>
>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>> UNSUBSCRIBE:
>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>> http://www.freeswitch.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Anthony Minessale II
>>>>>>>
>>>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>>> ClueCon http://www.cluecon.com/
>>>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>>>
>>>>>>> AIM: anthm
>>>>>>> MSN:anthony_minessale at hotmail.com
>>>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>>> IRC: irc.freenode.net #freeswitch
>>>>>>>
>>>>>>> FreeSWITCH Developer Conference
>>>>>>> sip:888 at conference.freeswitch.org
>>>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>>> pstn:+19193869900
>>>>>>>
>>>>>>>
>>>>>>> _________________________________________________________________________
>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>> consulting at freeswitch.org
>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>
>>>>>>> 
>>>>>>> 
>>>>>>>
>>>>>>> Official FreeSWITCH Sites
>>>>>>> http://www.freeswitch.org
>>>>>>> http://wiki.freeswitch.org
>>>>>>> http://www.cluecon.com
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>> UNSUBSCRIBE:
>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> 
>>>>> 
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://wiki.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Anthony Minessale II
>>>>
>>>> FreeSWITCH http://www.freeswitch.org/
>>>> ClueCon http://www.cluecon.com/
>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>
>>>> AIM: anthm
>>>> MSN:anthony_minessale at hotmail.com
>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>> IRC: irc.freenode.net #freeswitch
>>>>
>>>> FreeSWITCH Developer Conference
>>>> sip:888 at conference.freeswitch.org
>>>> googletalk:conf+888 at conference.freeswitch.org
>>>> pstn:+19193869900
>>>>
>>>>
>>>> _________________________________________________________________________
>>>> Professional FreeSWITCH Consulting Services:
>>>> consulting at freeswitch.org
>>>> http://www.freeswitchsolutions.com
>>>>
>>>> 
>>>> 
>>>>
>>>> Official FreeSWITCH Sites
>>>> http://www.freeswitch.org
>>>> http://wiki.freeswitch.org
>>>> http://www.cluecon.com
>>>>
>>>> FreeSWITCH-users mailing list
>>>> FreeSWITCH-users at lists.freeswitch.org
>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>> UNSUBSCRIBE:
>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>> http://www.freeswitch.org
>>>>
>>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> 
>>> 
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>>
>>
>> --
>> Anthony Minessale II
>>
>> FreeSWITCH http://www.freeswitch.org/
>> ClueCon http://www.cluecon.com/
>> Twitter: http://twitter.com/FreeSWITCH_wire
>>
>> AIM: anthm
>> MSN:anthony_minessale at hotmail.com
>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>> IRC: irc.freenode.net #freeswitch
>>
>> FreeSWITCH Developer Conference
>> sip:888 at conference.freeswitch.org
>> googletalk:conf+888 at conference.freeswitch.org
>> pstn:+19193869900
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20121026/74c3511e/attachment-0001.html 