[Freeswitch-users] ReloadXML takes very long time

Fri Oct 26 02:19:41 MSD 2012

Can you git diff it and post it to JIRA so we can have some people look at
it and test?

On Thu, Oct 25, 2012 at 5:11 PM, Yiftach Golan <yiftah at choochee.com> wrote:

> It may be a bit problematic since I will need 3 hooks :
> 1. *switch_xml_parse_fd :* Create the initial pre allocated buffer
> 2. *preprocess :* indexing the file names (adding a note before and after
> each file )
> 3. *switch_xml_parse_file :*  use my reload instead the regular one for
> creating the big xml and then I just let the function continue as usual
> through the "__switch_xml_open_root" function
> I am reloading the file to the memory and then let the regular
> "switch_xml.c" do its work, so using the existing functions in
> "switch_xml.c" will not do the work
> However, we do not have to add it I just thought it would be good
> improvement for FreeSWITCH as it was very helpful for us
>
> On Thu, Oct 25, 2012 at 2:58 PM, Anthony Minessale <
> anthony.minessale at gmail.com> wrote:
>
>> I thought you were going to move it to be a module?
>>
>> On Thu, Oct 25, 2012 at 4:50 PM, Yiftach Golan <yiftah at choochee.com>wrote:
>>
>>> I merged my code to the HEAD here are the results (I yet checked in the
>>> code) :
>>> # time /opt/freeswitch/bin/fs_cli -x "reloadxml"
>>> +OK [Success]
>>> real    *0m24.424s*
>>> user    0m0.000s
>>> sys    0m0.020s
>>>
>>> # time /opt/freeswitch/bin/fs_cli -x "reloadxml
>>> /repository/generated_configs/tenant_dialplan/156792923823_tenant_dialplan.xml"
>>> +OK [Success]
>>> real    *0m0.315s*
>>> user    0m0.000s
>>> sys    0m0.010s
>>>
>>> Looks like there is major improvement in the loading time
>>> How does it work, I just need to check in my code to the git? with my
>>> subscribe user?
>>> Thanks,
>>> Yiftach.
>>>
>>>
>>>
>>> On Tue, Oct 23, 2012 at 5:42 PM, Yiftach Golan <yiftah at choochee.com>wrote:
>>>
>>>> OK I will do so
>>>>
>>>> On Tue, Oct 23, 2012 at 5:41 PM, Anthony Minessale <
>>>> anthony.minessale at gmail.com> wrote:
>>>>
>>>>> Also, if you design it cleanly and generic enough you can check the
>>>>> module itself into HEAD and maintain it yourself and others can choose to
>>>>> use it.
>>>>>
>>>>>
>>>>> On Tue, Oct 23, 2012 at 5:54 PM, Yiftach Golan <yiftah at choochee.com>wrote:
>>>>>
>>>>>> Yes I agree, I can solve it this way for me, if anyone else in the
>>>>>> community is looking for a very fast way of loading a specific file from
>>>>>> the "reloadxml" command I can update it to the HEAD
>>>>>>
>>>>>>
>>>>>> On Tue, Oct 23, 2012 at 3:49 PM, Anthony Minessale <
>>>>>> anthony.minessale at gmail.com> wrote:
>>>>>>
>>>>>>> The man xml registry is already reloaded using the method described.
>>>>>>>  In fact the xml root has reference counting and everyone who was on the
>>>>>>> old one will be able to hold on to that root until they release it even
>>>>>>> when a new one has been loaded into place.
>>>>>>>
>>>>>>> I think if you pursue the custom C module with the XML hook, that
>>>>>>> you can put your same logic into a module and not bother the core as any
>>>>>>> access to the core xml will be redirected to your module callback where you
>>>>>>> can do whatever you want and not disturb the core of FS unless you have
>>>>>>> some kind of functionality that is useful to everyone in general.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Oct 23, 2012 at 4:32 PM, Yiftach Golan <yiftah at choochee.com>wrote:
>>>>>>>
>>>>>>>> You may be able to develop it as well but it will not solve our
>>>>>>>> problem of pushing configuration fast
>>>>>>>> We have a front end that pushes the configuration changes and it
>>>>>>>> needs to be fast, so even if we change it in the background it will be slow
>>>>>>>>
>>>>>>>> On Tue, Oct 23, 2012 at 2:03 PM, Puskás Zsolt <errotan at elder.hu>wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I'm not an expert ( correct me if I'm wrong ) but I suppose the
>>>>>>>>> xml config memory section is locked while the files are proccessed so other
>>>>>>>>> modules are waiting for it. Instead of loading just one file you could
>>>>>>>>> continue to load all the files into somekind of a new temporary config
>>>>>>>>> section without locking the main. Once the files are loaded and processed
>>>>>>>>> the temporary and main section could be swapped or the temporary is used
>>>>>>>>> and the other is deleted from memory. No idea how many time this could take
>>>>>>>>> to implement.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2012-10-23 02:56 keltezéssel, Yiftach Golan írta:
>>>>>>>>>
>>>>>>>>> Hi FreeSWITCH community,
>>>>>>>>> My name is Yiftah and I am working for ChooChee which is a hosted
>>>>>>>>> PBX solution company
>>>>>>>>> We have been using FreeSWITCH for almost 2 years and we added a
>>>>>>>>> lot of code changes to the core and to the different modules
>>>>>>>>> Lately we have made a decision to get more close to the community
>>>>>>>>> so that we can enjoy the bug fixes and the new feature from the community
>>>>>>>>> and we can contribute our knowledge and bug fixes
>>>>>>>>> I would please like to start with a very basic question :
>>>>>>>>> We are using dialplan xmls in order to load the configuration, we
>>>>>>>>> have a lot of users on each freeswitch (+4k), it took a very long time for
>>>>>>>>> the configuration to load (around 5 minutes) during this time no new calls
>>>>>>>>> could be served by FreeSWITCH
>>>>>>>>> In order to solve this problem we introduce reload xml that
>>>>>>>>> accepts file names and has the ability to load just one file name into the
>>>>>>>>> memory (we achieved 10ms loading time instead of 5 minutes)
>>>>>>>>> The problem is that we had to introduce a lot of code in order to
>>>>>>>>> solve this problem, has anyone else encounter this problem? we do not want
>>>>>>>>> to use mod_xml_curl as we want to take care bringing the file to the
>>>>>>>>> machine
>>>>>>>>> Another approach that we thought is that a database might be much
>>>>>>>>> better fit in here, because every xml update includes the entire tenant
>>>>>>>>> (business) and has the potential in harming other users in the same tenant,
>>>>>>>>> also DB has much higher capacities then XMLs, did anyone think of enabling
>>>>>>>>> MySQL as a directory of parameters instead of xml?
>>>>>>>>> Thanks,
>>>>>>>>> Yiftach.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _________________________________________________________________________
>>>>>>>>> Professional FreeSWITCH Consulting Services:consulting at freeswitch.orghttp://www.freeswitchsolutions.com
>>>>>>>>>
>>>>>>>>> FreeSWITCH-powered IP PBX: The CudaTel Communication Server
>>>>>>>>>
>>>>>>>>> Official FreeSWITCH Siteshttp://www.freeswitch.orghttp://wiki.freeswitch.orghttp://www.cluecon.com
>>>>>>>>>
>>>>>>>>> FreeSWITCH-users mailing listFreeSWITCH-users at lists.freeswitch.orghttp://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-usershttp://www.freeswitch.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _________________________________________________________________________
>>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>>> consulting at freeswitch.org
>>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>>
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>
>>>>>>>>> Official FreeSWITCH Sites
>>>>>>>>> http://www.freeswitch.org
>>>>>>>>> http://wiki.freeswitch.org
>>>>>>>>> http://www.cluecon.com
>>>>>>>>>
>>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>>> UNSUBSCRIBE:
>>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>>> http://www.freeswitch.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> _________________________________________________________________________
>>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>>> consulting at freeswitch.org
>>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>>
>>>>>>>> 
>>>>>>>> 
>>>>>>>>
>>>>>>>> Official FreeSWITCH Sites
>>>>>>>> http://www.freeswitch.org
>>>>>>>> http://wiki.freeswitch.org
>>>>>>>> http://www.cluecon.com
>>>>>>>>
>>>>>>>> FreeSWITCH-users mailing list
>>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>>> UNSUBSCRIBE:
>>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>>> http://www.freeswitch.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Anthony Minessale II
>>>>>>>
>>>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>>>> ClueCon http://www.cluecon.com/
>>>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>>>
>>>>>>> AIM: anthm
>>>>>>> MSN:anthony_minessale at hotmail.com
>>>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>>>> IRC: irc.freenode.net #freeswitch
>>>>>>>
>>>>>>> FreeSWITCH Developer Conference
>>>>>>> sip:888 at conference.freeswitch.org
>>>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>>>> pstn:+19193869900
>>>>>>>
>>>>>>>
>>>>>>> _________________________________________________________________________
>>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>>> consulting at freeswitch.org
>>>>>>> http://www.freeswitchsolutions.com
>>>>>>>
>>>>>>> 
>>>>>>> 
>>>>>>>
>>>>>>> Official FreeSWITCH Sites
>>>>>>> http://www.freeswitch.org
>>>>>>> http://wiki.freeswitch.org
>>>>>>> http://www.cluecon.com
>>>>>>>
>>>>>>> FreeSWITCH-users mailing list
>>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>>> UNSUBSCRIBE:
>>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>>> http://www.freeswitch.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> _________________________________________________________________________
>>>>>> Professional FreeSWITCH Consulting Services:
>>>>>> consulting at freeswitch.org
>>>>>> http://www.freeswitchsolutions.com
>>>>>>
>>>>>> 
>>>>>> 
>>>>>>
>>>>>> Official FreeSWITCH Sites
>>>>>> http://www.freeswitch.org
>>>>>> http://wiki.freeswitch.org
>>>>>> http://www.cluecon.com
>>>>>>
>>>>>> FreeSWITCH-users mailing list
>>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>>> UNSUBSCRIBE:
>>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>>> http://www.freeswitch.org
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Anthony Minessale II
>>>>>
>>>>> FreeSWITCH http://www.freeswitch.org/
>>>>> ClueCon http://www.cluecon.com/
>>>>> Twitter: http://twitter.com/FreeSWITCH_wire
>>>>>
>>>>> AIM: anthm
>>>>> MSN:anthony_minessale at hotmail.com
>>>>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>>>>> IRC: irc.freenode.net #freeswitch
>>>>>
>>>>> FreeSWITCH Developer Conference
>>>>> sip:888 at conference.freeswitch.org
>>>>> googletalk:conf+888 at conference.freeswitch.org
>>>>> pstn:+19193869900
>>>>>
>>>>>
>>>>> _________________________________________________________________________
>>>>> Professional FreeSWITCH Consulting Services:
>>>>> consulting at freeswitch.org
>>>>> http://www.freeswitchsolutions.com
>>>>>
>>>>> 
>>>>> 
>>>>>
>>>>> Official FreeSWITCH Sites
>>>>> http://www.freeswitch.org
>>>>> http://wiki.freeswitch.org
>>>>> http://www.cluecon.com
>>>>>
>>>>> FreeSWITCH-users mailing list
>>>>> FreeSWITCH-users at lists.freeswitch.org
>>>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>>>> UNSUBSCRIBE:
>>>>> http://lists.freeswitch.org/mailman/options/freeswitch-users
>>>>> http://www.freeswitch.org
>>>>>
>>>>>
>>>>
>>>
>>> _________________________________________________________________________
>>> Professional FreeSWITCH Consulting Services:
>>> consulting at freeswitch.org
>>> http://www.freeswitchsolutions.com
>>>
>>> 
>>> 
>>>
>>> Official FreeSWITCH Sites
>>> http://www.freeswitch.org
>>> http://wiki.freeswitch.org
>>> http://www.cluecon.com
>>>
>>> FreeSWITCH-users mailing list
>>> FreeSWITCH-users at lists.freeswitch.org
>>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>>> http://www.freeswitch.org
>>>
>>>
>>
>>
>> --
>> Anthony Minessale II
>>
>> FreeSWITCH http://www.freeswitch.org/
>> ClueCon http://www.cluecon.com/
>> Twitter: http://twitter.com/FreeSWITCH_wire
>>
>> AIM: anthm
>> MSN:anthony_minessale at hotmail.com
>> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
>> IRC: irc.freenode.net #freeswitch
>>
>> FreeSWITCH Developer Conference
>> sip:888 at conference.freeswitch.org
>> googletalk:conf+888 at conference.freeswitch.org
>> pstn:+19193869900
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> 
>> 
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://wiki.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>>
>>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> 
> 
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://wiki.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>
>

-- 
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/
Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm
MSN:anthony_minessale at hotmail.com
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org
googletalk:conf+888 at conference.freeswitch.org
pstn:+19193869900
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20121025/12f32e55/attachment-0001.html 