[Freeswitch-users] Are there any known memory leaks with reloadxml and large XML configs?

[Nathan Neulinger]

The indication I'm reacting to of leak isn't "one in memory".... The below is showing FIFTY THREE of them.

-- Nathan

On 04/14/2016 12:38 PM, Michael Jerris wrote:
> A parsed xml tree in a core file would be normal.  We always keep one in memory, thats why you have to reload to get a new one, so it would be in the core file every time, not an indication of a leak.
>
>
>> On Apr 14, 2016, at 12:49 PM, Nathan Neulinger <nneul at mst.edu> wrote:
>>
>> I've got a persistent issue with a slow (sometimes not that slow) memory leak in FS. Our environment does frequent
>> reconfigs built around reloadxml and when the leak gets bad enough, I start getting random reports of slipping audio
>> sync. I've put in weekly forced failovers, but if we have a week with more changes than typical (we're still in middle
>> of mass migration off of CCM) - the leak growth can be too fast:
>>
>> freeswi+ 17977 12.1 27.2 15247652 2213648 ?    S<l  Apr02 2155:56 /local/freeswitch/server/bin/freeswitch -u freeswitch
>> -core -cfgname freeswitch-prod.xml -base /local/freeswitch/server -conf /local/freeswitch/server/conf -db
>> /local/freeswitch/server/db -run /local/freeswitch/data -log /local/freeswitch/data -rp -nf -nc -nonat -nonatmap
>>
>>
>> The reason I ask if the leak is related to reloadxml - when doing some analysis on the resulting core dump on the (no
>> longer active) server, I'm finding examples like this when doing a 'strings corefile | sort | uniq -c | sort -n':
>>
>> ...
>>    74642 !-- type(skinny) --
>>    76811 /buttons
>>    76811 "InvalidHash
>>    76812 "a1-hash
>>    76836 /skinny
>>    76853 buttons
>> ...
>>   105277 !-- password won't be used by cisco devices --
>> ...
>>   105277 "vm-mailfrom
>>   105277 "vm-password
>>
>> [root at freesw-p1 data]# grep -c InvalidHash freeswitch-prod.xml.fsxml
>> 1431
>>
>> [root at freesw-p1 data]# ls -al *.fsxml
>> -rw------- 1 freeswitch freeswitch 4985669 Apr 14 08:28 freeswitch-prod.xml.fsxml
>> [root at freesw-p1 data]# wc -l *.fsxml
>> 105025 freeswitch-prod.xml.fsxml
>>
>
> this is just a copy of the compiled xml file.  it is 100k lines long.  I'm guessing you have lots of static users in here.
>
>>
>
> None of this would indicate what you are suggesting necessarily
>
>> It's like it's leaking large numbers of complete copies of the XML. When I look directly at the core dump, it looks to
>> me like the strings are in the parsed state of the XML. (Below slightly masked copy and paste from viewing dump with less.)
>
> Thats what i would accept.
>
>>
>>
>> -------------
>> ^@!-- xxx-xxx-xxxx --^@
>> ^@user^@id^@"xxxxxxxxxxxxxxxxxxxxxxxx^@^@
>>      ^@params^@
>>          ^@!-- password won't be used by cisco devices --^@
>>          ^@param^@name^@"password^@ value^@"XXXXXXXXXXXXXXXXX^@^@^@
>>          ^@param^@name^@"vm-mailfrom^@ value^@"voicemail at mst.edu^@^@^@
>> -----------
>>
>> Would really appreciate any ideas on how I might mitigate this leaking or if there is anything that could be done to
>> help diagnose it further to help address the underlying issue.
>>
>>
>> I'm happy to open a JIRA on this, but will NOT be able to test this with latest master as I can't just experiment with
>> the live production environment.
>>
>> -- Nathan
>>
>> ------------------------------------------------------------
>> Nathan Neulinger                       nneul at mst.edu
>> Missouri S&T Information Technology    (573) 612-1412
>> System Administrator - Architect
>>
>> _________________________________________________________________________
>> Professional FreeSWITCH Consulting Services:
>> consulting at freeswitch.org
>> http://www.freeswitchsolutions.com
>>
>> Official FreeSWITCH Sites
>> http://www.freeswitch.org
>> http://confluence.freeswitch.org
>> http://www.cluecon.com
>>
>> FreeSWITCH-users mailing list
>> FreeSWITCH-users at lists.freeswitch.org
>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
>> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
>> http://www.freeswitch.org
>
>
> _________________________________________________________________________
> Professional FreeSWITCH Consulting Services:
> consulting at freeswitch.org
> http://www.freeswitchsolutions.com
>
> Official FreeSWITCH Sites
> http://www.freeswitch.org
> http://confluence.freeswitch.org
> http://www.cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>

-- 
------------------------------------------------------------
Nathan Neulinger                       nneul at mst.edu
Missouri S&T Information Technology    (573) 612-1412
System Administrator - Architect