[Freeswitch-users] Freeswitch freezes on increasing call traffic

Anthony Minessale anthony.minessale at gmail.com
Sat Jan 31 08:50:03 PST 2009 

Clearly you have an issue with your javascript code.

You have the Garbage collector blocking in every thread.

Are you doing any endless loops in your code where you do not check
session.ready() as a condition for
continuing the script?

any time session.ready() fails you must immediately exit.

Are you using session.execute to execute long blocking operations like
bridging many calls or entering a conference?
You should avoid doing this as all the collective scripts on the system
share a common Garbage Collector provided by the
JS engine and it can lead to the exact issues you describe if the code is
not properly designed.

What else does you script do that are things provided by FS such as playing
files and executing applications.



On Sat, Jan 31, 2009 at 3:44 AM, shehzad p <pmhshz at gmail.com> wrote:

>
> Hi Anthony,
>
> Freeswitch 1.0.2 was crashed on last test again...
> BT is on http://pastebin.freeswitch.org/6979.
>
> I tried to use scripts/freeswitch-gcore, to capture resident memory, but
> before the command complete the process system hanged up so only half
> output
> was captured.< http://www.nabble.com/file/p21761523/gcore-fs.txt
> gcore-fs.txt >
>
>
> Now I have checkout from trunk and will post back if any new thing found...
>
> Thanks,
> msp
>
> Anthony Minessale-2 wrote:
> >
> > if you are using unix you can use the supplied script
> >
> > scripts/freeswitch-gcore
> >
> > to capture a copy of the resident memory and I can have a look perhaps.
> >
> > Trunk is safe for production as we are in beta stage for a release of
> > 1.0.3
> > at this time.
> >
> >
> >
> > On Fri, Jan 30, 2009 at 9:29 AM, shehzad p <pmhshz at gmail.com> wrote:
> >
> >>
> >> When freeswitch freezes, we can't connect to it to check sps status,
> >> but once we were able to connect and at that time it was showing 0/0
> sps.
> >>
> >> thanks...
> >>
> >> shehzad p wrote:
> >> >
> >> > Thanks, Anthony
> >> >
> >> > In my previous test sps did not changed,
> >> > but in recent test sps was dropped to 0 itself (as below).
> >> > ===============================================================
> >> > UP 0 years, 0 days, 5 hours, 1 minute, 53 seconds, 878 milliseconds,
> >> 190
> >> > microseconds
> >> > 5474 session(s) since startup
> >> > 75 session(s) 0/0
> >> > =============================================================
> >> >
> >> > My system is 32 bit.
> >> > CPU is Intel(R) Xeon(R) CPU X3220  @ 2.40GHz
> >> > And RAM is 4GB
> >> >
> >> > Output of ulimit -a is:
> >> > ulimit -a: (set after first test)
> >> > core file size          (blocks, -c) unlimited
> >> > data seg size           (kbytes, -d) unlimited
> >> > max nice                        (-e) 20
> >> > file size               (blocks, -f) unlimited
> >> > pending signals                 (-i) unlimited
> >> > max locked memory       (kbytes, -l) unlimited
> >> > max memory size         (kbytes, -m) unlimited
> >> > open files                      (-n) 999999
> >> > pipe size            (512 bytes, -p) 8
> >> > POSIX message queues     (bytes, -q) unlimited
> >> > max rt priority                 (-r) unlimited
> >> > stack size              (kbytes, -s) 244
> >> > cpu time               (seconds, -t) unlimited
> >> > max user processes              (-u) unlimited
> >> > virtual memory          (kbytes, -v) unlimited
> >> > file locks                      (-x) unlimited
> >> > ===================================================
> >> >
> >> >
> >> > BTW using trunk on production system is safe?
> >> >
> >> > Warm thanks for kind responses...
> >> >
> >> >
> >> >
> >> > Anthony Minessale-2 wrote:
> >> >>
> >> >> When you get it in that state what do you see when you execute
> >> >>
> >> >> fsctl sps
> >> >>
> >> >> is the sps a very low number?
> >> >>
> >> >> Did the sps drop by itself from the value you originally set it to?
> >> >>
> >> >> Are you using 32 bit?
> >> >>
> >> >> if so try all of these commands in your shell before starting FS
> >> >>
> >> >> ulimit -c unlimited
> >> >> ulimit -d unlimited
> >> >> ulimit -f unlimited
> >> >> ulimit -i unlimited
> >> >> ulimit -n 999999
> >> >> ulimit -q unlimited
> >> >> ulimit -u unlimited
> >> >> ulimit -v unlimited
> >> >> ulimit -x unlimited
> >> >> ulimit -s 244
> >> >> ulimit -l unlimited
> >> >>
> >> >>
> >> >> DO NOT put them in a script unless you source the script with .
> >> >> . myscript or they will be undone instantly when the script exits
> >> >>
> >> >> BTW, I said to try latest trunk not 1.0.2 We can only debug the
> >> >> development
> >> >> code at this point.
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> On Thu, Jan 29, 2009 at 10:06 AM, shehzad p <pmhshz at gmail.com>
> wrote:
> >> >>
> >> >>>
> >> >>> Hi Anthony,
> >> >>>
> >> >>> I found interesting result while testing Freeswitch, and it might be
> >> >>> cause
> >> >>> of freezing out of freeswitch,
> >> >>>
> >> >>> I updated my system (as you told) with latest stable version
> >> Freeswitch
> >> >>> 1.0.2
> >> >>> First of all I set sps to 100,
> >> >>> Then I sends call approximately 100 per seconds, Freeswitch works
> >> fine
> >> >>> and
> >> >>> handles all the calls very well.
> >> >>>
> >> >>> After that I send 130 calls per seconds, and magic happen now,
> >> >>> Freeswitch
> >> >>> handles first 100 calls only.
> >> >>> all the preceding calls were failed (even not appeared in freeswitch
> >> >>> console
> >> >>> why?)
> >> >>>
> >> >>> When I put ngrep trace, System responds with 503 Maximum Calls In
> >> >>> Progress.
> >> >>> (as below)
> >> >>> ###########################################################
> >> >>> #
> >> >>> U FSFSFSFSFS -> GWGWGWGWGW
> >> >>> SIP/2.0 503 Maximum Calls In Progress.
> >> >>> Via: SIP/2.0/UDP GWGWGWGWGW;branch=z9hG4bK53eafabb;rport=5060.
> >> >>> From: "99999" <sip:99999 at GWGWGWGWGW>;tag=as2e10c170.
> >> >>> To: <sip:1234567890 at FSFSFSFSFS >;tag=K3jSUFrDHpmmB.
> >> >>> Call-ID: 0feb229e58afe7be17110deb361dc234 at GWGWGWGWGW.
> >> >>> CSeq: 102 INVITE.
> >> >>> Retry-After: 300.
> >> >>> User-Agent: FreeSWITCH-mod_sofia/1.0.2-exported.
> >> >>> Accept: application/sdp.
> >> >>> Allow: INVITE, ACK, BYE, CANCEL, OPTIONS, PRACK, MESSAGE, SUBSCRIBE,
> >> >>> NOTIFY,
> >> >>> REFER, UPDATE, REGISTER, INFO, PUBLISH.
> >> >>> Supported: timer, precondition, path, replaces.
> >> >>> Allow-Events: talk, presence, dialog, call-info, sla,
> >> >>> include-session-description, presence.winfo, message-summary, refer.
> >> >>> Content-Length: 0.
> >> >>> .
> >> >>>
> #####################################################################
> >> >>>
> >> >>>
> >> >>> Now another issue to note down is that,
> >> >>> After all above happened and active calls comes to zero,
> >> >>> I just make a single call which also fails with response 503 -
> >> Maximum
> >> >>> Calls
> >> >>> In Progress.
> >> >>>
> >> >>>
> >> >>> Is this intended behaviour, should I increase SPS to overcome this.
> >> or
> >> >>> something like bug.
> >> >>>
> >> >>> Please let me know what should be the resolution for this.
> >> >>>
> >> >>> Thanks,
> >> >>> msp
> >> >>>
> >> >>>
> >> >>>
> >> >>> Anthony Minessale-2 wrote:
> >> >>> >
> >> >>> > Also remember,
> >> >>> > Actually completely uninstall and erase /usr/local/freeswitch and
> >> the
> >> >>> > 1.0.1
> >> >>> > source tree and freshly install the new one.
> >> >>> > If you try to upgrade on top of a release with trunk it will cause
> >> >>> more
> >> >>> > problems for you.
> >> >>> >
> >> >>> >
> >> >>> > On Wed, Jan 28, 2009 at 3:11 AM, Ken Rice <krice at suspicious.org>
> >> >>> wrote:
> >> >>> >
> >> >>> >> Upgrade to trunk... Many many issues have been resolved since
> >> 1.0.1
> >> >>> was
> >> >>> >> the
> >> >>> >> current release
> >> >>> >>
> >> >>> >>
> >> >>> >> > From: shehzad p <pmhshz at gmail.com>
> >> >>> >> > Reply-To: <freeswitch-users at lists.freeswitch.org>
> >> >>> >> > Date: Wed, 28 Jan 2009 00:54:13 -0800 (PST)
> >> >>> >> > To: <freeswitch-users at lists.freeswitch.org>
> >> >>> >> > Subject: [Freeswitch-users]  Freeswitch freezes on increasing
> >> call
> >> >>> >> traffic
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > Hi all,
> >> >>> >> >
> >> >>> >> > Yesterday my Freeswitch server faced a problem when call
> traffic
> >> >>> >> increased
> >> >>> >> > to more than 100.
> >> >>> >> >
> >> >>> >> > When I start Freeswitch, it works fine and then after some time
> >> >>> >> > (approximately 15 to 20 minutes)  it stops functioning (means
> no
> >> >>> call
> >> >>> >> is
> >> >>> >> > being processed, no CLI command is working and it just freezes)
> >> >>> until
> >> >>> I
> >> >>> >> > restart the freeswitch.
> >> >>> >> >
> >> >>> >> > I am using Freeswitch 1.0.1.
> >> >>> >> > Debug (gdb) trace as on wiki page
> >> >>> >> >
> >> http://wiki.freeswitch.org/wiki/Debugging_Freeswitch#fs_debug.shis
> >> >>> >> attached
> >> >>> >> > http://www.nabble.com/file/p21701744/fs_debgu.txt fs_debgu.txt
> >> >>> >> > --
> >> >>> >> > View this message in context:
> >> >>> >> >
> >> >>> >>
> >> >>>
> >>
> http://www.nabble.com/Freeswitch-freezes-on-increasing-call-traffic-tp21701744
> >> >>> >> > p21701744.html
> >> >>> >> > Sent from the Freeswitch-users mailing list archive at
> >> Nabble.com.
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > _______________________________________________
> >> >>> >> > Freeswitch-users mailing list
> >> >>> >> > Freeswitch-users at lists.freeswitch.org
> >> >>> >> > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> >> >>> >> >
> >> >>> >> UNSUBSCRIBE:
> >> >>> http://lists.freeswitch.org/mailman/options/freeswitch-users
> >> >>> >> > http://www.freeswitch.org
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >> _______________________________________________
> >> >>> >> Freeswitch-users mailing list
> >> >>> >> Freeswitch-users at lists.freeswitch.org
> >> >>> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> >> >>> >> UNSUBSCRIBE:
> >> >>> http://lists.freeswitch.org/mailman/options/freeswitch-users
> >> >>> >> http://www.freeswitch.org
> >> >>> >>
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > --
> >> >>> > Anthony Minessale II
> >> >>> >
> >> >>> > FreeSWITCH http://www.freeswitch.org/
> >> >>> > ClueCon http://www.cluecon.com/
> >> >>> >
> >> >>> > AIM: anthm
> >> >>> >
> >> MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
> <MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
> >
> >> >>>
> >> <MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
> <MSN%253Aanthony_minessale at hotmail.com<MSN%25253Aanthony_minessale at hotmail.com>
> >
> >> ><
> >> >>>
> >> MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
> <MSN%253Aanthony_minessale at hotmail.com<MSN%25253Aanthony_minessale at hotmail.com>
> >
> >> <MSN%253Aanthony_minessale at hotmail.com<MSN%25253Aanthony_minessale at hotmail.com>
> <MSN%25253Aanthony_minessale at hotmail.com<MSN%2525253Aanthony_minessale at hotmail.com>
> >
> >> >
> >> >>> >
> >> >>> >
> >> >>>
> >> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
> <PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
> >
> >> <PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
> <PAYPAL%253Aanthony.minessale at gmail.com<PAYPAL%25253Aanthony.minessale at gmail.com>
> >
> >> >
> >> >>>
> >> <PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
> <PAYPAL%253Aanthony.minessale at gmail.com<PAYPAL%25253Aanthony.minessale at gmail.com>
> >
> >> <PAYPAL%253Aanthony.minessale at gmail.com<PAYPAL%25253Aanthony.minessale at gmail.com>
> <PAYPAL%25253Aanthony.minessale at gmail.com<PAYPAL%2525253Aanthony.minessale at gmail.com>
> >
> >> >
> >> >>> >
> >> >>> > IRC: irc.freenode.net #freeswitch
> >> >>> >
> >> >>> > FreeSWITCH Developer Conference
> >> >>> >
> >> sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
> <sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
> >
> >> >>>
> >> <sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
> <sip%253A888 at conference.freeswitch.org<sip%25253A888 at conference.freeswitch.org>
> >
> >> ><
> >> >>>
> >> sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
> <sip%253A888 at conference.freeswitch.org<sip%25253A888 at conference.freeswitch.org>
> >
> >> <sip%253A888 at conference.freeswitch.org<sip%25253A888 at conference.freeswitch.org>
> <sip%25253A888 at conference.freeswitch.org<sip%2525253A888 at conference.freeswitch.org>
> >
> >> >
> >> >>> >
> >> >>> > iax:guest at conference.freeswitch.org/888
> >> >>> >
> >> >>>
> >> googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
> <googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
> >
> >> <googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
> <googletalk%253Aconf%252B888 at conference.freeswitch.org<googletalk%25253Aconf%25252B888 at conference.freeswitch.org>
> >
> >> >
> >> >>>
> >> <googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
> <googletalk%253Aconf%252B888 at conference.freeswitch.org<googletalk%25253Aconf%25252B888 at conference.freeswitch.org>
> >
> >> <googletalk%253Aconf%252B888 at conference.freeswitch.org<googletalk%25253Aconf%25252B888 at conference.freeswitch.org>
> <googletalk%25253Aconf%25252B888 at conference.freeswitch.org<googletalk%2525253Aconf%2525252B888 at conference.freeswitch.org>
> >
> >> >
> >> >>> >
> >> >>> > pstn:213-799-1400
> >> >>> >
> >> >>> > _______________________________________________
> >> >>> > Freeswitch-users mailing list
> >> >>> > Freeswitch-users at lists.freeswitch.org
> >> >>> > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> >> >>> >
> >> >>> UNSUBSCRIBE:
> >> http://lists.freeswitch.org/mailman/options/freeswitch-users
> >> >>> > http://www.freeswitch.org
> >> >>> >
> >> >>> >
> >> >>>
> >> >>> --
> >> >>> View this message in context:
> >> >>>
> >>
> http://www.nabble.com/Freeswitch-freezes-on-increasing-call-traffic-tp21701744p21729863.html
> >> >>> Sent from the Freeswitch-users mailing list archive at Nabble.com.
> >> >>>
> >> >>>
> >> >>> _______________________________________________
> >> >>> Freeswitch-users mailing list
> >> >>> Freeswitch-users at lists.freeswitch.org
> >> >>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> >> >>> UNSUBSCRIBE:
> >> http://lists.freeswitch.org/mailman/options/freeswitch-users
> >> >>> http://www.freeswitch.org
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Anthony Minessale II
> >> >>
> >> >> FreeSWITCH http://www.freeswitch.org/
> >> >> ClueCon http://www.cluecon.com/
> >> >>
> >> >> AIM: anthm
> >> >> MSN:anthony_minessale at hotmail.com<MSN%3Aanthony_minessale at hotmail.com>
> >> <MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
> ><
> >> MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
> <MSN%253Aanthony_minessale at hotmail.com<MSN%25253Aanthony_minessale at hotmail.com>
> >
> >> >
> >> >>
> >> GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
> <PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
> >
> >> <PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
> <PAYPAL%253Aanthony.minessale at gmail.com<PAYPAL%25253Aanthony.minessale at gmail.com>
> >
> >> >
> >> >> IRC: irc.freenode.net #freeswitch
> >> >>
> >> >> FreeSWITCH Developer Conference
> >> >> sip:888 at conference.freeswitch.org<sip%3A888 at conference.freeswitch.org>
> >> <sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
> ><
> >> sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
> <sip%253A888 at conference.freeswitch.org<sip%25253A888 at conference.freeswitch.org>
> >
> >> >
> >> >> iax:guest at conference.freeswitch.org/888
> >> >>
> >> googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
> <googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
> >
> >> <googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
> <googletalk%253Aconf%252B888 at conference.freeswitch.org<googletalk%25253Aconf%25252B888 at conference.freeswitch.org>
> >
> >> >
> >> >> pstn:213-799-1400
> >> >>
> >> >> _______________________________________________
> >> >> Freeswitch-users mailing list
> >> >> Freeswitch-users at lists.freeswitch.org
> >> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> >> >> UNSUBSCRIBE:
> >> http://lists.freeswitch.org/mailman/options/freeswitch-users
> >> >> http://www.freeswitch.org
> >> >>
> >> >>
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://www.nabble.com/Freeswitch-freezes-on-increasing-call-traffic-tp21701744p21749375.html
> >> Sent from the Freeswitch-users mailing list archive at Nabble.com.
> >>
> >>
> >> _______________________________________________
> >> Freeswitch-users mailing list
> >> Freeswitch-users at lists.freeswitch.org
> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> >> UNSUBSCRIBE:
> http://lists.freeswitch.org/mailman/options/freeswitch-users
> >> http://www.freeswitch.org
> >>
> >
> >
> >
> > --
> > Anthony Minessale II
> >
> > FreeSWITCH http://www.freeswitch.org/
> > ClueCon http://www.cluecon.com/
> >
> > AIM: anthm
> > MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com><
> MSN%3Aanthony_minessale at hotmail.com<MSN%253Aanthony_minessale at hotmail.com>
> >
> > GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
> <PAYPAL%3Aanthony.minessale at gmail.com<PAYPAL%253Aanthony.minessale at gmail.com>
> >
> > IRC: irc.freenode.net #freeswitch
> >
> > FreeSWITCH Developer Conference
> > sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org><
> sip%3A888 at conference.freeswitch.org<sip%253A888 at conference.freeswitch.org>
> >
> > iax:guest at conference.freeswitch.org/888
> > googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
> <googletalk%3Aconf%2B888 at conference.freeswitch.org<googletalk%253Aconf%252B888 at conference.freeswitch.org>
> >
> > pstn:213-799-1400
> >
> > _______________________________________________
> > Freeswitch-users mailing list
> > Freeswitch-users at lists.freeswitch.org
> > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> > UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> > http://www.freeswitch.org
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Freeswitch-freezes-on-increasing-call-traffic-tp21701744p21761523.html
> Sent from the Freeswitch-users mailing list archive at Nabble.com.
>
>
> _______________________________________________
> Freeswitch-users mailing list
> Freeswitch-users at lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> http://www.freeswitch.org
>



-- 
Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/
ClueCon http://www.cluecon.com/

AIM: anthm
MSN:anthony_minessale at hotmail.com <MSN%3Aanthony_minessale at hotmail.com>
GTALK/JABBER/PAYPAL:anthony.minessale at gmail.com<PAYPAL%3Aanthony.minessale at gmail.com>
IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference
sip:888 at conference.freeswitch.org <sip%3A888 at conference.freeswitch.org>
iax:guest at conference.freeswitch.org/888
googletalk:conf+888 at conference.freeswitch.org<googletalk%3Aconf%2B888 at conference.freeswitch.org>
pstn:213-799-1400
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freeswitch.org/pipermail/freeswitch-users/attachments/20090131/dcaaabe3/attachment-0002.html

More information about the FreeSWITCH-users mailing list