High CPU usage and Slow BLFs with only 8 channels?
Hello all, I'm having some interesting problems with our Freeswitch system. I have found Tim's posting athttps://lists.freeswitch.org/pipermail/freeswitch-users/2012-February/080422... It seems very similar but does not help me... Wish it did. Hardware: Dell R610, dual 6 core Xeon X5680 @ 3.3GHz with 10 Gig of RAM Network: Dual 10Gig Fiber to 40Gig fabric Databases: also Dell R610 with dual 10Gig and Mariadb 10.5.9 NO ISSUES - VERY FAST - ULTRA LOW LATENCY We have 2 different FS systems 1) One acting as an SBC of sorts - process SIP/RTP 2) One acting as a PBX - process SIP/RTP Providers are connected to SBC (public - context) PBX (sbc - context) connects to SBC (internal - context) End Points connect to PBX (device - context) picographically: ITSP --- SBC --- PBX --- EndPoints The machine running FreeSWITCH as an SBC is fine - 50 calls (100 sessions) and near zero load average - CPU idle 98% - ALL GREAT About 100 devices registered, all NAT'd, approx 50 subscription via BLF buttons Registration period is maxed to 60 minutes and most devices use the default of 60 minutes Typically, We see only a few CPS, maybe 5 if it's really busy All our trunks are SIP. Only using PCMU/8000 codec. We use MARIADB in the core - as well as for sofia, voicemail, and anything else that can use it. Config, Directory and dialplan are done using CURL and handled via apache2 on the same server over 127.0.0.1. I keep hearing about FS systems handling thousands of users on one box. I'm nowhere near that and it seems to be maxed out. Freeswitch on both systems is Version 1.10.6-release git 1ff9d0a 2021-03-25 13:16:09Z 64bit So... The problem The machine running FreeSWITCH as a PBX seems to be struggling during periods of any usage - even only 1 call (2 channels). Just a moment ago, load average is 4.5 - CPU Idle is 65% - 8 sessions active - 315 sessions since startup - 45 threads - 8 hours running The audio seems fine - no complaints reported.. There is a very noticable chocking on the BLFs though... Every few minutes, the BLFs just freeze - yes - freeze... Then they catch up for example, for testing this, I have programmed a BLF on one phone. I run manually "luarun /etc/freeswitch/scripts/on.lua *8896@domain.com" - the lights turns red then after a few seconds, I run manually "luarun /etc/freeswitch/scripts/off.lua *8896@domain.com" - the light turns green Sometimes, when I run these manually, the lights take 30 seconds to change - sometimes - usually, under a second. They change the state of the subscribe target Here is the LUA script to turn the light ON(RED) ############################################################## #!/lua local random = math.random math.randomseed(os.time()) local function uuid() local template ='xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx' return string.gsub(template, '[xy]', function (c) local v = (c == 'x') and random(0, 0xf) or random(8, 0xb) return string.format('%x', v) end) end local function SendCallDirectorBLFevent() Uuid = uuid() local event = freeswitch.Event("PRESENCE_IN") event:addHeader("proto", "sip") event:addHeader("event_type", "presence") event:addHeader("alt_event_type", "dialog") event:addHeader("Presence-Call-Direction", "outbound") event:addHeader("from", argv[1]) event:addHeader("login", argv[1]) event:addHeader("unique-id", Uuid ) event:addHeader("answer-state", "confirmed") event:fire(); end SendCallDirectorBLFevent() ############################################################## The issue is the choking on the BLFs and the HIGH load avg for such small amounts of calls on the PBX - the SBC seems fine. SO..... I have been trying to resolve this for about a month - looking everywhere I could. I have monitored the network stats, disk IO stats, cpu usage, processes, etc.... I don't see anything different between the SBC and PBX stats... I would like the BLFs to show the correct state a little faster (more like half a second, not 10-60 seconds after the event) and would I like to know how to fix the high CPU load on the PBX. I have confirmed 2 things today: 1) it's not using mod_mariadb as I put back the ODBC and not help 2) this is seems most noticeable when new calls come in. The BLFs pause to 5-8 seconds and continue - there is a noticeable spike in load avg when calls keep coming in for 5 minutes... Any information that would be helpful? Thanks, Jerry
just keep in mind registration and blf subs etc count as CPS (sip calls/sec) why not offload that stuff to opensips? Sent from my iPhone
On Sep 26, 2021, at 09:18, Jerry Kendall <Jerry.Kendall@bishophosting.com> wrote:
Hello all,
I'm having some interesting problems with our Freeswitch system.
I have found Tim's posting at https://lists.freeswitch.org/pipermail/freeswitch-users/2012-February/080422...
It seems very similar but does not help me... Wish it did.
Hardware: Dell R610, dual 6 core Xeon X5680 @ 3.3GHz with 10 Gig of RAM Network: Dual 10Gig Fiber to 40Gig fabric
Databases: also Dell R610 with dual 10Gig and Mariadb 10.5.9 NO ISSUES - VERY FAST - ULTRA LOW LATENCY
We have 2 different FS systems 1) One acting as an SBC of sorts - process SIP/RTP 2) One acting as a PBX - process SIP/RTP
Providers are connected to SBC (public - context) PBX (sbc - context) connects to SBC (internal - context) End Points connect to PBX (device - context)
picographically:
ITSP --- SBC --- PBX --- EndPoints
The machine running FreeSWITCH as an SBC is fine - 50 calls (100 sessions) and near zero load average - CPU idle 98% - ALL GREAT
About 100 devices registered, all NAT'd, approx 50 subscription via BLF buttons Registration period is maxed to 60 minutes and most devices use the default of 60 minutes
Typically, We see only a few CPS, maybe 5 if it's really busy
All our trunks are SIP. Only using PCMU/8000 codec.
We use MARIADB in the core - as well as for sofia, voicemail, and anything else that can use it.
Config, Directory and dialplan are done using CURL and handled via apache2 on the same server over 127.0.0.1.
I keep hearing about FS systems handling thousands of users on one box. I'm nowhere near that and it seems to be maxed out.
Freeswitch on both systems is Version 1.10.6-release git 1ff9d0a 2021-03-25 13:16:09Z 64bit
So... The problem
The machine running FreeSWITCH as a PBX seems to be struggling during periods of any usage - even only 1 call (2 channels).
Just a moment ago, load average is 4.5 - CPU Idle is 65% - 8 sessions active - 315 sessions since startup - 45 threads - 8 hours running
The audio seems fine - no complaints reported.. There is a very noticable chocking on the BLFs though... Every few minutes, the BLFs just freeze - yes - freeze... Then they catch up
for example, for testing this, I have programmed a BLF on one phone.
I run manually "luarun /etc/freeswitch/scripts/on.lua *8896@domain.com" - the lights turns red then after a few seconds, I run manually "luarun /etc/freeswitch/scripts/off.lua *8896@domain.com" - the light turns green Sometimes, when I run these manually, the lights take 30 seconds to change - sometimes - usually, under a second.
They change the state of the subscribe target Here is the LUA script to turn the light ON(RED)
############################################################## #!/lua
local random = math.random math.randomseed(os.time())
local function uuid() local template ='xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx' return string.gsub(template, '[xy]', function (c) local v = (c == 'x') and random(0, 0xf) or random(8, 0xb) return string.format('%x', v) end) end
local function SendCallDirectorBLFevent()
Uuid = uuid()
local event = freeswitch.Event("PRESENCE_IN") event:addHeader("proto", "sip") event:addHeader("event_type", "presence") event:addHeader("alt_event_type", "dialog") event:addHeader("Presence-Call-Direction", "outbound") event:addHeader("from", argv[1]) event:addHeader("login", argv[1]) event:addHeader("unique-id", Uuid ) event:addHeader("answer-state", "confirmed") event:fire(); end
SendCallDirectorBLFevent() ##############################################################
The issue is the choking on the BLFs and the HIGH load avg for such small amounts of calls on the PBX - the SBC seems fine.
SO.....
I have been trying to resolve this for about a month - looking everywhere I could. I have monitored the network stats, disk IO stats, cpu usage, processes, etc.... I don't see anything different between the SBC and PBX stats...
I would like the BLFs to show the correct state a little faster (more like half a second, not 10-60 seconds after the event) and would I like to know how to fix the high CPU load on the PBX.
I have confirmed 2 things today: 1) it's not using mod_mariadb as I put back the ODBC and not help 2) this is seems most noticeable when new calls come in. The BLFs pause to 5-8 seconds and continue - there is a noticeable spike in load avg when calls keep coming in for 5 minutes...
Any information that would be helpful?
Thanks, Jerry _________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
Thanks Ken, I have actually played with putting Kamailio in between the End Devices and FS but could not get Kamailio to do the REG/SUB and let all other things hit FS... I could'nt even get the SUBs to work of Kamailio.. I get the REG working but that was all. Do you have a cfg file that can get me started ? Does anyone ? Thanks, Jerry On 9/26/2021 4:25 PM, Ken Rice wrote:
just keep in mind registration and blf subs etc count as CPS (sip calls/sec) why not offload that stuff to opensips?
Sent from my iPhone
On Sep 26, 2021, at 09:18, Jerry Kendall <Jerry.Kendall@bishophosting.com> wrote:
Hello all,
I'm having some interesting problems with our Freeswitch system.
I have found Tim's posting athttps://lists.freeswitch.org/pipermail/freeswitch-users/2012-February/080422...
It seems very similar but does not help me... Wish it did.
Hardware: Dell R610, dual 6 core Xeon X5680 @ 3.3GHz with 10 Gig of RAM Network: Dual 10Gig Fiber to 40Gig fabric
Databases: also Dell R610 with dual 10Gig and Mariadb 10.5.9 NO ISSUES - VERY FAST - ULTRA LOW LATENCY
We have 2 different FS systems 1) One acting as an SBC of sorts - process SIP/RTP 2) One acting as a PBX - process SIP/RTP
Providers are connected to SBC (public - context) PBX (sbc - context) connects to SBC (internal - context) End Points connect to PBX (device - context)
picographically:
ITSP --- SBC --- PBX --- EndPoints
The machine running FreeSWITCH as an SBC is fine - 50 calls (100 sessions) and near zero load average - CPU idle 98% - ALL GREAT
About 100 devices registered, all NAT'd, approx 50 subscription via BLF buttons Registration period is maxed to 60 minutes and most devices use the default of 60 minutes
Typically, We see only a few CPS, maybe 5 if it's really busy
All our trunks are SIP. Only using PCMU/8000 codec.
We use MARIADB in the core - as well as for sofia, voicemail, and anything else that can use it.
Config, Directory and dialplan are done using CURL and handled via apache2 on the same server over 127.0.0.1.
I keep hearing about FS systems handling thousands of users on one box. I'm nowhere near that and it seems to be maxed out.
Freeswitch on both systems is Version 1.10.6-release git 1ff9d0a 2021-03-25 13:16:09Z 64bit
So... The problem
The machine running FreeSWITCH as a PBX seems to be struggling during periods of any usage - even only 1 call (2 channels).
Just a moment ago, load average is 4.5 - CPU Idle is 65% - 8 sessions active - 315 sessions since startup - 45 threads - 8 hours running
The audio seems fine - no complaints reported.. There is a very noticable chocking on the BLFs though... Every few minutes, the BLFs just freeze - yes - freeze... Then they catch up
for example, for testing this, I have programmed a BLF on one phone.
I run manually "luarun /etc/freeswitch/scripts/on.lua *8896@domain.com" - the lights turns red then after a few seconds, I run manually "luarun /etc/freeswitch/scripts/off.lua *8896@domain.com" - the light turns green Sometimes, when I run these manually, the lights take 30 seconds to change - sometimes - usually, under a second.
They change the state of the subscribe target Here is the LUA script to turn the light ON(RED)
############################################################## #!/lua
local random = math.random math.randomseed(os.time())
local function uuid() local template ='xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx' return string.gsub(template, '[xy]', function (c) local v = (c == 'x') and random(0, 0xf) or random(8, 0xb) return string.format('%x', v) end) end
local function SendCallDirectorBLFevent()
Uuid = uuid()
local event = freeswitch.Event("PRESENCE_IN") event:addHeader("proto", "sip") event:addHeader("event_type", "presence") event:addHeader("alt_event_type", "dialog") event:addHeader("Presence-Call-Direction", "outbound") event:addHeader("from", argv[1]) event:addHeader("login", argv[1]) event:addHeader("unique-id", Uuid ) event:addHeader("answer-state", "confirmed") event:fire(); end
SendCallDirectorBLFevent() ##############################################################
The issue is the choking on the BLFs and the HIGH load avg for such small amounts of calls on the PBX - the SBC seems fine.
SO.....
I have been trying to resolve this for about a month - looking everywhere I could. I have monitored the network stats, disk IO stats, cpu usage, processes, etc.... I don't see anything different between the SBC and PBX stats...
I would like the BLFs to show the correct state a little faster (more like half a second, not 10-60 seconds after the event) and would I like to know how to fix the high CPU load on the PBX.
I have confirmed 2 things today: 1) it's not using mod_mariadb as I put back the ODBC and not help 2) this is seems most noticeable when new calls come in. The BLFs pause to 5-8 seconds and continue - there is a noticeable spike in load avg when calls keep coming in for 5 minutes...
Any information that would be helpful?
Thanks, Jerry _________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
Sorry i dont have a config. However with OpenSips they have this midregister proxy option that eats a lot of the registration load for nat busting. you might read up on that and see if that helps any Sent from my iPhone
On Sep 26, 2021, at 20:01, Jerry Kendall <Jerry.Kendall@bishophosting.com> wrote:
Thanks Ken,
I have actually played with putting Kamailio in between the End Devices and FS but could not get Kamailio to do the REG/SUB and let all other things hit FS... I could'nt even get the SUBs to work of Kamailio..
I get the REG working but that was all.
Do you have a cfg file that can get me started ? Does anyone ?
Thanks, Jerry
On 9/26/2021 4:25 PM, Ken Rice wrote: just keep in mind registration and blf subs etc count as CPS (sip calls/sec) why not offload that stuff to opensips?
Sent from my iPhone
On Sep 26, 2021, at 09:18, Jerry Kendall <Jerry.Kendall@bishophosting.com> wrote:
Hello all,
I'm having some interesting problems with our Freeswitch system.
I have found Tim's posting athttps://lists.freeswitch.org/pipermail/freeswitch-users/2012-February/080422...
It seems very similar but does not help me... Wish it did.
Hardware: Dell R610, dual 6 core Xeon X5680 @ 3.3GHz with 10 Gig of RAM Network: Dual 10Gig Fiber to 40Gig fabric
Databases: also Dell R610 with dual 10Gig and Mariadb 10.5.9 NO ISSUES - VERY FAST - ULTRA LOW LATENCY
We have 2 different FS systems 1) One acting as an SBC of sorts - process SIP/RTP 2) One acting as a PBX - process SIP/RTP
Providers are connected to SBC (public - context) PBX (sbc - context) connects to SBC (internal - context) End Points connect to PBX (device - context)
picographically:
ITSP --- SBC --- PBX --- EndPoints
The machine running FreeSWITCH as an SBC is fine - 50 calls (100 sessions) and near zero load average - CPU idle 98% - ALL GREAT
About 100 devices registered, all NAT'd, approx 50 subscription via BLF buttons Registration period is maxed to 60 minutes and most devices use the default of 60 minutes
Typically, We see only a few CPS, maybe 5 if it's really busy
All our trunks are SIP. Only using PCMU/8000 codec.
We use MARIADB in the core - as well as for sofia, voicemail, and anything else that can use it.
Config, Directory and dialplan are done using CURL and handled via apache2 on the same server over 127.0.0.1.
I keep hearing about FS systems handling thousands of users on one box. I'm nowhere near that and it seems to be maxed out.
Freeswitch on both systems is Version 1.10.6-release git 1ff9d0a 2021-03-25 13:16:09Z 64bit
So... The problem
The machine running FreeSWITCH as a PBX seems to be struggling during periods of any usage - even only 1 call (2 channels).
Just a moment ago, load average is 4.5 - CPU Idle is 65% - 8 sessions active - 315 sessions since startup - 45 threads - 8 hours running
The audio seems fine - no complaints reported.. There is a very noticable chocking on the BLFs though... Every few minutes, the BLFs just freeze - yes - freeze... Then they catch up
for example, for testing this, I have programmed a BLF on one phone.
I run manually "luarun /etc/freeswitch/scripts/on.lua *8896@domain.com" - the lights turns red then after a few seconds, I run manually "luarun /etc/freeswitch/scripts/off.lua *8896@domain.com" - the light turns green Sometimes, when I run these manually, the lights take 30 seconds to change - sometimes - usually, under a second.
They change the state of the subscribe target Here is the LUA script to turn the light ON(RED)
############################################################## #!/lua
local random = math.random math.randomseed(os.time())
local function uuid() local template ='xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx' return string.gsub(template, '[xy]', function (c) local v = (c == 'x') and random(0, 0xf) or random(8, 0xb) return string.format('%x', v) end) end
local function SendCallDirectorBLFevent()
Uuid = uuid()
local event = freeswitch.Event("PRESENCE_IN") event:addHeader("proto", "sip") event:addHeader("event_type", "presence") event:addHeader("alt_event_type", "dialog") event:addHeader("Presence-Call-Direction", "outbound") event:addHeader("from", argv[1]) event:addHeader("login", argv[1]) event:addHeader("unique-id", Uuid ) event:addHeader("answer-state", "confirmed") event:fire(); end
SendCallDirectorBLFevent() ##############################################################
The issue is the choking on the BLFs and the HIGH load avg for such small amounts of calls on the PBX - the SBC seems fine.
SO.....
I have been trying to resolve this for about a month - looking everywhere I could. I have monitored the network stats, disk IO stats, cpu usage, processes, etc.... I don't see anything different between the SBC and PBX stats...
I would like the BLFs to show the correct state a little faster (more like half a second, not 10-60 seconds after the event) and would I like to know how to fix the high CPU load on the PBX.
I have confirmed 2 things today: 1) it's not using mod_mariadb as I put back the ODBC and not help 2) this is seems most noticeable when new calls come in. The BLFs pause to 5-8 seconds and continue - there is a noticeable spike in load avg when calls keep coming in for 5 minutes...
Any information that would be helpful?
Thanks, Jerry _________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
Hi List,
the sip proxy Kamailio can also act as mid register you can stack the
pieces together, there is not a separate module.
Here is Fred Posner example:
https://github.com/fredposner/scripts/blob/master/kamailio/mid-registrar.cfg
Kind regards
Karsten Horsmann
Ken Rice <krice@freeswitch.org> schrieb am Mo., 27. Sept. 2021, 03:40:
> Sorry i dont have a config. However with OpenSips they have this
> midregister proxy option that eats a lot of the registration load for nat
> busting. you might read up on that and see if that helps any
>
> Sent from my iPhone
>
> > On Sep 26, 2021, at 20:01, Jerry Kendall <
> Jerry.Kendall@bishophosting.com> wrote:
> >
> > Thanks Ken,
> >
> > I have actually played with putting Kamailio in between the End Devices
> and FS but could not get Kamailio to do the REG/SUB and let all other
> things hit FS...
> > I could'nt even get the SUBs to work of Kamailio..
> >
> > I get the REG working but that was all.
> >
> > Do you have a cfg file that can get me started ? Does anyone ?
> >
> > Thanks,
> > Jerry
> >
> >> On 9/26/2021 4:25 PM, Ken Rice wrote:
> >> just keep in mind registration and blf subs etc count as CPS (sip
> calls/sec) why not offload that stuff to opensips?
> >>
> >> Sent from my iPhone
> >>
> >>>> On Sep 26, 2021, at 09:18, Jerry Kendall <
> Jerry.Kendall@bishophosting.com> wrote:
> >>>
> >>>
> >>> Hello all,
> >>>
> >>> I'm having some interesting problems with our Freeswitch system.
> >>>
> >>> I have found Tim's posting athttps://
> lists.freeswitch.org/pipermail/freeswitch-users/2012-February/080422.html
> >>>
> >>> It seems very similar but does not help me... Wish it did.
> >>>
> >>>
> >>> Hardware: Dell R610, dual 6 core Xeon X5680 @ 3.3GHz with 10 Gig of RAM
> >>> Network: Dual 10Gig Fiber to 40Gig fabric
> >>>
> >>> Databases: also Dell R610 with dual 10Gig and Mariadb 10.5.9 NO ISSUES
> - VERY FAST - ULTRA LOW LATENCY
> >>>
> >>>
> >>>
> >>> We have 2 different FS systems
> >>> 1) One acting as an SBC of sorts - process SIP/RTP
> >>> 2) One acting as a PBX - process SIP/RTP
> >>>
> >>> Providers are connected to SBC (public - context)
> >>> PBX (sbc - context) connects to SBC (internal - context)
> >>> End Points connect to PBX (device - context)
> >>>
> >>> picographically:
> >>>
> >>> ITSP --- SBC --- PBX --- EndPoints
> >>>
> >>> The machine running FreeSWITCH as an SBC is fine - 50 calls (100
> sessions) and near zero load average - CPU idle 98% - ALL GREAT
> >>>
> >>> About 100 devices registered, all NAT'd, approx 50 subscription via
> BLF buttons
> >>> Registration period is maxed to 60 minutes and most devices use the
> default of 60 minutes
> >>>
> >>> Typically, We see only a few CPS, maybe 5 if it's really busy
> >>>
> >>> All our trunks are SIP. Only using PCMU/8000 codec.
> >>>
> >>> We use MARIADB in the core - as well as for sofia, voicemail, and
> anything else that can use it.
> >>>
> >>> Config, Directory and dialplan are done using CURL and handled via
> apache2 on the same server over 127.0.0.1.
> >>>
> >>> I keep hearing about FS systems handling thousands of users on one box.
> >>> I'm nowhere near that and it seems to be maxed out.
> >>>
> >>> Freeswitch on both systems is Version 1.10.6-release git 1ff9d0a
> 2021-03-25 13:16:09Z 64bit
> >>>
> >>> So... The problem
> >>>
> >>> The machine running FreeSWITCH as a PBX seems to be struggling during
> periods of any usage - even only 1 call (2 channels).
> >>>
> >>> Just a moment ago, load average is 4.5 - CPU Idle is 65% - 8 sessions
> active - 315 sessions since startup - 45 threads - 8 hours running
> >>>
> >>> The audio seems fine - no complaints reported..
> >>> There is a very noticable chocking on the BLFs though... Every few
> minutes, the BLFs just freeze - yes - freeze... Then they catch up
> >>>
> >>> for example, for testing this, I have programmed a BLF on one phone.
> >>>
> >>> I run manually "luarun /etc/freeswitch/scripts/on.lua *
> 8896@domain.com" - the lights turns red
> >>> then after a few seconds,
> >>> I run manually "luarun /etc/freeswitch/scripts/off.lua *
> 8896@domain.com" - the light turns green
> >>> Sometimes, when I run these manually, the lights take 30 seconds to
> change - sometimes - usually, under a second.
> >>>
> >>>
> >>> They change the state of the subscribe target
> >>> Here is the LUA script to turn the light ON(RED)
> >>>
> >>> ##############################################################
> >>> #!/lua
> >>>
> >>> local random = math.random
> >>> math.randomseed(os.time())
> >>>
> >>> local function uuid()
> >>> local template ='xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'
> >>> return string.gsub(template, '[xy]', function (c)
> >>> local v = (c == 'x') and random(0, 0xf) or random(8, 0xb)
> >>> return string.format('%x', v)
> >>> end)
> >>> end
> >>>
> >>> local function SendCallDirectorBLFevent()
> >>>
> >>> Uuid = uuid()
> >>>
> >>> local event = freeswitch.Event("PRESENCE_IN")
> >>> event:addHeader("proto", "sip")
> >>> event:addHeader("event_type", "presence")
> >>> event:addHeader("alt_event_type", "dialog")
> >>> event:addHeader("Presence-Call-Direction", "outbound")
> >>> event:addHeader("from", argv[1])
> >>> event:addHeader("login", argv[1])
> >>> event:addHeader("unique-id", Uuid )
> >>> event:addHeader("answer-state", "confirmed")
> >>> event:fire();
> >>> end
> >>>
> >>> SendCallDirectorBLFevent()
> >>> ##############################################################
> >>>
> >>> The issue is the choking on the BLFs and the HIGH load avg for such
> small amounts of calls on the PBX - the SBC seems fine.
> >>>
> >>> SO.....
> >>>
> >>> I have been trying to resolve this for about a month - looking
> everywhere I could.
> >>> I have monitored the network stats, disk IO stats, cpu usage,
> processes, etc.... I don't see anything different between the SBC and PBX
> stats...
> >>>
> >>>
> >>> I would like the BLFs to show the correct state a little faster (more
> like half a second, not 10-60 seconds after the event) and would I like to
> know how to fix the high CPU load on the PBX.
> >>>
> >>> I have confirmed 2 things today:
> >>> 1) it's not using mod_mariadb as I put back the ODBC and not help
> >>> 2) this is seems most noticeable when new calls come in. The BLFs
> pause to 5-8 seconds and continue - there is a noticeable spike in load avg
> when calls keep coming in for 5 minutes...
> >>>
> >>>
> >>> Any information that would be helpful?
> >>>
> >>>
> >>> Thanks,
> >>> Jerry
> >>>
> _________________________________________________________________________
> >>>
> >>> The FreeSWITCH project is sponsored by SignalWire
> https://signalwire.com
> >>> Enhance your FreeSWITCH install with disruptive priced SMS and PSTN
> services.
> >>> Build your next product on our scalable cloud platform.
> >>>
> >>> Join our online community to chat in real time
> https://signalwire.community
> >>>
> >>> Professional FreeSWITCH Services
> >>> sales@freeswitch.com
> >>> https://freeswitch.com
> >>>
> >>> Official FreeSWITCH Sites
> >>> https://freeswitch.com/oss
> >>> https://freeswitch.org/confluence
> >>> https://cluecon.com
> >>>
> >>> FreeSWITCH-users mailing list
> >>> FreeSWITCH-users@lists.freeswitch.org
> >>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> >>> UNSUBSCRIBE:
> http://lists.freeswitch.org/mailman/options/freeswitch-users
> >>> https://freeswitch.com
> >>
> >>
> _________________________________________________________________________
> >>
> >> The FreeSWITCH project is sponsored by SignalWire
> https://signalwire.com
> >> Enhance your FreeSWITCH install with disruptive priced SMS and PSTN
> services.
> >> Build your next product on our scalable cloud platform.
> >>
> >> Join our online community to chat in real time
> https://signalwire.community
> >>
> >> Professional FreeSWITCH Services
> >> sales@freeswitch.com
> >> https://freeswitch.com
> >>
> >> Official FreeSWITCH Sites
> >> https://freeswitch.com/oss
> >> https://freeswitch.org/confluence
> >> https://cluecon.com
> >>
> >> FreeSWITCH-users mailing list
> >> FreeSWITCH-users@lists.freeswitch.org
> >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> >> UNSUBSCRIBE:
> http://lists.freeswitch.org/mailman/options/freeswitch-users
> >> https://freeswitch.com
> >
> >
> >
> > _________________________________________________________________________
> >
> > The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
> > Enhance your FreeSWITCH install with disruptive priced SMS and PSTN
> services.
> > Build your next product on our scalable cloud platform.
> >
> > Join our online community to chat in real time
> https://signalwire.community
> >
> > Professional FreeSWITCH Services
> > sales@freeswitch.com
> > https://freeswitch.com
> >
> > Official FreeSWITCH Sites
> > https://freeswitch.com/oss
> > https://freeswitch.org/confluence
> > https://cluecon.com
> >
> > FreeSWITCH-users mailing list
> > FreeSWITCH-users@lists.freeswitch.org
> > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> > UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> > https://freeswitch.com
>
> _________________________________________________________________________
>
> The FreeSWITCH project is sponsored by SignalWire https://signalwire.com
> Enhance your FreeSWITCH install with disruptive priced SMS and PSTN
> services.
> Build your next product on our scalable cloud platform.
>
> Join our online community to chat in real time
> https://signalwire.community
>
> Professional FreeSWITCH Services
> sales@freeswitch.com
> https://freeswitch.com
>
> Official FreeSWITCH Sites
> https://freeswitch.com/oss
> https://freeswitch.org/confluence
> https://cluecon.com
>
> FreeSWITCH-users mailing list
> FreeSWITCH-users@lists.freeswitch.org
> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users
> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users
> https://freeswitch.com
participants (3)
-
Jerry Kendall
-
Karsten Horsmann
-
Ken Rice