Hi, We run FreeSWITCH on Amazon AWS EC2, using the c5.xlarge machine size, meaning 4 CPUs with hyper-threading, sums up to 8 vCpu. In time of high-traffic, the system is reaching 100% CPU at about 400 concurrent sessions, which is typically 200 calls (2 sessions per call). RTP is sent as G-711. SIP signaling is this test is UDP (no TCP or other protocols). These numbers are after reducing the log level and moving the core-dsn to memory. The dialplan in this test is very simple. ulimit values are as recommended in FreeSWITCH documentation. On thread level, we see many threads that consume <1.5% CPU. Several threads consume more, apparently the signaling. Question: Do these numbers make sense? We expected to see much more traffic before the CPU is congested. Are there simple tricks to reduce the CPU usage (as we did with changing the log level) Is there a way to analyze the CPU consumption so we can try and improve? Thanks
my test results using sipp script + FreeSWITCH docker container + PCMU on c5.4xlarge instance only audio stream. Each call duration is 200 seconds. Tested Kazoo all-in-one install. So, here some overhead exit for Elang components. According to load test on c5.4xlarge Amazon instance after 282 seconds is reached 1258 simultaneous calls at 6 calls per second rate. During the test all calls and registrations are successful. During the test CPU load is reached 79% value, memory usage 7Gb, and 150 mb/s total network usage over all docker containers on "all-in-one" Kazoo configuration. Sergey On Sun, Oct 11, 2020 at 1:51 PM <ziv@ip-com.co.il> wrote:
Hi,
We run FreeSWITCH on Amazon AWS EC2, using the c5.xlarge machine size, meaning 4 CPUs with hyper-threading, sums up to 8 vCpu.
In time of high-traffic, the system is reaching 100% CPU at about 400 concurrent sessions, which is typically 200 calls (2 sessions per call).
RTP is sent as G-711. SIP signaling is this test is UDP (no TCP or other protocols).
These numbers are after reducing the log level and moving the core-dsn to memory.
The dialplan in this test is very simple.
ulimit values are as recommended in FreeSWITCH documentation.
On thread level, we see many threads that consume <1.5% CPU. Several threads consume more, apparently the signaling.
Question:
Do these numbers make sense? We expected to see much more traffic before the CPU is congested.
Are there simple tricks to reduce the CPU usage (as we did with changing the log level)
Is there a way to analyze the CPU consumption so we can try and improve?
Thanks _________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
Assuming it’s 1258 one-session calls, and remembering that c5.4xlarge has 4 times more CPU than c5.xlarge, and taking into account that my test was on 30 seconds call vs. 200 seconds call in your test, it seems that our findings are similar. This makes the FreeSWITCH deployment costly. There must be a way to reduce CPU consumption. � From: FreeSWITCH-users <freeswitch-users-bounces@lists.freeswitch.org> On Behalf Of Sergey Safarov Sent: Monday, October 12, 2020 10:30 AM To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] Freeswitch CPU consumption on AWS EC2 � my test results using sipp script �+ �FreeSWITCH docker container �+ PCMU on �c5.4xlarge instance only audio stream. Each call duration is 200 seconds. Tested Kazoo all-in-one install. So, here some overhead exit for Elang components. According to load test on c5.4xlarge Amazon instance after 282 seconds is reached 1258 simultaneous calls at 6 calls per second rate. During the test all calls and registrations are successful. During the test CPU load is reached 79% value, memory usage 7Gb, and 150 mb/s total network usage over all docker containers on "all-in-one" Kazoo configuration. � Sergey � On Sun, Oct 11, 2020 at 1:51 PM <ziv@ip-com.co.il <mailto:ziv@ip-com.co.il> > wrote: Hi, � We run FreeSWITCH on Amazon AWS EC2, using the c5.xlarge machine size, meaning 4 CPUs with hyper-threading, sums up to 8 vCpu. In time of high-traffic, the system is reaching 100% CPU at about 400 concurrent sessions, which is typically 200 calls (2 sessions per call). RTP is sent as G-711. SIP signaling is this test is UDP (no TCP or other protocols). These numbers are after reducing the log level and moving the core-dsn to memory. The dialplan in this test is very simple. ulimit values are as recommended in FreeSWITCH documentation. On thread level, we see many threads that consume <1.5% CPU. Several threads consume more, apparently the signaling. � Question: Do these numbers make sense? We expected to see much more traffic before the CPU is congested. Are there simple tricks to reduce the CPU usage (as we did with changing the log level) Is there a way to analyze the CPU consumption so we can try and improve? � Thanks _________________________________________________________________________ The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform. Join our online community to chat in real time https://signalwire.community Professional FreeSWITCH Services sales@freeswitch.com <mailto:sales@freeswitch.com> https://freeswitch.com Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
You can double decrease AWS cost by using "reserved" instances or "spot" instances. Also, important call duration. Think reals calls longer, I think about 1-2 mins average. I expect, if you increase call duration two times (decrease CPS two times), then you will increase the number of calls. Also important to evaluate how complex your dialplan. Is used codec conversion or not. All of this may impact total calls handled by FreeSwitch. Sergey On Mon, Oct 12, 2020 at 12:00 PM <ziv@ip-com.co.il> wrote:
Assuming it’s 1258 one-session calls, and remembering that c5.4xlarge has 4 times more CPU than c5.xlarge, and taking into account that my test was on 30 seconds call vs. 200 seconds call in your test, it seems that our findings are similar.
This makes the FreeSWITCH deployment costly. There must be a way to reduce CPU consumption.
*From:* FreeSWITCH-users <freeswitch-users-bounces@lists.freeswitch.org> *On Behalf Of *Sergey Safarov *Sent:* Monday, October 12, 2020 10:30 AM *To:* FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> *Subject:* Re: [Freeswitch-users] Freeswitch CPU consumption on AWS EC2
my test results using sipp script + FreeSWITCH docker container + PCMU on c5.4xlarge instance only audio stream. Each call duration is 200 seconds.
Tested Kazoo all-in-one install. So, here some overhead exit for Elang components.
According to load test on c5.4xlarge Amazon instance after 282 seconds is reached 1258 simultaneous calls at 6 calls per second rate. During the test all calls and registrations are successful. During the test CPU load is reached 79% value, memory usage 7Gb, and 150 mb/s total network usage over all docker containers on "all-in-one" Kazoo configuration.
Sergey
On Sun, Oct 11, 2020 at 1:51 PM <ziv@ip-com.co.il> wrote:
Hi,
We run FreeSWITCH on Amazon AWS EC2, using the c5.xlarge machine size, meaning 4 CPUs with hyper-threading, sums up to 8 vCpu.
In time of high-traffic, the system is reaching 100% CPU at about 400 concurrent sessions, which is typically 200 calls (2 sessions per call).
RTP is sent as G-711. SIP signaling is this test is UDP (no TCP or other protocols).
These numbers are after reducing the log level and moving the core-dsn to memory.
The dialplan in this test is very simple.
ulimit values are as recommended in FreeSWITCH documentation.
On thread level, we see many threads that consume <1.5% CPU. Several threads consume more, apparently the signaling.
Question:
Do these numbers make sense? We expected to see much more traffic before the CPU is congested.
Are there simple tricks to reduce the CPU usage (as we did with changing the log level)
Is there a way to analyze the CPU consumption so we can try and improve?
Thanks
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
Are you using the call centre module? If so, we've ran into issues with the FS SQLite thread (single child process) becoming CPU bound, you can fix it by using an RDBMS such as PGSQL which is multi-threaded. ________________________________ From: FreeSWITCH-users <freeswitch-users-bounces@lists.freeswitch.org> on behalf of ziv@ip-com.co.il <ziv@ip-com.co.il> Sent: 12 October 2020 10:34 To: 'FreeSWITCH Users Help' <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] Freeswitch CPU consumption on AWS EC2 Assuming it’s 1258 one-session calls, and remembering that c5.4xlarge has 4 times more CPU than c5.xlarge, and taking into account that my test was on 30 seconds call vs. 200 seconds call in your test, it seems that our findings are similar. This makes the FreeSWITCH deployment costly. There must be a way to reduce CPU consumption. From: FreeSWITCH-users <freeswitch-users-bounces@lists.freeswitch.org> On Behalf Of Sergey Safarov Sent: Monday, October 12, 2020 10:30 AM To: FreeSWITCH Users Help <freeswitch-users@lists.freeswitch.org> Subject: Re: [Freeswitch-users] Freeswitch CPU consumption on AWS EC2 my test results using sipp script + FreeSWITCH docker container + PCMU on c5.4xlarge instance only audio stream. Each call duration is 200 seconds. Tested Kazoo all-in-one install. So, here some overhead exit for Elang components. According to load test on c5.4xlarge Amazon instance after 282 seconds is reached 1258 simultaneous calls at 6 calls per second rate. During the test all calls and registrations are successful. During the test CPU load is reached 79% value, memory usage 7Gb, and 150 mb/s total network usage over all docker containers on "all-in-one" Kazoo configuration. Sergey On Sun, Oct 11, 2020 at 1:51 PM <ziv@ip-com.co.il<mailto:ziv@ip-com.co.il>> wrote: Hi, We run FreeSWITCH on Amazon AWS EC2, using the c5.xlarge machine size, meaning 4 CPUs with hyper-threading, sums up to 8 vCpu. In time of high-traffic, the system is reaching 100% CPU at about 400 concurrent sessions, which is typically 200 calls (2 sessions per call). RTP is sent as G-711. SIP signaling is this test is UDP (no TCP or other protocols). These numbers are after reducing the log level and moving the core-dsn to memory. The dialplan in this test is very simple. ulimit values are as recommended in FreeSWITCH documentation. On thread level, we see many threads that consume <1.5% CPU. Several threads consume more, apparently the signaling. Question: Do these numbers make sense? We expected to see much more traffic before the CPU is congested. Are there simple tricks to reduce the CPU usage (as we did with changing the log level) Is there a way to analyze the CPU consumption so we can try and improve? Thanks _________________________________________________________________________ The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform. Join our online community to chat in real time https://signalwire.community Professional FreeSWITCH Services sales@freeswitch.com<mailto:sales@freeswitch.com> https://freeswitch.com Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org<mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
participants (3)
-
Sergey Safarov
-
Shaun Stokes
-
ziv＠ip-com.co.il