Oh, looks like the timers are also used for streaming local data in read_stream_thread(). Due to this there is always one timer active with 20ms interval. But wait a sec, why is freeswitch periodically trying to stream /opt/freeswitch/sounds/music/8000/ponce-preludio-in-e-major.wav somewhere? Every minute or so? Did I misconfigure it? eaf wrote:

Say, what if that thread is made to suspend on a condition variable in case if there are no timers registered in TIMER_MATRIX? Then, if some other thread comes up and adds its timer into the matrix, it could wake up the timer thread and enjoy accurate timing as needed, on demand? And in-between the calls, when there is no RTP or IVR, it will all go silent? I mean, sitting on a wait queue in the kernel is way better than go back and forth incrementing counters that nobody even needs at the moment?

Anthony Minessale-2 wrote:

...

idle is a 4 letter word to a realtime application.

The core keeps a single high-priority thread to keep 1ms timing and expands that broadcasting to hundreds or thousand of threads who need accurate timing.

Your choppy audio is caused by linksys lying about the packet len that it's using and we set our timer to the wrong speed.

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26620518.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

This is keeping track of a place in the music on hold so your hold music does not start back up at the same place every time. If you don't want to do this it is a module that you don't need to load and you can get your moh from any soundfile at your choice in configuration. Mike On Dec 2, 2009, at 10:35 PM, eaf <erandr-junk@usa.net> wrote:

Oh, looks like the timers are also used for streaming local data in read_stream_thread(). Due to this there is always one timer active with 20ms interval.

But wait a sec, why is freeswitch periodically trying to stream /opt/freeswitch/sounds/music/8000/ponce-preludio-in-e-major.wav somewhere? Every minute or so? Did I misconfigure it?

eaf wrote:

...

Say, what if that thread is made to suspend on a condition variable in case if there are no timers registered in TIMER_MATRIX? Then, if some other thread comes up and adds its timer into the matrix, it could wake up the timer thread and enjoy accurate timing as needed, on demand? And in-between the calls, when there is no RTP or IVR, it will all go silent? I mean, sitting on a wait queue in the kernel is way better than go back and forth incrementing counters that nobody even needs at the moment?

Anthony Minessale-2 wrote:

...

idle is a 4 letter word to a realtime application.

The core keeps a single high-priority thread to keep 1ms timing and expands that broadcasting to hundreds or thousand of threads who need accurate timing.

Your choppy audio is caused by linksys lying about the packet len that it's using and we set our timer to the wrong speed.

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26620518.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org

As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes, it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND" overrides that. Yeah, there is a global timestamp... It's easy to workaround that for RTP who calls switch_micro_time_now()... But if somebody accesses runtime.timestamp directly, it's gonna be tough to grep for that. If only this was C++... I'll play around. Never liked polling too much. Never could've guessed that polling could be so useful for scalability ;) My naive implementation would've pulled timestamp via system calls and would've done sleeping by passing exact interval to select() instead of syncing with a pacing thread. Which would be dead-quiet at idle time, but, of course, would stop scaling at some point due to excessive number of system calls. Thanks. Michael Jerris wrote:

In short. No, you can not for many reasons. The milisecond tic is used throughout the code even when there is not any calls up. You can grep for switch_cond_next if you would like to see where but it is required to keep our global timestamp and for pacing the scheduler among other services that run all the time.

Mike

On Dec 2, 2009, at 7:31 PM, eaf <erandr-junk@usa.net> wrote:

...

Can I reduce resolution of that timer thread 10 times? I mean, I glanced through the code, and see that among others (are there others?) RTP and IVR set up their timers that are subsequently managed by this thread. RTP timers should be eliminated by that setting you've suggested. IVR timers are set at 20ms... So, if the thread is set to wake up every 10ms instead of 1ms it should be able to wake up those IVR timers just fine. Right?

That's a cool design to have one dedicated thread that maintains accurate timing and then broadcasts via condition variables to hundreds of other threads events that they can register for. I'm sure it's one of the reasons why FS scales so much better than Asterisk. But for poor low-end setups that sit in the closet, eat only 6W of power and hardly ever run more than two calls at the same time, can I hack it somehow to be more UNIX- friendly? I.e. make it stuck in select() or recv() when there is nothing to do, call clock_gettime() right from the thread that wants and when it wants to know current time?

Say, what if that thread is made to suspend on a condition variable in case if there are no timers registered in TIMER_MATRIX? Then, if some other thread comes up and adds its timer into the matrix, it could wake up the timer thread and enjoy accurate timing as needed, on demand? And in- between the calls, when there is no RTP or IVR, it will all go silent? I mean, sitting on a wait queue in the kernel is way better than go back and forth incrementing counters that nobody even needs at the moment?

Anthony Minessale-2 wrote:

...

idle is a 4 letter word to a realtime application.

The core keeps a single high-priority thread to keep 1ms timing and expands that broadcasting to hundreds or thousand of threads who need accurate timing.

Your choppy audio is caused by linksys lying about the packet len that it's using and we set our timer to the wrong speed.

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26621005.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

Oh, it's not just one timer thread... Why, why is sql_thread keeps on checking for messages every millisecond? Couldn't there be some signalling implemented that will make the thread suspend on condition variable or a socket/pipe in between? #0 do_sleep (t=1000) at src/switch_time.c:109 #1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0) at src/switch_core_sqldb.c:783 Why does this sofia_profile_worker_thread keeps on looping checking for the queue? Have a semaphore! #0 do_sleep (t=1000) at src/switch_time.c:109 #1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30, obj=0x80f2490) at sofia.c:978 Nothing's happening on the box, but there are three threads that pretend to be actively busy with smth. Others at least sleep for hundreds of milliseconds, not for one. And there is even infrastructure present to do blocking pops: i.e. why couldn't sqldb thread do queue_pop() instead of queue_trypop() intermixed with 1ms sleeps? This looping is such a waste... eaf wrote:

As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes, it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND" overrides that.

Yeah, there is a global timestamp... It's easy to workaround that for RTP who calls switch_micro_time_now()... But if somebody accesses runtime.timestamp directly, it's gonna be tough to grep for that. If only this was C++...

I'll play around. Never liked polling too much. Never could've guessed that polling could be so useful for scalability ;) My naive implementation would've pulled timestamp via system calls and would've done sleeping by passing exact interval to select() instead of syncing with a pacing thread. Which would be dead-quiet at idle time, but, of course, would stop scaling at some point due to excessive number of system calls.

Thanks.

Michael Jerris wrote:

...

In short. No, you can not for many reasons. The milisecond tic is used throughout the code even when there is not any calls up. You can grep for switch_cond_next if you would like to see where but it is required to keep our global timestamp and for pacing the scheduler among other services that run all the time.

Mike

On Dec 2, 2009, at 7:31 PM, eaf <erandr-junk@usa.net> wrote:

...

Can I reduce resolution of that timer thread 10 times? I mean, I glanced through the code, and see that among others (are there others?) RTP and IVR set up their timers that are subsequently managed by this thread. RTP timers should be eliminated by that setting you've suggested. IVR timers are set at 20ms... So, if the thread is set to wake up every 10ms instead of 1ms it should be able to wake up those IVR timers just fine. Right?

That's a cool design to have one dedicated thread that maintains accurate timing and then broadcasts via condition variables to hundreds of other threads events that they can register for. I'm sure it's one of the reasons why FS scales so much better than Asterisk. But for poor low-end setups that sit in the closet, eat only 6W of power and hardly ever run more than two calls at the same time, can I hack it somehow to be more UNIX- friendly? I.e. make it stuck in select() or recv() when there is nothing to do, call clock_gettime() right from the thread that wants and when it wants to know current time?

Say, what if that thread is made to suspend on a condition variable in case if there are no timers registered in TIMER_MATRIX? Then, if some other thread comes up and adds its timer into the matrix, it could wake up the timer thread and enjoy accurate timing as needed, on demand? And in- between the calls, when there is no RTP or IVR, it will all go silent? I mean, sitting on a wait queue in the kernel is way better than go back and forth incrementing counters that nobody even needs at the moment?

Anthony Minessale-2 wrote:

...

idle is a 4 letter word to a realtime application.

The core keeps a single high-priority thread to keep 1ms timing and expands that broadcasting to hundreds or thousand of threads who need accurate timing.

Your choppy audio is caused by linksys lying about the packet len that it's using and we set our timer to the wrong speed.

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26626634.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

If you see that message then your machine/os/combo is having some problems keeping up. It's not the timer missing anything its the monotonic clock detecting a 1 second or more differential from what its next prediction for the time should be. The best way to trigger this would be to suspend FS with control-z or attach to it with gdb blocking the entire process, that 1ms thread would have to miss 1000 iterations to trigger that warning. Btw, that error message is at line 471 not 473 so you are using modified code. Its possible your box has a bad monotonic timer, you can set <param name="disable-monotonic-timing" value="true"/> under <settings> in switch.conf.xml We are now starting to guess you are using some small embedded type platform perhaps? I've run FS even on a nokia n810 and never caused that message to fire. if 1 call can interrupt the cpu enough to cause noticeable issues you might want to consider running the process at a greater priority by using the -hp command line arg or at least nice it Why don't you tell us the whole story about what OS/platform you are using here rather that form conjectures about what is wrong with our code that thousands of people are happy with. On Thu, Dec 3, 2009 at 8:55 AM, eaf <erandr-junk@usa.net> wrote:

Btw, I have these popping up in my logs from time to time:

2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314 (sofia/external/xxxxx@4.68.250.148) Running State Change CS_HANGUP 2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration Detected! Syncing Clock

In this case an incoming call rang to both FS and Asterisk, Asterisk picked up, but the surge of activity made FS timer thread miss a beat or two.

eaf wrote:

...

Oh, it's not just one timer thread... Why, why is sql_thread keeps on checking for messages every millisecond? Couldn't there be some

signalling

...

implemented that will make the thread suspend on condition variable or a socket/pipe in between?

#0 do_sleep (t=1000) at src/switch_time.c:109 #1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0) at src/switch_core_sqldb.c:783

Why does this sofia_profile_worker_thread keeps on looping checking for the queue? Have a semaphore!

#0 do_sleep (t=1000) at src/switch_time.c:109 #1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30, obj=0x80f2490) at sofia.c:978

Nothing's happening on the box, but there are three threads that pretend to be actively busy with smth. Others at least sleep for hundreds of milliseconds, not for one.

And there is even infrastructure present to do blocking pops: i.e. why couldn't sqldb thread do queue_pop() instead of queue_trypop() intermixed with 1ms sleeps? This looping is such a waste...

eaf wrote:

...

As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes, it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND" overrides that.

Yeah, there is a global timestamp... It's easy to workaround that for

RTP

...

who calls switch_micro_time_now()... But if somebody accesses runtime.timestamp directly, it's gonna be tough to grep for that. If only this was C++...

I'll play around. Never liked polling too much. Never could've guessed that polling could be so useful for scalability ;) My naive implementation would've pulled timestamp via system calls and would've done sleeping by passing exact interval to select() instead of syncing with a pacing thread. Which would be dead-quiet at idle time, but, of course, would stop scaling at some point due to excessive number of system calls.

Thanks.

Michael Jerris wrote:

...

In short. No, you can not for many reasons. The milisecond tic is used throughout the code even when there is not any calls up. You can grep for switch_cond_next if you would like to see where but it is required to keep our global timestamp and for pacing the scheduler among other services that run all the time.

Mike

On Dec 2, 2009, at 7:31 PM, eaf <erandr-junk@usa.net> wrote:

...

Can I reduce resolution of that timer thread 10 times? I mean, I glanced through the code, and see that among others (are there others?) RTP and IVR set up their timers that are subsequently managed by this thread. RTP timers should be eliminated by that setting you've suggested. IVR timers are set at 20ms... So, if the thread is set to wake up every 10ms instead of 1ms it should be able to wake up those IVR timers just fine. Right?

That's a cool design to have one dedicated thread that maintains accurate timing and then broadcasts via condition variables to hundreds of other threads events that they can register for. I'm sure it's one of the reasons why FS scales so much better than Asterisk. But for poor low-end setups that sit in the closet, eat only 6W of power and hardly ever run more than two calls at the same time, can I hack it somehow to be more UNIX- friendly? I.e. make it stuck in select() or recv() when there is nothing to do, call clock_gettime() right from the thread that wants and when it wants to know current time?

Say, what if that thread is made to suspend on a condition variable in case if there are no timers registered in TIMER_MATRIX? Then, if some other thread comes up and adds its timer into the matrix, it could wake up the timer thread and enjoy accurate timing as needed, on demand? And in- between the calls, when there is no RTP or IVR, it will all go silent? I mean, sitting on a wait queue in the kernel is way better than go back and forth incrementing counters that nobody even needs at the moment?

Anthony Minessale-2 wrote:

...

idle is a 4 letter word to a realtime application.

The core keeps a single high-priority thread to keep 1ms timing and expands that broadcasting to hundreds or thousand of threads who need accurate timing.

Your choppy audio is caused by linksys lying about the packet len that it's using and we set our timer to the wrong speed.

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch- users http://www.freeswitch.org

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:

http://lists.freeswitch.org/mailman/options/freeswitch-users

...

http://www.freeswitch.org

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:213-799-1400

I don't think it's the board itself... We have extensively tested FreeSwitch (no modifications) on that exact board with AstLinux and have it running at multiple customer locations. No timing errors, no warnings or errors of any kind. Pretty standard really just don't expect too much from the LX800 (transcoding, resampling, massive numbers of calls, etc). On Thu, Dec 3, 2009 at 12:29 PM, eaf <erandr-junk@usa.net> wrote:

I'm sorry if I sounded that way. Did mean to. :)

Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm

Line offset difference is due to some minor logging changes I made to see who's allocating timers and how often. This way I found MOH streaming and that RTP still allocates timers even when it's set to none in the profile.

I feel that this platform turned out to be underpowered for FS because it cannot meet its scheduling expectations. I guess, some degree of kernel tweaking or setting priorities will fix that. Meanwhile I just got rid of the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms thread in two (one blocked and waiting for new commands in the SQL queue, the other one checking registrations and gateways with 1sec interval), and don't know yet what to do about the timer thread.

Again, I apologize for stupid or accusing questions, I'm just trying to see how FS can be made friendlier to this board. Or the board be made friendlier to FS ;)

-- Kristian Kielhofner http://www.astlinux.org http://blog.krisk.org http://www.star2star.com http://www.submityoursip.com http://www.voalte.com

What about the things I spent time suggesting in my last email? Did you try them because I was actually curious if they made any impact. On Thu, Dec 3, 2009 at 11:29 AM, eaf <erandr-junk@usa.net> wrote:

I'm sorry if I sounded that way. Did mean to. :)

Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm

Line offset difference is due to some minor logging changes I made to see who's allocating timers and how often. This way I found MOH streaming and that RTP still allocates timers even when it's set to none in the profile.

I feel that this platform turned out to be underpowered for FS because it cannot meet its scheduling expectations. I guess, some degree of kernel tweaking or setting priorities will fix that. Meanwhile I just got rid of the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms thread in two (one blocked and waiting for new commands in the SQL queue, the other one checking registrations and gateways with 1sec interval), and don't know yet what to do about the timer thread.

Again, I apologize for stupid or accusing questions, I'm just trying to see how FS can be made friendlier to this board. Or the board be made friendlier to FS ;)

Anthony Minessale-2 wrote:

...

If you see that message then your machine/os/combo is having some

problems

...

keeping up. It's not the timer missing anything its the monotonic clock detecting a 1 second or more differential from what its next prediction for the time should be. The best way to trigger this would be to suspend FS with control-z or attach to it with gdb blocking the entire process, that 1ms thread would have to miss 1000 iterations to trigger that warning.

Btw, that error message is at line 471 not 473 so you are using modified code.

Its possible your box has a bad monotonic timer, you can set

under <settings> in switch.conf.xml

We are now starting to guess you are using some small embedded type platform perhaps? I've run FS even on a nokia n810 and never caused that message to fire.

if 1 call can interrupt the cpu enough to cause noticeable issues you might want to consider running the process at a greater priority by using the -hp command line arg or at least nice it

Why don't you tell us the whole story about what OS/platform you are using here rather that form conjectures about what is wrong with our code that thousands of people are happy with.

On Thu, Dec 3, 2009 at 8:55 AM, eaf <erandr-junk@usa.net> wrote:

...

Btw, I have these popping up in my logs from time to time:

2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314 (sofia/external/xxxxx@4.68.250.148) Running State Change CS_HANGUP 2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration Detected! Syncing Clock

In this case an incoming call rang to both FS and Asterisk, Asterisk picked up, but the surge of activity made FS timer thread miss a beat or two.

eaf wrote:

...

Oh, it's not just one timer thread... Why, why is sql_thread keeps on checking for messages every millisecond? Couldn't there be some

signalling

...

implemented that will make the thread suspend on condition variable or a socket/pipe in between?

#0 do_sleep (t=1000) at src/switch_time.c:109 #1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0) at src/switch_core_sqldb.c:783

Why does this sofia_profile_worker_thread keeps on looping checking

for

...

...

the queue? Have a semaphore!

#0 do_sleep (t=1000) at src/switch_time.c:109 #1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30, obj=0x80f2490) at sofia.c:978

Nothing's happening on the box, but there are three threads that pretend to be actively busy with smth. Others at least sleep for hundreds of milliseconds, not for one.

And there is even infrastructure present to do blocking pops: i.e. why couldn't sqldb thread do queue_pop() instead of queue_trypop() intermixed with 1ms sleeps? This looping is such a waste...

eaf wrote:

...

As I see it, switch_cond_next() currently is just a do_sleep(1000).

Yes,

...

it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND" overrides that.

Yeah, there is a global timestamp... It's easy to workaround that for RTP who calls switch_micro_time_now()... But if somebody accesses runtime.timestamp directly, it's gonna be tough to grep for that. If only this was C++...

I'll play around. Never liked polling too much. Never could've guessed that polling could be so useful for scalability ;) My naive implementation would've pulled timestamp via system calls and would've done sleeping by passing exact interval to select() instead of syncing with a pacing thread. Which would be dead-quiet at idle time, but, of course, would stop scaling at some point due to excessive number of system calls.

Thanks.

Michael Jerris wrote:

...

In short. No, you can not for many reasons. The milisecond tic is used throughout the code even when there is not any calls up. You

can

...

grep for switch_cond_next if you would like to see where but it is required to keep our global timestamp and for pacing the scheduler among other services that run all the time.

Mike

On Dec 2, 2009, at 7:31 PM, eaf <erandr-junk@usa.net> wrote:

> > Can I reduce resolution of that timer thread 10 times? I mean, I > glanced > through the code, and see that among others (are there others?) RTP > and IVR > set up their timers that are subsequently managed by this thread. > RTP timers > should be eliminated by that setting you've suggested. IVR timers > are set at > 20ms... So, if the thread is set to wake up every 10ms instead of > 1ms it > should be able to wake up those IVR timers just fine. Right? > > That's a cool design to have one dedicated thread that maintains > accurate > timing and then broadcasts via condition variables to hundreds of > other > threads events that they can register for. I'm sure it's one of the > reasons > why FS scales so much better than Asterisk. But for poor low-end > setups that > sit in the closet, eat only 6W of power and hardly ever run more > than two > calls at the same time, can I hack it somehow to be more UNIX- > friendly? I.e. > make it stuck in select() or recv() when there is nothing to do, call > clock_gettime() right from the thread that wants and when it wants > to know > current time? > > Say, what if that thread is made to suspend on a condition variable > in case > if there are no timers registered in TIMER_MATRIX? Then, if some other > thread comes up and adds its timer into the matrix, it could wake up > the > timer thread and enjoy accurate timing as needed, on demand? And in- > between > the calls, when there is no RTP or IVR, it will all go silent? I mean, > sitting on a wait queue in the kernel is way better than go back and > forth > incrementing counters that nobody even needs at the moment? > > > Anthony Minessale-2 wrote: >> >> idle is a 4 letter word to a realtime application. >> >> The core keeps a single high-priority thread to keep 1ms timing and >> expands >> that broadcasting >> to hundreds or thousand of threads who need accurate timing. >> >> Your choppy audio is caused by linksys lying about the packet len >> that >> it's >> using and we set our timer >> to the wrong speed. >> >> > > -- > View this message in context: > http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html > Sent from the Freeswitch-users mailing list archive at Nabble.com. > > > _______________________________________________ > FreeSWITCH-users mailing list > FreeSWITCH-users@lists.freeswitch.org > http://lists.freeswitch.org/mailman/listinfo/freeswitch-users > UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch- > users > http://www.freeswitch.org

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com>< MSN%3Aanthony_minessale@hotmail.com<MSN%253Aanthony_minessale@hotmail.com>

GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> <PAYPAL%3Aanthony.minessale@gmail.com<PAYPAL%253Aanthony.minessale@gmail.com>

IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org>< sip%3A888@conference.freeswitch.org<sip%253A888@conference.freeswitch.org>

iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> <googletalk%3Aconf%2B888@conference.freeswitch.org<googletalk%253Aconf%252B888@conference.freeswitch.org>

pstn:213-799-1400

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:213-799-1400

no, I mean the one after that that you must have completely skipped with a command line option to try and a param to set in the config. It somewhat annoys me for taking the time to compose it now. I wrote all of the code you are talking about myself and I was trying to give you some suggestions.... Well, actually, you did answer my question about the platform so you must have seen it..... The loops are not the cause of that migration message, something wrong with the hardware or the kernel is. Another guy just told you he does not see that problem on the same exact hardware. Even if you have a point about the sql threads, you could make a patch to slow them down but you cant slow down too much or you will not be able to handle 400 cps all asking to send updates to transactions in batches of thousands of sql stmts. Every line of that code is carefully designed so I don't know what else to tell you but to stop being so arrogant and re-read this thread for all the advice you have totally ignored. I started out trying to help you but I have a lot of work to do. I thoroughly explained it to you and you are choosing to ignore me so I guess I'm done. You can do whatever you want with your working copy, i'll see you in 3 or 4 years when you get up to speed with the rest of us........ On Thu, Dec 3, 2009 at 12:43 PM, eaf <erandr-junk@usa.net> wrote:

You mean, upgrading to the trunk and disabling RTP timers? Yes, I did. I thought I responded back. Perhaps it didn't make through though, as I just emailed back to the list instead of using nabble.com...

Anyway, upgrading to the trunk didn't change much, forcing SPA to 30ms went w/o any effect either, but disabling RTP timers did the trick. I don't have the original "choppy sound with PCMU" problem any more, thanks a lot for the quick turnaround on that question.

But your suggestions made me look, into logs, strace, code, etc, so now I'm just checking on how to quiet down those busy loops a little and how to get rid of periodic CRIT messages about Virtual Machine Migration.

Anthony Minessale-2 wrote:

...

What about the things I spent time suggesting in my last email? Did you try them because I was actually curious if they made any impact.

On Thu, Dec 3, 2009 at 11:29 AM, eaf <erandr-junk@usa.net> wrote:

...

I'm sorry if I sounded that way. Did mean to. :)

Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip and 256MB of RAM. http://www.pcengines.ch/alix2d3.htm

Line offset difference is due to some minor logging changes I made to

see

...

...

who's allocating timers and how often. This way I found MOH streaming and that RTP still allocates timers even when it's set to none in the profile.

I feel that this platform turned out to be underpowered for FS because it cannot meet its scheduling expectations. I guess, some degree of kernel tweaking or setting priorities will fix that. Meanwhile I just got rid of the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms thread in two (one blocked and waiting for new commands in the SQL queue, the other one checking registrations and gateways with 1sec interval), and don't know yet what to do about the timer thread.

Again, I apologize for stupid or accusing questions, I'm just trying to see how FS can be made friendlier to this board. Or the board be made friendlier to FS ;)

Anthony Minessale-2 wrote:

...

If you see that message then your machine/os/combo is having some

problems

...

keeping up. It's not the timer missing anything its the monotonic clock detecting a 1 second or more differential from what its next prediction for the time should be. The best way to trigger this would be to suspend FS with control-z or attach to it with gdb blocking the entire process, that 1ms thread would have to miss 1000 iterations to trigger that warning.

Btw, that error message is at line 471 not 473 so you are using modified code.

Its possible your box has a bad monotonic timer, you can set

under <settings> in switch.conf.xml

We are now starting to guess you are using some small embedded type platform perhaps? I've run FS even on a nokia n810 and never caused that message to fire.

if 1 call can interrupt the cpu enough to cause noticeable issues you might want to consider running the process at a greater priority by using the -hp command line arg or at least nice it

Why don't you tell us the whole story about what OS/platform you are using here rather that form conjectures about what is wrong with our code that thousands of people are happy with.

On Thu, Dec 3, 2009 at 8:55 AM, eaf <erandr-junk@usa.net> wrote:

...

Btw, I have these popping up in my logs from time to time:

2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314 (sofia/external/xxxxx@4.68.250.148) Running State Change CS_HANGUP 2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration Detected! Syncing Clock

In this case an incoming call rang to both FS and Asterisk, Asterisk picked up, but the surge of activity made FS timer thread miss a beat or

two.

...

eaf wrote:

...

Oh, it's not just one timer thread... Why, why is sql_thread keeps

on

...

...

checking for messages every millisecond? Couldn't there be some signalling implemented that will make the thread suspend on condition variable or a socket/pipe in between?

#0 do_sleep (t=1000) at src/switch_time.c:109 #1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0) at src/switch_core_sqldb.c:783

Why does this sofia_profile_worker_thread keeps on looping checking for the queue? Have a semaphore!

#0 do_sleep (t=1000) at src/switch_time.c:109 #1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30, obj=0x80f2490) at sofia.c:978

Nothing's happening on the box, but there are three threads that pretend to be actively busy with smth. Others at least sleep for hundreds of milliseconds, not for one.

And there is even infrastructure present to do blocking pops: i.e. why couldn't sqldb thread do queue_pop() instead of queue_trypop() intermixed with 1ms sleeps? This looping is such a waste...

eaf wrote: > > As I see it, switch_cond_next() currently is just a do_sleep(1000). Yes, > it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND" > overrides that. > > Yeah, there is a global timestamp... It's easy to workaround that for RTP > who calls switch_micro_time_now()... But if somebody accesses > runtime.timestamp directly, it's gonna be tough to grep for that. If only > this was C++... > > I'll play around. Never liked polling too much. Never could've guessed > that polling could be so useful for scalability ;) My naive > implementation would've pulled timestamp via system calls and would've > done sleeping by passing exact interval to select() instead of syncing > with a pacing thread. Which would be dead-quiet at idle time, but, of > course, would stop scaling at some point due to excessive number of > system calls. > > Thanks. > > > Michael Jerris wrote: >> >> In short. No, you can not for many reasons. The milisecond tic is >> used throughout the code even when there is not any calls up. You can >> grep for switch_cond_next if you would like to see where but it is >> required to keep our global timestamp and for pacing the scheduler >> among other services that run all the time. >> >> Mike >> >> On Dec 2, 2009, at 7:31 PM, eaf <erandr-junk@usa.net> wrote: >> >>> >>> Can I reduce resolution of that timer thread 10 times? I mean, I >>> glanced >>> through the code, and see that among others (are there others?) RTP >>> and IVR >>> set up their timers that are subsequently managed by this thread. >>> RTP timers >>> should be eliminated by that setting you've suggested. IVR timers >>> are set at >>> 20ms... So, if the thread is set to wake up every 10ms instead of >>> 1ms it >>> should be able to wake up those IVR timers just fine. Right? >>> >>> That's a cool design to have one dedicated thread that maintains >>> accurate >>> timing and then broadcasts via condition variables to hundreds of >>> other >>> threads events that they can register for. I'm sure it's one of the >>> reasons >>> why FS scales so much better than Asterisk. But for poor low-end >>> setups that >>> sit in the closet, eat only 6W of power and hardly ever run more >>> than two >>> calls at the same time, can I hack it somehow to be more UNIX- >>> friendly? I.e. >>> make it stuck in select() or recv() when there is nothing to do, call >>> clock_gettime() right from the thread that wants and when it wants >>> to know >>> current time? >>> >>> Say, what if that thread is made to suspend on a condition variable >>> in case >>> if there are no timers registered in TIMER_MATRIX? Then, if some other >>> thread comes up and adds its timer into the matrix, it could wake up >>> the >>> timer thread and enjoy accurate timing as needed, on demand? And in- >>> between >>> the calls, when there is no RTP or IVR, it will all go silent? I mean, >>> sitting on a wait queue in the kernel is way better than go back and >>> forth >>> incrementing counters that nobody even needs at the moment? >>> >>> >>> Anthony Minessale-2 wrote: >>>> >>>> idle is a 4 letter word to a realtime application. >>>> >>>> The core keeps a single high-priority thread to keep 1ms timing and >>>> expands >>>> that broadcasting >>>> to hundreds or thousand of threads who need accurate timing. >>>> >>>> Your choppy audio is caused by linksys lying about the packet len >>>> that >>>> it's >>>> using and we set our timer >>>> to the wrong speed. >>>> >>>> >>> >>> -- >>> View this message in context: >>>

http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html

...

...

>>> Sent from the Freeswitch-users mailing list archive at Nabble.com. >>> >>> >>> _______________________________________________ >>> FreeSWITCH-users mailing list >>> FreeSWITCH-users@lists.freeswitch.org >>> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users >>> UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch- >>> users >>> http://www.freeswitch.org >> >> _______________________________________________ >> FreeSWITCH-users mailing list >> FreeSWITCH-users@lists.freeswitch.org >> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users >> UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users >> http://www.freeswitch.org >> >> > >

-- View this message in context:

http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html

...

Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm MSN:anthony_minessale@hotmail.com<MSN%3Aanthony_minessale@hotmail.com> <MSN%3Aanthony_minessale@hotmail.com<MSN%253Aanthony_minessale@hotmail.com> < MSN%3Aanthony_minessale@hotmail.com<MSN%253Aanthony_minessale@hotmail.com> <MSN%253Aanthony_minessale@hotmail.com<MSN%25253Aanthony_minessale@hotmail.com>

...

...

GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> <PAYPAL%3Aanthony.minessale@gmail.com<PAYPAL%253Aanthony.minessale@gmail.com>

...

<PAYPAL%3Aanthony.minessale@gmail.com<PAYPAL%253Aanthony.minessale@gmail.com> <PAYPAL%253Aanthony.minessale@gmail.com<PAYPAL%25253Aanthony.minessale@gmail.com>

...

...

IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference sip:888@conference.freeswitch.org<sip%3A888@conference.freeswitch.org>

<sip%3A888@conference.freeswitch.org<sip%253A888@conference.freeswitch.org> < sip%3A888@conference.freeswitch.org<sip%253A888@conference.freeswitch.org> <sip%253A888@conference.freeswitch.org<sip%25253A888@conference.freeswitch.org>

...

...

iax:guest@conference.freeswitch.org/888

googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> <googletalk%3Aconf%2B888@conference.freeswitch.org<googletalk%253Aconf%252B888@conference.freeswitch.org>

...

<googletalk%3Aconf%2B888@conference.freeswitch.org<googletalk%253Aconf%252B888@conference.freeswitch.org> <googletalk%253Aconf%252B888@conference.freeswitch.org<googletalk%25253Aconf%25252B888@conference.freeswitch.org>

...

...

pstn:213-799-1400

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users

UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users

...

http://www.freeswitch.org

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com>< MSN%3Aanthony_minessale@hotmail.com<MSN%253Aanthony_minessale@hotmail.com>

GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> <PAYPAL%3Aanthony.minessale@gmail.com<PAYPAL%253Aanthony.minessale@gmail.com>

IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org>< sip%3A888@conference.freeswitch.org<sip%253A888@conference.freeswitch.org>

iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> <googletalk%3Aconf%2B888@conference.freeswitch.org<googletalk%253Aconf%252B888@conference.freeswitch.org>

pstn:213-799-1400

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26630994.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:213-799-1400

Sigh, You just took it up a notch in terms of disdain and sarcasm. Why do people always only apologize sarcastically? I asked you to try the -hp and turn off the monotonic clock just to gather the results to help you. You completely missed it and just went on about the threads. Please save the "ok fine the code is perfect, blah blah" if you would have just read the email and answered the question I might have cared more about the status of your problem. I told you both of those threads need to be on their toes because they try to balance between a certian number of sql stmts or 500ms whatever comes first. When there are thousands of events per second being turned into SQL statements which are in turn compiled into large sql transactions. If you want to come up with a way that they can sleep longer until there is a sign of activity and stay busy for a few seconds then slow down again, that's probably possible but the process is already idle at 0% cpu so maybe you can appreciate why we are not rushing to work on it. Maybe I'll give it a go just to show you it has nothing to do with your problem. Please don't mock our comment about several years. You have no idea how hard this code was to develop and it's truly insulting. Its clear to see you are locked into assuming that the busy threads that are not all that busy because they are constantly yielding to the scheduler is breaking the timing code. I begged you to understand me when i told you that the err is not normal, most boxes do not see it doing nothing and there has to be a specific problem on your box or configuration. So instead of working with us you want to escalate to snotty comments. That's pretty normal on the internet I guess..... If you want to have a constructive conversation about our core, install FS on a normal box, use it for a few weeks, figure out everything about how it works then try.... There was pure speculation and conjecture in your original emails and I never said a word about it until you kept pushing. Kristian mentioned he never sees that on that same hardware did you even consider following up on why that is? I don't have your device, but I assume if you get it working well it will certainly help you more than it helps me so you could at least have the decency to believe what we are trying to tell you. On Thu, Dec 3, 2009 at 3:44 PM, eaf <erandr-junk@usa.net> wrote:

Oh, you mean giving FS higher priority? Yeah, as a last resort I'll do that. At the moment, I hope it won't be necessary as I can make those "hyper" threads behave, and will see how that goes first. I see where your implementation could be coming from. There is a queue of SQL queries in sofia.c processed by the worker thread. There are only two pop functions available in APR: queue_pop() and queue_trypop(), so alas no option with a timeout here. You don't want to block the thread in pop() indefinitely because you chose that same worker needs to do ireg and gw processing once in a while (separated by tens or hundreds of seconds, btw). You also want to be able to detect shutdown condition so that the worker doesn't hold up profile thread. So you chose to poll for events every millisecond instead of just creating an apr_thread_cond_t for resource friendly signalling.

I agree that the timer thread philosophy is great and was the right choice for scaling, but I just don't comprehend responses to things like these other SQL or sofia worker threads. Did somebody even remotely acknowledge that busy loops at least in those areas that I showed may probably be a bad idea and could've been eliminated? I've heard suggestions to bump up priority, I've heard that the code was perfect already, that it's the result of 4-year effort, that I am arrogant, don't listen and don't understand squat.

I'm sorry if I gave you impression that I was looking for the bad parts in the software. I apologized for that already. All I wanted was to have constructive conversation, perhaps I'm not too good at it. Code is already perfect according to you? Fine with me.

Anthony Minessale-2 wrote:

...

no,

I mean the one after that that you must have completely skipped with a command line option to try and a param to set in the config. It somewhat annoys me for taking the time to compose it now. I wrote all of the code you are talking about myself and I was trying to give you some suggestions....

Well, actually, you did answer my question about the platform so you

must

...

have seen it.....

The loops are not the cause of that migration message, something wrong with the hardware or the kernel is. Another guy just told you he does not see that problem on the same exact hardware.

Even if you have a point about the sql threads, you could make a patch to slow them down but you cant slow down too much or you will not be able to handle 400 cps all asking to send updates to transactions in batches of thousands of sql stmts. Every line of that code is carefully designed so I don't know what else to tell you but to stop being so arrogant and re-read this thread for all the advice you have totally ignored. I started out trying to help you but I have a lot of work to do. I thoroughly explained it to you and you are choosing to ignore me so I guess I'm done. You can do whatever you want with your working copy, i'll see you in 3 or 4 years when you get up to speed with the rest of us........

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26633739.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:213-799-1400

There is another user here with a 300mhz box. I am willing to investigate this improved performance for weak devices but I need to do it in a sane cross-platform way. On Fri, Dec 4, 2009 at 1:32 PM, Yossi Neiman <freeswitch@cartissolutions.com

wrote:

A word to the wise to the general FreeSWITCH community: If Anthony Minessale suggests that you try to do any number of things, it's a very good idea to try all those ideas before continuing on. I've known him, MikeJ, and bkw for several years, and they almost always have very good ideas as to troubleshoot a problem in FreeSWITCH. It's extremely frustrating to try to help people out who won't try the provided suggestions first.

And note directly to "eaf" - bogomips is quite possibly the least significant bit of data about a cpu that you will get out of /proc/cpuinfo... The name itself - bogo, means bogus. http://en.wikipedia.org/wiki/Bogomips

-Yossi

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:213-799-1400

Did you do each thing alone too to tell the difference? -hp alone, disable monotonic alone (i did not see you mention the disable monotonic) as for your 4ms thing, yes we require high resolution timing, if we ask to sleep 1000 microseconds that is what we need it to sleep for or at least as close as possible, and the main reason that thread is never sleeping is because you can't actually count on it to run every 1ms but you mostly can. Hence the whole philosophy on only making 1 thread run hot all the time to ensure that the rest don't have to repeat the same algorithm. We focus on high end performance this was the point of your experimentation because we will need to use a compile time defines and other logic to make it more efficient on your platform, a platform which we are not using. I am curious what would happen if you install Kristian's astlinux on one of your devices, i think you should also compare the kernel versions. What OS are you running anyway? Here are some more things to try (running plain trunk with no mods) do these systematically each alone and all together with/without -hp or disable monotonic etc to see what different combos create comment out this line (line 10) #define DISABLE_1MS_COND rebuild, this tells it to run a conditional at 1ms in the same timer thread which will make all the switch_cond_next share a 1ms conditional instead of doing microsleeps next some kernels/devices work better using select(0) for sleep where others work better using usleep. comment out line 109 apr_sleep(t); and try usleep(t) also mac works better using nanosleep so you could try changing it so it uses the code starting at 101 instead. also your claim about JS should be investigated because I do not think it should be the case. but you may want to move this to a jira http://jira.freeswitch.org As for the asterisk comparison, not sure how to answer you, that's your decision. On Mon, Dec 7, 2009 at 9:28 AM, eaf <erandr-junk@usa.net> wrote:

Here is what I found...

I tried high-priority scheduling as per your suggestion, reniced the program explicitly, rewrote timer thread to sleep on cond. variable and activate only when there are timers and only when the timer actually had to be clicked, turned off SQL thread and removed polling from sofia profile thread.

That pretty much eliminated all idle 1ms sleepers that were there except for three in sofia itself (su_epoll_port). And when I was about to be happy, I found that two outgoing calls through my VOIP providers when bridged together showed terrible distortions. I undid all my changes, tried 1.0.4, trunk (noticed btw that when I bridge two calls via loopback in JS in the trunk I must keep JS running, or the calls get terminated - NOT the same as in 1.0.4 where exitting JS left calls running), got pretty much the same sad results. At the same time calls bridged by freeswitch between LAN and any of the VOIP providers behaved just fine. And calls bridged by Asterisk any way were fine too. So that pretty much looked like the end of the freeswitch trials for me.

But then I timed your code, mine and found that all those 1ms sleeps that your timer thread was doing (and all those pollers were doing as well) were actually 4ms sleeps because you know what unless kernel is configured with HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms (HZ=100). Mine was 250.

This actually meant that the original timer thread was firing once, sleeping for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4 times back-to-back, etc. It was still firing 20ms timers on time, but 30ms ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever relied on runtime.reference or switch_micro_time_now() were kind of screwed because both were running jumpy. Plus whoever assumed that apr_sleep(1000) or cond_yield() was sleeping for 1ms were also in for a surprise. It felt satisfying to find that, however it didn't explain why the same distortions were observed with rewritten timer thread and disabled RTP timers.

Anyway, I sighed (pretty much like you) and recompiled the kernel with HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south, you need to hook up serial console and see what the heck went wrong.

That eliminated distortions, ha! But made freeswitch more CPU hungry. Now the remaining 1ms threads sitting in sofia epoll were really polling for 1ms, not 4, and freeswitch was consistently sitting in the first line of the top chart showing 3% CPU utilization when idle.

Don't know whether it's because of the remaining epolls in sofia or whether it's because there are still some threads left in freeswitch that I neglected to change because they were sleeping with 100ms interval, so I figured, who cares. Maybe when all things come together (sofia, 100ms*N) freeswitch ends up spending 3% of CPU while doing pretty much nothing.

Btw, compared with Asterisk, the latter is not even visible on the first top's screen and spends 1% CPU when bridging two G711 calls and recording them to disk.

So, at this time I have both original Asterisk and FS setups running. One is seemless but clumsy in configuration, the other one is neat and stylish but too preoccupied with smth... Should I look into sofia epollers? That's kind of deep in the code. Or should I just stick with Asterisk?

Anthony Minessale-2 wrote:

...

There is another user here with a 300mhz box. I am willing to

investigate

...

this improved performance for weak devices but I need to do it in a sane cross-platform way.

On Fri, Dec 4, 2009 at 1:32 PM, Yossi Neiman <freeswitch@cartissolutions.com

...

wrote:

...

A word to the wise to the general FreeSWITCH community: If Anthony Minessale suggests that you try to do any number of things, it's a very good idea to try all those ideas before continuing on. I've known him, MikeJ, and bkw for several years, and they almost always have very good ideas as to troubleshoot a problem in FreeSWITCH. It's extremely frustrating to try to help people out who won't try the provided suggestions first.

And note directly to "eaf" - bogomips is quite possibly the least significant bit of data about a cpu that you will get out of /proc/cpuinfo... The name itself - bogo, means bogus. http://en.wikipedia.org/wiki/Bogomips

-Yossi

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com>< MSN%3Aanthony_minessale@hotmail.com<MSN%253Aanthony_minessale@hotmail.com>

GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> <PAYPAL%3Aanthony.minessale@gmail.com<PAYPAL%253Aanthony.minessale@gmail.com>

IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org>< sip%3A888@conference.freeswitch.org<sip%253A888@conference.freeswitch.org>

iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> <googletalk%3Aconf%2B888@conference.freeswitch.org<googletalk%253Aconf%252B888@conference.freeswitch.org>

pstn:213-799-1400

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26678873.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:213-799-1400

oh and also use top -H to see which threads are using specific CPU and try to cross reference them by attaching with gdb and dumping all the thread bt On Mon, Dec 7, 2009 at 10:16 AM, Michael Jerris <mike@jerris.com> wrote:

Also I have seen some people reporting that the new tickless timers in newer kernels work better. You may want to try those.

Mike

On Dec 7, 2009, at 11:00 AM, Anthony Minessale wrote:

Did you do each thing alone too to tell the difference? -hp alone, disable monotonic alone (i did not see you mention the disable monotonic)

as for your 4ms thing, yes we require high resolution timing, if we ask to sleep 1000 microseconds that is what we need it to sleep for or at least as close as possible, and the main reason that thread is never sleeping is because you can't actually count on it to run every 1ms but you mostly can. Hence the whole philosophy on only making 1 thread run hot all the time to ensure that the rest don't have to repeat the same algorithm. We focus on high end performance this was the point of your experimentation because we will need to use a compile time defines and other logic to make it more efficient on your platform, a platform which we are not using. I am curious what would happen if you install Kristian's astlinux on one of your devices, i think you should also compare the kernel versions.

What OS are you running anyway?

Here are some more things to try (running plain trunk with no mods) do these systematically each alone and all together with/without -hp or disable monotonic etc to see what different combos create

comment out this line (line 10) #define DISABLE_1MS_COND

rebuild, this tells it to run a conditional at 1ms in the same timer thread which will make all the switch_cond_next share a 1ms conditional instead of doing microsleeps

next

some kernels/devices work better using select(0) for sleep where others work better using usleep. comment out line 109 apr_sleep(t);

and try usleep(t)

also mac works better using nanosleep so you could try changing it so it uses the code starting at 101 instead.

also your claim about JS should be investigated because I do not think it should be the case. but you may want to move this to a jira http://jira.freeswitch.org

As for the asterisk comparison, not sure how to answer you, that's your decision.

On Mon, Dec 7, 2009 at 9:28 AM, eaf <erandr-junk@usa.net> wrote:

...

Here is what I found...

I tried high-priority scheduling as per your suggestion, reniced the program explicitly, rewrote timer thread to sleep on cond. variable and activate only when there are timers and only when the timer actually had to be clicked, turned off SQL thread and removed polling from sofia profile thread.

That pretty much eliminated all idle 1ms sleepers that were there except for three in sofia itself (su_epoll_port). And when I was about to be happy, I found that two outgoing calls through my VOIP providers when bridged together showed terrible distortions. I undid all my changes, tried 1.0.4, trunk (noticed btw that when I bridge two calls via loopback in JS in the trunk I must keep JS running, or the calls get terminated - NOT the same as in 1.0.4 where exitting JS left calls running), got pretty much the same sad results. At the same time calls bridged by freeswitch between LAN and any of the VOIP providers behaved just fine. And calls bridged by Asterisk any way were fine too. So that pretty much looked like the end of the freeswitch trials for me.

But then I timed your code, mine and found that all those 1ms sleeps that your timer thread was doing (and all those pollers were doing as well) were actually 4ms sleeps because you know what unless kernel is configured with HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms (HZ=100). Mine was 250.

This actually meant that the original timer thread was firing once, sleeping for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4 times back-to-back, etc. It was still firing 20ms timers on time, but 30ms ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever relied on runtime.reference or switch_micro_time_now() were kind of screwed because both were running jumpy. Plus whoever assumed that apr_sleep(1000) or cond_yield() was sleeping for 1ms were also in for a surprise. It felt satisfying to find that, however it didn't explain why the same distortions were observed with rewritten timer thread and disabled RTP timers.

Anyway, I sighed (pretty much like you) and recompiled the kernel with HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south, you need to hook up serial console and see what the heck went wrong.

That eliminated distortions, ha! But made freeswitch more CPU hungry. Now the remaining 1ms threads sitting in sofia epoll were really polling for 1ms, not 4, and freeswitch was consistently sitting in the first line of the top chart showing 3% CPU utilization when idle.

Don't know whether it's because of the remaining epolls in sofia or whether it's because there are still some threads left in freeswitch that I neglected to change because they were sleeping with 100ms interval, so I figured, who cares. Maybe when all things come together (sofia, 100ms*N) freeswitch ends up spending 3% of CPU while doing pretty much nothing.

Btw, compared with Asterisk, the latter is not even visible on the first top's screen and spends 1% CPU when bridging two G711 calls and recording them to disk.

So, at this time I have both original Asterisk and FS setups running. One is seemless but clumsy in configuration, the other one is neat and stylish but too preoccupied with smth... Should I look into sofia epollers? That's kind of deep in the code. Or should I just stick with Asterisk?

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:213-799-1400

Both, if it always sounds ok then I guess CPU usage. On Mon, Dec 7, 2009 at 2:58 PM, eaf <erandr-junk@usa.net> wrote:

What do you want me to check while running these tests? Sound quality (it's good now even with original 1.0.4). Or CPU utilization?

It's Debian 4.

Anthony Minessale-2 wrote:

...

Did you do each thing alone too to tell the difference? -hp alone, disable monotonic alone (i did not see you mention the disable monotonic)

as for your 4ms thing, yes we require high resolution timing, if we ask

to

...

sleep 1000 microseconds that is what we need it to sleep for or at least as close as possible, and the main reason that thread is never sleeping is because you can't actually count on it to run every 1ms but you mostly can. Hence the whole philosophy on only making 1 thread run hot all the time to ensure that the rest don't have to repeat the same algorithm. We focus on high end performance this was the point of your experimentation because we will need to use a compile time defines and other logic to make it more efficient on your platform, a platform which we are not using. I am curious what would happen if you install Kristian's astlinux on one of your devices, i think you should also compare the kernel versions.

What OS are you running anyway?

Here are some more things to try (running plain trunk with no mods) do these systematically each alone and all together with/without -hp or disable monotonic etc to see what different combos create

comment out this line (line 10) #define DISABLE_1MS_COND

rebuild, this tells it to run a conditional at 1ms in the same timer thread which will make all the switch_cond_next share a 1ms conditional instead of doing microsleeps

next

some kernels/devices work better using select(0) for sleep where others work better using usleep. comment out line 109 apr_sleep(t);

and try usleep(t)

also mac works better using nanosleep so you could try changing it so it uses the code starting at 101 instead.

also your claim about JS should be investigated because I do not think it should be the case. but you may want to move this to a jira http://jira.freeswitch.org

As for the asterisk comparison, not sure how to answer you, that's your decision.

On Mon, Dec 7, 2009 at 9:28 AM, eaf <erandr-junk@usa.net> wrote:

...

Here is what I found...

I tried high-priority scheduling as per your suggestion, reniced the program explicitly, rewrote timer thread to sleep on cond. variable and activate only when there are timers and only when the timer actually had to be clicked, turned off SQL thread and removed polling from sofia profile thread.

That pretty much eliminated all idle 1ms sleepers that were there except for three in sofia itself (su_epoll_port). And when I was about to be happy, I found that two outgoing calls through my VOIP providers when bridged together showed terrible distortions. I undid all my changes, tried 1.0.4, trunk (noticed btw that when I bridge two calls via loopback in JS in

the

...

trunk I must keep JS running, or the calls get terminated - NOT the same as in 1.0.4 where exitting JS left calls running), got pretty much the same sad results. At the same time calls bridged by freeswitch between LAN and any of the VOIP providers behaved just fine. And calls bridged by Asterisk any way were fine too. So that pretty much looked like the end of the freeswitch trials for me.

But then I timed your code, mine and found that all those 1ms sleeps that your timer thread was doing (and all those pollers were doing as well) were actually 4ms sleeps because you know what unless kernel is configured with HZ=1000, you can't sleep for less than 4ms (HZ=250) or perhaps even 10ms (HZ=100). Mine was 250.

This actually meant that the original timer thread was firing once, sleeping for 4ms, firing 3 more times back-to-back, sleeping for 4ms more, firing 4 times back-to-back, etc. It was still firing 20ms timers on time, but 30ms ones of course were not, since 30ms doesn't divide by 4 evenly. Plus whoever relied on runtime.reference or switch_micro_time_now() were kind of screwed because both were running jumpy. Plus whoever assumed that apr_sleep(1000) or cond_yield() was sleeping for 1ms were also in for a surprise. It felt satisfying to find that, however it didn't explain why the same distortions were observed with rewritten timer thread and disabled RTP timers.

Anyway, I sighed (pretty much like you) and recompiled the kernel with HZ=1000. Recompiling kernel on these ALIX boards is fun. If smth goes south, you need to hook up serial console and see what the heck went wrong.

That eliminated distortions, ha! But made freeswitch more CPU hungry. Now the remaining 1ms threads sitting in sofia epoll were really polling for 1ms, not 4, and freeswitch was consistently sitting in the first line of the top chart showing 3% CPU utilization when idle.

Don't know whether it's because of the remaining epolls in sofia or whether it's because there are still some threads left in freeswitch that I neglected to change because they were sleeping with 100ms interval, so I figured, who cares. Maybe when all things come together (sofia, 100ms*N) freeswitch ends up spending 3% of CPU while doing pretty much nothing.

Btw, compared with Asterisk, the latter is not even visible on the first top's screen and spends 1% CPU when bridging two G711 calls and recording them to disk.

So, at this time I have both original Asterisk and FS setups running. One is seemless but clumsy in configuration, the other one is neat and stylish but too preoccupied with smth... Should I look into sofia epollers? That's kind of deep in the code. Or should I just stick with Asterisk?

Anthony Minessale-2 wrote:

...

There is another user here with a 300mhz box. I am willing to

investigate

...

this improved performance for weak devices but I need to do it in a sane cross-platform way.

On Fri, Dec 4, 2009 at 1:32 PM, Yossi Neiman <freeswitch@cartissolutions.com

...

wrote:

...

A word to the wise to the general FreeSWITCH community: If Anthony Minessale suggests that you try to do any number of things, it's a very good idea to try all those ideas before continuing on. I've known him, MikeJ, and bkw for several years, and they almost always have very good ideas as to troubleshoot a problem in FreeSWITCH. It's extremely frustrating to try to help people out who won't try the provided suggestions first.

And note directly to "eaf" - bogomips is quite possibly the least significant bit of data about a cpu that you will get out of /proc/cpuinfo... The name itself - bogo, means bogus. http://en.wikipedia.org/wiki/Bogomips

-Yossi

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm MSN:anthony_minessale@hotmail.com<MSN%3Aanthony_minessale@hotmail.com> <MSN%3Aanthony_minessale@hotmail.com<MSN%253Aanthony_minessale@hotmail.com> < MSN%3Aanthony_minessale@hotmail.com<MSN%253Aanthony_minessale@hotmail.com> <MSN%253Aanthony_minessale@hotmail.com<MSN%25253Aanthony_minessale@hotmail.com>

...

...

GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> <PAYPAL%3Aanthony.minessale@gmail.com<PAYPAL%253Aanthony.minessale@gmail.com>

...

<PAYPAL%3Aanthony.minessale@gmail.com<PAYPAL%253Aanthony.minessale@gmail.com> <PAYPAL%253Aanthony.minessale@gmail.com<PAYPAL%25253Aanthony.minessale@gmail.com>

...

...

IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference sip:888@conference.freeswitch.org<sip%3A888@conference.freeswitch.org>

<sip%3A888@conference.freeswitch.org<sip%253A888@conference.freeswitch.org> < sip%3A888@conference.freeswitch.org<sip%253A888@conference.freeswitch.org> <sip%253A888@conference.freeswitch.org<sip%25253A888@conference.freeswitch.org>

...

...

iax:guest@conference.freeswitch.org/888

googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> <googletalk%3Aconf%2B888@conference.freeswitch.org<googletalk%253Aconf%252B888@conference.freeswitch.org>

...

<googletalk%3Aconf%2B888@conference.freeswitch.org<googletalk%253Aconf%252B888@conference.freeswitch.org> <googletalk%253Aconf%252B888@conference.freeswitch.org<googletalk%25253Aconf%25252B888@conference.freeswitch.org>

...

...

pstn:213-799-1400

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users

UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users

...

http://www.freeswitch.org

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26678873.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE: http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com>< MSN%3Aanthony_minessale@hotmail.com<MSN%253Aanthony_minessale@hotmail.com>

GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> <PAYPAL%3Aanthony.minessale@gmail.com<PAYPAL%253Aanthony.minessale@gmail.com>

IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org>< sip%3A888@conference.freeswitch.org<sip%253A888@conference.freeswitch.org>

iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> <googletalk%3Aconf%2B888@conference.freeswitch.org<googletalk%253Aconf%252B888@conference.freeswitch.org>

pstn:213-799-1400

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26684048.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com <MSN%3Aanthony_minessale@hotmail.com> GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com<PAYPAL%3Aanthony.minessale@gmail.com> IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org <sip%3A888@conference.freeswitch.org> iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org<googletalk%3Aconf%2B888@conference.freeswitch.org> pstn:213-799-1400

A little more data from one of my (our) boxes: starbox_352 ~ # uname -a Linux starbox_352 2.6.26.8-astlinux #1 PREEMPT Tue Nov 24 16:20:52 EST 2009 i586 unknown starbox_352 ~ # starbox_352 ~ # cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 5 model : 10 model name : Geode(TM) Integrated Processor by AMD PCS stepping : 2 cpu MHz : 498.053 cache size : 128 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 1 wp : yes flags : fpu de pse tsc msr cx8 sep pge cmov clflush mmx mmxext 3dnowext 3dnow bogomips : 997.21 clflush size : 32 power management: starbox_352 ~ # cat /etc/astlinux-release astlinux-s2s-3491 starbox_352 ~ # I'll find one that has been in production for a while with some active calls... On Thu, Dec 3, 2009 at 6:49 PM, Anthony Minessale <anthony.minessale@gmail.com> wrote:

Sigh,

You just took it up a notch in terms of disdain and sarcasm. Why do people always only apologize sarcastically?

I asked you to try the -hp and turn off the monotonic clock just to gather the results to help you.  You completely missed it and just went on about the threads.   Please save the "ok fine the code is perfect, blah blah" if you would have just read the email and answered the question I might have cared more about the status of your problem.

I told you both of those threads need to be on their toes because they try to balance between a certian number of sql stmts or 500ms whatever comes first.  When there are thousands of events per second being turned into SQL statements which are in turn compiled into large sql transactions.

If you want to come up with a way that they can sleep longer until there is a sign of activity and stay busy for a few seconds then slow down again, that's probably possible but the process is already idle at 0% cpu so maybe you can appreciate why we are not rushing to work on it.  Maybe I'll give it a go just to show you it has nothing to do with your problem.

Please don't mock our comment about several years.  You have no idea how hard this code was to develop and it's truly insulting.  Its clear to see you are locked into assuming that the busy threads that are not all that busy because they are constantly yielding to the scheduler is breaking the timing code.  I begged you to understand me when i told you that the err is not normal, most boxes do not see it doing nothing and there has to be a specific problem on your box or configuration.  So instead of working with us you want to escalate to snotty comments.  That's pretty normal on the internet I guess.....  If you want to have a constructive conversation about our core, install FS on a normal box, use it for a few weeks, figure out everything about how it works then try.... There was pure speculation and conjecture in your original emails and I never said a word about it until you kept pushing.

Kristian mentioned he never sees that on that same hardware did you even consider following up on why that is?

I don't have your device, but I assume if you get it working well it will certainly help you more than it helps me so you could at least have the decency to believe what we are trying to tell you.

On Thu, Dec 3, 2009 at 3:44 PM, eaf <erandr-junk@usa.net> wrote:

...

Oh, you mean giving FS higher priority? Yeah, as a last resort I'll do that. At the moment, I hope it won't be necessary as I can make those "hyper" threads behave, and will see how that goes first. I see where your implementation could be coming from. There is a queue of SQL queries in sofia.c processed by the worker thread. There are only two pop functions available in APR: queue_pop() and queue_trypop(), so alas no option with a timeout here. You don't want to block the thread in pop() indefinitely because you chose that same worker needs to do ireg and gw processing once in a while (separated by tens or hundreds of seconds, btw). You also want to be able to detect shutdown condition so that the worker doesn't hold up profile thread. So you chose to poll for events every millisecond instead of just creating an apr_thread_cond_t for resource friendly signalling.

I agree that the timer thread philosophy is great and was the right choice for scaling, but I just don't comprehend responses to things like these other SQL or sofia worker threads. Did somebody even remotely acknowledge that busy loops at least in those areas that I showed may probably be a bad idea and could've been eliminated? I've heard suggestions to bump up priority, I've heard that the code was perfect already, that it's the result of 4-year effort, that I am arrogant, don't listen and don't understand squat.

I'm sorry if I gave you impression that I was looking for the bad parts in the software. I apologized for that already. All I wanted was to have constructive conversation, perhaps I'm not too good at it. Code is already perfect according to you? Fine with me.

Anthony Minessale-2 wrote:

...

no,

I mean the one after that that you must have completely skipped with a command line option to try and a param to set in the config. It somewhat annoys me for taking the time to compose it now.  I wrote all of the code you are talking about myself and I was trying to give you some suggestions....

Well, actually,  you did answer my question about the platform so you must have seen it.....

The loops are not the cause of that migration message, something wrong with the hardware or the kernel is. Another guy just told you he does not see that problem on the same exact hardware.

Even if you have a point about the sql threads, you could make a patch to slow them down but you cant slow down too much or you will not be able to handle 400 cps all asking to send updates to transactions in batches of thousands of sql stmts.  Every line of that code is carefully designed so I don't know what else to tell you but to stop being so arrogant and re-read this thread for all the advice you have totally ignored.  I started out trying to help you but I have a lot of work to do.  I thoroughly explained it to you and you are choosing to ignore me so I guess I'm done. You can do whatever you want with your working copy, i'll see you in 3 or 4 years when you get up to speed with the rest of us........

-- View this message in context: http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26633739.html Sent from the Freeswitch-users mailing list archive at Nabble.com.

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Anthony Minessale II

FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire

AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch

FreeSWITCH Developer Conference sip:888@conference.freeswitch.org iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org pstn:213-799-1400

_______________________________________________ FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org

-- Kristian Kielhofner http://www.astlinux.org http://blog.krisk.org http://www.star2star.com http://www.submityoursip.com http://www.voalte.com