What about the things I spent time suggesting in my last email?<br>Did you try them because I was actually curious if they made any impact.<br><br><br><div class="gmail_quote">On Thu, Dec 3, 2009 at 11:29 AM, eaf <span dir="ltr"><<a href="mailto:erandr-junk@usa.net">erandr-junk@usa.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br>
I'm sorry if I sounded that way. Did mean to. :)<br>
<br>
Yes, it's an embedded platform. It's an ALIX board with AMD Geode LX800 chip<br>
and 256MB of RAM. <a href="http://www.pcengines.ch/alix2d3.htm" target="_blank">http://www.pcengines.ch/alix2d3.htm</a><br>
<br>
Line offset difference is due to some minor logging changes I made to see<br>
who's allocating timers and how often. This way I found MOH streaming and<br>
that RTP still allocates timers even when it's set to none in the profile.<br>
<br>
I feel that this platform turned out to be underpowered for FS because it<br>
cannot meet its scheduling expectations. I guess, some degree of kernel<br>
tweaking or setting priorities will fix that. Meanwhile I just got rid of<br>
the SQLDB 1ms thread via -nosql command line option, split sofia worker 1ms<br>
thread in two (one blocked and waiting for new commands in the SQL queue,<br>
the other one checking registrations and gateways with 1sec interval), and<br>
don't know yet what to do about the timer thread.<br>
<br>
Again, I apologize for stupid or accusing questions, I'm just trying to see<br>
how FS can be made friendlier to this board. Or the board be made friendlier<br>
to FS ;)<br>
<div class="im"><br>
<br>
Anthony Minessale-2 wrote:<br>
><br>
> If you see that message then your machine/os/combo is having some problems<br>
> keeping up.<br>
> It's not the timer missing anything its the monotonic clock detecting a 1<br>
> second or more differential from what its next prediction for the time<br>
> should be. The best way to trigger this would be to suspend FS with<br>
> control-z or attach to it with gdb blocking the entire process, that 1ms<br>
> thread would have to miss 1000 iterations to trigger that warning.<br>
><br>
> Btw, that error message is at line 471 not 473 so you are using modified<br>
> code.<br>
><br>
> Its possible your box has a bad monotonic timer, you can set<br>
><br>
><br>
><br>
</div><div><div></div><div class="h5">> under <settings> in switch.conf.xml<br>
><br>
> We are now starting to guess you are using some small embedded type<br>
> platform<br>
> perhaps?<br>
> I've run FS even on a nokia n810 and never caused that message to fire.<br>
><br>
> if 1 call can interrupt the cpu enough to cause noticeable issues you<br>
> might<br>
> want to consider running the process at a<br>
> greater priority by using the -hp command line arg or at least nice it<br>
><br>
> Why don't you tell us the whole story about what OS/platform you are using<br>
> here rather that form conjectures about what is wrong with our code that<br>
> thousands of people are happy with.<br>
><br>
><br>
><br>
><br>
><br>
><br>
><br>
> On Thu, Dec 3, 2009 at 8:55 AM, eaf <<a href="mailto:erandr-junk@usa.net">erandr-junk@usa.net</a>> wrote:<br>
><br>
>><br>
>> Btw, I have these popping up in my logs from time to time:<br>
>><br>
>> 2009-12-03 09:42:06.035294 [DEBUG] switch_core_state_machine.c:314<br>
>> (sofia/external/<a href="mailto:xxxxx@4.68.250.148">xxxxx@4.68.250.148</a>) Running State Change CS_HANGUP<br>
>> 2009-12-03 09:42:06.035294 [CRIT] switch_time.c:473 Virtual Migration<br>
>> Detected! Syncing Clock<br>
>><br>
>> In this case an incoming call rang to both FS and Asterisk, Asterisk<br>
>> picked<br>
>> up, but the surge of activity made FS timer thread miss a beat or two.<br>
>><br>
>><br>
>> eaf wrote:<br>
>> ><br>
>> > Oh, it's not just one timer thread... Why, why is sql_thread keeps on<br>
>> > checking for messages every millisecond? Couldn't there be some<br>
>> signalling<br>
>> > implemented that will make the thread suspend on condition variable or<br>
>> a<br>
>> > socket/pipe in between?<br>
>> ><br>
>> > #0 do_sleep (t=1000) at src/switch_time.c:109<br>
>> > #1 0xb7e5e215 in switch_core_sql_thread (thread=0xb7586ae8, obj=0x0)<br>
>> at<br>
>> > src/switch_core_sqldb.c:783<br>
>> ><br>
>> > Why does this sofia_profile_worker_thread keeps on looping checking for<br>
>> > the queue? Have a semaphore!<br>
>> ><br>
>> > #0 do_sleep (t=1000) at src/switch_time.c:109<br>
>> > #1 0xb73a4701 in sofia_profile_worker_thread_run (thread=0x80f3a30,<br>
>> > obj=0x80f2490) at sofia.c:978<br>
>> ><br>
>> > Nothing's happening on the box, but there are three threads that<br>
>> pretend<br>
>> > to be actively busy with smth. Others at least sleep for hundreds of<br>
>> > milliseconds, not for one.<br>
>> ><br>
>> > And there is even infrastructure present to do blocking pops: i.e. why<br>
>> > couldn't sqldb thread do queue_pop() instead of queue_trypop()<br>
>> intermixed<br>
>> > with 1ms sleeps? This looping is such a waste...<br>
>> ><br>
>> ><br>
>> > eaf wrote:<br>
>> >><br>
>> >> As I see it, switch_cond_next() currently is just a do_sleep(1000).<br>
>> Yes,<br>
>> >> it could be mapped to a 1ms timer, but "#define DISABLE_1MS_COND"<br>
>> >> overrides that.<br>
>> >><br>
>> >> Yeah, there is a global timestamp... It's easy to workaround that for<br>
>> RTP<br>
>> >> who calls switch_micro_time_now()... But if somebody accesses<br>
>> >> runtime.timestamp directly, it's gonna be tough to grep for that. If<br>
>> only<br>
>> >> this was C++...<br>
>> >><br>
>> >> I'll play around. Never liked polling too much. Never could've guessed<br>
>> >> that polling could be so useful for scalability ;) My naive<br>
>> >> implementation would've pulled timestamp via system calls and would've<br>
>> >> done sleeping by passing exact interval to select() instead of syncing<br>
>> >> with a pacing thread. Which would be dead-quiet at idle time, but, of<br>
>> >> course, would stop scaling at some point due to excessive number of<br>
>> >> system calls.<br>
>> >><br>
>> >> Thanks.<br>
>> >><br>
>> >><br>
>> >> Michael Jerris wrote:<br>
>> >>><br>
>> >>> In short. No, you can not for many reasons. The milisecond tic is<br>
>> >>> used throughout the code even when there is not any calls up. You<br>
>> can<br>
>> >>> grep for switch_cond_next if you would like to see where but it is<br>
>> >>> required to keep our global timestamp and for pacing the scheduler<br>
>> >>> among other services that run all the time.<br>
>> >>><br>
>> >>> Mike<br>
>> >>><br>
>> >>> On Dec 2, 2009, at 7:31 PM, eaf <<a href="mailto:erandr-junk@usa.net">erandr-junk@usa.net</a>> wrote:<br>
>> >>><br>
>> >>>><br>
>> >>>> Can I reduce resolution of that timer thread 10 times? I mean, I<br>
>> >>>> glanced<br>
>> >>>> through the code, and see that among others (are there others?) RTP<br>
>> >>>> and IVR<br>
>> >>>> set up their timers that are subsequently managed by this thread.<br>
>> >>>> RTP timers<br>
>> >>>> should be eliminated by that setting you've suggested. IVR timers<br>
>> >>>> are set at<br>
>> >>>> 20ms... So, if the thread is set to wake up every 10ms instead of<br>
>> >>>> 1ms it<br>
>> >>>> should be able to wake up those IVR timers just fine. Right?<br>
>> >>>><br>
>> >>>> That's a cool design to have one dedicated thread that maintains<br>
>> >>>> accurate<br>
>> >>>> timing and then broadcasts via condition variables to hundreds of<br>
>> >>>> other<br>
>> >>>> threads events that they can register for. I'm sure it's one of the<br>
>> >>>> reasons<br>
>> >>>> why FS scales so much better than Asterisk. But for poor low-end<br>
>> >>>> setups that<br>
>> >>>> sit in the closet, eat only 6W of power and hardly ever run more<br>
>> >>>> than two<br>
>> >>>> calls at the same time, can I hack it somehow to be more UNIX-<br>
>> >>>> friendly? I.e.<br>
>> >>>> make it stuck in select() or recv() when there is nothing to do,<br>
>> call<br>
>> >>>> clock_gettime() right from the thread that wants and when it wants<br>
>> >>>> to know<br>
>> >>>> current time?<br>
>> >>>><br>
>> >>>> Say, what if that thread is made to suspend on a condition variable<br>
>> >>>> in case<br>
>> >>>> if there are no timers registered in TIMER_MATRIX? Then, if some<br>
>> other<br>
>> >>>> thread comes up and adds its timer into the matrix, it could wake up<br>
>> >>>> the<br>
>> >>>> timer thread and enjoy accurate timing as needed, on demand? And in-<br>
>> >>>> between<br>
>> >>>> the calls, when there is no RTP or IVR, it will all go silent? I<br>
>> mean,<br>
>> >>>> sitting on a wait queue in the kernel is way better than go back and<br>
>> >>>> forth<br>
>> >>>> incrementing counters that nobody even needs at the moment?<br>
>> >>>><br>
>> >>>><br>
>> >>>> Anthony Minessale-2 wrote:<br>
>> >>>>><br>
>> >>>>> idle is a 4 letter word to a realtime application.<br>
>> >>>>><br>
>> >>>>> The core keeps a single high-priority thread to keep 1ms timing and<br>
>> >>>>> expands<br>
>> >>>>> that broadcasting<br>
>> >>>>> to hundreds or thousand of threads who need accurate timing.<br>
>> >>>>><br>
>> >>>>> Your choppy audio is caused by linksys lying about the packet len<br>
>> >>>>> that<br>
>> >>>>> it's<br>
>> >>>>> using and we set our timer<br>
>> >>>>> to the wrong speed.<br>
>> >>>>><br>
>> >>>>><br>
>> >>>><br>
>> >>>> --<br>
>> >>>> View this message in context:<br>
>> >>>><br>
>> <a href="http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html" target="_blank">http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26619085.html</a><br>
>> >>>> Sent from the Freeswitch-users mailing list archive at Nabble.com.<br>
>> >>>><br>
>> >>>><br>
>> >>>> _______________________________________________<br>
>> >>>> FreeSWITCH-users mailing list<br>
>> >>>> <a href="mailto:FreeSWITCH-users@lists.freeswitch.org">FreeSWITCH-users@lists.freeswitch.org</a><br>
>> >>>> <a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
>> >>>> UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-</a><br>
>> >>>> users<br>
>> >>>> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
>> >>><br>
>> >>> _______________________________________________<br>
>> >>> FreeSWITCH-users mailing list<br>
>> >>> <a href="mailto:FreeSWITCH-users@lists.freeswitch.org">FreeSWITCH-users@lists.freeswitch.org</a><br>
>> >>> <a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
>> >>> UNSUBSCRIBE:<br>
>> <a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
>> >>> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
>> >>><br>
>> >>><br>
>> >><br>
>> >><br>
>> ><br>
>> ><br>
>><br>
>> --<br>
>> View this message in context:<br>
>> <a href="http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html" target="_blank">http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26627246.html</a><br>
>> Sent from the Freeswitch-users mailing list archive at Nabble.com.<br>
>><br>
>><br>
>> _______________________________________________<br>
>> FreeSWITCH-users mailing list<br>
>> <a href="mailto:FreeSWITCH-users@lists.freeswitch.org">FreeSWITCH-users@lists.freeswitch.org</a><br>
>> <a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
>> UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
>> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
>><br>
><br>
><br>
><br>
> --<br>
> Anthony Minessale II<br>
><br>
> FreeSWITCH <a href="http://www.freeswitch.org/" target="_blank">http://www.freeswitch.org/</a><br>
> ClueCon <a href="http://www.cluecon.com/" target="_blank">http://www.cluecon.com/</a><br>
> Twitter: <a href="http://twitter.com/FreeSWITCH_wire" target="_blank">http://twitter.com/FreeSWITCH_wire</a><br>
><br>
> AIM: anthm<br>
</div></div>> <a href="mailto:MSN%3Aanthony_minessale@hotmail.com">MSN:anthony_minessale@hotmail.com</a> <<a href="mailto:MSN%253Aanthony_minessale@hotmail.com">MSN%3Aanthony_minessale@hotmail.com</a>><br>
> GTALK/JABBER/<a href="mailto:PAYPAL%3Aanthony.minessale@gmail.com">PAYPAL:anthony.minessale@gmail.com</a><<a href="mailto:PAYPAL%253Aanthony.minessale@gmail.com">PAYPAL%3Aanthony.minessale@gmail.com</a>><br>
<div class="im">> IRC: <a href="http://irc.freenode.net" target="_blank">irc.freenode.net</a> #freeswitch<br>
><br>
> FreeSWITCH Developer Conference<br>
</div>> <a href="mailto:sip%3A888@conference.freeswitch.org">sip:888@conference.freeswitch.org</a> <<a href="mailto:sip%253A888@conference.freeswitch.org">sip%3A888@conference.freeswitch.org</a>><br>
<div class="im">> <a href="http://iax:guest@conference.freeswitch.org/888" target="_blank">iax:guest@conference.freeswitch.org/888</a><br>
</div>> <a href="mailto:googletalk%3Aconf%2B888@conference.freeswitch.org">googletalk:conf+888@conference.freeswitch.org</a><<a href="mailto:googletalk%253Aconf%252B888@conference.freeswitch.org">googletalk%3Aconf%2B888@conference.freeswitch.org</a>><br>
> pstn:213-799-1400<br>
><br>
<div class="im">> _______________________________________________<br>
> FreeSWITCH-users mailing list<br>
> <a href="mailto:FreeSWITCH-users@lists.freeswitch.org">FreeSWITCH-users@lists.freeswitch.org</a><br>
> <a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
> UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
> <a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
><br>
><br>
<br>
--<br>
</div>View this message in context: <a href="http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html" target="_blank">http://old.nabble.com/Choppy-sound-with-PCMU-tp26594250p26629856.html</a><br>
<div><div></div><div class="h5">Sent from the Freeswitch-users mailing list archive at Nabble.com.<br>
<br>
<br>
_______________________________________________<br>
FreeSWITCH-users mailing list<br>
<a href="mailto:FreeSWITCH-users@lists.freeswitch.org">FreeSWITCH-users@lists.freeswitch.org</a><br>
<a href="http://lists.freeswitch.org/mailman/listinfo/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/listinfo/freeswitch-users</a><br>
UNSUBSCRIBE:<a href="http://lists.freeswitch.org/mailman/options/freeswitch-users" target="_blank">http://lists.freeswitch.org/mailman/options/freeswitch-users</a><br>
<a href="http://www.freeswitch.org" target="_blank">http://www.freeswitch.org</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>Anthony Minessale II<br><br>FreeSWITCH <a href="http://www.freeswitch.org/">http://www.freeswitch.org/</a><br>ClueCon <a href="http://www.cluecon.com/">http://www.cluecon.com/</a><br>
Twitter: <a href="http://twitter.com/FreeSWITCH_wire">http://twitter.com/FreeSWITCH_wire</a><br><br>AIM: anthm<br><a href="mailto:MSN%3Aanthony_minessale@hotmail.com">MSN:anthony_minessale@hotmail.com</a><br>GTALK/JABBER/<a href="mailto:PAYPAL%3Aanthony.minessale@gmail.com">PAYPAL:anthony.minessale@gmail.com</a><br>
IRC: <a href="http://irc.freenode.net">irc.freenode.net</a> #freeswitch<br><br>FreeSWITCH Developer Conference<br><a href="mailto:sip%3A888@conference.freeswitch.org">sip:888@conference.freeswitch.org</a><br><a href="http://iax:guest@conference.freeswitch.org/888">iax:guest@conference.freeswitch.org/888</a><br>
<a href="mailto:googletalk%3Aconf%2B888@conference.freeswitch.org">googletalk:conf+888@conference.freeswitch.org</a><br>pstn:213-799-1400<br>