Conference timer synchronization with T1 FreeTDM - how?
Looking at the source for mod_conference.c, I see that the timer_name is hard-coded to "soft", i.e. the 1000 Hz. system clock. How can this work with a T1-based FreeTDM system, where the FreeTDM channels are presumably synchronized to the T1 clock source? I'm seeing some weirdness with my FreeTDM / Dahdi system, where only FreeTDM channels are hearing distortion on conference, and I'm almost certain that this is the culprit, but I have no idea how to proceed to fix it. I'm pretty new to FreeTDM, so I'm hoping that I'm missing something simple. Any thoughts? Thanks, Bob
Hi Bob, On Tue, Sep 24, 2013 at 6:30 PM, Bob Hartwig <bobjectsfreeswitch@gmail.com>wrote:
Looking at the source for mod_conference.c, I see that the timer_name is hard-coded to "soft", i.e. the 1000 Hz. system clock. How can this work with a T1-based FreeTDM system, where the FreeTDM channels are presumably synchronized to the T1 clock source?
Yeah I think something is fishy there. As I see the code, it ends up being timed by the software timer on top of the hardware timer. I've not used the conferencing module of FreeSWITCH in TDM much, but in the few instances where I used it in the past seemed to work, mind you it was a small setup with a few callers, so I may have missed the distortion problem and our customer never complained (but the issue may be lurking there). mod_conference ends up using one main thread for mixing (for the whole conference) and then each member of the conference has 2 threads, input and output threads, the output thread is the same as the session thread, the input thread is launched when the member joins. The input thread IO timing seems to be completely based on the endpoint read clock (in FreeTDM case, using hardware timer). The output thread on the other hand uses the soft-timer to time the calls to switch_core_session_write() (and then internally the write will be timed by the hardware timer, again).
I'm seeing some weirdness with my FreeTDM / Dahdi system, where only FreeTDM channels are hearing distortion on conference, and I'm almost certain that this is the culprit, but I have no idea how to proceed to fix it. I'm pretty new to FreeTDM, so I'm hoping that I'm missing something simple. Any thoughts?
What kind of distortion exactly? You may want to use the "ftdm trace" command to record the audio read/written to the TDM endpoint and see if the audio is clean when written to the tdm endopoint. You will have to add some debugging to see if read/write is being done at the proper times. The uuid_debug_media command would have been handy for that. Unfortunately seems like that is implemented for RTP only, we would have to add some code in mod_freetdm.c to handle that command and print timing information for read/write for a given session. What sampling rate is your conference? what packetization size? (20ms, 30ms, 40ms??) what kind of hardware are you using? Moy *Moises Silva **Manager, Software Engineering*** msilva@sangoma.com Sangoma Technologies 100 Renfrew Drive, Suite 100, Markham, ON L3R 9R6 Canada t. +1 800 388 2475 (N. America) t. +1 905 474 1990 x128 f. +1 905 474 9223 **<http://www.sangoma.com/contact?utm_source=signature&utm_medium=email&utm_campaign=email+signatures> Products<http://sangoma.com/products?utm_source=signature&utm_medium=email&utm_campaign=email%2Bsignatures> | Solutions<http://sangoma.com/solutions?utm_source=signature&utm_medium=email&utm_campaign=email%2Bsignatures> | Events<http://sangoma.com/about_us/events?utm_source=signature&utm_medium=email&utm_campaign=email%2Bsignatures> | Contact<http://www.sangoma.com/contact?utm_source=signature&utm_medium=email&utm_campaign=email%2Bsignatures> | Wiki<http://wiki.sangoma.com/?utm_source=signature&utm_medium=email&utm_campaign=email%2Bsignatures> | Facebook<http://www.facebook.com/pages/Sangoma-VoIP-Cards/43578453335?utm_source=signature&utm_medium=email&utm_campaign=email%2Bsignatures> | Twitter<http://www.twitter.com/sangoma?utm_source=signature&utm_medium=email&utm_campaign=email%2Bsignatures>`| | YouTube<http://www.youtube.com/sangomatechnologies?utm_source=signature&utm_medium=email&utm_campaign=email%2Bsignatures>
Hi Moises, On Tue, Sep 24, 2013 at 9:55 PM, Moises Silva <moises.silva@gmail.com>wrote:
The input thread IO timing seems to be completely based on the endpoint read clock (in FreeTDM case, using hardware timer). The output thread on the other hand uses the soft-timer to time the calls to switch_core_session_write() (and then internally the write will be timed by the hardware timer, again).
Sorry if this is a dumb question, but is there any way to hack things to make all of FreeSWITCH sync up with the FreeTDM clock source? I.e. the "soft" timer would be synchronized to the FreeTDM clock source?
I'm seeing some weirdness with my FreeTDM / Dahdi system, where only FreeTDM channels are hearing distortion on conference, and I'm almost certain that this is the culprit, but I have no idea how to proceed to fix it. I'm pretty new to FreeTDM, so I'm hoping that I'm missing something simple. Any thoughts?
What kind of distortion exactly?
Here is how I described it in an earlier email: "I would classify the distortion as a "crackle", but it is very regularly periodic. Repeating ticks at (roughly) 10-20 Hz repeat rate, I would say. I can only hear them when audio is present, i.e. the audio seems to be modulated with the ticks. And that conference audio can be the MOH guitar recording." It is only on conference, and only for TDM channels. WAV file playback does not have the problem (but I assume that playing a WAV on conference would, since MOH does).
You may want to use the "ftdm trace" command to record the audio read/written to the TDM endpoint and see if the audio is clean when written to the tdm endopoint.
OK, I'll try that.
What sampling rate is your conference? what packetization size? (20ms, 30ms, 40ms??) what kind of hardware are you using?
8000, 20 mS (and I verified that everything is 20), and a quad Rhino board using Dahdi. Thanks, Bob
You could test your theory more by making an extension like this: <extension name="test"> <condition field="destination_number" expression="^1234$"> <action application="answer"/> <action application="set" data="time_name=soft"/> <action application="playback" data="tone_stream://%(251,0,1004);loops=-1"/> </condition> </extension> This would produce a timer and use it for playback instead of the read stream. If you wanted to make a timer you could use from the TDM you would have to make a FS module with raw dahdi code in it that opened a socket and clocked off of it and pushed it out to the channels like how some of the other timer modules do but I think if you are on a modern kernel with timerfd that this should not be necessary. If you are on an older platform like centos5 you could try the old -heavy-timer command line arg. Also don't rule out the zt mod for freetdm or dahdi itself. Most people use FreeTDM with sangoma cards so the zt mod code is more of a variable. On Tue, Sep 24, 2013 at 9:32 PM, Bob Hartwig <bobjectsfreeswitch@gmail.com>wrote:
Hi Moises,
On Tue, Sep 24, 2013 at 9:55 PM, Moises Silva <moises.silva@gmail.com>wrote:
The input thread IO timing seems to be completely based on the endpoint read clock (in FreeTDM case, using hardware timer). The output thread on the other hand uses the soft-timer to time the calls to switch_core_session_write() (and then internally the write will be timed by the hardware timer, again).
Sorry if this is a dumb question, but is there any way to hack things to make all of FreeSWITCH sync up with the FreeTDM clock source? I.e. the "soft" timer would be synchronized to the FreeTDM clock source?
I'm seeing some weirdness with my FreeTDM / Dahdi system, where only FreeTDM channels are hearing distortion on conference, and I'm almost certain that this is the culprit, but I have no idea how to proceed to fix it. I'm pretty new to FreeTDM, so I'm hoping that I'm missing something simple. Any thoughts?
What kind of distortion exactly?
Here is how I described it in an earlier email: "I would classify the distortion as a "crackle", but it is very regularly periodic. Repeating ticks at (roughly) 10-20 Hz repeat rate, I would say. I can only hear them when audio is present, i.e. the audio seems to be modulated with the ticks. And that conference audio can be the MOH guitar recording." It is only on conference, and only for TDM channels. WAV file playback does not have the problem (but I assume that playing a WAV on conference would, since MOH does).
You may want to use the "ftdm trace" command to record the audio read/written to the TDM endpoint and see if the audio is clean when written to the tdm endopoint.
OK, I'll try that.
What sampling rate is your conference? what packetization size? (20ms, 30ms, 40ms??) what kind of hardware are you using?
8000, 20 mS (and I verified that everything is 20), and a quad Rhino board using Dahdi.
Thanks, Bob
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
Hello Anthony, On Tue, Sep 24, 2013 at 11:45 PM, Anthony Minessale < anthony.minessale@gmail.com> wrote:
You could test your theory more by making an extension like this:
<extension name="test"> <condition field="destination_number" expression="^1234$"> <action application="answer"/> <action application="set" data="time_name=soft"/> <action application="playback" data="tone_stream://%(251,0,1004);loops=-1"/> </condition> </extension>
This would produce a timer and use it for playback instead of the read stream.
Yes, I did just that yesterday, and the playback is similarly distorted then. That experiment is what made me pretty certain that the mod_conference timer was my culprit.
If you wanted to make a timer you could use from the TDM you would have to make a FS module with raw dahdi code in it that opened a socket and clocked off of it and pushed it out to the channels like how some of the other timer modules do but I think if you are on a modern kernel with timerfd that this should not be necessary. If you are on an older platform like centos5 you could try the old -heavy-timer command line arg.
-heavy-timer is new to me, thanks for the tip. My problem is not timer accuracy, but timer synchronization with the T1 clock. But grepping for heavy-timer may give me some insight on how to replace the main timer if it turns out that I need to go that way. Thanks, Bob
There are buffers in here… so as long as we are always a chunk ahead in the conference, the timer accuracy shouldn't matter unless there is drift over time. The only way it would cause distortion would be if there was not a packet ready to write to the t1 when it needed. How frequent are the blips, and are they at a fairly regular schedule? On Sep 25, 2013, at 5:36 AM, Bob Hartwig <bobjectsfreeswitch@gmail.com> wrote:
Hello Anthony,
On Tue, Sep 24, 2013 at 11:45 PM, Anthony Minessale <anthony.minessale@gmail.com> wrote: You could test your theory more by making an extension like this:
<extension name="test"> <condition field="destination_number" expression="^1234$"> <action application="answer"/> <action application="set" data="time_name=soft"/> <action application="playback" data="tone_stream://%(251,0,1004);loops=-1"/> </condition> </extension>
This would produce a timer and use it for playback instead of the read stream.
Yes, I did just that yesterday, and the playback is similarly distorted then. That experiment is what made me pretty certain that the mod_conference timer was my culprit.
If you wanted to make a timer you could use from the TDM you would have to make a FS module with raw dahdi code in it that opened a socket and clocked off of it and pushed it out to the channels like how some of the other timer modules do but I think if you are on a modern kernel with timerfd that this should not be necessary. If you are on an older platform like centos5 you could try the old -heavy-timer command line arg.
-heavy-timer is new to me, thanks for the tip. My problem is not timer accuracy, but timer synchronization with the T1 clock. But grepping for heavy-timer may give me some insight on how to replace the main timer if it turns out that I need to go that way.
Thanks, Bob
On Wed, Sep 25, 2013 at 7:36 AM, Bob Hartwig <bobjectsfreeswitch@gmail.com>wrote:
Hello Anthony,
On Tue, Sep 24, 2013 at 11:45 PM, Anthony Minessale < anthony.minessale@gmail.com> wrote:
You could test your theory more by making an extension like this:
<extension name="test"> <condition field="destination_number" expression="^1234$"> <action application="answer"/> <action application="set" data="time_name=soft"/> <action application="playback" data="tone_stream://%(251,0,1004);loops=-1"/> </condition> </extension>
This would produce a timer and use it for playback instead of the read stream.
Yes, I did just that yesterday, and the playback is similarly distorted then. That experiment is what made me pretty certain that the mod_conference timer was my culprit.
If you wanted to make a timer you could use from the TDM you would have to make a FS module with raw dahdi code in it that opened a socket and clocked off of it and pushed it out to the channels like how some of the other timer modules do but I think if you are on a modern kernel with timerfd that this should not be necessary. If you are on an older platform like centos5 you could try the old -heavy-timer command line arg.
-heavy-timer is new to me, thanks for the tip. My problem is not timer accuracy, but timer synchronization with the T1 clock. But grepping for heavy-timer may give me some insight on how to replace the main timer if it turns out that I need to go that way.
Thanks, Bob
Not replace the main timer ...., we have timer modules you can create and configure. I gave you the recipe above. You didn't note the distro really. One more warning, dahdi runs at 1000hz internally because it's paranoid that it won't accurately catch the 8000hz. Asterisk and all of its satellites like zaptel/dahdi etc always ran on a notion of using the read stream to clock the write. It would be really nice if you could reproduce this issue on a Sangoma setup to rule out every moving part we don't control between FreeTDM and your T1 line. I know Sangoma did a lot of work inspired by ClueCon 1 to produce slower interrupt timing sync. I'm sure you could find someone to run the test for you on other boxes from the extension I pasted above and see if the problem is reproducible elsewhere.
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
FreeSWITCH-powered IP PBX: The CudaTel Communication Server http://www.cudatel.com
Official FreeSWITCH Sites http://www.freeswitch.org http://wiki.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ Twitter: http://twitter.com/FreeSWITCH_wire AIM: anthm MSN:anthony_minessale@hotmail.com GTALK/JABBER/PAYPAL:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org googletalk:conf+888@conference.freeswitch.org pstn:+19193869900
Hello Anthony, I'm refering to this thread from September 2013: http://lists.freeswitch.org/pipermail/freeswitch-users/2013-September/100068... We are experiencing the same problems like Bob, but with Sangoma cards, not Dahdi. We already contacted Sangoma and asked them to find a solution for this, but so far they didn't feel responsible. Since Sangoma cards are recommended by FreeSWITCH there should be a working timer module for Sangoma, shouldn't it? So it would be great if you could help and talk to Sangoma asking them to create this timer module! Thanks Melanie
participants (5)
-
Anthony Minessale
-
Bob Hartwig
-
Melanie Treitinger
-
Michael Jerris
-
Moises Silva