Re: [Freeswitch-dev] adding support for incoming streaming audio
The proper approach for this would be to use the media_bugs which are used to record the session have have been recently enhanced to displace the actual audio with audio you can replace. each streaming protocol can implement it's own mod say "mod_shoutcast" and have an application or api command interface hook to attach itself to the channel. media bugs is a concept where you can latch a struct and a callback onto a channel. Your callback will be called continuously when audio is read or written to a channel and you can choose on setup if you want it to be called on read/write with muxed audio from both sides of a call or on write giving you a chance to replace the audio. http://www.freeswitch.org/docs/group__mb1.html If you look in switch_ivr.c in switch_ivr_record_file http://fisheye.freeswitch.org/browse/FreeSWITCH/trunk/src/switch_ivr.c?r=448... You can see this is how it's possible to record from the background. If you look at mod_ivrtest.c (basically a scratch pad to test code) http://fisheye.freeswitch.org/browse/FreeSWITCH/trunk/src/mod/applications/m... the bugtest function shows how to set the channel up to run a bug that gives you a chance to replace the frames as they are supposed to be written to the channel so you could attach your shoutcast socket and whenever audio was to be written to the channel you could replace the frame with one from your stream so the caller would hear the stream instead of silence or whatever the channel was really writing etc. When a bug is active on a channel it forces trancoding even when it's not necessary for a bridge so you can always get the audio as raw SLIN which is the only common audio format you can work with when combining media. Anthony Minessale II FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/ AIM: anthm MSN:anthony_minessale@hotmail.com JABBER:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch FreeSWITCH Developer Conference sip:888@conference.freeswitch.org iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org pstn:213-799-1400 ----- Original Message ---- From: Mark D. Anderson <mda@discerning.com> To: freeswitch-dev@lists.freeswitch.org Sent: Thursday, March 8, 2007 5:04:50 PM Subject: [Freeswitch-dev] adding support for incoming streaming audio I'd like to have a streaming audio source (not just playing a local file) for users while they are parked or waiting for a conference to start. In digging into FS it seems there are several ways this might be approached. Of course this is based on only a superficial understanding of the FS internals. But I've come up with 6 (count them, 6) possibilities: 1) hack the "playback" application. Right now, the lookup of driver in switch-core.c is by file suffix (with no override in xml conf). Currently the only real format module is mod_sndfile. (The string constants for metadata like SWITCH_AUDIO_COL_STR_TITLE are conveniently set to be identical to the enum used by libsndfile.) It could be done by hacking mod_sndfile, or instead by making a new format driver, say "mod_url". libsndfile does not support mp3 (and won't), and is oriented around local files. So a mod_url driver would be better, with support for some suffixes like .pipe or .m3u or .mp3. It would need to do content-type sniffing in some cases. One issue is that it seems that mod_playback is blocking til completion. There is a timer in switch_ivr_play_file; it would be necessary to invert control there to allow the processing to proceed without the file ending. Pros: most useful; wherever a local file is used, a remote could be used instead. Cons: non-trivial amount of hacking. 2) Use an external process to convert streaming audio into sip. For example, using mjua or pjsua as sip clients, along with some command-line streaming audio client such as streamripper. Pros: no change to FS required Cons: lots of moving parts, and the SIP gateway utility will probably require some hacking. 3) implement a "mod_shout" as an endpoint for shoutcast/icecast protocol Pros: fits in well with FS architecture Cons: requires understanding FS internals concerning threading, buffering, etc. 4) implement a "mod_rtsp" as an endpoint. This would mean I'd have to be satisfied with supporting only rtsp streaming audio, not shoutcast/icecast streaming audio. Pros: In theory there wouldn't be much to do, since there is already RTSP support in the FS stack. Cons: requires underestanding FS internals *and* RTSP streaming audio 5) hack mod_conference to do stream-in just like record-out, as a thread. [I'm not sure why mod_conference does it that way, rather than at the core switch level (which *also* has session recording ability) -- isn't the point of a softswitch framework that it can allow for arbitrary combinations of modules? Shouldn't there be a way to supply an arbitrary channel as a member?] Pros: relatively easy to do by blindly copying the session record case. Cons: specific to conferencing, and makes mod_conference even bulkier. 6) implement mod_park with specific support for streaming audio [right now mod_park is just an empty stub.] Pros: standalone so easier to implement Cons: limits its usefulness to just that "application" _______________________________________________ Freeswitch-dev mailing list Freeswitch-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org ____________________________________________________________________________________ Expecting? Get great news right away with email Auto-Check. Try the Yahoo! Mail Beta. http://advision.webevents.yahoo.com/mailbeta/newmail_tools.html
On Fri, 9 Mar 2007 11:35:15 -0800 (PST), "Anthony Minessale" <anthmct@yahoo.com> said:
The proper approach for this would be to use the media_bugs which are used to record the session have have been recently enhanced to displace the actual audio with audio you can replace.
hmmm, I can see this for the case where someone might want to just sniff the frames (like recording) or if it wants to do arbitrary signal manipulation (such as noise cleanup or whatever). (perhaps a more evocative name would be better, like "filter" or "interceptor"?) but this case seems very similar in theory to what it would be to mux audio in a conference, or bridge to a hardware endpoint -- something the core should be able to do without me having to write all the by-frame signal muxing?
each streaming protocol can implement it's own mod say "mod_shoutcast" and have an application or api command interface hook to attach itself to the channel.
but for example that isn't what mod_iax does, yet an IAX call can participate in a conference as a member, right? And so can dingaling. so why would mod_shoutcast be different? far be it to be to argue architecture with the architect :), but i'm still at a loss as to how/why this should be a "media bug" entity instead of a "file" entity or an "endpoint" entity. -mda
media bugs is a concept where you can latch a struct and a callback onto a channel. Your callback will be called continuously when audio is read or written to a channel and you can choose on setup if you want it to be called on read/write with muxed audio from both sides of a call or on write giving you a chance to replace the audio.
http://www.freeswitch.org/docs/group__mb1.html
If you look in switch_ivr.c in switch_ivr_record_file http://fisheye.freeswitch.org/browse/FreeSWITCH/trunk/src/switch_ivr.c?r=448...
You can see this is how it's possible to record from the background.
If you look at mod_ivrtest.c (basically a scratch pad to test code) http://fisheye.freeswitch.org/browse/FreeSWITCH/trunk/src/mod/applications/m...
the bugtest function shows how to set the channel up to run a bug that gives you a chance to replace the frames as they are supposed to be written to the channel so you could attach your shoutcast socket and whenever audio was to be written to the channel you could replace the frame with one from your stream so the caller would hear the stream instead of silence or whatever the channel was really writing etc.
When a bug is active on a channel it forces trancoding even when it's not necessary for a bridge so you can always get the audio as raw SLIN which is the only common audio format you can work with when combining media.
Anthony Minessale II
FreeSWITCH http://www.freeswitch.org/ ClueCon http://www.cluecon.com/
AIM: anthm MSN:anthony_minessale@hotmail.com JABBER:anthony.minessale@gmail.com IRC: irc.freenode.net #freeswitch
FreeSWITCH Developer Conference sip:888@conference.freeswitch.org iax:guest@conference.freeswitch.org/888 googletalk:conf+888@conference.freeswitch.org pstn:213-799-1400
----- Original Message ---- From: Mark D. Anderson <mda@discerning.com> To: freeswitch-dev@lists.freeswitch.org Sent: Thursday, March 8, 2007 5:04:50 PM Subject: [Freeswitch-dev] adding support for incoming streaming audio
I'd like to have a streaming audio source (not just playing a local file) for users while they are parked or waiting for a conference to start.
In digging into FS it seems there are several ways this might be approached. Of course this is based on only a superficial understanding of the FS internals. But I've come up with 6 (count them, 6) possibilities:
1) hack the "playback" application. Right now, the lookup of driver in switch-core.c is by file suffix (with no override in xml conf). Currently the only real format module is mod_sndfile. (The string constants for metadata like SWITCH_AUDIO_COL_STR_TITLE are conveniently set to be identical to the enum used by libsndfile.)
It could be done by hacking mod_sndfile, or instead by making a new format driver, say "mod_url". libsndfile does not support mp3 (and won't), and is oriented around local files. So a mod_url driver would be better, with support for some suffixes like .pipe or .m3u or .mp3. It would need to do content-type sniffing in some cases.
One issue is that it seems that mod_playback is blocking til completion. There is a timer in switch_ivr_play_file; it would be necessary to invert control there to allow the processing to proceed without the file ending.
Pros: most useful; wherever a local file is used, a remote could be used instead. Cons: non-trivial amount of hacking.
2) Use an external process to convert streaming audio into sip. For example, using mjua or pjsua as sip clients, along with some command-line streaming audio client such as streamripper.
Pros: no change to FS required Cons: lots of moving parts, and the SIP gateway utility will probably require some hacking.
3) implement a "mod_shout" as an endpoint for shoutcast/icecast protocol
Pros: fits in well with FS architecture Cons: requires understanding FS internals concerning threading, buffering, etc.
4) implement a "mod_rtsp" as an endpoint. This would mean I'd have to be satisfied with supporting only rtsp streaming audio, not shoutcast/icecast streaming audio.
Pros: In theory there wouldn't be much to do, since there is already RTSP support in the FS stack. Cons: requires underestanding FS internals *and* RTSP streaming audio
5) hack mod_conference to do stream-in just like record-out, as a thread. [I'm not sure why mod_conference does it that way, rather than at the core switch level (which *also* has session recording ability) -- isn't the point of a softswitch framework that it can allow for arbitrary combinations of modules? Shouldn't there be a way to supply an arbitrary channel as a member?]
Pros: relatively easy to do by blindly copying the session record case. Cons: specific to conferencing, and makes mod_conference even bulkier.
6) implement mod_park with specific support for streaming audio [right now mod_park is just an empty stub.]
Pros: standalone so easier to implement Cons: limits its usefulness to just that "application"
_______________________________________________ Freeswitch-dev mailing list Freeswitch-dev@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-dev UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-dev http://www.freeswitch.org
____________________________________________________________________________________ Expecting? Get great news right away with email Auto-Check. Try the Yahoo! Mail Beta. http://advision.webevents.yahoo.com/mailbeta/newmail_tools.html
participants (2)
-
Anthony Minessale
-
mda＠discerning.com