Copy incoming audio from participants for ASR
Hello everyone, i have a technical question about freeswitch and would like to know if my idea is technically possible. I've got a project at the university and my goal is to improve automatic speech recognition for bigbluebutton. In the summer semester a group of students of my university made a subtitling plugin[1] for bigbluebutton. This plugin adds a participant into the meeting and grabs his audiostream to send it to the kaldi ASR Software. When more than one person speaks at a time the detection decreases. My idea is to copy the incoming freeswitch audiostreams from every participant and send them directly as a copy to the ASR (kaldi can handle multiple audiostreams at once). In the Confluence-Wiki i searched for something in this direction but only found things like Dialplan to ring multiple telephones at once. I hope my idea is formulated understandably. Kind regards, Robert [1] https://github.com/3wille/bbb-kaldi-connector
Hi Robert, FreeSWITCH can be extended by writing custom modules. Modules can register a callback to get every audio frame passed by the core FS to them. You can find a lot of examples in src/mod (best look into src/mod/applications, e.g. mod_avmd works with audio frames). kind regards, Piotr Gregor Software Engineer M: (+44) 07483 866 525 L: (+44) 01256 597 470 www: dataandsignal.com On Mon, Dec 14, 2020 at 4:56 PM R G <mitachundkrach@gmail.com> wrote:
Hello everyone,
i have a technical question about freeswitch and would like to know if my idea is technically possible.
I've got a project at the university and my goal is to improve automatic speech recognition for bigbluebutton.
In the summer semester a group of students of my university made a subtitling plugin[1] for bigbluebutton. This plugin adds a participant into the meeting and grabs his audiostream to send it to the kaldi ASR Software. When more than one person speaks at a time the detection decreases.
My idea is to copy the incoming freeswitch audiostreams from every participant and send them directly as a copy to the ASR (kaldi can handle multiple audiostreams at once).
In the Confluence-Wiki i searched for something in this direction but only found things like Dialplan to ring multiple telephones at once.
I hope my idea is formulated understandably.
Kind regards,
Robert
[1] https://github.com/3wille/bbb-kaldi-connector
_________________________________________________________________________
The FreeSWITCH project is sponsored by SignalWire https://signalwire.com Enhance your FreeSWITCH install with disruptive priced SMS and PSTN services. Build your next product on our scalable cloud platform.
Join our online community to chat in real time https://signalwire.community
Professional FreeSWITCH Services sales@freeswitch.com https://freeswitch.com
Official FreeSWITCH Sites https://freeswitch.com/oss https://freeswitch.org/confluence https://cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users https://freeswitch.com
participants (2)
-
Piotr Gregor
-
R G