Hello, following a discussion on IRC, I thought I would move this here so it is more async. The need is to be able to use Nuance/IBM Watson/Google Speech APIs in streaming mode for real-time transcription (essentially, captioning a video conference and getting a transcript of the proceedings). My first idea was to simply get the frames from a recording as it is being written, though there might be some obstacles there. It was suggested the media bugs infrastructure would work, at the C level, though I need to check which formats it supports. Do you have any other suggestions? Thank you! Luca
This should be implemented the way we implement other speech interfaces.
On Oct 8, 2016, at 4:53 PM, Luca Pradovera <luca.pradovera@gmail.com> wrote:
Hello, following a discussion on IRC, I thought I would move this here so it is more async.
The need is to be able to use Nuance/IBM Watson/Google Speech APIs in streaming mode for real-time transcription (essentially, captioning a video conference and getting a transcript of the proceedings).
My first idea was to simply get the frames from a recording as it is being written, though there might be some obstacles there.
It was suggested the media bugs infrastructure would work, at the C level, though I need to check which formats it supports.
Do you have any other suggestions?
Thank you!
Luca
Just an FYI on a couple of points: 1) I’ve tried to implment support for Nuance NCS API as an asr_tts module. tts works fine, but asr / transcription does not seem possible because the NCS API is not really suited for a streaming application. Based on my experience (and I opened tickets with them to try to resolve, to no avail), their server does not handle streaming because when I start sending small-ish chunks of audio using chunked-encoding, they immediately return me a “no speech” result. They seem to expect that you have a recorded file on disk that you can ship over in much larger chunks. 2) I’m not completely sure I agree that transcription fits into the asr part of the asr_tts modules. Some of these cloud-based services support providing a stream of interim results while the speaker is talking, and this would be very useful to propogate up to the freeswitch application (or esl) but the asr_tts pattern seems to be to return a single speech result at the end of a segment of speech. Dave On Oct 10, 2016, at 11:54 AM, Michael Jerris <mike@jerris.com> wrote: This should be implemented the way we implement other speech interfaces.
On Oct 8, 2016, at 4:53 PM, Luca Pradovera <luca.pradovera@gmail.com> wrote:
Hello, following a discussion on IRC, I thought I would move this here so it is more async.
The need is to be able to use Nuance/IBM Watson/Google Speech APIs in streaming mode for real-time transcription (essentially, captioning a video conference and getting a transcript of the proceedings).
My first idea was to simply get the frames from a recording as it is being written, though there might be some obstacles there.
It was suggested the media bugs infrastructure would work, at the C level, though I need to check which formats it supports.
Do you have any other suggestions?
Thank you!
Luca
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
It can return as many as it wants, as it gets them from the speech server, there really is no limit in the ASR/TTS interface, We reuse handles in the MRCP interface module. /b On Mon, Oct 10, 2016 at 11:29 AM, Dave Horton <daveh@beachdognet.com> wrote:
Just an FYI on a couple of points:
1) I’ve tried to implment support for Nuance NCS API as an asr_tts module. tts works fine, but asr / transcription does not seem possible because the NCS API is not really suited for a streaming application. Based on my experience (and I opened tickets with them to try to resolve, to no avail), their server does not handle streaming because when I start sending small-ish chunks of audio using chunked-encoding, they immediately return me a “no speech” result. They seem to expect that you have a recorded file on disk that you can ship over in much larger chunks.
2) I’m not completely sure I agree that transcription fits into the asr part of the asr_tts modules. Some of these cloud-based services support providing a stream of interim results while the speaker is talking, and this would be very useful to propogate up to the freeswitch application (or esl) but the asr_tts pattern seems to be to return a single speech result at the end of a segment of speech.
Dave
On Oct 10, 2016, at 11:54 AM, Michael Jerris <mike@jerris.com> wrote:
This should be implemented the way we implement other speech interfaces.
On Oct 8, 2016, at 4:53 PM, Luca Pradovera <luca.pradovera@gmail.com> wrote:
Hello, following a discussion on IRC, I thought I would move this here so it is more async.
The need is to be able to use Nuance/IBM Watson/Google Speech APIs in streaming mode for real-time transcription (essentially, captioning a video conference and getting a transcript of the proceedings).
My first idea was to simply get the frames from a recording as it is being written, though there might be some obstacles there.
It was suggested the media bugs infrastructure would work, at the C level, though I need to check which formats it supports.
Do you have any other suggestions?
Thank you!
Luca
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- *Brian West* brian@freeswitch.org *Twitter: @FreeSWITCH , @briankwest* http://www.freeswitchbook.com (50% Discount using code FreeSwitch50) http://www.freeswitchcookbook.com (50% Discount using code FreeSwitch50) https://www.gofundme.com/freeswitch_ubuntu Got Bugs? Report them here <https://freeswitch.org/jira>! | Reddit: /r/freeswitch <https://www.reddit.com/r/freeswitch> *T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378) *iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
Brian, Mike, as far as I understand, they talk about speech to text, not the other way round. TTS is something completely different ) On 10 Oct 2016 18:37, "Brian West" <brian@freeswitch.org> wrote:
It can return as many as it wants, as it gets them from the speech server, there really is no limit in the ASR/TTS interface, We reuse handles in the MRCP interface module.
/b
On Mon, Oct 10, 2016 at 11:29 AM, Dave Horton <daveh@beachdognet.com> wrote:
Just an FYI on a couple of points:
1) I’ve tried to implment support for Nuance NCS API as an asr_tts module. tts works fine, but asr / transcription does not seem possible because the NCS API is not really suited for a streaming application. Based on my experience (and I opened tickets with them to try to resolve, to no avail), their server does not handle streaming because when I start sending small-ish chunks of audio using chunked-encoding, they immediately return me a “no speech” result. They seem to expect that you have a recorded file on disk that you can ship over in much larger chunks.
2) I’m not completely sure I agree that transcription fits into the asr part of the asr_tts modules. Some of these cloud-based services support providing a stream of interim results while the speaker is talking, and this would be very useful to propogate up to the freeswitch application (or esl) but the asr_tts pattern seems to be to return a single speech result at the end of a segment of speech.
Dave
On Oct 10, 2016, at 11:54 AM, Michael Jerris <mike@jerris.com> wrote:
This should be implemented the way we implement other speech interfaces.
On Oct 8, 2016, at 4:53 PM, Luca Pradovera <luca.pradovera@gmail.com> wrote:
Hello, following a discussion on IRC, I thought I would move this here so it is more async.
The need is to be able to use Nuance/IBM Watson/Google Speech APIs in streaming mode for real-time transcription (essentially, captioning a video conference and getting a transcript of the proceedings).
My first idea was to simply get the frames from a recording as it is being written, though there might be some obstacles there.
It was suggested the media bugs infrastructure would work, at the C level, though I need to check which formats it supports.
Do you have any other suggestions?
Thank you!
Luca
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
--
*Brian West* brian@freeswitch.org
*Twitter: @FreeSWITCH , @briankwest* http://www.freeswitchbook.com (50% Discount using code FreeSwitch50) http://www.freeswitchcookbook.com (50% Discount using code FreeSwitch50) https://www.gofundme.com/freeswitch_ubuntu
Got Bugs? Report them here <https://freeswitch.org/jira>! | Reddit: /r/freeswitch <https://www.reddit.com/r/freeswitch>
*T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378) *iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
ASR is implemented like a media bug. On Mon, Oct 10, 2016 at 3:34 PM, Stanislav Sinyagin <ssinyagin@gmail.com> wrote:
Brian, Mike, as far as I understand, they talk about speech to text, not the other way round. TTS is something completely different )
On 10 Oct 2016 18:37, "Brian West" <brian@freeswitch.org> wrote:
It can return as many as it wants, as it gets them from the speech server, there really is no limit in the ASR/TTS interface, We reuse handles in the MRCP interface module.
/b
On Mon, Oct 10, 2016 at 11:29 AM, Dave Horton <daveh@beachdognet.com> wrote:
Just an FYI on a couple of points:
1) I’ve tried to implment support for Nuance NCS API as an asr_tts module. tts works fine, but asr / transcription does not seem possible because the NCS API is not really suited for a streaming application. Based on my experience (and I opened tickets with them to try to resolve, to no avail), their server does not handle streaming because when I start sending small-ish chunks of audio using chunked-encoding, they immediately return me a “no speech” result. They seem to expect that you have a recorded file on disk that you can ship over in much larger chunks.
2) I’m not completely sure I agree that transcription fits into the asr part of the asr_tts modules. Some of these cloud-based services support providing a stream of interim results while the speaker is talking, and this would be very useful to propogate up to the freeswitch application (or esl) but the asr_tts pattern seems to be to return a single speech result at the end of a segment of speech.
Dave
On Oct 10, 2016, at 11:54 AM, Michael Jerris <mike@jerris.com> wrote:
This should be implemented the way we implement other speech interfaces.
On Oct 8, 2016, at 4:53 PM, Luca Pradovera <luca.pradovera@gmail.com> wrote:
Hello, following a discussion on IRC, I thought I would move this here so it is more async.
The need is to be able to use Nuance/IBM Watson/Google Speech APIs in streaming mode for real-time transcription (essentially, captioning a video conference and getting a transcript of the proceedings).
My first idea was to simply get the frames from a recording as it is being written, though there might be some obstacles there.
It was suggested the media bugs infrastructure would work, at the C level, though I need to check which formats it supports.
Do you have any other suggestions?
Thank you!
Luca
____________________________________________________________ _____________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
____________________________________________________________ _____________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
--
*Brian West* brian@freeswitch.org
*Twitter: @FreeSWITCH , @briankwest* http://www.freeswitchbook.com (50% Discount using code FreeSwitch50) http://www.freeswitchcookbook.com (50% Discount using code FreeSwitch50) https://www.gofundme.com/freeswitch_ubuntu
Got Bugs? Report them here <https://freeswitch.org/jira>! | Reddit: /r/freeswitch <https://www.reddit.com/r/freeswitch>
*T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378) *iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
-- *Brian West* brian@freeswitch.org *Twitter: @FreeSWITCH , @briankwest* http://www.freeswitchbook.com (50% Discount using code FreeSwitch50) http://www.freeswitchcookbook.com (50% Discount using code FreeSwitch50) https://www.gofundme.com/freeswitch_ubuntu Got Bugs? Report them here <https://freeswitch.org/jira>! | Reddit: /r/freeswitch <https://www.reddit.com/r/freeswitch> *T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378) *iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
Hello, sorry, somehow my mail client had hidden this conversation. IBM Watson (my current use case for this) supports a Session type of recognition, similar to HTTP Chunked upload, where the difference is that each chunk is sent as its own POST. I understand that the goal here could/would be to get this working with FS alone, but even having the ability to get raw frames on disk would be enough for now. Do you think that would be feasible? Best regards, Luca On Tue, Oct 11, 2016 at 1:26 AM, Brian West <brian@freeswitch.org> wrote:
ASR is implemented like a media bug.
On Mon, Oct 10, 2016 at 3:34 PM, Stanislav Sinyagin <ssinyagin@gmail.com> wrote:
Brian, Mike, as far as I understand, they talk about speech to text, not the other way round. TTS is something completely different )
On 10 Oct 2016 18:37, "Brian West" <brian@freeswitch.org> wrote:
It can return as many as it wants, as it gets them from the speech server, there really is no limit in the ASR/TTS interface, We reuse handles in the MRCP interface module.
/b
On Mon, Oct 10, 2016 at 11:29 AM, Dave Horton <daveh@beachdognet.com> wrote:
Just an FYI on a couple of points:
1) I’ve tried to implment support for Nuance NCS API as an asr_tts module. tts works fine, but asr / transcription does not seem possible because the NCS API is not really suited for a streaming application. Based on my experience (and I opened tickets with them to try to resolve, to no avail), their server does not handle streaming because when I start sending small-ish chunks of audio using chunked-encoding, they immediately return me a “no speech” result. They seem to expect that you have a recorded file on disk that you can ship over in much larger chunks.
2) I’m not completely sure I agree that transcription fits into the asr part of the asr_tts modules. Some of these cloud-based services support providing a stream of interim results while the speaker is talking, and this would be very useful to propogate up to the freeswitch application (or esl) but the asr_tts pattern seems to be to return a single speech result at the end of a segment of speech.
Dave
On Oct 10, 2016, at 11:54 AM, Michael Jerris <mike@jerris.com> wrote:
This should be implemented the way we implement other speech interfaces.
On Oct 8, 2016, at 4:53 PM, Luca Pradovera <luca.pradovera@gmail.com> wrote:
Hello, following a discussion on IRC, I thought I would move this here so it is more async.
The need is to be able to use Nuance/IBM Watson/Google Speech APIs in streaming mode for real-time transcription (essentially, captioning a video conference and getting a transcript of the proceedings).
My first idea was to simply get the frames from a recording as it is being written, though there might be some obstacles there.
It was suggested the media bugs infrastructure would work, at the C level, though I need to check which formats it supports.
Do you have any other suggestions?
Thank you!
Luca
____________________________________________________________ _____________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/free switch-users http://www.freeswitch.org
____________________________________________________________ _____________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/free switch-users http://www.freeswitch.org
--
*Brian West* brian@freeswitch.org
*Twitter: @FreeSWITCH , @briankwest* http://www.freeswitchbook.com (50% Discount using code FreeSwitch50) http://www.freeswitchcookbook.com (50% Discount using code FreeSwitch50) https://www.gofundme.com/freeswitch_ubuntu
Got Bugs? Report them here <https://freeswitch.org/jira>! | Reddit: /r/freeswitch <https://www.reddit.com/r/freeswitch>
*T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378) *iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
____________________________________________________________ _____________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
--
*Brian West* brian@freeswitch.org
*Twitter: @FreeSWITCH , @briankwest* http://www.freeswitchbook.com (50% Discount using code FreeSwitch50) http://www.freeswitchcookbook.com (50% Discount using code FreeSwitch50) https://www.gofundme.com/freeswitch_ubuntu
Got Bugs? Report them here <https://freeswitch.org/jira>! | Reddit: /r/freeswitch <https://www.reddit.com/r/freeswitch>
*T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378) *iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Wouldn’t it be better to use their websockets interface? http://www.ibm.com/watson/developercloud/speech-to-text/api/v1/?curl#websock... <http://www.ibm.com/watson/developercloud/speech-to-text/api/v1/?curl#websockets> Dave On Oct 17, 2016, at 9:35 PM, Luca Pradovera <luca.pradovera@gmail.com> wrote: Hello, sorry, somehow my mail client had hidden this conversation. IBM Watson (my current use case for this) supports a Session type of recognition, similar to HTTP Chunked upload, where the difference is that each chunk is sent as its own POST. I understand that the goal here could/would be to get this working with FS alone, but even having the ability to get raw frames on disk would be enough for now. Do you think that would be feasible? Best regards, Luca On Tue, Oct 11, 2016 at 1:26 AM, Brian West <brian@freeswitch.org <mailto:brian@freeswitch.org>> wrote: ASR is implemented like a media bug. On Mon, Oct 10, 2016 at 3:34 PM, Stanislav Sinyagin <ssinyagin@gmail.com <mailto:ssinyagin@gmail.com>> wrote: Brian, Mike, as far as I understand, they talk about speech to text, not the other way round. TTS is something completely different ) On 10 Oct 2016 18:37, "Brian West" <brian@freeswitch.org <mailto:brian@freeswitch.org>> wrote: It can return as many as it wants, as it gets them from the speech server, there really is no limit in the ASR/TTS interface, We reuse handles in the MRCP interface module. /b On Mon, Oct 10, 2016 at 11:29 AM, Dave Horton <daveh@beachdognet.com <mailto:daveh@beachdognet.com>> wrote: Just an FYI on a couple of points: 1) I’ve tried to implment support for Nuance NCS API as an asr_tts module. tts works fine, but asr / transcription does not seem possible because the NCS API is not really suited for a streaming application. Based on my experience (and I opened tickets with them to try to resolve, to no avail), their server does not handle streaming because when I start sending small-ish chunks of audio using chunked-encoding, they immediately return me a “no speech” result. They seem to expect that you have a recorded file on disk that you can ship over in much larger chunks. 2) I’m not completely sure I agree that transcription fits into the asr part of the asr_tts modules. Some of these cloud-based services support providing a stream of interim results while the speaker is talking, and this would be very useful to propogate up to the freeswitch application (or esl) but the asr_tts pattern seems to be to return a single speech result at the end of a segment of speech. Dave On Oct 10, 2016, at 11:54 AM, Michael Jerris <mike@jerris.com <mailto:mike@jerris.com>> wrote: This should be implemented the way we implement other speech interfaces.
On Oct 8, 2016, at 4:53 PM, Luca Pradovera <luca.pradovera@gmail.com <mailto:luca.pradovera@gmail.com>> wrote:
Hello, following a discussion on IRC, I thought I would move this here so it is more async.
The need is to be able to use Nuance/IBM Watson/Google Speech APIs in streaming mode for real-time transcription (essentially, captioning a video conference and getting a transcript of the proceedings).
My first idea was to simply get the frames from a recording as it is being written, though there might be some obstacles there.
It was suggested the media bugs infrastructure would work, at the C level, though I need to check which formats it supports.
Do you have any other suggestions?
Thank you!
Luca
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com <http://www.freeswitchsolutions.com/> Official FreeSWITCH Sites http://www.freeswitch.org <http://www.freeswitch.org/> http://confluence.freeswitch.org <http://confluence.freeswitch.org/> http://www.cluecon.com <http://www.cluecon.com/> FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users <http://lists.freeswitch.org/mailman/listinfo/freeswitch-users> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users <http://lists.freeswitch.org/mailman/options/freeswitch-users> http://www.freeswitch.org <http://www.freeswitch.org/> _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com <http://www.freeswitchsolutions.com/> Official FreeSWITCH Sites http://www.freeswitch.org <http://www.freeswitch.org/> http://confluence.freeswitch.org <http://confluence.freeswitch.org/> http://www.cluecon.com <http://www.cluecon.com/> FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users <http://lists.freeswitch.org/mailman/listinfo/freeswitch-users> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users <http://lists.freeswitch.org/mailman/options/freeswitch-users> http://www.freeswitch.org <http://www.freeswitch.org/> -- Brian West brian@freeswitch.org <mailto:brian@freeswitch.org> Twitter: @FreeSWITCH , @briankwest http://www.freeswitchbook.com <http://www.freeswitchbook.com/> (50% Discount using code FreeSwitch50) http://www.freeswitchcookbook.com <http://www.freeswitchcookbook.com/> (50% Discount using code FreeSwitch50) https://www.gofundme.com/freeswitch_ubuntu <https://www.gofundme.com/freeswitch_ubuntu> Got Bugs? Report them here <https://freeswitch.org/jira>! | Reddit: /r/freeswitch <https://www.reddit.com/r/freeswitch> T:+19184209001 <tel:%2B19184209001> | F:+19184209002 <tel:%2B19184209002> | M:+1918424WEST (9378) iNUM:+883 5100 1420 9001 | ISN:410*543 | Skype:briankwest _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com <http://www.freeswitchsolutions.com/> Official FreeSWITCH Sites http://www.freeswitch.org <http://www.freeswitch.org/> http://confluence.freeswitch.org <http://confluence.freeswitch.org/> http://www.cluecon.com <http://www.cluecon.com/> FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users <http://lists.freeswitch.org/mailman/listinfo/freeswitch-users> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users <http://lists.freeswitch.org/mailman/options/freeswitch-users> http://www.freeswitch.org <http://www.freeswitch.org/> _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com <http://www.freeswitchsolutions.com/> Official FreeSWITCH Sites http://www.freeswitch.org <http://www.freeswitch.org/> http://confluence.freeswitch.org <http://confluence.freeswitch.org/> http://www.cluecon.com <http://www.cluecon.com/> FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users <http://lists.freeswitch.org/mailman/listinfo/freeswitch-users> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users <http://lists.freeswitch.org/mailman/options/freeswitch-users> http://www.freeswitch.org <http://www.freeswitch.org/> -- Brian West brian@freeswitch.org <mailto:brian@freeswitch.org> Twitter: @FreeSWITCH , @briankwest http://www.freeswitchbook.com <http://www.freeswitchbook.com/> (50% Discount using code FreeSwitch50) http://www.freeswitchcookbook.com <http://www.freeswitchcookbook.com/> (50% Discount using code FreeSwitch50) https://www.gofundme.com/freeswitch_ubuntu <https://www.gofundme.com/freeswitch_ubuntu> Got Bugs? Report them here <https://freeswitch.org/jira>! | Reddit: /r/freeswitch <https://www.reddit.com/r/freeswitch> T:+19184209001 <tel:%2B19184209001> | F:+19184209002 <tel:%2B19184209002> | M:+1918424WEST (9378) iNUM:+883 5100 1420 9001 | ISN:410*543 | Skype:briankwest _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org <mailto:consulting@freeswitch.org> http://www.freeswitchsolutions.com <http://www.freeswitchsolutions.com/> Official FreeSWITCH Sites http://www.freeswitch.org <http://www.freeswitch.org/> http://confluence.freeswitch.org <http://confluence.freeswitch.org/> http://www.cluecon.com <http://www.cluecon.com/> FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org <mailto:FreeSWITCH-users@lists.freeswitch.org> http://lists.freeswitch.org/mailman/listinfo/freeswitch-users <http://lists.freeswitch.org/mailman/listinfo/freeswitch-users> UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users <http://lists.freeswitch.org/mailman/options/freeswitch-users> http://www.freeswitch.org <http://www.freeswitch.org/> _________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
Hello, that would work too, but isn't the problem still the same? I am trying to find an angle to attack the problem. Where could I start? Luca On Mon, Oct 17, 2016 at 11:24 PM, Dave Horton <daveh@beachdognet.com> wrote:
Wouldn’t it be better to use their websockets interface?
http://www.ibm.com/watson/developercloud/speech-to-text/ api/v1/?curl#websockets
Dave
On Oct 17, 2016, at 9:35 PM, Luca Pradovera <luca.pradovera@gmail.com> wrote:
Hello, sorry, somehow my mail client had hidden this conversation.
IBM Watson (my current use case for this) supports a Session type of recognition, similar to HTTP Chunked upload, where the difference is that each chunk is sent as its own POST.
I understand that the goal here could/would be to get this working with FS alone, but even having the ability to get raw frames on disk would be enough for now.
Do you think that would be feasible?
Best regards,
Luca
On Tue, Oct 11, 2016 at 1:26 AM, Brian West <brian@freeswitch.org> wrote:
ASR is implemented like a media bug.
On Mon, Oct 10, 2016 at 3:34 PM, Stanislav Sinyagin <ssinyagin@gmail.com> wrote:
Brian, Mike, as far as I understand, they talk about speech to text, not the other way round. TTS is something completely different )
On 10 Oct 2016 18:37, "Brian West" <brian@freeswitch.org> wrote:
It can return as many as it wants, as it gets them from the speech server, there really is no limit in the ASR/TTS interface, We reuse handles in the MRCP interface module.
/b
On Mon, Oct 10, 2016 at 11:29 AM, Dave Horton <daveh@beachdognet.com> wrote:
Just an FYI on a couple of points:
1) I’ve tried to implment support for Nuance NCS API as an asr_tts module. tts works fine, but asr / transcription does not seem possible because the NCS API is not really suited for a streaming application. Based on my experience (and I opened tickets with them to try to resolve, to no avail), their server does not handle streaming because when I start sending small-ish chunks of audio using chunked-encoding, they immediately return me a “no speech” result. They seem to expect that you have a recorded file on disk that you can ship over in much larger chunks.
2) I’m not completely sure I agree that transcription fits into the asr part of the asr_tts modules. Some of these cloud-based services support providing a stream of interim results while the speaker is talking, and this would be very useful to propogate up to the freeswitch application (or esl) but the asr_tts pattern seems to be to return a single speech result at the end of a segment of speech.
Dave
On Oct 10, 2016, at 11:54 AM, Michael Jerris <mike@jerris.com> wrote:
This should be implemented the way we implement other speech interfaces.
On Oct 8, 2016, at 4:53 PM, Luca Pradovera <luca.pradovera@gmail.com> wrote:
Hello, following a discussion on IRC, I thought I would move this here so it is more async.
The need is to be able to use Nuance/IBM Watson/Google Speech APIs in streaming mode for real-time transcription (essentially, captioning a video conference and getting a transcript of the proceedings).
My first idea was to simply get the frames from a recording as it is being written, though there might be some obstacles there.
It was suggested the media bugs infrastructure would work, at the C level, though I need to check which formats it supports.
Do you have any other suggestions?
Thank you!
Luca
____________________________________________________________ _____________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/free switch-users http://www.freeswitch.org
____________________________________________________________ _____________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/free switch-users http://www.freeswitch.org
--
*Brian West* brian@freeswitch.org
*Twitter: @FreeSWITCH , @briankwest* http://www.freeswitchbook.com (50% Discount using code FreeSwitch50) http://www.freeswitchcookbook.com (50% Discount using code FreeSwitch50) https://www.gofundme.com/freeswitch_ubuntu
Got Bugs? Report them here <https://freeswitch.org/jira>! | Reddit: /r/freeswitch <https://www.reddit.com/r/freeswitch>
*T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378) *iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
____________________________________________________________ _____________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/free switch-users http://www.freeswitch.org
____________________________________________________________ _____________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
--
*Brian West* brian@freeswitch.org
*Twitter: @FreeSWITCH , @briankwest* http://www.freeswitchbook.com (50% Discount using code FreeSwitch50) http://www.freeswitchcookbook.com (50% Discount using code FreeSwitch50) https://www.gofundme.com/freeswitch_ubuntu
Got Bugs? Report them here <https://freeswitch.org/jira>! | Reddit: /r/freeswitch <https://www.reddit.com/r/freeswitch>
*T:*+19184209001 | *F:*+19184209002 | *M:*+1918424WEST (9378) *iNUM:*+883 5100 1420 9001 | *ISN:*410*543 | *Skype:*briankwest
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
_________________________________________________________________________ Professional FreeSWITCH Consulting Services: consulting@freeswitch.org http://www.freeswitchsolutions.com
Official FreeSWITCH Sites http://www.freeswitch.org http://confluence.freeswitch.org http://www.cluecon.com
FreeSWITCH-users mailing list FreeSWITCH-users@lists.freeswitch.org http://lists.freeswitch.org/mailman/listinfo/freeswitch-users UNSUBSCRIBE:http://lists.freeswitch.org/mailman/options/freeswitch-users http://www.freeswitch.org
participants (5)
-
Brian West
-
Dave Horton
-
Luca Pradovera
-
Michael Jerris
-
Stanislav Sinyagin