[Freeswitch-users] ASR Weirdness
Noel Morgan
noel at vwci.com
Wed May 16 17:16:50 MSD 2012
Sorry for the cross-thread; it was an accident.
FreeSWITCH Version 1.2.0 (git-44fd0de 2012-05-14 02-04-36 +0200) on CentOS
6.2
Here is the scenario:
Caller calls into dialplan, lua script gets called, their number gets
parsed, does a db look up, then the caller is transferred to a recording
then eventually hold music in dialplan. During the original callers
session, script 2 gets called (see script 1). Script 2 executes lua script
3 on the origination. Speech is always detected on the session:answer(). I
have even tried separate/new/independent sessions of the original sessionŠ
No go. The idea is to bridge the calls in a uuid bridge eventually and it
works fine when it gets there, but I cannot trap when the call is answered
under any scenario (no early media signaling and disposition is always
'ANSWER'), hence the ASR (tried all the execute_on_answer variants plus
nolocal: yada yada, even tried it with python and javascript as well). I
am cross-eyed by now so it is probably missing something stupid. I need
the async behavior to keep the user on hold for as little time as
possible. Any ideas are appreciated.
TIA,
Noel
<script 1>
caller_leg = session:getVariable("uuid")
dealer = get_dealer_number(area_code)
session:execute("export","store_name="..dealer["name"])
session:execute("export","store_id="..dealer["store_id"])
set_store_call(caller_leg, caller_num, tostring(dealer["store_number"]))
api:executeString("luarun originate_dealer.lua " .. caller_leg .. " " ..
dealer["store_number"] .. " " .. caller_num)
session:execute("export", "store_number=" .. dealer["store_number"])
session:execute("export","caller_leg="..caller_leg)
session:execute("export","caller_num="..caller_num)
session:setHangupHook("session_hangup_hook")
session:transfer("xxxxxx_park", "XML", "default")
</script 1>
<script 2>
api = freeswitch.API()
api:executeString("bgapi originate {caller_leg=".. argv[1]
..",origination_caller_id_number=".. argv[3]
.."}sofia/gateway/provider/".. argv[2] .. " &lua(dealer.lua)")
</script 2>
<script 3>
log("dealer...")
caller_leg = session:getVariable("caller_leg")
my_leg = session:getVariable("uuid")
store_caller = get_caller(caller_leg)
function onInput(s, type, obj)
freeswitch.consoleLog("info", "Callback with type " .. type .. "\n");
if (type == "dtmf") then
freeswitch.consoleLog("info", "DTMF Digit: " .. obj.digit .. "\n");
else if (type == "event") then
local event = obj:getHeader("Speech-Type");
if (event == "begin-speaking") then
freeswitch.consoleLog("info", "\n" .. obj:serialize() .. "\n");
-- Return break on begin-speaking events to stop playback of
the fire or tts.
call_answered=true
return "break";
end
if (event == "detected-speech") then
freeswitch.consoleLog("info", "\n" .. obj:serialize() .. "\n");
if (obj:getBody()) then
-- Pause speech detection (this is on auto but pausing it
just in case)
session:execute("detect_speech", "pause");
-- Parse the results from the event into the results table
for later use.
call_answered=true
end
return "break";
end
end
end
end
called_leg = session:getVariable("uuid")
session:answer();
session:setInputCallback("onInput");
session:sleep(1000)
session:execute("detect_speech", "pocketsphinx myapp default")
while call_answered==false and tries <= 10 do
session:sleep(2000)
log("call answered is false...")
tries = tries + 1
end
if session:ready()==true then
log(session:getVariable("endpoint_disposition"))
digits = session:playAndGetDigits(1, 1, 1, 4000, '',
'/usr/local/freeswitch/recordings/DealerGreeting.wav', '', '\\d+')
if digits then
if digits == "1" then
log("In Dealer IVR and 1 was pressed...")
api:executeString("uuid_bridge " ..called_leg .. " " ..
caller_leg)
else
log("In Dealer IVR and 1 was NOT pressed...")
api:executeString("uuid_bridge " ..called_leg .. " " ..
caller_leg)
end
else
n_session = freeswitch.Session()
ses = freeswitch.Session("{effective_caller_id_number="..
caller_leg
..",ignore_early_media=true,originate_timeout=40}sofia/gateway/provider/214
3909113", n_session)
called_leg = ses:getVariable("uuid")
sess:waitForAnswer(sess)
sess:answer()
api:executeString("uuid_bridge " ..called_leg .. " " .. caller_leg)
end
end
</script 3>
2012-05-15 17:01:09.748559 [DEBUG] switch_rtp.c:3253 Correct ip/port
confirmed.
INFO: cmd_ln.c(691): Parsing command line:
\
-samprate 8000 \
-hmm /usr/local/freeswitch/grammar/model/communicator \
-jsgf /usr/local/freeswitch/grammar/xxxxxx.gram \
-lw 6.5 \
-dict /usr/local/freeswitch/grammar/default.dic \
-frate 50 \
-silprob 0.005
Current configuration:
[NAME] [DEFLT] [VALUE]
-agc none none
-agcthresh 2.0 2.000000e+00
-alpha 0.97 9.700000e-01
-ascale 20.0 2.000000e+01
-aw 1 1
-backtrace no no
-beam 1e-48 1.000000e-48
-bestpath yes yes
-bestpathlw 9.5 9.500000e+00
-bghist no no
-ceplen 13 13
-cmn current current
-cmninit 8.0 8.0
-compallsen no no
-debug 0
-dict /usr/local/freeswitch/grammar/default.dic
-dictcase no no
-dither no no
-doublebw no no
-ds 1 1
-fdict
-feat 1s_c_d_dd 1s_c_d_dd
-featparams
-fillprob 1e-8 1.000000e-08
-frate 100 50
-fsg
-fsgusealtpron yes yes
-fsgusefiller yes yes
-fwdflat yes yes
-fwdflatbeam 1e-64 1.000000e-64
-fwdflatefwid 4 4
-fwdflatlw 8.5 8.500000e+00
-fwdflatsfwin 25 25
-fwdflatwbeam 7e-29 7.000000e-29
-fwdtree yes yes
-hmm /usr/local/freeswitch/grammar/model/communicator
-input_endian little little
-jsgf /usr/local/freeswitch/grammar/xxxxxx.gram
-kdmaxbbi -1 -1
-kdmaxdepth 0 0
-kdtree
-latsize 5000 5000
-lda
-ldadim 0 0
-lextreedump 0 0
-lifter 0 0
-lm
-lmctl
-lmname default default
-logbase 1.0001 1.000100e+00
-logfn
-logspec no no
-lowerf 133.33334 1.333333e+02
-lpbeam 1e-40 1.000000e-40
-lponlybeam 7e-29 7.000000e-29
-lw 6.5 6.500000e+00
-maxhmmpf -1 -1
-maxnewoov 20 20
-maxwpf -1 -1
-mdef
-mean
-mfclogdir
-min_endfr 0 0
-mixw
-mixwfloor 0.0000001 1.000000e-07
-mllr
-mmap yes yes
-ncep 13 13
-nfft 512 512
-nfilt 40 40
-nwpen 1.0 1.000000e+00
-pbeam 1e-48 1.000000e-48
-pip 1.0 1.000000e+00
-pl_beam 1e-10 1.000000e-10
-pl_pbeam 1e-5 1.000000e-05
-pl_window 0 0
-rawlogdir
-remove_dc no no
-round_filters yes yes
-samprate 16000 8.000000e+03
-seed -1 -1
-sendump
-senlogdir
-senmgau
-silprob 0.005 5.000000e-03
-smoothspec no no
-svspec
-tmat
-tmatfloor 0.0001 1.000000e-04
-topn 4 4
-topn_beam 0 0
-toprule
-transform legacy legacy
-unit_area yes yes
-upperf 6855.4976 6.855498e+03
-usewdphones no no
-uw 1.0 1.000000e+00
-var
-varfloor 0.0001 1.000000e-04
-varnorm no no
-verbose no no
-warp_params
-warp_type inverse_linear inverse_linear
-wbeam 7e-29 7.000000e-29
-wip 0.65 6.500000e-01
-wlen 0.025625 2.562500e-02
EXECUTE sofia/xxxxxx/2143909113 detect_speech(pocketsphinx xxxxxx default)
INFO: cmd_ln.c(691): Parsing command line:
\
-alpha 0.97 \
-dither yes \
-doublebw no \
-nfilt 31 \
-ncep 13 \
-lowerf 200 \
-upperf 3500 \
-nfft 256 \
-wlen 0.0256 \
-transform legacy \
-feat s2_4x \
-agc none \
-cmn current \
-varnorm no
Current configuration:
[NAME] [DEFLT] [VALUE]
-agc none none
-agcthresh 2.0 2.000000e+00
-alpha 0.97 9.700000e-01
-ceplen 13 13
-cmn current current
-cmninit 8.0 8.0
-dither no yes
-doublebw no no
-feat 1s_c_d_dd s2_4x
-frate 100 50
-input_endian little little
-lda
-ldadim 0 0
-lifter 0 0
-logspec no no
-lowerf 133.33334 2.000000e+02
-ncep 13 13
-nfft 512 256
-nfilt 40 31
-remove_dc no no
-round_filters yes yes
-samprate 16000 8.000000e+03
-seed -1 -1
-smoothspec no no
-svspec
-transform legacy legacy
-unit_area yes yes
-upperf 6855.4976 3.500000e+03
-varnorm no no
-verbose no no
-warp_params
-warp_type inverse_linear inverse_linear
-wlen 0.025625 2.560000e-02
INFO: acmod.c(242): Parsed model-specific feature parameters from
/usr/local/freeswitch/grammar/model/communicator/feat.params
INFO: fe_interface.c(289): You are using the internal mechanism to
generate the seed.
INFO: feat.c(684): Initializing feature stream to type: 's2_4x',
ceplen=13, CMN='current', VARNORM='no', AGC='none'
INFO: cmn.c(142): mean[0]= 12.00, mean[1..12]= 0.0
INFO: mdef.c(520): Reading model definition:
/usr/local/freeswitch/grammar/model/communicator/mdef
INFO: bin_mdef.c(173): Allocating 104160 * 8 bytes (813 KiB) for CD tree
INFO: tmat.c(205): Reading HMM transition probability matrices:
/usr/local/freeswitch/grammar/model/communicator/transition_matrices
INFO: acmod.c(117): Attempting to use SCHMM computation module
INFO: ms_gauden.c(198): Reading mixture gaussian parameter:
/usr/local/freeswitch/grammar/model/communicator/means
INFO: ms_gauden.c(292): 1 codebook, 4 feature, size:
INFO: ms_gauden.c(294): 256x12
INFO: ms_gauden.c(294): 256x24
INFO: ms_gauden.c(294): 256x3
INFO: ms_gauden.c(294): 256x12
INFO: ms_gauden.c(198): Reading mixture gaussian parameter:
/usr/local/freeswitch/grammar/model/communicator/variances
INFO: ms_gauden.c(292): 1 codebook, 4 feature, size:
INFO: ms_gauden.c(294): 256x12
INFO: ms_gauden.c(294): 256x24
INFO: ms_gauden.c(294): 256x3
INFO: ms_gauden.c(294): 256x12
INFO: ms_gauden.c(354): 59 variance values floored
INFO: s2_semi_mgau.c(908): Loading senones from dump file
/usr/local/freeswitch/grammar/model/communicator/sendump
INFO: s2_semi_mgau.c(932): BEGIN FILE FORMAT DESCRIPTION
INFO: s2_semi_mgau.c(995): Rows: 256, Columns: 6256
INFO: s2_semi_mgau.c(1027): Using memory-mapped I/O for senones
INFO: s2_semi_mgau.c(1304): Maximum top-N: 4 Top-N beams: 0 0 0 0
INFO: dict.c(306): Allocating 137548 * 32 bytes (4298 KiB) for word entries
INFO: dict.c(321): Reading main dictionary:
/usr/local/freeswitch/grammar/default.dic
INFO: dict.c(212): Allocated 1010 KiB for strings, 1664 KiB for phones
INFO: dict.c(324): 133436 words read
INFO: dict.c(330): Reading filler dictionary:
/usr/local/freeswitch/grammar/model/communicator/noisedict
INFO: dict.c(212): Allocated 0 KiB for strings, 0 KiB for phones
INFO: dict.c(333): 17 words read
INFO: dict2pid.c(396): Building PID tables for dictionary
INFO: dict2pid.c(404): Allocating 51^3 * 2 bytes (259 KiB) for
word-initial triphones
INFO: dict2pid.c(131): Allocated 62832 bytes (61 KiB) for word-final
triphones
INFO: dict2pid.c(195): Allocated 62832 bytes (61 KiB) for single-phone
word triphones
INFO: fsg_search.c(145): FSG(beam: -1080, pbeam: -1080, wbeam: -634; wip:
-26, pip: 0)
INFO: jsgf.c(546): Defined rule: <xxxxxx.g00000>
INFO: jsgf.c(546): Defined rule: <xxxxxx.yes>
INFO: jsgf.c(546): Defined rule: <xxxxxx.g00002>
INFO: jsgf.c(546): Defined rule: <xxxxxx.no>
INFO: jsgf.c(546): Defined rule: <xxxxxx.g00004>
INFO: jsgf.c(546): Defined rule: <xxxxxx.hello>
INFO: jsgf.c(546): Defined rule: <xxxxxx.g00006>
INFO: jsgf.c(546): Defined rule: PUBLIC <xxxxxx.results>
INFO: fsg_model.c(213): Computing transitive closure for null transitions
INFO: fsg_model.c(264): 87 null transitions added
INFO: fsg_model.c(411): Adding silence transitions for <sil> to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++AE++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++AH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++BACKGROUND++ to
FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++BREATH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++COUGH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++EH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++ER++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++LAUGH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++MM++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++MUMBLE++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++NOISE++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++OH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++SMACK++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++UH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++UH_NOISE++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++UM++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++UM_NOISE++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_search.c(364): Added 1 alternate word transitions
INFO: fsg_lextree.c(108): Allocated 2392 bytes (2 KiB) for left and right
context phones
INFO: fsg_lextree.c(251): 505 HMM nodes in lextree (496 leaves)
INFO: fsg_lextree.c(253): Allocated 64640 bytes (63 KiB) for all lextree
nodes
INFO: fsg_lextree.c(256): Allocated 63488 bytes (62 KiB) for lextree
leafnodes
2012-05-15 17:01:11.428551 [DEBUG] switch_core_media_bug.c:502 Attaching
BUG to sofia/xxxxxx/2143909113
2012-05-15 17:01:11.448552 [DEBUG] switch_core_io.c:353 Setting BUG Codec
PCMU:0
2012-05-15 17:01:13.448551 [NOTICE] switch_cpp.cpp:1227
----------------->call answered is false...<-------------
EXECUTE sofia/xxxxxx/2143909113 detect_speech(resume)
2012-05-15 17:01:15.008548 [INFO] switch_cpp.cpp:1227 Callback with type
event
2012-05-15 17:01:15.008548 [INFO] switch_cpp.cpp:1227
'Event-Name: DETECTED_SPEECH
Core-UUID: 0a387574-9ee8-11e1-8fa1-43045b098f7a
FreeSWITCH-Hostname: xxxxxx
FreeSWITCH-Switchname: xxxxxx
FreeSWITCH-IPv4: 199.15.96.29
FreeSWITCH-IPv6: %3A%3A1
Event-Date-Local: 2012-05-15%2017%3A01%3A14
Event-Date-GMT: Wed,%2016%20May%202012%2000%3A01%3A14%20GMT
Event-Date-Timestamp: 1337126474988540
Event-Calling-File: switch_ivr_async.c
Event-Calling-Function: speech_thread
Event-Calling-Line-Number: 3549
Event-Sequence: 1204
Speech-Type: begin-speaking
'
2012-05-15 17:01:15.008548 [NOTICE] switch_cpp.cpp:1227
----------------->call answered is false...<-------------
EXECUTE sofia/xxxxxx/2143909113 detect_speech(resume)
2012-05-15 17:01:17.008548 [NOTICE] switch_cpp.cpp:1227
----------------->call answered is false...<-------------
EXECUTE sofia/xxxxxx/2143909113 detect_speech(resume)
INFO: cmn_prior.c(121): cmn_prior_update: from < 8.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 >
INFO: cmn_prior.c(139): cmn_prior_update: to < 8.11 -0.00 0.40 -0.00
-0.39 -0.64 -0.79 -0.65 -0.51 -0.32 -0.08 0.12 0.11 >
INFO: fsg_search.c(1030): 133 frames, 1846 HMMs (13/fr), 5417 senones
(40/fr), 1099 history entries (8/fr)
INFO: fsg_search.c(1407): Start node hello.0:14:96
INFO: fsg_search.c(1446): End node <sil>.95:105:132 (-2846)
INFO: fsg_search.c(1662): lattice start node hello.0 end node <sil>.95
INFO: ps_lattice.c(1352): Normalizer P(O) = alpha(<sil>:95:132) = -886169
INFO: ps_lattice.c(1390): Joint P(O,S) = -886169 P(S|O) = 0
2012-05-15 17:01:17.648543 [DEBUG] mod_pocketsphinx.c:383 Recognized:
hello, Confidence: 100
2012-05-15 17:01:17.648543 [INFO] switch_cpp.cpp:1227 Callback with type
event
2012-05-15 17:01:17.648543 [INFO] switch_cpp.cpp:1227
'Event-Name: DETECTED_SPEECH
Core-UUID: 0a387574-9ee8-11e1-8fa1-43045b098f7a
FreeSWITCH-Hostname: xxxxxx
FreeSWITCH-Switchname: xxxxxx
FreeSWITCH-IPv4: 199.15.96.29
FreeSWITCH-IPv6: %3A%3A1
Event-Date-Local: 2012-05-15%2017%3A01%3A17
Event-Date-GMT: Wed,%2016%20May%202012%2000%3A01%3A17%20GMT
Event-Date-Timestamp: 1337126477648543
Event-Calling-File: switch_ivr_async.c
Event-Calling-Function: speech_thread
Event-Calling-Line-Number: 3549
Event-Sequence: 1209
Speech-Type: detected-speech
Content-Length: 172
<?xml version="1.0"?>
<result grammar="xxxxxx">
<interpretation grammar="xxxxxx" confidence="100">
<input mode="speech">hello</input>
</interpretation>
</result>
'
EXECUTE sofia/xxxxxx/2143909113 detect_speech(pause)
Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users
mailing list