[Freeswitch-users] ASR Weirdness

Noel Morgan noel at vwci.com
Wed May 16 17:16:50 MSD 2012


Sorry for the cross-thread; it was an accident.

FreeSWITCH Version 1.2.0 (git-44fd0de 2012-05-14 02-04-36 +0200) on CentOS
6.2


Here is the scenario:

Caller calls into dialplan, lua script gets called, their number gets
parsed, does a db look up, then the caller is transferred to a recording
then eventually hold music in dialplan. During the original callers
session, script 2 gets called (see script 1). Script 2 executes lua script
3 on the origination. Speech is always detected on the session:answer(). I
have even tried separate/new/independent sessions of the original sessionŠ
No go. The idea is to bridge the calls in a uuid bridge eventually and it
works fine when it gets there, but I cannot trap when the call is answered
under any scenario (no early media signaling and disposition is always
'ANSWER'), hence the ASR (tried all the execute_on_answer variants plus
nolocal: yada yada, even tried it with python and javascript as well). I
am cross-eyed by now so it is probably missing something stupid. I need
the async behavior to keep the user on hold for as little time as
possible. Any ideas are appreciated.

TIA,
Noel



<script 1>
caller_leg = session:getVariable("uuid")
dealer = get_dealer_number(area_code)
session:execute("export","store_name="..dealer["name"])
session:execute("export","store_id="..dealer["store_id"])
set_store_call(caller_leg, caller_num, tostring(dealer["store_number"]))

api:executeString("luarun originate_dealer.lua " .. caller_leg .. " " ..
dealer["store_number"] .. " " .. caller_num)

session:execute("export", "store_number=" .. dealer["store_number"])
session:execute("export","caller_leg="..caller_leg)
session:execute("export","caller_num="..caller_num)
session:setHangupHook("session_hangup_hook")
session:transfer("xxxxxx_park", "XML", "default")

</script 1>


<script 2>
api = freeswitch.API()
api:executeString("bgapi originate {caller_leg=".. argv[1]
..",origination_caller_id_number=".. argv[3]
.."}sofia/gateway/provider/".. argv[2] .. " &lua(dealer.lua)")

</script 2>

<script 3>
log("dealer...")

caller_leg = session:getVariable("caller_leg")
my_leg = session:getVariable("uuid")
store_caller = get_caller(caller_leg)

function onInput(s, type, obj)
    freeswitch.consoleLog("info", "Callback with type " .. type .. "\n");
    if (type == "dtmf") then
        freeswitch.consoleLog("info", "DTMF Digit: " .. obj.digit .. "\n");
    else if (type == "event") then
        local event = obj:getHeader("Speech-Type");
        if (event == "begin-speaking") then
            freeswitch.consoleLog("info", "\n" .. obj:serialize() .. "\n");
            -- Return break on begin-speaking events to stop playback of
the fire or tts.
     call_answered=true
            return "break";
        end
        if (event == "detected-speech") then
            freeswitch.consoleLog("info", "\n" .. obj:serialize() .. "\n");
            if (obj:getBody()) then
                -- Pause speech detection (this is on auto but pausing it
just in case)
                session:execute("detect_speech", "pause");
                -- Parse the results from the event into the results table
for later use.
                call_answered=true
            end
            return "break";
        end
    end
    end
end

called_leg = session:getVariable("uuid")

session:answer();
session:setInputCallback("onInput");
session:sleep(1000)
session:execute("detect_speech", "pocketsphinx myapp default")


while call_answered==false and tries <= 10 do
    session:sleep(2000)
    log("call answered is false...")
    tries = tries + 1
end

if session:ready()==true then
    log(session:getVariable("endpoint_disposition"))
    digits = session:playAndGetDigits(1, 1, 1, 4000, '',
'/usr/local/freeswitch/recordings/DealerGreeting.wav', '', '\\d+')
    if digits then
        if digits == "1" then
            log("In Dealer IVR and 1 was pressed...")
            api:executeString("uuid_bridge " ..called_leg .. " " ..
caller_leg)
        else
            log("In Dealer IVR and 1 was NOT pressed...")
            api:executeString("uuid_bridge " ..called_leg .. " " ..
caller_leg)
        end
    else
        n_session = freeswitch.Session()
        ses = freeswitch.Session("{effective_caller_id_number="..
caller_leg 
..",ignore_early_media=true,originate_timeout=40}sofia/gateway/provider/214
3909113", n_session)
        called_leg = ses:getVariable("uuid")
        sess:waitForAnswer(sess)
        sess:answer()
        api:executeString("uuid_bridge " ..called_leg .. " " .. caller_leg)
    end
end

</script 3>




2012-05-15 17:01:09.748559 [DEBUG] switch_rtp.c:3253 Correct ip/port
confirmed.
INFO: cmd_ln.c(691): Parsing command line:
\
 -samprate 8000 \
 -hmm /usr/local/freeswitch/grammar/model/communicator \
 -jsgf /usr/local/freeswitch/grammar/xxxxxx.gram \
 -lw 6.5 \
 -dict /usr/local/freeswitch/grammar/default.dic \
 -frate 50 \
 -silprob 0.005 

Current configuration:
[NAME]  [DEFLT]  [VALUE]
-agc  none  none
-agcthresh 2.0  2.000000e+00
-alpha  0.97  9.700000e-01
-ascale  20.0  2.000000e+01
-aw  1  1
-backtrace no  no
-beam  1e-48  1.000000e-48
-bestpath yes  yes
-bestpathlw 9.5  9.500000e+00
-bghist  no  no
-ceplen  13  13
-cmn  current  current
-cmninit 8.0  8.0
-compallsen no  no
-debug    0
-dict    /usr/local/freeswitch/grammar/default.dic
-dictcase no  no
-dither  no  no
-doublebw no  no
-ds  1  1
-fdict    
-feat  1s_c_d_dd 1s_c_d_dd
-featparams   
-fillprob 1e-8  1.000000e-08
-frate  100  50
-fsg    
-fsgusealtpron yes  yes
-fsgusefiller yes  yes
-fwdflat yes  yes
-fwdflatbeam 1e-64  1.000000e-64
-fwdflatefwid 4  4
-fwdflatlw 8.5  8.500000e+00
-fwdflatsfwin 25  25
-fwdflatwbeam 7e-29  7.000000e-29
-fwdtree yes  yes
-hmm    /usr/local/freeswitch/grammar/model/communicator
-input_endian little  little
-jsgf    /usr/local/freeswitch/grammar/xxxxxx.gram
-kdmaxbbi -1  -1
-kdmaxdepth 0  0
-kdtree    
-latsize 5000  5000
-lda    
-ldadim  0  0
-lextreedump 0  0
-lifter  0  0
-lm    
-lmctl    
-lmname  default  default
-logbase 1.0001  1.000100e+00
-logfn    
-logspec no  no
-lowerf  133.33334 1.333333e+02
-lpbeam  1e-40  1.000000e-40
-lponlybeam 7e-29  7.000000e-29
-lw  6.5  6.500000e+00
-maxhmmpf -1  -1
-maxnewoov 20  20
-maxwpf  -1  -1
-mdef    
-mean    
-mfclogdir   
-min_endfr 0  0
-mixw    
-mixwfloor 0.0000001 1.000000e-07
-mllr    
-mmap  yes  yes
-ncep  13  13
-nfft  512  512
-nfilt  40  40
-nwpen  1.0  1.000000e+00
-pbeam  1e-48  1.000000e-48
-pip  1.0  1.000000e+00
-pl_beam 1e-10  1.000000e-10
-pl_pbeam 1e-5  1.000000e-05
-pl_window 0  0
-rawlogdir   
-remove_dc no  no
-round_filters yes  yes
-samprate 16000  8.000000e+03
-seed  -1  -1
-sendump   
-senlogdir   
-senmgau   
-silprob 0.005  5.000000e-03
-smoothspec no  no
-svspec    
-tmat    
-tmatfloor 0.0001  1.000000e-04
-topn  4  4
-topn_beam 0  0
-toprule   
-transform legacy  legacy
-unit_area yes  yes
-upperf  6855.4976 6.855498e+03
-usewdphones no  no
-uw  1.0  1.000000e+00
-var    
-varfloor 0.0001  1.000000e-04
-varnorm no  no
-verbose no  no
-warp_params   
-warp_type inverse_linear inverse_linear
-wbeam  7e-29  7.000000e-29
-wip  0.65  6.500000e-01
-wlen  0.025625 2.562500e-02

EXECUTE sofia/xxxxxx/2143909113 detect_speech(pocketsphinx xxxxxx default)
INFO: cmd_ln.c(691): Parsing command line:
\
 -alpha 0.97 \
 -dither yes \
 -doublebw no \
 -nfilt 31 \
 -ncep 13 \
 -lowerf 200 \
 -upperf 3500 \
 -nfft 256 \
 -wlen 0.0256 \
 -transform legacy \
 -feat s2_4x \
 -agc none \
 -cmn current \
 -varnorm no 

Current configuration:
[NAME]  [DEFLT]  [VALUE]
-agc  none  none
-agcthresh 2.0  2.000000e+00
-alpha  0.97  9.700000e-01
-ceplen  13  13
-cmn  current  current
-cmninit 8.0  8.0
-dither  no  yes
-doublebw no  no
-feat  1s_c_d_dd s2_4x
-frate  100  50
-input_endian little  little
-lda    
-ldadim  0  0
-lifter  0  0
-logspec no  no
-lowerf  133.33334 2.000000e+02
-ncep  13  13
-nfft  512  256
-nfilt  40  31
-remove_dc no  no
-round_filters yes  yes
-samprate 16000  8.000000e+03
-seed  -1  -1
-smoothspec no  no
-svspec    
-transform legacy  legacy
-unit_area yes  yes
-upperf  6855.4976 3.500000e+03
-varnorm no  no
-verbose no  no
-warp_params   
-warp_type inverse_linear inverse_linear
-wlen  0.025625 2.560000e-02

INFO: acmod.c(242): Parsed model-specific feature parameters from
/usr/local/freeswitch/grammar/model/communicator/feat.params
INFO: fe_interface.c(289): You are using the internal mechanism to
generate the seed.
INFO: feat.c(684): Initializing feature stream to type: 's2_4x',
ceplen=13, CMN='current', VARNORM='no', AGC='none'
INFO: cmn.c(142): mean[0]= 12.00, mean[1..12]= 0.0
INFO: mdef.c(520): Reading model definition:
/usr/local/freeswitch/grammar/model/communicator/mdef
INFO: bin_mdef.c(173): Allocating 104160 * 8 bytes (813 KiB) for CD tree
INFO: tmat.c(205): Reading HMM transition probability matrices:
/usr/local/freeswitch/grammar/model/communicator/transition_matrices
INFO: acmod.c(117): Attempting to use SCHMM computation module
INFO: ms_gauden.c(198): Reading mixture gaussian parameter:
/usr/local/freeswitch/grammar/model/communicator/means
INFO: ms_gauden.c(292): 1 codebook, 4 feature, size:
INFO: ms_gauden.c(294):  256x12
INFO: ms_gauden.c(294):  256x24
INFO: ms_gauden.c(294):  256x3
INFO: ms_gauden.c(294):  256x12
INFO: ms_gauden.c(198): Reading mixture gaussian parameter:
/usr/local/freeswitch/grammar/model/communicator/variances
INFO: ms_gauden.c(292): 1 codebook, 4 feature, size:
INFO: ms_gauden.c(294):  256x12
INFO: ms_gauden.c(294):  256x24
INFO: ms_gauden.c(294):  256x3
INFO: ms_gauden.c(294):  256x12
INFO: ms_gauden.c(354): 59 variance values floored
INFO: s2_semi_mgau.c(908): Loading senones from dump file
/usr/local/freeswitch/grammar/model/communicator/sendump
INFO: s2_semi_mgau.c(932): BEGIN FILE FORMAT DESCRIPTION
INFO: s2_semi_mgau.c(995): Rows: 256, Columns: 6256
INFO: s2_semi_mgau.c(1027): Using memory-mapped I/O for senones
INFO: s2_semi_mgau.c(1304): Maximum top-N: 4 Top-N beams: 0 0 0 0
INFO: dict.c(306): Allocating 137548 * 32 bytes (4298 KiB) for word entries
INFO: dict.c(321): Reading main dictionary:
/usr/local/freeswitch/grammar/default.dic
INFO: dict.c(212): Allocated 1010 KiB for strings, 1664 KiB for phones
INFO: dict.c(324): 133436 words read
INFO: dict.c(330): Reading filler dictionary:
/usr/local/freeswitch/grammar/model/communicator/noisedict
INFO: dict.c(212): Allocated 0 KiB for strings, 0 KiB for phones
INFO: dict.c(333): 17 words read
INFO: dict2pid.c(396): Building PID tables for dictionary
INFO: dict2pid.c(404): Allocating 51^3 * 2 bytes (259 KiB) for
word-initial triphones
INFO: dict2pid.c(131): Allocated 62832 bytes (61 KiB) for word-final
triphones
INFO: dict2pid.c(195): Allocated 62832 bytes (61 KiB) for single-phone
word triphones
INFO: fsg_search.c(145): FSG(beam: -1080, pbeam: -1080, wbeam: -634; wip:
-26, pip: 0)
INFO: jsgf.c(546): Defined rule: <xxxxxx.g00000>
INFO: jsgf.c(546): Defined rule: <xxxxxx.yes>
INFO: jsgf.c(546): Defined rule: <xxxxxx.g00002>
INFO: jsgf.c(546): Defined rule: <xxxxxx.no>
INFO: jsgf.c(546): Defined rule: <xxxxxx.g00004>
INFO: jsgf.c(546): Defined rule: <xxxxxx.hello>
INFO: jsgf.c(546): Defined rule: <xxxxxx.g00006>
INFO: jsgf.c(546): Defined rule: PUBLIC <xxxxxx.results>
INFO: fsg_model.c(213): Computing transitive closure for null transitions
INFO: fsg_model.c(264): 87 null transitions added
INFO: fsg_model.c(411): Adding silence transitions for <sil> to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++AE++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++AH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++BACKGROUND++ to
FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++BREATH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++COUGH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++EH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++ER++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++LAUGH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++MM++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++MUMBLE++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++NOISE++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++OH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++SMACK++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++UH++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++UH_NOISE++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++UM++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_model.c(411): Adding silence transitions for ++UM_NOISE++ to FSG
INFO: fsg_model.c(431): Added 23 silence word transitions
INFO: fsg_search.c(364): Added 1 alternate word transitions
INFO: fsg_lextree.c(108): Allocated 2392 bytes (2 KiB) for left and right
context phones
INFO: fsg_lextree.c(251): 505 HMM nodes in lextree (496 leaves)
INFO: fsg_lextree.c(253): Allocated 64640 bytes (63 KiB) for all lextree
nodes
INFO: fsg_lextree.c(256): Allocated 63488 bytes (62 KiB) for lextree
leafnodes
2012-05-15 17:01:11.428551 [DEBUG] switch_core_media_bug.c:502 Attaching
BUG to sofia/xxxxxx/2143909113
2012-05-15 17:01:11.448552 [DEBUG] switch_core_io.c:353 Setting BUG Codec
PCMU:0
2012-05-15 17:01:13.448551 [NOTICE] switch_cpp.cpp:1227
----------------->call answered is false...<-------------
EXECUTE sofia/xxxxxx/2143909113 detect_speech(resume)
2012-05-15 17:01:15.008548 [INFO] switch_cpp.cpp:1227 Callback with type
event
2012-05-15 17:01:15.008548 [INFO] switch_cpp.cpp:1227
'Event-Name: DETECTED_SPEECH
Core-UUID: 0a387574-9ee8-11e1-8fa1-43045b098f7a
FreeSWITCH-Hostname: xxxxxx
FreeSWITCH-Switchname: xxxxxx
FreeSWITCH-IPv4: 199.15.96.29
FreeSWITCH-IPv6: %3A%3A1
Event-Date-Local: 2012-05-15%2017%3A01%3A14
Event-Date-GMT: Wed,%2016%20May%202012%2000%3A01%3A14%20GMT
Event-Date-Timestamp: 1337126474988540
Event-Calling-File: switch_ivr_async.c
Event-Calling-Function: speech_thread
Event-Calling-Line-Number: 3549
Event-Sequence: 1204
Speech-Type: begin-speaking

'
2012-05-15 17:01:15.008548 [NOTICE] switch_cpp.cpp:1227
----------------->call answered is false...<-------------
EXECUTE sofia/xxxxxx/2143909113 detect_speech(resume)
2012-05-15 17:01:17.008548 [NOTICE] switch_cpp.cpp:1227
----------------->call answered is false...<-------------
EXECUTE sofia/xxxxxx/2143909113 detect_speech(resume)
INFO: cmn_prior.c(121): cmn_prior_update: from <  8.00  0.00  0.00  0.00
0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00  0.00 >
INFO: cmn_prior.c(139): cmn_prior_update: to   <  8.11 -0.00  0.40 -0.00
-0.39 -0.64 -0.79 -0.65 -0.51 -0.32 -0.08  0.12  0.11 >
INFO: fsg_search.c(1030): 133 frames, 1846 HMMs (13/fr), 5417 senones
(40/fr), 1099 history entries (8/fr)

INFO: fsg_search.c(1407): Start node hello.0:14:96
INFO: fsg_search.c(1446): End node <sil>.95:105:132 (-2846)
INFO: fsg_search.c(1662): lattice start node hello.0 end node <sil>.95
INFO: ps_lattice.c(1352): Normalizer P(O) = alpha(<sil>:95:132) = -886169
INFO: ps_lattice.c(1390): Joint P(O,S) = -886169 P(S|O) = 0
2012-05-15 17:01:17.648543 [DEBUG] mod_pocketsphinx.c:383 Recognized:
hello, Confidence: 100
2012-05-15 17:01:17.648543 [INFO] switch_cpp.cpp:1227 Callback with type
event
2012-05-15 17:01:17.648543 [INFO] switch_cpp.cpp:1227
'Event-Name: DETECTED_SPEECH
Core-UUID: 0a387574-9ee8-11e1-8fa1-43045b098f7a
FreeSWITCH-Hostname: xxxxxx
FreeSWITCH-Switchname: xxxxxx
FreeSWITCH-IPv4: 199.15.96.29
FreeSWITCH-IPv6: %3A%3A1
Event-Date-Local: 2012-05-15%2017%3A01%3A17
Event-Date-GMT: Wed,%2016%20May%202012%2000%3A01%3A17%20GMT
Event-Date-Timestamp: 1337126477648543
Event-Calling-File: switch_ivr_async.c
Event-Calling-Function: speech_thread
Event-Calling-Line-Number: 3549
Event-Sequence: 1209
Speech-Type: detected-speech
Content-Length: 172

<?xml version="1.0"?>
<result grammar="xxxxxx">
  <interpretation grammar="xxxxxx" confidence="100">
    <input mode="speech">hello</input>
  </interpretation>
</result>
'
EXECUTE sofia/xxxxxx/2143909113 detect_speech(pause)






Join us at ClueCon 2011 Aug 9-11, 2011
More information about the FreeSWITCH-users mailing list