Thanks Antoine,<br>
<br>
Helios does not have access to the Phoenix grammar does it? It only gets the string and the output parse, right?<br>
Here is my grammar excerpt. (the full grammar file is attached)<br>
<br>
I am binding Question and EventTypeSpec concepts in separate agents, so phoenix outputs:<br>
(Maybe this is a problem?)<br>
<br>
[Question] ( [where_event] ( WHERE ) )<br>
Features<br>
[EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) )<br>
<br>
Question and EventTypeSpec are declared in form<br>
<br>
<br>
[Question]<br>
([whattime_event]) <br>
([describe_event])<br>
([where_event])<br>
;<br>
<br>
[where_event]<br>
(where)<br>
(at which location)<br>
(what is the location)<br>
(which location)<br>
(what location)<br>
;<br>
<br>
# SS: I used to have this, but it did not matter:<br>
# (where *is *the *EVENT)<br>
# (*at which location *is *the *EVENT)<br>
<br>
<br>[EventTypeSpec]<br>
([event_type])<br>
;<br><div><span class="gmail_quote">[event_type]<br>
([event_type_singular])<br>
([event_type_plural])<br>
;<br>
<br>
On 11/8/06, <b class="gmail_sendername">Antoine Raux</b> <<a href="mailto:antoine@cs.cmu.edu">antoine@cs.cmu.edu</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div link="blue" vlink="blue" lang="FR">
<div>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US">I think the problem is either
in the grammar or in Helios. Helios uses features such as "how many top level
slots does the parse contain", "how many words are covered by the parse", … and
combines them (along with more typical speech recognition features) to produce
the confidence score. What seems to be happening here is that the grammar is
such that the parse for "WHERE IS CLASS", while producing the right concepts,
leads to low confidence.</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US">Three ways to solve this:
1) hack Helios to always give perfect confidence for parsed input (that is *<b><span style="font-weight: bold;">not</span></b>* a long term solution, I wouldn't
recommend it), 2) fix your grammar so that it produces parses that are more
like what Helios expect, 3) fix the Helios model to match your grammar. The
problem with 3) is that the confidence model in Helios is trained on data so
you'd need data for your system before being able to retrain Helios. So for now
I'd try to go for 2).</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US"> </span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US">Could you send the parse
for these two utterances and you grammar?</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US"> </span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US">antoine</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US"> </span></font></p>
<div>
<div style="text-align: center;" align="center"><font face="Times New Roman" size="3"><span style="font-size: 12pt;" lang="EN-US">
<hr align="center" size="2" width="100%">
</span></font></div>
<p><b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma; font-weight: bold;" lang="EN-US">From:</span></font></b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;" lang="EN-US">
Svetlana Stenchikova [mailto:<a href="mailto:svetastenchikova@gmail.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">svetastenchikova@gmail.com</a>] <br>
<b><span style="font-weight: bold;">Sent:</span></b> Wednesday, November 08, 2006
5:45 PM<span class="q"><br>
<b><span style="font-weight: bold;">To:</span></b> Antoine Raux<br>
<b><span style="font-weight: bold;">Cc:</span></b> <a href="mailto:ravenclaw-developers@cs.cmu.edu" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">ravenclaw-developers@cs.cmu.edu</a><br></span>
<b><span style="font-weight: bold;">Subject:</span></b> Re: [RavenclawDev 182]
Re: grounding_manager_configuration setting inconfig file of DM</span></font></p>
</div><div><span class="e" id="q_10eca6b7447f6a11_3">
<p><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
<p style="margin-bottom: 12pt;"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">In the case of WHERE
CLASS, the conversation continues correctly, but in WHERE IS CLASS it it
goes into default explicit confirmation for CLASS (I just added it - thanks for
the hint)<br>
<br>
I don't think it is the desirable behavior though. These typed (or perfectly
recognized) concepts. Why should it need to confirm them?<br>
<br>
thanks<br>
Svetlana</span></font></p>
<div>
<p><span><font face="Times New Roman" size="3"><span style="font-size: 12pt;">On 11/8/06, <b><span style="font-weight: bold;">Antoine
Raux</span></b> <<a href="mailto:antoine@cs.cmu.edu" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">
antoine@cs.cmu.edu</a>> wrote:</span></font></span></p>
<div link="blue" vlink="blue">
<div>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US">What exactly is happening after you type either
of those?</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US">In the second case (WHERE IS CLASS) RC is trying
to perform an explicit confirmation "I think you said you wanted
information about a class. Is this correct?", whereas in the first case
it's just trying to do an implicit confirmation "Information about a
class… <moves on to the next task related question>".</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US">For any of these to work properly, you need to
have the ExplicitConfirm.pm and ImplicitConfirm.pm updated in the NLG. They
should map the concept name to the proper confirmation prompt (these Rosetta
modeuls work the same as <a href="http://Inform.pm" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">Inform.pm</a>
or <a href="http://Request.pm" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">Request.pm</a>). Maybe the
problem is that you're missing those? It wouldn't be a major problem in the
implicit confirmation case because the dialog moves on anyway but it would in
the explicit confirmation case because RC waits for a user answer (it creates a
new request agent and puts it on top of the stack).</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US"> </span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US">So the thing is, when you start dealing with
speech, you need to have all those grounding models and their associated
prompts in the NLG set correctly. It takes a little while but then RC does the
whole grounding job for you…</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US">I guess we really need some documentation about
this (Dan??)…</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US"> </span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US">Hope this helps…</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US">antoine</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US"> </span></font></p>
<div>
<div style="text-align: center;" align="center"><font face="Times New Roman" size="3"><span style="font-size: 12pt;" lang="EN-US">
<hr align="center" size="2" width="100%">
</span></font></div>
<p><b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma; font-weight: bold;" lang="EN-US">From:</span></font></b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;" lang="EN-US">
Svetlana Stenchikova [mailto:<a href="mailto:svetastenchikova@gmail.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">svetastenchikova@gmail.com</a>] <br>
<b><span style="font-weight: bold;">Sent:</span></b> Wednesday, November 08, 2006
4:51 PM<br>
<b><span style="font-weight: bold;">To:</span></b> Antoine Raux<br>
<b><span style="font-weight: bold;">Cc:</span></b> <a href="mailto:ravenclaw-developers@cs.cmu.edu" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">ravenclaw-developers@cs.cmu.edu</a><br>
<b><span style="font-weight: bold;">Subject:</span></b> Re: [RavenclawDev 179]
grounding_manager_configuration setting inconfig file of DM</span></font></p>
</div>
<div>
<p><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
<p style="margin-bottom: 12pt;"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">I am not sure which entry to look at. <br>
<br>
Could this be because I am binding multiple concepts?<br>
It is not a problem for me now as I can fix it by taking out
grounding_manager_configuration from config. <br>
I am just concerned that I will deal with these issues with speech.<br>
<br>
<br>
>From a first glance at the helios log:<br>
WHERE CLASS<br>
<br>
confidence 0.62288<br>
<br>
WHERE IS CLASS<br>
<br>
confidence 0.28883<br>
<br>
<br>
In the dialog.log I get entries:<br>
<br>
I think this is for WHERE CLASS:<br>
****************************************<br>
<br>
[GMX@16:11:10.083] Grounding model /RavenCalendar/question_type_string
[TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]:<br>
Full state: <br>
grounded = false<br>
top_confidence = 0.622880<br>
updated = true<br>
Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38
GROUNDED:0.00 <br>
Action values (dumped below):<br>
ACCEPT:2.4576<br>
EXPL_CONF:0.6568<br>
<br>
Suggested action: ACCEPT<br>
<br>
[GMX@16:11:10.083] Concept /RavenCalendar/the_event.event_type dumped below:<br>
CLASS|0.62<br>
[GMX@16:11:10.083] Grounding model /RavenCalendar/the_event.event_type
[TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]:<br>
Full state: <br>
grounded = false<br>
top_confidence = 0.622880<br>
updated = true<br>
Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38
GROUNDED:0.00 <br>
Action values (dumped below):<br>
ACCEPT:-2.1822<br>
EXPL_CONF:0.6568<br>
IMPL_CONF:1.8856<br>
<br>
Suggested action: IMPL_CONF<br>
<br>
<br>
<br>
*****************************************<br>
<br>
and this for WHERE IS CLASS<br>
<br>
<br>
<br>
[GMX@16:11:13.314] Grounding model /RavenCalendar/question_type_string
[TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]:<br>
Full state: <br>
grounded = false<br>
top_confidence = 0.000000<br>
updated = false<br>
Belief state: INACTIVE:1.00 CONFIDENT:0.00 UNCONFIDENT:0.00
GROUNDED:0.00 <br>
Action values (dumped below):<br>
ACCEPT:10.0000<br>
EXPL_CONF: -<br>
<br>
Suggested action: ACCEPT<br>
<br>
[GMX@16:11:13.314] Concept /RavenCalendar/the_event.event_type dumped below:<br>
<UNDEFINED><br>
[GMX@16:11:13.315] Grounding model /RavenCalendar/the_event.event_type
[TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]:<br>
Full state: <br>
grounded = false<br>
top_confidence = 0.000000<br>
<br>
<br>
[GMX@16:11:37.467] Concept /RavenCalendar/question_type_string dumped below:<br>
where|0.29<br>
[GMX@16:11:37.467] Grounding model /RavenCalendar/question_type_string
[TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]:<br>
Full state: <br>
grounded = false<br>
top_confidence = 0.288830<br>
updated = true<br>
Belief state: INACTIVE:0.00 CONFIDENT:0.29 UNCONFIDENT:0.71
GROUNDED:0.00 <br>
Action values (dumped below):<br>
ACCEPT:-4.2234<br>
EXPL_CONF:5.6675<br>
<br>
Suggested action: EXPL_CONF<br>
<br>
<br>
<br>
<br>
<br>
</span></font></p>
<div>
<p><font face="Times New Roman" size="3"><span style="font-size: 12pt;">On
11/8/06, <b><span style="font-weight: bold;">Antoine Raux</span></b> <<a href="mailto:antoine@cs.cmu.edu" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">antoine@cs.cmu.edu</a>>
wrote:</span></font></p>
<div link="blue" vlink="purple">
<div>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Hi Svetlana,</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US">Have you looked at the confidence value for the
input in both cases? This is one of the features that Helios sends to the DM
for each input. It is possible that "where class" has a low confidence
for some reason (e.g. segmented parse…) and RC rejects it. Normally if your
grammar gives a full parse for "where class" that shouldn't happen
(at least on typed input) but…</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US"> </span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US">antoine</span></font></p>
<p><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;" lang="EN-US"> </span></font></p>
<div>
<div style="text-align: center;" align="center"><font face="Times New Roman" size="3"><span style="font-size: 12pt;" lang="EN-US">
<hr align="center" size="2" width="100%">
</span></font></div>
<p><b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma; font-weight: bold;" lang="EN-US">From:</span></font></b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;" lang="EN-US">
<a href="mailto:ravenclaw-developers-bounces@LOGANBERRY.srv.cs.cmu.edu" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">ravenclaw-developers-bounces@LOGANBERRY.srv.cs.cmu.edu</a>
[mailto:<a href="mailto:ravenclaw-developers-bounces@LOGANBERRY.srv.cs.cmu.edu" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">ravenclaw-developers-bounces@LOGANBERRY.srv.cs.cmu.edu</a>] <b><span style="font-weight: bold;">
On Behalf Of </span></b>Svetlana Stenchikova<br>
<b><span style="font-weight: bold;">Sent:</span></b> Wednesday, November 08, 2006
2:50 PM<br>
<b><span style="font-weight: bold;">To:</span></b> <a href="mailto:ravenclaw-developers@cs.cmu.edu" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">ravenclaw-developers@cs.cmu.edu</a><br>
<b><span style="font-weight: bold;">Subject:</span></b> [RavenclawDev 179]
grounding_manager_configuration setting inconfig file of DM</span></font></p>
</div>
<div>
<p><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
<p><font face="Times New Roman" size="3"><span style="font-size: 12pt;"><br>
I have encountered an issue with binding a concept in DM depending on a user
utterance.<br>
<br>
in my DM I have a grammar mapping.
"@(...)[Question.where_event]>where,"<br>
<br>
<br>
The user can ask a question in multiple ways:<br>
where class<br>
where is class<br>
where is the class<br>
what is the location the class<br>
..<br>
<br>
for all of these entries PHOENIX produces identical parse<br>
<br>
[Question] ( [where_event] ( WHERE ) )<br>
Features<br>
[EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) )<br>
<br>
" } )<br>
<br>
When a configuration file passed to the DM in -config has an entry:
grounding_manager_configuration =
turns:default;concepts:default;beliefupdatingmodel:npu<br>
<br>
<br>
if a user types:<br>
"where is the class" DM does not bind the concepts,
but if the user types <br>
"where class"
DM binds the concepts.<br>
<br>
When the grounding_manager_configuration is taken out of DM, the binding
succeeds for both utterances.<br>
<br>
Can this be explained? Do you think there may be a problem with my
grammar or something in my DM? <br>
The grammar of the Phoenix parser should not matter because in all cases the
parses are identical, right?<br>
<br>
thank you<br>
<br>
Svetlana</span></font></p>
</div>
</div>
</div>
</div>
<p><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
</div>
</div>
</div>
</div>
<p><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
</span></div></div>
</div>
</blockquote></div><br>