From dbohus at cs.cmu.edu Sat Nov 4 15:53:51 2006 From: dbohus at cs.cmu.edu (Dan Bohus) Date: Sat, 4 Nov 2006 15:53:51 -0500 Subject: [RavenclawDev 171] Re: reopening concepts In-Reply-To: <31cecd6b0610272041n68ec79aes98f21f80907e2abc@mail.gmail.com> Message-ID: <1546B8BE23C96940B5B4929DB7CC2343AC523C@e2k3.srv.cs.cmu.edu> Hi Svetlana, There's a ReOpenConcepts() method class on each agent. If you call it, all the concept contained in the subtree starting at that agent will be reopened. Reopen pushes the values into the history. It does not set the concept to UNAVAILABLE. A concept is AVAILABLE if a value exists for that concept somewhere (current value or in history) A concept is UPDATED if a value exists for that concept in the current value. So once you Reopen in general the concept will still be available but it will no longer be updated. The answer to the last question escapes me right now (I'll have to look at the Roomline task tree). I suspect that either there is some reason I might still want to hold on to the histories for some reason, or it might be just a poor implementation issue... Hope this helps, -Dan ________________________________ From: ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto:ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] On Behalf Of Svetlana Stenchikova Sent: Friday, October 27, 2006 11:42 PM To: ravenclaw-developers at cs.cmu.edu Subject: [RavenclawDev 169] reopening concepts Hi, is it possible to REOPEN all concepts that are declared in the above agent without explicitly specifying each concept? I understand that REOPEN concept pushes the concept value into the history. Does it also set the status of the concept to UNAVAILABLE? In Roomline you also call ClearExplicitlyConfirmedHyp() and ClearExplicitlyDisconfirmedHyp() in ResetTask. Why not call clear() which is supposed to clear everything for the concept? thank you, Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061104/c2c61d1c/attachment.html From svetastenchikova at gmail.com Sun Nov 5 18:43:42 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Sun, 5 Nov 2006 18:43:42 -0500 Subject: [RavenclawDev 172] Re: bypassing the parser Message-ID: <31cecd6b0611051543g32f60440p9e40c1652247557@mail.gmail.com> Dan, thank you for your answer. I have another question. We are interested in the functionality where a free text can be entered through TTY (or through free speech recognition). For example, a user can be asked to describe the event. The grammar for this description is not available, so we want to store all of the typed ( or recognized) text, and/or a URL of the recorded speech into the concept in a dialog manager. We need to bypass the parser. I would need to make changes to the RavenClaw to accommodate this, right? Currently the user input from the TTY (or ASR) is processed by the parser. The control flow is determined by the .pgm file. This entry causes it to call process_parse when an output from the parser is available: ;; If we have a confidence annotated hypothesis, but ;; no parse, call phoenix to parse it ;; --- RULE: :confhyps & !:parses --> phoenix.phoenixparse IN: :confhyps :npow :pow :clip_info OUT: :total_numparses :parses :input_source LOG_OUT: :parses So I plan to add another entry: ;; If we have a confidence annotated hypothesis, but ;; no parse, call DM directly ;; --- RULE: :confhyps & !:parses --> DM.process_raw IN: :confhyps :npow :pow :clip_info OUT: LOG_OUT: Add process_raw() to Libraries/RavenClaw/DMInterfaces/GalaxyInterfaceFunctions.h& implement in GalaxyInterface.cpp process_raw() function will be called whenever an output from the TTY (or ASR) is received. So, this output will go to both parser and DialogManager.process_raw. process_raw() assigns the text to a concept (but only in the case when the concept is currently expected): For every expected concept as compiled by CDMCoreAgent::compileExpectationAgenda(): If a concept is a "raw speech" concept, set its value to the raw text received in an input frame to process_raw I would have to add a type to a concept to indicate that it allows "raw text" input. Does this implementation plan make sense? Could you please make any suggestions? Jahanzeb must have done something similar because his application can record user's input. Do you know how that was implemented? Thank you, Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061105/d2d5ae0f/attachment-0001.html From svetastenchikova at gmail.com Sun Nov 5 19:52:53 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Sun, 5 Nov 2006 19:52:53 -0500 Subject: [RavenclawDev 173] Re: bypassing the parser In-Reply-To: <31cecd6b0611051543g32f60440p9e40c1652247557@mail.gmail.com> References: <31cecd6b0611051543g32f60440p9e40c1652247557@mail.gmail.com> Message-ID: <31cecd6b0611051652o6c16170fub32a9d393664b6cc@mail.gmail.com> I just thought of another way to implement this without changing RavenClaw: Add another server ProcessRawData which receives the raw data input (in parallel with the parser) ProcessRawData outputs the input text in the format of the Parser: [RawText] ( the typed in text ) The output is passed to the DialogManager.process_parse message The the text will bind to all concept that are expected in the [RawText] grammar mapping. Please let me know your thoughts on this. thank you Svetlana On 11/5/06, Svetlana Stenchikova wrote: > > Dan, thank you for your answer. I have another question. > > We are interested in the functionality where a free text can be entered > through TTY (or through free speech recognition). > > For example, a user can be asked to describe the event. The grammar for > this description is not available, so we want to store all of the typed ( or > recognized) text, and/or a URL of the recorded speech into the concept in > a dialog manager. > > We need to bypass the parser. I would need to make changes to the > RavenClaw to accommodate this, right? > > Currently the user input from the TTY (or ASR) is processed by the > parser. > > The control flow is determined by the .pgm file. This entry causes it to > call process_parse when an output from the parser is available: > > > > ;; If we have a confidence annotated hypothesis, but > ;; no parse, call phoenix to parse it > ;; --- > RULE: :confhyps & !:parses --> phoenix.phoenixparse > IN: :confhyps :npow :pow :clip_info > OUT: :total_numparses :parses :input_source > LOG_OUT: :parses > > > > So I plan to add another entry: > > ;; If we have a confidence annotated hypothesis, but > ;; no parse, call DM directly > ;; --- > RULE: :confhyps & !:parses --> DM.process_raw > IN: :confhyps :npow :pow :clip_info > OUT: > LOG_OUT: > > > > Add process_raw() to > > Libraries/RavenClaw/DMInterfaces/GalaxyInterfaceFunctions.h& implement in > GalaxyInterface.cpp > > > process_raw() function will be called whenever an output from the TTY (or > ASR) is received. So, this output will go to both parser and > DialogManager.process_raw. > process_raw() assigns the text to a concept (but only in the case when > the concept is currently expected): > For every expected concept as compiled by > CDMCoreAgent::compileExpectationAgenda(): > If a concept is a "raw speech" concept, set its value to the raw text > received in an input frame to process_raw > I would have to add a type to a concept to indicate that it allows "raw > text" input. > > > Does this implementation plan make sense? > > Could you please make any suggestions? > > Jahanzeb must have done something similar because his application can > record user's input. Do you know how that was implemented? > > > Thank you, > Svetlana > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061105/1bfdd3f3/attachment.html From antoine at cs.cmu.edu Sun Nov 5 20:47:28 2006 From: antoine at cs.cmu.edu (Antoine Raux) Date: Sun, 05 Nov 2006 20:47:28 -0500 Subject: [RavenclawDev 174] Re: bypassing the parser In-Reply-To: <31cecd6b0611051652o6c16170fub32a9d393664b6cc@mail.gmail.com> References: <31cecd6b0611051543g32f60440p9e40c1652247557@mail.gmail.com> <31cecd6b0611051652o6c16170fub32a9d393664b6cc@mail.gmail.com> Message-ID: <454E9430.3030801@cs.cmu.edu> Hi Svetlana, RavenClaw actually has access to the (recognized) text corresponding to each input, whether it's parsable or not. It's part of the set of features that Helios passes to the DM. I forgot exactly how to access these features but I believe it's basically a grammar mapping with a special prefix that indicates that you're not binding a grammar slot but a Helios input structure slot. Sorry I don't remember the exact syntax now... If nobody else (Dan?) replies with more precision, I'll find it out and email that later in the week. In any case, you might be able to get what you want (the text of the input) without modifying anything at all but your task specification (grammar mappings)... Otherwise, the second approach you are proposing (basically adding a second "parser") sounds definitely much better than the first one, in the sense that it preserves the integrity of the input line (which is important for many things in the DM like grounding, reference to previous turns....). I hope Dan will come up with a more precise answer soon :) antoine Svetlana Stenchikova wrote: > I just thought of another way to implement this without changing > RavenClaw: > > Add another server ProcessRawData which receives the raw data input > (in parallel with the parser) > ProcessRawData outputs the input text in the format of the Parser: > > [RawText] ( the typed in text ) > > The output is passed to the DialogManager.process_parse message > > The the text will bind to all concept that are expected in the > [RawText] grammar mapping. > > Please let me know your thoughts on this. > > thank you > Svetlana > > > On 11/5/06, *Svetlana Stenchikova* > wrote: > > Dan, thank you for your answer. I have another question. > > We are interested in the functionality where a free text can be > entered through TTY (or through free speech recognition). > > For example, a user can be asked to describe the event. The > grammar for this description is not available, so we want to store > all of the typed ( or recognized) text, and/or a URL of the > recorded speech into the concept in a dialog manager. > > We need to bypass the parser. I would need to make changes to the > RavenClaw to accommodate this, right? > > Currently the user input from the TTY (or ASR) is processed by > the parser. > > The control flow is determined by the .pgm file. This entry > causes it to call process_parse when an output from the parser is > available: > > > > ;; If we have a confidence annotated hypothesis, but > ;; no parse, call phoenix to parse it > ;; --- > RULE: :confhyps & !:parses --> phoenix.phoenixparse > IN: :confhyps :npow :pow :clip_info > OUT: :total_numparses :parses :input_source > LOG_OUT: :parses > > > > So I plan to add another entry: > > ;; If we have a confidence annotated hypothesis, but > ;; no parse, call DM directly > ;; --- > RULE: :confhyps & !:parses --> DM.process_raw > IN: :confhyps :npow :pow :clip_info > OUT: > LOG_OUT: > > > > Add process_raw() to > > Libraries/RavenClaw/DMInterfaces/GalaxyInterfaceFunctions.h& > implement in GalaxyInterface.cpp > > > process_raw() function will be called whenever an output from the > TTY (or ASR) is received. So, this output will go to both parser > and DialogManager.process_raw. > process_raw() assigns the text to a concept (but only in the case > when the concept is currently expected): > For every expected concept as compiled by > CDMCoreAgent::compileExpectationAgenda(): > If a concept is a "raw speech" concept, set its value to the raw > text received in an input frame to process_raw > I would have to add a type to a concept to indicate that it allows > "raw text" input. > > > Does this implementation plan make sense? > > Could you please make any suggestions? > > Jahanzeb must have done something similar because his application > can record user's input. Do you know how that was implemented? > > > > Thank you, > Svetlana > > > > From svetastenchikova at gmail.com Sun Nov 5 22:06:18 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Sun, 5 Nov 2006 22:06:18 -0500 Subject: [RavenclawDev 175] Re: bypassing the parser In-Reply-To: <454E9430.3030801@cs.cmu.edu> References: <31cecd6b0611051543g32f60440p9e40c1652247557@mail.gmail.com> <31cecd6b0611051652o6c16170fub32a9d393664b6cc@mail.gmail.com> <454E9430.3030801@cs.cmu.edu> Message-ID: <31cecd6b0611051906o2acee8b8t1c0cf28d93671168@mail.gmail.com> Thank you Dan and Antoine for your quick response. Antoine, your suggestion sounds interesting. DM does get called even if there is no parse found (I did not think it was) So, the change would be in a RequestAgent, right? I will probably still go for the "second parser" because eventually we will want to record and save the audio, where, I guess, the audio input will go directly to this "parser" which will output the location of the saved file. Thanks! Svetlana On 11/5/06, Antoine Raux wrote: > > Hi Svetlana, > > RavenClaw actually has access to the (recognized) text corresponding to > each input, whether it's parsable or not. It's part of the set of > features that Helios passes to the DM. I forgot exactly how to access > these features but I believe it's basically a grammar mapping with a > special prefix that indicates that you're not binding a grammar slot but > a Helios input structure slot. Sorry I don't remember the exact syntax > now... If nobody else (Dan?) replies with more precision, I'll find it > out and email that later in the week. In any case, you might be able to > get what you want (the text of the input) without modifying anything at > all but your task specification (grammar mappings)... > > Otherwise, the second approach you are proposing (basically adding a > second "parser") sounds definitely much better than the first one, in > the sense that it preserves the integrity of the input line (which is > important for many things in the DM like grounding, reference to > previous turns....). > > I hope Dan will come up with a more precise answer soon :) > > antoine > > Svetlana Stenchikova wrote: > > > I just thought of another way to implement this without changing > > RavenClaw: > > > > Add another server ProcessRawData which receives the raw data input > > (in parallel with the parser) > > ProcessRawData outputs the input text in the format of the Parser: > > > > [RawText] ( the typed in text ) > > > > The output is passed to the DialogManager.process_parse message > > > > The the text will bind to all concept that are expected in the > > [RawText] grammar mapping. > > > > Please let me know your thoughts on this. > > > > thank you > > Svetlana > > > > > > On 11/5/06, *Svetlana Stenchikova* > > wrote: > > > > Dan, thank you for your answer. I have another question. > > > > We are interested in the functionality where a free text can be > > entered through TTY (or through free speech recognition). > > > > For example, a user can be asked to describe the event. The > > grammar for this description is not available, so we want to store > > all of the typed ( or recognized) text, and/or a URL of the > > recorded speech into the concept in a dialog manager. > > > > We need to bypass the parser. I would need to make changes to the > > RavenClaw to accommodate this, right? > > > > Currently the user input from the TTY (or ASR) is processed by > > the parser. > > > > The control flow is determined by the .pgm file. This entry > > causes it to call process_parse when an output from the parser is > > available: > > > > > > > > ;; If we have a confidence annotated hypothesis, but > > ;; no parse, call phoenix to parse it > > ;; --- > > RULE: :confhyps & !:parses --> phoenix.phoenixparse > > IN: :confhyps :npow :pow :clip_info > > OUT: :total_numparses :parses :input_source > > LOG_OUT: :parses > > > > > > > > So I plan to add another entry: > > > > ;; If we have a confidence annotated hypothesis, but > > ;; no parse, call DM directly > > ;; --- > > RULE: :confhyps & !:parses --> DM.process_raw > > IN: :confhyps :npow :pow :clip_info > > OUT: > > LOG_OUT: > > > > > > > > Add process_raw() to > > > > Libraries/RavenClaw/DMInterfaces/GalaxyInterfaceFunctions.h& > > implement in GalaxyInterface.cpp > > > > > > process_raw() function will be called whenever an output from the > > TTY (or ASR) is received. So, this output will go to both parser > > and DialogManager.process_raw. > > process_raw() assigns the text to a concept (but only in the case > > when the concept is currently expected): > > For every expected concept as compiled by > > CDMCoreAgent::compileExpectationAgenda(): > > If a concept is a "raw speech" concept, set its value to the raw > > text received in an input frame to process_raw > > I would have to add a type to a concept to indicate that it allows > > "raw text" input. > > > > > > Does this implementation plan make sense? > > > > Could you please make any suggestions? > > > > Jahanzeb must have done something similar because his application > > can record user's input. Do you know how that was implemented? > > > > > > > > Thank you, > > Svetlana > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061105/ff14ec9d/attachment-0001.html From svetastenchikova at gmail.com Sun Nov 5 23:19:36 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Sun, 5 Nov 2006 23:19:36 -0500 Subject: [RavenclawDev 176] Re: bypassing the parser In-Reply-To: <31cecd6b0611051906o2acee8b8t1c0cf28d93671168@mail.gmail.com> References: <31cecd6b0611051543g32f60440p9e40c1652247557@mail.gmail.com> <31cecd6b0611051652o6c16170fub32a9d393664b6cc@mail.gmail.com> <454E9430.3030801@cs.cmu.edu> <31cecd6b0611051906o2acee8b8t1c0cf28d93671168@mail.gmail.com> Message-ID: <31cecd6b0611052019u2e8194c4k7637d36bf9ed614b@mail.gmail.com> Antoine, I think this would be still useful for me. Even if I use another server for recording, I still need to bind recognized text to a concept. So, in case you remember how to do it with changes just in DM, please let me know. thank you, Svetlana On 11/5/06, Svetlana Stenchikova wrote: > > Thank you Dan and Antoine for your quick response. > > Antoine, your suggestion sounds interesting. DM does get called even if > there is no parse found (I did not think it was) > > So, the change would be in a RequestAgent, right? > > I will probably still go for the "second parser" because eventually we > will want to record and save the audio, where, I guess, the audio input will > go directly to this "parser" which will output the location of the saved > file. > > Thanks! > Svetlana > > > On 11/5/06, Antoine Raux wrote: > > > > Hi Svetlana, > > > > RavenClaw actually has access to the (recognized) text corresponding to > > each input, whether it's parsable or not. It's part of the set of > > features that Helios passes to the DM. I forgot exactly how to access > > these features but I believe it's basically a grammar mapping with a > > special prefix that indicates that you're not binding a grammar slot but > > a Helios input structure slot. Sorry I don't remember the exact syntax > > now... If nobody else (Dan?) replies with more precision, I'll find it > > out and email that later in the week. In any case, you might be able to > > get what you want (the text of the input) without modifying anything at > > all but your task specification (grammar mappings)... > > > > Otherwise, the second approach you are proposing (basically adding a > > second "parser") sounds definitely much better than the first one, in > > the sense that it preserves the integrity of the input line (which is > > important for many things in the DM like grounding, reference to > > previous turns....). > > > > I hope Dan will come up with a more precise answer soon :) > > > > antoine > > > > Svetlana Stenchikova wrote: > > > > > I just thought of another way to implement this without changing > > > RavenClaw: > > > > > > Add another server ProcessRawData which receives the raw data input > > > (in parallel with the parser) > > > ProcessRawData outputs the input text in the format of the Parser: > > > > > > [RawText] ( the typed in text ) > > > > > > The output is passed to the DialogManager.process_parse message > > > > > > The the text will bind to all concept that are expected in the > > > [RawText] grammar mapping. > > > > > > Please let me know your thoughts on this. > > > > > > thank you > > > Svetlana > > > > > > > > > On 11/5/06, *Svetlana Stenchikova* > > > wrote: > > > > > > Dan, thank you for your answer. I have another question. > > > > > > We are interested in the functionality where a free text can be > > > entered through TTY (or through free speech recognition). > > > > > > For example, a user can be asked to describe the event. The > > > grammar for this description is not available, so we want to store > > > > > all of the typed ( or recognized) text, and/or a URL of the > > > recorded speech into the concept in a dialog manager. > > > > > > We need to bypass the parser. I would need to make changes to the > > > RavenClaw to accommodate this, right? > > > > > > Currently the user input from the TTY (or ASR) is processed by > > > the parser. > > > > > > The control flow is determined by the .pgm file. This entry > > > causes it to call process_parse when an output from the parser is > > > available: > > > > > > > > > > > > ;; If we have a confidence annotated hypothesis, but > > > ;; no parse, call phoenix to parse it > > > ;; --- > > > RULE: :confhyps & !:parses --> phoenix.phoenixparse > > > IN: :confhyps :npow :pow :clip_info > > > OUT: :total_numparses :parses :input_source > > > LOG_OUT: :parses > > > > > > > > > > > > So I plan to add another entry: > > > > > > ;; If we have a confidence annotated hypothesis, but > > > ;; no parse, call DM directly > > > ;; --- > > > RULE: :confhyps & !:parses --> DM.process_raw > > > IN: :confhyps :npow :pow :clip_info > > > OUT: > > > LOG_OUT: > > > > > > > > > > > > Add process_raw() to > > > > > > Libraries/RavenClaw/DMInterfaces/GalaxyInterfaceFunctions.h& > > > implement in GalaxyInterface.cpp > > > > > > > > > process_raw() function will be called whenever an output from the > > > TTY (or ASR) is received. So, this output will go to both parser > > > and DialogManager.process_raw. > > > process_raw() assigns the text to a concept (but only in the case > > > when the concept is currently expected): > > > For every expected concept as compiled by > > > CDMCoreAgent::compileExpectationAgenda(): > > > If a concept is a "raw speech" concept, set its value to the raw > > > text received in an input frame to process_raw > > > I would have to add a type to a concept to indicate that it allows > > > > > "raw text" input. > > > > > > > > > Does this implementation plan make sense? > > > > > > Could you please make any suggestions? > > > > > > Jahanzeb must have done something similar because his application > > > can record user's input. Do you know how that was implemented? > > > > > > > > > > > > Thank you, > > > Svetlana > > > > > > > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061105/06e78db9/attachment.html From svetastenchikova at gmail.com Tue Nov 7 15:04:41 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Tue, 7 Nov 2006 15:04:41 -0500 Subject: [RavenclawDev 177] binding multiple concepts Message-ID: <31cecd6b0611071204sa13e962s2cdd62f23b6afce5@mail.gmail.com> Hi, I thought it would be possible to bind multiple concepts at one utterance, but when I try it it does not work. The phoenix gets this: "Queries [Question] ( [where_event] ( WHERE ) ) Features [EventTypeSpec] ( [event_type] ( [event_type_singular] ( CLASS ) ) ) The 2 separate expect agents that are both active at this time bind Question and EventTypeSpec. but I get an error in DM [WAR at 14:55:02.773] Multiple binding for grammar concept [event:timeout]. Agents dumped below. Binding performed just for the first agent. /RavenCalendar/TimeoutTerminate/TT_HandleFirstTimeout tries to bind to /RavenCal endar/TimeoutTerminate/TT_HandleFirstTimeout/_TT_HandleFirstTimeout_trigger /RavenCalendar/TimeoutTerminate/TT_ReestablishChannel/TT_TerminateOnTimeout trie s to bind to /RavenCalendar/TimeoutTerminate/TT_ReestablishChannel/TT_TerminateO nTimeout/_TT_TerminateOnTimeout_trigger . Could you please let me know whether it is intended this way and why? thank you Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061107/4cd6c92c/attachment.html From svetastenchikova at gmail.com Tue Nov 7 21:39:11 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Tue, 7 Nov 2006 21:39:11 -0500 Subject: [RavenclawDev 178] Re: binding multiple concepts In-Reply-To: <31cecd6b0611071204sa13e962s2cdd62f23b6afce5@mail.gmail.com> References: <31cecd6b0611071204sa13e962s2cdd62f23b6afce5@mail.gmail.com> Message-ID: <31cecd6b0611071839j2ea9d7d1j1fbe66362ca0b93@mail.gmail.com> Please, never mind my question. It seems to be binding just fine now. Svetlana On 11/7/06, Svetlana Stenchikova wrote: > > Hi, > > I thought it would be possible to bind multiple concepts at one > utterance, but when I try it it does not work. > > The phoenix gets this: > > "Queries > [Question] ( [where_event] ( WHERE ) ) > Features > [EventTypeSpec] ( [event_type] ( [event_type_singular] ( CLASS ) ) ) > > The 2 separate expect agents that are both active at this time bind > Question and EventTypeSpec. > > > but I get an error in DM > > [WAR at 14:55:02.773] Multiple binding for grammar concept [event:timeout]. > Agents > dumped below. Binding performed just for the first agent. > /RavenCalendar/TimeoutTerminate/TT_HandleFirstTimeout tries to bind to > /RavenCal > > endar/TimeoutTerminate/TT_HandleFirstTimeout/_TT_HandleFirstTimeout_trigger > /RavenCalendar/TimeoutTerminate/TT_ReestablishChannel/TT_TerminateOnTimeout > trie > s to bind to > /RavenCalendar/TimeoutTerminate/TT_ReestablishChannel/TT_TerminateO > nTimeout/_TT_TerminateOnTimeout_trigger > documents\work\calendar\ravencalendar\ > libraries\ravenclaw\dmcore\agents\coreagents\dmcoreagent.cpp, line 1080>. > > Could you please let me know whether it is intended this way and why? > > thank you > > Svetlana > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061107/2ac7c6ab/attachment.html From svetastenchikova at gmail.com Wed Nov 8 14:49:38 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Wed, 8 Nov 2006 14:49:38 -0500 Subject: [RavenclawDev 179] grounding_manager_configuration setting in config file of DM Message-ID: <31cecd6b0611081149g48d27674g30cb1870f6f19b5a@mail.gmail.com> I have encountered an issue with binding a concept in DM depending on a user utterance. in my DM I have a grammar mapping. "@(...)[Question.where_event]>where," The user can ask a question in multiple ways: where class where is class where is the class what is the location the class .. for all of these entries PHOENIX produces identical parse [Question] ( [where_event] ( WHERE ) ) Features [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) " } ) When a configuration file passed to the DM in -config has an entry: grounding_manager_configuration = turns:default;concepts:default;beliefupdatingmodel:npu if a user types: "where is the class" DM does not bind the concepts, but if the user types "where class" DM binds the concepts. When the grounding_manager_configuration is taken out of DM, the binding succeeds for both utterances. Can this be explained? Do you think there may be a problem with my grammar or something in my DM? The grammar of the Phoenix parser should not matter because in all cases the parses are identical, right? thank you Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061108/cac530b8/attachment-0001.html From antoine at cs.cmu.edu Wed Nov 8 15:46:50 2006 From: antoine at cs.cmu.edu (Antoine Raux) Date: Wed, 8 Nov 2006 15:46:50 -0500 Subject: [RavenclawDev 180] Re: grounding_manager_configuration setting inconfig file of DM In-Reply-To: <31cecd6b0611081149g48d27674g30cb1870f6f19b5a@mail.gmail.com> Message-ID: <001901c70377$03d1c860$03bd0280@sp.cs.cmu.edu> Hi Svetlana, Have you looked at the confidence value for the input in both cases? This is one of the features that Helios sends to the DM for each input. It is possible that "where class" has a low confidence for some reason (e.g. segmented parse.) and RC rejects it. Normally if your grammar gives a full parse for "where class" that shouldn't happen (at least on typed input) but. antoine _____ From: ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto:ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] On Behalf Of Svetlana Stenchikova Sent: Wednesday, November 08, 2006 2:50 PM To: ravenclaw-developers at cs.cmu.edu Subject: [RavenclawDev 179] grounding_manager_configuration setting inconfig file of DM I have encountered an issue with binding a concept in DM depending on a user utterance. in my DM I have a grammar mapping. "@(...)[Question.where_event]>where," The user can ask a question in multiple ways: where class where is class where is the class what is the location the class .. for all of these entries PHOENIX produces identical parse [Question] ( [where_event] ( WHERE ) ) Features [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) " } ) When a configuration file passed to the DM in -config has an entry: grounding_manager_configuration = turns:default;concepts:default;beliefupdatingmodel:npu if a user types: "where is the class" DM does not bind the concepts, but if the user types "where class" DM binds the concepts. When the grounding_manager_configuration is taken out of DM, the binding succeeds for both utterances. Can this be explained? Do you think there may be a problem with my grammar or something in my DM? The grammar of the Phoenix parser should not matter because in all cases the parses are identical, right? thank you Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061108/bc9a3fb8/attachment.html From svetastenchikova at gmail.com Wed Nov 8 16:50:31 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Wed, 8 Nov 2006 16:50:31 -0500 Subject: [RavenclawDev 181] Re: grounding_manager_configuration setting inconfig file of DM In-Reply-To: <001901c70377$03d1c860$03bd0280@sp.cs.cmu.edu> References: <31cecd6b0611081149g48d27674g30cb1870f6f19b5a@mail.gmail.com> <001901c70377$03d1c860$03bd0280@sp.cs.cmu.edu> Message-ID: <31cecd6b0611081350y79279453n1d70cc0b73a9973f@mail.gmail.com> I am not sure which entry to look at. Could this be because I am binding multiple concepts? It is not a problem for me now as I can fix it by taking out grounding_manager_configuration from config. I am just concerned that I will deal with these issues with speech. >From a first glance at the helios log: WHERE CLASS confidence 0.62288 WHERE IS CLASS confidence 0.28883 In the dialog.log I get entries: I think this is for WHERE CLASS: **************************************** [GMX at 16:11:10.083] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.622880 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED:0.00 Action values (dumped below): ACCEPT:2.4576 EXPL_CONF:0.6568 Suggested action: ACCEPT [GMX at 16:11:10.083] Concept /RavenCalendar/the_event.event_type dumped below: CLASS|0.62 [GMX at 16:11:10.083] Grounding model /RavenCalendar/the_event.event_type [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.622880 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED:0.00 Action values (dumped below): ACCEPT:-2.1822 EXPL_CONF:0.6568 IMPL_CONF:1.8856 Suggested action: IMPL_CONF ***************************************** and this for WHERE IS CLASS [GMX at 16:11:13.314] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.000000 updated = false Belief state: INACTIVE:1.00 CONFIDENT:0.00 UNCONFIDENT:0.00 GROUNDED:0.00 Action values (dumped below): ACCEPT:10.0000 EXPL_CONF: - Suggested action: ACCEPT [GMX at 16:11:13.314] Concept /RavenCalendar/the_event.event_type dumped below: [GMX at 16:11:13.315] Grounding model /RavenCalendar/the_event.event_type [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.000000 [GMX at 16:11:37.467] Concept /RavenCalendar/question_type_string dumped below: where|0.29 [GMX at 16:11:37.467] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.288830 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.29 UNCONFIDENT:0.71 GROUNDED:0.00 Action values (dumped below): ACCEPT:-4.2234 EXPL_CONF:5.6675 Suggested action: EXPL_CONF On 11/8/06, Antoine Raux wrote: > > Hi Svetlana, > > > > Have you looked at the confidence value for the input in both cases? This > is one of the features that Helios sends to the DM for each input. It is > possible that "where class" has a low confidence for some reason (e.g. > segmented parse?) and RC rejects it. Normally if your grammar gives a full > parse for "where class" that shouldn't happen (at least on typed input) but? > > > > antoine > > > ------------------------------ > > *From:* ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto: > ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] *On Behalf Of *Svetlana > Stenchikova > *Sent:* Wednesday, November 08, 2006 2:50 PM > *To:* ravenclaw-developers at cs.cmu.edu > *Subject:* [RavenclawDev 179] grounding_manager_configuration setting > inconfig file of DM > > > > > I have encountered an issue with binding a concept in DM depending on a > user utterance. > > in my DM I have a grammar mapping. "@(...)[Question.where_event]>where," > > > The user can ask a question in multiple ways: > where class > where is class > where is the class > what is the location the class > .. > > for all of these entries PHOENIX produces identical parse > > [Question] ( [where_event] ( WHERE ) ) > Features > [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) > > " } ) > > When a configuration file passed to the DM in -config has an entry: > grounding_manager_configuration = > turns:default;concepts:default;beliefupdatingmodel:npu > > > if a user types: > "where is the class" DM does not bind the concepts, but if the user > types > "where class" DM binds the concepts. > > When the grounding_manager_configuration is taken out of DM, the binding > succeeds for both utterances. > > Can this be explained? Do you think there may be a problem with my > grammar or something in my DM? > The grammar of the Phoenix parser should not matter because in all cases > the parses are identical, right? > > thank you > > Svetlana > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061108/69fc9951/attachment.html From antoine at cs.cmu.edu Wed Nov 8 17:00:12 2006 From: antoine at cs.cmu.edu (Antoine Raux) Date: Wed, 8 Nov 2006 17:00:12 -0500 Subject: [RavenclawDev 182] Re: grounding_manager_configuration setting inconfig file of DM In-Reply-To: <31cecd6b0611081350y79279453n1d70cc0b73a9973f@mail.gmail.com> Message-ID: <002401c70381$43be2db0$03bd0280@sp.cs.cmu.edu> What exactly is happening after you type either of those? In the second case (WHERE IS CLASS) RC is trying to perform an explicit confirmation "I think you said you wanted information about a class. Is this correct?", whereas in the first case it's just trying to do an implicit confirmation "Information about a class. ". For any of these to work properly, you need to have the ExplicitConfirm.pm and ImplicitConfirm.pm updated in the NLG. They should map the concept name to the proper confirmation prompt (these Rosetta modeuls work the same as Inform.pm or Request.pm). Maybe the problem is that you're missing those? It wouldn't be a major problem in the implicit confirmation case because the dialog moves on anyway but it would in the explicit confirmation case because RC waits for a user answer (it creates a new request agent and puts it on top of the stack). So the thing is, when you start dealing with speech, you need to have all those grounding models and their associated prompts in the NLG set correctly. It takes a little while but then RC does the whole grounding job for you. I guess we really need some documentation about this (Dan??). Hope this helps. antoine _____ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Wednesday, November 08, 2006 4:51 PM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 179] grounding_manager_configuration setting inconfig file of DM I am not sure which entry to look at. Could this be because I am binding multiple concepts? It is not a problem for me now as I can fix it by taking out grounding_manager_configuration from config. I am just concerned that I will deal with these issues with speech. >From a first glance at the helios log: WHERE CLASS confidence 0.62288 WHERE IS CLASS confidence 0.28883 In the dialog.log I get entries: I think this is for WHERE CLASS: **************************************** [GMX at 16:11:10.083] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.622880 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED:0.00 Action values (dumped below): ACCEPT:2.4576 EXPL_CONF:0.6568 Suggested action: ACCEPT [GMX at 16:11:10.083] Concept /RavenCalendar/the_event.event_type dumped below: CLASS|0.62 [GMX at 16:11:10.083] Grounding model /RavenCalendar/the_event.event_type [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.622880 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED:0.00 Action values (dumped below): ACCEPT:-2.1822 EXPL_CONF:0.6568 IMPL_CONF:1.8856 Suggested action: IMPL_CONF ***************************************** and this for WHERE IS CLASS [GMX at 16:11:13.314] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.000000 updated = false Belief state: INACTIVE:1.00 CONFIDENT:0.00 UNCONFIDENT:0.00 GROUNDED:0.00 Action values (dumped below): ACCEPT:10.0000 EXPL_CONF: - Suggested action: ACCEPT [GMX at 16:11:13.314] Concept /RavenCalendar/the_event.event_type dumped below: [GMX at 16:11:13.315] Grounding model /RavenCalendar/the_event.event_type [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.000000 [GMX at 16:11:37.467] Concept /RavenCalendar/question_type_string dumped below: where|0.29 [GMX at 16:11:37.467] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.288830 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.29 UNCONFIDENT:0.71 GROUNDED:0.00 Action values (dumped below): ACCEPT:-4.2234 EXPL_CONF:5.6675 Suggested action: EXPL_CONF On 11/8/06, Antoine Raux wrote: Hi Svetlana, Have you looked at the confidence value for the input in both cases? This is one of the features that Helios sends to the DM for each input. It is possible that "where class" has a low confidence for some reason (e.g. segmented parse.) and RC rejects it. Normally if your grammar gives a full parse for "where class" that shouldn't happen (at least on typed input) but. antoine _____ From: ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto:ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] On Behalf Of Svetlana Stenchikova Sent: Wednesday, November 08, 2006 2:50 PM To: ravenclaw-developers at cs.cmu.edu Subject: [RavenclawDev 179] grounding_manager_configuration setting inconfig file of DM I have encountered an issue with binding a concept in DM depending on a user utterance. in my DM I have a grammar mapping. "@(...)[Question.where_event]>where," The user can ask a question in multiple ways: where class where is class where is the class what is the location the class .. for all of these entries PHOENIX produces identical parse [Question] ( [where_event] ( WHERE ) ) Features [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) " } ) When a configuration file passed to the DM in -config has an entry: grounding_manager_configuration = turns:default;concepts:default;beliefupdatingmodel:npu if a user types: "where is the class" DM does not bind the concepts, but if the user types "where class" DM binds the concepts. When the grounding_manager_configuration is taken out of DM, the binding succeeds for both utterances. Can this be explained? Do you think there may be a problem with my grammar or something in my DM? The grammar of the Phoenix parser should not matter because in all cases the parses are identical, right? thank you Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061108/a0790bbc/attachment-0001.html From svetastenchikova at gmail.com Wed Nov 8 17:45:27 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Wed, 8 Nov 2006 17:45:27 -0500 Subject: [RavenclawDev 183] Re: grounding_manager_configuration setting inconfig file of DM In-Reply-To: <002401c70381$43be2db0$03bd0280@sp.cs.cmu.edu> References: <31cecd6b0611081350y79279453n1d70cc0b73a9973f@mail.gmail.com> <002401c70381$43be2db0$03bd0280@sp.cs.cmu.edu> Message-ID: <31cecd6b0611081445r5597c16fhb6363204c0aa684c@mail.gmail.com> In the case of WHERE CLASS, the conversation continues correctly, but in WHERE IS CLASS it it goes into default explicit confirmation for CLASS (I just added it - thanks for the hint) I don't think it is the desirable behavior though. These typed (or perfectly recognized) concepts. Why should it need to confirm them? thanks Svetlana On 11/8/06, Antoine Raux < antoine at cs.cmu.edu> wrote: > > What exactly is happening after you type either of those? > > In the second case (WHERE IS CLASS) RC is trying to perform an explicit > confirmation "I think you said you wanted information about a class. Is this > correct?", whereas in the first case it's just trying to do an implicit > confirmation "Information about a class? question>". > > For any of these to work properly, you need to have the ExplicitConfirm.pmand > ImplicitConfirm.pm updated in the NLG. They should map the concept name to > the proper confirmation prompt (these Rosetta modeuls work the same as > Inform.pm or Request.pm). Maybe the problem is that you're missing those? > It wouldn't be a major problem in the implicit confirmation case because the > dialog moves on anyway but it would in the explicit confirmation case > because RC waits for a user answer (it creates a new request agent and puts > it on top of the stack). > > > > So the thing is, when you start dealing with speech, you need to have all > those grounding models and their associated prompts in the NLG set > correctly. It takes a little while but then RC does the whole grounding job > for you? > > I guess we really need some documentation about this (Dan??)? > > > > Hope this helps? > > antoine > > > ------------------------------ > > *From:* Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] > *Sent:* Wednesday, November 08, 2006 4:51 PM > *To:* Antoine Raux > *Cc:* ravenclaw-developers at cs.cmu.edu > *Subject:* Re: [RavenclawDev 179] grounding_manager_configuration setting > inconfig file of DM > > > > I am not sure which entry to look at. > > Could this be because I am binding multiple concepts? > It is not a problem for me now as I can fix it by taking out grounding_manager_configuration > from config. > I am just concerned that I will deal with these issues with speech. > > > >From a first glance at the helios log: > WHERE CLASS > > confidence 0.62288 > > WHERE IS CLASS > > confidence 0.28883 > > > In the dialog.log I get entries: > > I think this is for WHERE CLASS: > **************************************** > > [GMX at 16:11:10.083] Grounding model /RavenCalendar/question_type_string > [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.622880 > updated = true > Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED: > 0.00 > Action values (dumped below): > ACCEPT:2.4576 > EXPL_CONF:0.6568 > > Suggested action: ACCEPT > > [GMX at 16:11:10.083] Concept /RavenCalendar/the_event.event_type dumped > below: > CLASS|0.62 > [GMX at 16:11:10.083] Grounding model /RavenCalendar/the_event.event_type > [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.622880 > updated = true > Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED: > 0.00 > Action values (dumped below): > ACCEPT:-2.1822 > EXPL_CONF:0.6568 > IMPL_CONF:1.8856 > > Suggested action: IMPL_CONF > > > > ***************************************** > > and this for WHERE IS CLASS > > > > [GMX at 16:11:13.314] Grounding model /RavenCalendar/question_type_string > [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.000000 > updated = false > Belief state: INACTIVE:1.00 CONFIDENT:0.00 UNCONFIDENT:0.00 GROUNDED: > 0.00 > Action values (dumped below): > ACCEPT:10.0000 > EXPL_CONF: - > > Suggested action: ACCEPT > > [GMX at 16:11:13.314] Concept /RavenCalendar/the_event.event_type dumped > below: > > [GMX at 16:11:13.315] Grounding model /RavenCalendar/the_event.event_type > [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.000000 > > > [GMX at 16:11:37.467] Concept /RavenCalendar/question_type_string dumped > below: > where|0.29 > [GMX at 16:11:37.467] Grounding model /RavenCalendar/question_type_string > [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.288830 > updated = true > Belief state: INACTIVE:0.00 CONFIDENT:0.29 UNCONFIDENT:0.71 GROUNDED: > 0.00 > Action values (dumped below): > ACCEPT:-4.2234 > EXPL_CONF:5.6675 > > Suggested action: EXPL_CONF > > > > > > > On 11/8/06, *Antoine Raux* wrote: > > Hi Svetlana, > > > > Have you looked at the confidence value for the input in both cases? This > is one of the features that Helios sends to the DM for each input. It is > possible that "where class" has a low confidence for some reason (e.g. > segmented parse?) and RC rejects it. Normally if your grammar gives a full > parse for "where class" that shouldn't happen (at least on typed input) but? > > > > antoine > > > ------------------------------ > > *From:* ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto: > ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] * On Behalf Of *Svetlana > Stenchikova > *Sent:* Wednesday, November 08, 2006 2:50 PM > *To:* ravenclaw-developers at cs.cmu.edu > *Subject:* [RavenclawDev 179] grounding_manager_configuration setting > inconfig file of DM > > > > > I have encountered an issue with binding a concept in DM depending on a > user utterance. > > in my DM I have a grammar mapping. "@(...)[Question.where_event]>where," > > > The user can ask a question in multiple ways: > where class > where is class > where is the class > what is the location the class > .. > > for all of these entries PHOENIX produces identical parse > > [Question] ( [where_event] ( WHERE ) ) > Features > [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) > > " } ) > > When a configuration file passed to the DM in -config has an entry: > grounding_manager_configuration = > turns:default;concepts:default;beliefupdatingmodel:npu > > > if a user types: > "where is the class" DM does not bind the concepts, but if the user > types > "where class" DM binds the concepts. > > When the grounding_manager_configuration is taken out of DM, the binding > succeeds for both utterances. > > Can this be explained? Do you think there may be a problem with my > grammar or something in my DM? > The grammar of the Phoenix parser should not matter because in all cases > the parses are identical, right? > > thank you > > Svetlana > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061108/6b2a222f/attachment.html From antoine at cs.cmu.edu Wed Nov 8 20:53:19 2006 From: antoine at cs.cmu.edu (Antoine Raux) Date: Wed, 8 Nov 2006 20:53:19 -0500 Subject: [RavenclawDev 184] Re: grounding_manager_configuration setting inconfig file of DM In-Reply-To: <31cecd6b0611081445r5597c16fhb6363204c0aa684c@mail.gmail.com> Message-ID: <003401c703a1$d4f54410$03bd0280@sp.cs.cmu.edu> I think the problem is either in the grammar or in Helios. Helios uses features such as "how many top level slots does the parse contain", "how many words are covered by the parse", . and combines them (along with more typical speech recognition features) to produce the confidence score. What seems to be happening here is that the grammar is such that the parse for "WHERE IS CLASS", while producing the right concepts, leads to low confidence. Three ways to solve this: 1) hack Helios to always give perfect confidence for parsed input (that is *not* a long term solution, I wouldn't recommend it), 2) fix your grammar so that it produces parses that are more like what Helios expect, 3) fix the Helios model to match your grammar. The problem with 3) is that the confidence model in Helios is trained on data so you'd need data for your system before being able to retrain Helios. So for now I'd try to go for 2). Could you send the parse for these two utterances and you grammar? antoine _____ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Wednesday, November 08, 2006 5:45 PM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 182] Re: grounding_manager_configuration setting inconfig file of DM In the case of WHERE CLASS, the conversation continues correctly, but in WHERE IS CLASS it it goes into default explicit confirmation for CLASS (I just added it - thanks for the hint) I don't think it is the desirable behavior though. These typed (or perfectly recognized) concepts. Why should it need to confirm them? thanks Svetlana On 11/8/06, Antoine Raux < antoine at cs.cmu.edu> wrote: What exactly is happening after you type either of those? In the second case (WHERE IS CLASS) RC is trying to perform an explicit confirmation "I think you said you wanted information about a class. Is this correct?", whereas in the first case it's just trying to do an implicit confirmation "Information about a class. ". For any of these to work properly, you need to have the ExplicitConfirm.pm and ImplicitConfirm.pm updated in the NLG. They should map the concept name to the proper confirmation prompt (these Rosetta modeuls work the same as Inform.pm or Request.pm). Maybe the problem is that you're missing those? It wouldn't be a major problem in the implicit confirmation case because the dialog moves on anyway but it would in the explicit confirmation case because RC waits for a user answer (it creates a new request agent and puts it on top of the stack). So the thing is, when you start dealing with speech, you need to have all those grounding models and their associated prompts in the NLG set correctly. It takes a little while but then RC does the whole grounding job for you. I guess we really need some documentation about this (Dan??). Hope this helps. antoine _____ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Wednesday, November 08, 2006 4:51 PM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 179] grounding_manager_configuration setting inconfig file of DM I am not sure which entry to look at. Could this be because I am binding multiple concepts? It is not a problem for me now as I can fix it by taking out grounding_manager_configuration from config. I am just concerned that I will deal with these issues with speech. >From a first glance at the helios log: WHERE CLASS confidence 0.62288 WHERE IS CLASS confidence 0.28883 In the dialog.log I get entries: I think this is for WHERE CLASS: **************************************** [GMX at 16:11:10.083] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.622880 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED:0.00 Action values (dumped below): ACCEPT:2.4576 EXPL_CONF:0.6568 Suggested action: ACCEPT [GMX at 16:11:10.083] Concept /RavenCalendar/the_event.event_type dumped below: CLASS|0.62 [GMX at 16:11:10.083] Grounding model /RavenCalendar/the_event.event_type [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.622880 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED:0.00 Action values (dumped below): ACCEPT:-2.1822 EXPL_CONF:0.6568 IMPL_CONF:1.8856 Suggested action: IMPL_CONF ***************************************** and this for WHERE IS CLASS [GMX at 16:11:13.314] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.000000 updated = false Belief state: INACTIVE:1.00 CONFIDENT:0.00 UNCONFIDENT:0.00 GROUNDED:0.00 Action values (dumped below): ACCEPT:10.0000 EXPL_CONF: - Suggested action: ACCEPT [GMX at 16:11:13.314] Concept /RavenCalendar/the_event.event_type dumped below: [GMX at 16:11:13.315] Grounding model /RavenCalendar/the_event.event_type [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.000000 [GMX at 16:11:37.467] Concept /RavenCalendar/question_type_string dumped below: where|0.29 [GMX at 16:11:37.467] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.288830 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.29 UNCONFIDENT:0.71 GROUNDED:0.00 Action values (dumped below): ACCEPT:-4.2234 EXPL_CONF:5.6675 Suggested action: EXPL_CONF On 11/8/06, Antoine Raux wrote: Hi Svetlana, Have you looked at the confidence value for the input in both cases? This is one of the features that Helios sends to the DM for each input. It is possible that "where class" has a low confidence for some reason (e.g. segmented parse.) and RC rejects it. Normally if your grammar gives a full parse for "where class" that shouldn't happen (at least on typed input) but. antoine _____ From: ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto:ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] On Behalf Of Svetlana Stenchikova Sent: Wednesday, November 08, 2006 2:50 PM To: ravenclaw-developers at cs.cmu.edu Subject: [RavenclawDev 179] grounding_manager_configuration setting inconfig file of DM I have encountered an issue with binding a concept in DM depending on a user utterance. in my DM I have a grammar mapping. "@(...)[Question.where_event]>where," The user can ask a question in multiple ways: where class where is class where is the class what is the location the class .. for all of these entries PHOENIX produces identical parse [Question] ( [where_event] ( WHERE ) ) Features [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) " } ) When a configuration file passed to the DM in -config has an entry: grounding_manager_configuration = turns:default;concepts:default;beliefupdatingmodel:npu if a user types: "where is the class" DM does not bind the concepts, but if the user types "where class" DM binds the concepts. When the grounding_manager_configuration is taken out of DM, the binding succeeds for both utterances. Can this be explained? Do you think there may be a problem with my grammar or something in my DM? The grammar of the Phoenix parser should not matter because in all cases the parses are identical, right? thank you Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061108/650f792f/attachment-0001.html From svetastenchikova at gmail.com Wed Nov 8 21:16:40 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Wed, 8 Nov 2006 21:16:40 -0500 Subject: [RavenclawDev 185] Re: grounding_manager_configuration setting inconfig file of DM In-Reply-To: <003401c703a1$d4f54410$03bd0280@sp.cs.cmu.edu> References: <31cecd6b0611081445r5597c16fhb6363204c0aa684c@mail.gmail.com> <003401c703a1$d4f54410$03bd0280@sp.cs.cmu.edu> Message-ID: <31cecd6b0611081816q1f9a7695ydbb7f6ab741d0d76@mail.gmail.com> Thanks Antoine, Helios does not have access to the Phoenix grammar does it? It only gets the string and the output parse, right? Here is my grammar excerpt. (the full grammar file is attached) I am binding Question and EventTypeSpec concepts in separate agents, so phoenix outputs: (Maybe this is a problem?) [Question] ( [where_event] ( WHERE ) ) Features [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) Question and EventTypeSpec are declared in form [Question] ([whattime_event]) ([describe_event]) ([where_event]) ; [where_event] (where) (at which location) (what is the location) (which location) (what location) ; # SS: I used to have this, but it did not matter: # (where *is *the *EVENT) # (*at which location *is *the *EVENT) [EventTypeSpec] ([event_type]) ; [event_type] ([event_type_singular]) ([event_type_plural]) ; On 11/8/06, Antoine Raux wrote: > > I think the problem is either in the grammar or in Helios. Helios uses > features such as "how many top level slots does the parse contain", "how > many words are covered by the parse", ? and combines them (along with more > typical speech recognition features) to produce the confidence score. What > seems to be happening here is that the grammar is such that the parse for > "WHERE IS CLASS", while producing the right concepts, leads to low > confidence. > > Three ways to solve this: 1) hack Helios to always give perfect confidence > for parsed input (that is **not** a long term solution, I wouldn't > recommend it), 2) fix your grammar so that it produces parses that are more > like what Helios expect, 3) fix the Helios model to match your grammar. The > problem with 3) is that the confidence model in Helios is trained on data so > you'd need data for your system before being able to retrain Helios. So for > now I'd try to go for 2). > > > > Could you send the parse for these two utterances and you grammar? > > > > antoine > > > ------------------------------ > > *From:* Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] > *Sent:* Wednesday, November 08, 2006 5:45 PM > *To:* Antoine Raux > *Cc:* ravenclaw-developers at cs.cmu.edu > *Subject:* Re: [RavenclawDev 182] Re: grounding_manager_configuration > setting inconfig file of DM > > > > In the case of WHERE CLASS, the conversation continues correctly, but in > WHERE IS CLASS it it goes into default explicit confirmation for CLASS (I > just added it - thanks for the hint) > > I don't think it is the desirable behavior though. These typed (or > perfectly recognized) concepts. Why should it need to confirm them? > > thanks > Svetlana > > On 11/8/06, *Antoine Raux* < antoine at cs.cmu.edu> wrote: > > What exactly is happening after you type either of those? > > In the second case (WHERE IS CLASS) RC is trying to perform an explicit > confirmation "I think you said you wanted information about a class. Is this > correct?", whereas in the first case it's just trying to do an implicit > confirmation "Information about a class? question>". > > For any of these to work properly, you need to have the ExplicitConfirm.pmand > ImplicitConfirm.pm updated in the NLG. They should map the concept name to > the proper confirmation prompt (these Rosetta modeuls work the same as > Inform.pm or Request.pm). Maybe the problem is that you're missing those? > It wouldn't be a major problem in the implicit confirmation case because the > dialog moves on anyway but it would in the explicit confirmation case > because RC waits for a user answer (it creates a new request agent and puts > it on top of the stack). > > > > So the thing is, when you start dealing with speech, you need to have all > those grounding models and their associated prompts in the NLG set > correctly. It takes a little while but then RC does the whole grounding job > for you? > > I guess we really need some documentation about this (Dan??)? > > > > Hope this helps? > > antoine > > > ------------------------------ > > *From:* Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] > *Sent:* Wednesday, November 08, 2006 4:51 PM > *To:* Antoine Raux > *Cc:* ravenclaw-developers at cs.cmu.edu > *Subject:* Re: [RavenclawDev 179] grounding_manager_configuration setting > inconfig file of DM > > > > I am not sure which entry to look at. > > Could this be because I am binding multiple concepts? > It is not a problem for me now as I can fix it by taking out > grounding_manager_configuration from config. > I am just concerned that I will deal with these issues with speech. > > > >From a first glance at the helios log: > WHERE CLASS > > confidence 0.62288 > > WHERE IS CLASS > > confidence 0.28883 > > > In the dialog.log I get entries: > > I think this is for WHERE CLASS: > **************************************** > > [GMX at 16:11:10.083] Grounding model /RavenCalendar/question_type_string > [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.622880 > updated = true > Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED: > 0.00 > Action values (dumped below): > ACCEPT:2.4576 > EXPL_CONF:0.6568 > > Suggested action: ACCEPT > > [GMX at 16:11:10.083] Concept /RavenCalendar/the_event.event_type dumped > below: > CLASS|0.62 > [GMX at 16:11:10.083] Grounding model /RavenCalendar/the_event.event_type > [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.622880 > updated = true > Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED: > 0.00 > Action values (dumped below): > ACCEPT:-2.1822 > EXPL_CONF:0.6568 > IMPL_CONF:1.8856 > > Suggested action: IMPL_CONF > > > > ***************************************** > > and this for WHERE IS CLASS > > > > [GMX at 16:11:13.314] Grounding model /RavenCalendar/question_type_string > [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.000000 > updated = false > Belief state: INACTIVE:1.00 CONFIDENT:0.00 UNCONFIDENT:0.00 GROUNDED: > 0.00 > Action values (dumped below): > ACCEPT:10.0000 > EXPL_CONF: - > > Suggested action: ACCEPT > > [GMX at 16:11:13.314] Concept /RavenCalendar/the_event.event_type dumped > below: > > [GMX at 16:11:13.315] Grounding model /RavenCalendar/the_event.event_type > [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.000000 > > > [GMX at 16:11:37.467] Concept /RavenCalendar/question_type_string dumped > below: > where|0.29 > [GMX at 16:11:37.467] Grounding model /RavenCalendar/question_type_string > [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.288830 > updated = true > Belief state: INACTIVE:0.00 CONFIDENT:0.29 UNCONFIDENT:0.71 GROUNDED: > 0.00 > Action values (dumped below): > ACCEPT:-4.2234 > EXPL_CONF:5.6675 > > Suggested action: EXPL_CONF > > > > > > On 11/8/06, *Antoine Raux* wrote: > > Hi Svetlana, > > > > Have you looked at the confidence value for the input in both cases? This > is one of the features that Helios sends to the DM for each input. It is > possible that "where class" has a low confidence for some reason (e.g. > segmented parse?) and RC rejects it. Normally if your grammar gives a full > parse for "where class" that shouldn't happen (at least on typed input) but? > > > > antoine > > > ------------------------------ > > *From:* ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto: > ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] *On Behalf Of *Svetlana > Stenchikova > *Sent:* Wednesday, November 08, 2006 2:50 PM > *To:* ravenclaw-developers at cs.cmu.edu > *Subject:* [RavenclawDev 179] grounding_manager_configuration setting > inconfig file of DM > > > > > I have encountered an issue with binding a concept in DM depending on a > user utterance. > > in my DM I have a grammar mapping. "@(...)[Question.where_event]>where," > > > The user can ask a question in multiple ways: > where class > where is class > where is the class > what is the location the class > .. > > for all of these entries PHOENIX produces identical parse > > [Question] ( [where_event] ( WHERE ) ) > Features > [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) > > " } ) > > When a configuration file passed to the DM in -config has an entry: > grounding_manager_configuration = > turns:default;concepts:default;beliefupdatingmodel:npu > > > if a user types: > "where is the class" DM does not bind the concepts, but if the user > types > "where class" DM binds the concepts. > > When the grounding_manager_configuration is taken out of DM, the binding > succeeds for both utterances. > > Can this be explained? Do you think there may be a problem with my > grammar or something in my DM? > The grammar of the Phoenix parser should not matter because in all cases > the parses are identical, right? > > thank you > > Svetlana > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061108/d9beebf3/attachment-0001.html -------------- next part -------------- A non-text attachment was scrubbed... Name: RavenCalendarTask.gra Type: application/octet-stream Size: 21178 bytes Desc: not available Url : http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061108/d9beebf3/RavenCalendarTask-0002.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: RavenCalendarTask.forms Type: application/octet-stream Size: 1384 bytes Desc: not available Url : http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061108/d9beebf3/RavenCalendarTask-0003.obj From antoine at cs.cmu.edu Wed Nov 8 21:39:43 2006 From: antoine at cs.cmu.edu (Antoine Raux) Date: Wed, 8 Nov 2006 21:39:43 -0500 Subject: [RavenclawDev 186] Re: grounding_manager_configuration setting inconfig file of DM In-Reply-To: <31cecd6b0611081816q1f9a7695ydbb7f6ab741d0d76@mail.gmail.com> Message-ID: <004301c703a8$50b53370$03bd0280@sp.cs.cmu.edu> I see. I assume [Question] and [EventTypeSpec] are top-level slots in your grammar right? The "IS" in "WHERE IS CLASS" doesn't get covered by any parse? This looks like a key-phrase-spotting-like approach. While this is perfectly okay in the sense that Phoenix will return the right semantic structure, we try in general to get our grammars to cover as much as possible of the input. The idea is that the fact that we get a full parse or a fragmented parse gives an indication on how confident we should be in the recognizer output (assuming your user speaks natural English, I'd be more confident in an ASR result like "WHERE IS THE CLASS" than in "WHERE CLASS" or "WHERE BLAHBLAH CLASS"). So in your case, you can either extend your [where_event] rules to include an (optional) IS: [where_event] (where *is) (at which location *is) (what is the location *is) (which location *is) (what location *is) ; or (even better) you can create a top-level rule like: [Question] ([where_event] *is [EventType]) ([whattime_event] *is [EventType]) ([describe_event] [EventType]) ; This way you can still bind [where_event] and [EventType] to different concepts (in different agents) but your parse will cover more of your input, under a single top-level slot (which means that you can be pretty confident that your input was correctly recognized). Of course it requires more work to actually cover most of what users might say (all the variants of "WHERE IS THE MEETING", "IN WHICH ROOM IS THE MEETING", "WHERE WILL THE MEETING BE HELD AT".) but the best you can do that, the better your system will behave, both in terms of understanding what the user means and in terms of engaging in confirmations at the right time. Does that clarify things a bit? antoine _____ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Wednesday, November 08, 2006 9:17 PM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 184] Re: grounding_manager_configuration setting inconfig file of DM Thanks Antoine, Helios does not have access to the Phoenix grammar does it? It only gets the string and the output parse, right? Here is my grammar excerpt. (the full grammar file is attached) I am binding Question and EventTypeSpec concepts in separate agents, so phoenix outputs: (Maybe this is a problem?) [Question] ( [where_event] ( WHERE ) ) Features [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) Question and EventTypeSpec are declared in form [Question] ([whattime_event]) ([describe_event]) ([where_event]) ; [where_event] (where) (at which location) (what is the location) (which location) (what location) ; # SS: I used to have this, but it did not matter: # (where *is *the *EVENT) # (*at which location *is *the *EVENT) [EventTypeSpec] ([event_type]) ; [event_type] ([event_type_singular]) ([event_type_plural]) ; On 11/8/06, Antoine Raux wrote: I think the problem is either in the grammar or in Helios. Helios uses features such as "how many top level slots does the parse contain", "how many words are covered by the parse", . and combines them (along with more typical speech recognition features) to produce the confidence score. What seems to be happening here is that the grammar is such that the parse for "WHERE IS CLASS", while producing the right concepts, leads to low confidence. Three ways to solve this: 1) hack Helios to always give perfect confidence for parsed input (that is *not* a long term solution, I wouldn't recommend it), 2) fix your grammar so that it produces parses that are more like what Helios expect, 3) fix the Helios model to match your grammar. The problem with 3) is that the confidence model in Helios is trained on data so you'd need data for your system before being able to retrain Helios. So for now I'd try to go for 2). Could you send the parse for these two utterances and you grammar? antoine _____ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Wednesday, November 08, 2006 5:45 PM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 182] Re: grounding_manager_configuration setting inconfig file of DM In the case of WHERE CLASS, the conversation continues correctly, but in WHERE IS CLASS it it goes into default explicit confirmation for CLASS (I just added it - thanks for the hint) I don't think it is the desirable behavior though. These typed (or perfectly recognized) concepts. Why should it need to confirm them? thanks Svetlana On 11/8/06, Antoine Raux < antoine at cs.cmu.edu > wrote: What exactly is happening after you type either of those? In the second case (WHERE IS CLASS) RC is trying to perform an explicit confirmation "I think you said you wanted information about a class. Is this correct?", whereas in the first case it's just trying to do an implicit confirmation "Information about a class. ". For any of these to work properly, you need to have the ExplicitConfirm.pm and ImplicitConfirm.pm updated in the NLG. They should map the concept name to the proper confirmation prompt (these Rosetta modeuls work the same as Inform.pm or Request.pm). Maybe the problem is that you're missing those? It wouldn't be a major problem in the implicit confirmation case because the dialog moves on anyway but it would in the explicit confirmation case because RC waits for a user answer (it creates a new request agent and puts it on top of the stack). So the thing is, when you start dealing with speech, you need to have all those grounding models and their associated prompts in the NLG set correctly. It takes a little while but then RC does the whole grounding job for you. I guess we really need some documentation about this (Dan??). Hope this helps. antoine _____ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Wednesday, November 08, 2006 4:51 PM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 179] grounding_manager_configuration setting inconfig file of DM I am not sure which entry to look at. Could this be because I am binding multiple concepts? It is not a problem for me now as I can fix it by taking out grounding_manager_configuration from config. I am just concerned that I will deal with these issues with speech. >From a first glance at the helios log: WHERE CLASS confidence 0.62288 WHERE IS CLASS confidence 0.28883 In the dialog.log I get entries: I think this is for WHERE CLASS: **************************************** [GMX at 16:11:10.083] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.622880 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED:0.00 Action values (dumped below): ACCEPT:2.4576 EXPL_CONF:0.6568 Suggested action: ACCEPT [GMX at 16:11:10.083] Concept /RavenCalendar/the_event.event_type dumped below: CLASS|0.62 [GMX at 16:11:10.083] Grounding model /RavenCalendar/the_event.event_type [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.622880 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED:0.00 Action values (dumped below): ACCEPT:-2.1822 EXPL_CONF:0.6568 IMPL_CONF:1.8856 Suggested action: IMPL_CONF ***************************************** and this for WHERE IS CLASS [GMX at 16:11:13.314] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.000000 updated = false Belief state: INACTIVE:1.00 CONFIDENT:0.00 UNCONFIDENT:0.00 GROUNDED:0.00 Action values (dumped below): ACCEPT:10.0000 EXPL_CONF: - Suggested action: ACCEPT [GMX at 16:11:13.314] Concept /RavenCalendar/the_event.event_type dumped below: [GMX at 16:11:13.315] Grounding model /RavenCalendar/the_event.event_type [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.000000 [GMX at 16:11:37.467] Concept /RavenCalendar/question_type_string dumped below: where|0.29 [GMX at 16:11:37.467] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.288830 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.29 UNCONFIDENT:0.71 GROUNDED:0.00 Action values (dumped below): ACCEPT:-4.2234 EXPL_CONF:5.6675 Suggested action: EXPL_CONF On 11/8/06, Antoine Raux wrote: Hi Svetlana, Have you looked at the confidence value for the input in both cases? This is one of the features that Helios sends to the DM for each input. It is possible that "where class" has a low confidence for some reason (e.g. segmented parse.) and RC rejects it. Normally if your grammar gives a full parse for "where class" that shouldn't happen (at least on typed input) but. antoine _____ From: ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto:ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] On Behalf Of Svetlana Stenchikova Sent: Wednesday, November 08, 2006 2:50 PM To: ravenclaw-developers at cs.cmu.edu Subject: [RavenclawDev 179] grounding_manager_configuration setting inconfig file of DM I have encountered an issue with binding a concept in DM depending on a user utterance. in my DM I have a grammar mapping. "@(...)[Question.where_event]>where," The user can ask a question in multiple ways: where class where is class where is the class what is the location the class .. for all of these entries PHOENIX produces identical parse [Question] ( [where_event] ( WHERE ) ) Features [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) " } ) When a configuration file passed to the DM in -config has an entry: grounding_manager_configuration = turns:default;concepts:default;beliefupdatingmodel:npu if a user types: "where is the class" DM does not bind the concepts, but if the user types "where class" DM binds the concepts. When the grounding_manager_configuration is taken out of DM, the binding succeeds for both utterances. Can this be explained? Do you think there may be a problem with my grammar or something in my DM? The grammar of the Phoenix parser should not matter because in all cases the parses are identical, right? thank you Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061108/2c7e8bac/attachment-0001.html From svetastenchikova at gmail.com Thu Nov 9 09:35:41 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Thu, 9 Nov 2006 09:35:41 -0500 Subject: [RavenclawDev 187] Re: grounding_manager_configuration setting inconfig file of DM In-Reply-To: <004301c703a8$50b53370$03bd0280@sp.cs.cmu.edu> References: <31cecd6b0611081816q1f9a7695ydbb7f6ab741d0d76@mail.gmail.com> <004301c703a8$50b53370$03bd0280@sp.cs.cmu.edu> Message-ID: <31cecd6b0611090635q3d078099t9e560a57eb5390d2@mail.gmail.com> Thank you Antoine, this clarifies the things. So, currently Helios assumes that there should be a single parse for an utterance. I can make these changes. Do you think that if Helios did not have this requirement, the recognition would deteriorate? I though that since in RavenClaw you can set the nodes of the dialog tree where a concept is recognized, this limits the number of concepts that are in focus at each point and recognition of a full sentence may not be necessary to disambiguate between hypotheses. I think it would be interesting to investigate if RavenClaw would work with simple grammars that pick out concepts from an utterance. (Assuming Helios is modified to allow this). Or do you think there is a flaw in this idea? thanks, Svetlana On 11/8/06, Antoine Raux wrote: > > I see. I assume [Question] and [EventTypeSpec] are top-level slots in > your grammar right? The "IS" in "WHERE IS CLASS" doesn't get covered by any > parse? This looks like a key-phrase-spotting-like approach. While this is > perfectly okay in the sense that Phoenix will return the right semantic > structure, we try in general to get our grammars to cover as much as > possible of the input. The idea is that the fact that we get a full parse or > a fragmented parse gives an indication on how confident we should be in the > recognizer output (assuming your user speaks natural English, I'd be more > confident in an ASR result like "WHERE IS THE CLASS" than in "WHERE CLASS" > or "WHERE BLAHBLAH CLASS"). So in your case, you can either extend your > [where_event] rules to include an (optional) IS: > > > > [where_event] > (where *is) > (at which location *is) > (what is the location *is) > (which location *is) > (what location *is) > ; > > or (even better) you can create a top-level rule like: > > [Question] > > ([where_event] *is [EventType]) > > ([whattime_event] *is [EventType]) > > ([describe_event] [EventType]) > > ; > > > > This way you can still bind [where_event] and [EventType] to different > concepts (in different agents) but your parse will cover more of your input, > under a single top-level slot (which means that you can be pretty confident > that your input was correctly recognized). Of course it requires more work > to actually cover most of what users might say (all the variants of "WHERE > IS THE MEETING", "IN WHICH ROOM IS THE MEETING", "WHERE WILL THE MEETING BE > HELD AT"?) but the best you can do that, the better your system will behave, > both in terms of understanding what the user means and in terms of engaging > in confirmations at the right time. > > > > Does that clarify things a bit? > > > > antoine > > > ------------------------------ > > *From:* Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] > *Sent:* Wednesday, November 08, 2006 9:17 PM > *To:* Antoine Raux > *Cc:* ravenclaw-developers at cs.cmu.edu > *Subject:* Re: [RavenclawDev 184] Re: grounding_manager_configuration > setting inconfig file of DM > > > > Thanks Antoine, > > Helios does not have access to the Phoenix grammar does it? It only gets > the string and the output parse, right? > Here is my grammar excerpt. (the full grammar file is attached) > > I am binding Question and EventTypeSpec concepts in separate agents, so > phoenix outputs: > (Maybe this is a problem?) > > [Question] ( [where_event] ( WHERE ) ) > Features > [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) > > Question and EventTypeSpec are declared in form > > > [Question] > ([whattime_event]) > ([describe_event]) > ([where_event]) > ; > > [where_event] > (where) > (at which location) > (what is the location) > (which location) > (what location) > ; > > # SS: I used to have this, but it did not matter: > # (where *is *the *EVENT) > # (*at which location *is *the *EVENT) > > > [EventTypeSpec] > ([event_type]) > ; > > [event_type] > ([event_type_singular]) > ([event_type_plural]) > ; > > On 11/8/06, *Antoine Raux* wrote: > > I think the problem is either in the grammar or in Helios. Helios uses > features such as "how many top level slots does the parse contain", "how > many words are covered by the parse", ? and combines them (along with more > typical speech recognition features) to produce the confidence score. What > seems to be happening here is that the grammar is such that the parse for > "WHERE IS CLASS", while producing the right concepts, leads to low > confidence. > > Three ways to solve this: 1) hack Helios to always give perfect confidence > for parsed input (that is **not** a long term solution, I wouldn't > recommend it), 2) fix your grammar so that it produces parses that are more > like what Helios expect, 3) fix the Helios model to match your grammar. The > problem with 3) is that the confidence model in Helios is trained on data so > you'd need data for your system before being able to retrain Helios. So for > now I'd try to go for 2). > > > > Could you send the parse for these two utterances and you grammar? > > > > antoine > > > ------------------------------ > > *From:* Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] > *Sent:* Wednesday, November 08, 2006 5:45 PM > *To:* Antoine Raux > *Cc:* ravenclaw-developers at cs.cmu.edu > *Subject:* Re: [RavenclawDev 182] Re: grounding_manager_configuration > setting inconfig file of DM > > > > In the case of WHERE CLASS, the conversation continues correctly, but in > WHERE IS CLASS it it goes into default explicit confirmation for CLASS (I > just added it - thanks for the hint) > > I don't think it is the desirable behavior though. These typed (or > perfectly recognized) concepts. Why should it need to confirm them? > > thanks > Svetlana > > On 11/8/06, *Antoine Raux* < antoine at cs.cmu.edu> wrote: > > What exactly is happening after you type either of those? > > In the second case (WHERE IS CLASS) RC is trying to perform an explicit > confirmation "I think you said you wanted information about a class. Is this > correct?", whereas in the first case it's just trying to do an implicit > confirmation "Information about a class? question>". > > For any of these to work properly, you need to have the ExplicitConfirm.pmand > ImplicitConfirm.pm updated in the NLG. They should map the concept name to > the proper confirmation prompt (these Rosetta modeuls work the same as > Inform.pm or Request.pm). Maybe the problem is that you're missing those? > It wouldn't be a major problem in the implicit confirmation case because the > dialog moves on anyway but it would in the explicit confirmation case > because RC waits for a user answer (it creates a new request agent and puts > it on top of the stack). > > > > So the thing is, when you start dealing with speech, you need to have all > those grounding models and their associated prompts in the NLG set > correctly. It takes a little while but then RC does the whole grounding job > for you? > > I guess we really need some documentation about this (Dan??)? > > > > Hope this helps? > > antoine > > > ------------------------------ > > *From:* Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] > *Sent:* Wednesday, November 08, 2006 4:51 PM > *To:* Antoine Raux > *Cc:* ravenclaw-developers at cs.cmu.edu > *Subject:* Re: [RavenclawDev 179] grounding_manager_configuration setting > inconfig file of DM > > > > I am not sure which entry to look at. > > Could this be because I am binding multiple concepts? > It is not a problem for me now as I can fix it by taking out > grounding_manager_configuration from config. > I am just concerned that I will deal with these issues with speech. > > > >From a first glance at the helios log: > WHERE CLASS > > confidence 0.62288 > > WHERE IS CLASS > > confidence 0.28883 > > > In the dialog.log I get entries: > > I think this is for WHERE CLASS: > **************************************** > > [GMX at 16:11:10.083] Grounding model /RavenCalendar/question_type_string > [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.622880 > updated = true > Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED: > 0.00 > Action values (dumped below): > ACCEPT:2.4576 > EXPL_CONF:0.6568 > > Suggested action: ACCEPT > > [GMX at 16:11:10.083] Concept /RavenCalendar/the_event.event_type dumped > below: > CLASS|0.62 > [GMX at 16:11:10.083] Grounding model /RavenCalendar/the_event.event_type > [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.622880 > updated = true > Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED: > 0.00 > Action values (dumped below): > ACCEPT:-2.1822 > EXPL_CONF:0.6568 > IMPL_CONF:1.8856 > > Suggested action: IMPL_CONF > > > > ***************************************** > > and this for WHERE IS CLASS > > > > [GMX at 16:11:13.314] Grounding model /RavenCalendar/question_type_string > [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.000000 > updated = false > Belief state: INACTIVE:1.00 CONFIDENT:0.00 UNCONFIDENT:0.00 GROUNDED: > 0.00 > Action values (dumped below): > ACCEPT:10.0000 > EXPL_CONF: - > > Suggested action: ACCEPT > > [GMX at 16:11:13.314] Concept /RavenCalendar/the_event.event_type dumped > below: > > [GMX at 16:11:13.315] Grounding model /RavenCalendar/the_event.event_type > [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.000000 > > > [GMX at 16:11:37.467] Concept /RavenCalendar/question_type_string dumped > below: > where|0.29 > [GMX at 16:11:37.467] Grounding model /RavenCalendar/question_type_string > [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: > Full state: > grounded = false > top_confidence = 0.288830 > updated = true > Belief state: INACTIVE:0.00 CONFIDENT:0.29 UNCONFIDENT:0.71 GROUNDED: > 0.00 > Action values (dumped below): > ACCEPT:-4.2234 > EXPL_CONF:5.6675 > > Suggested action: EXPL_CONF > > > > > On 11/8/06, *Antoine Raux* wrote: > > Hi Svetlana, > > > > Have you looked at the confidence value for the input in both cases? This > is one of the features that Helios sends to the DM for each input. It is > possible that "where class" has a low confidence for some reason (e.g. > segmented parse?) and RC rejects it. Normally if your grammar gives a full > parse for "where class" that shouldn't happen (at least on typed input) but? > > > > antoine > > > ------------------------------ > > *From:* ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto: > ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] *On Behalf Of *Svetlana > Stenchikova > *Sent:* Wednesday, November 08, 2006 2:50 PM > *To:* ravenclaw-developers at cs.cmu.edu > *Subject:* [RavenclawDev 179] grounding_manager_configuration setting > inconfig file of DM > > > > > I have encountered an issue with binding a concept in DM depending on a > user utterance. > > in my DM I have a grammar mapping. "@(...)[Question.where_event]>where," > > > The user can ask a question in multiple ways: > where class > where is class > where is the class > what is the location the class > .. > > for all of these entries PHOENIX produces identical parse > > [Question] ( [where_event] ( WHERE ) ) > Features > [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) > > " } ) > > When a configuration file passed to the DM in -config has an entry: > grounding_manager_configuration = > turns:default;concepts:default;beliefupdatingmodel:npu > > > if a user types: > "where is the class" DM does not bind the concepts, but if the user > types > "where class" DM binds the concepts. > > When the grounding_manager_configuration is taken out of DM, the binding > succeeds for both utterances. > > Can this be explained? Do you think there may be a problem with my > grammar or something in my DM? > The grammar of the Phoenix parser should not matter because in all cases > the parses are identical, right? > > thank you > > Svetlana > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061109/9b0354e0/attachment-0001.html From svetastenchikova at gmail.com Thu Nov 9 09:39:34 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Thu, 9 Nov 2006 09:39:34 -0500 Subject: [RavenclawDev 188] Re: bypassing the parser In-Reply-To: <454E9430.3030801@cs.cmu.edu> References: <31cecd6b0611051543g32f60440p9e40c1652247557@mail.gmail.com> <31cecd6b0611051652o6c16170fub32a9d393664b6cc@mail.gmail.com> <454E9430.3030801@cs.cmu.edu> Message-ID: <31cecd6b0611090639i1b59bbdbsc33970514afb4b6b@mail.gmail.com> Hi Antoine (or Dan), did you, by any chance, remember the syntax for the grammar mapping to get text instead of a parse in a request/expect agent? Thank you Svetlana On 11/5/06, Antoine Raux wrote: > > Hi Svetlana, > > RavenClaw actually has access to the (recognized) text corresponding to > each input, whether it's parsable or not. It's part of the set of > features that Helios passes to the DM. I forgot exactly how to access > these features but I believe it's basically a grammar mapping with a > special prefix that indicates that you're not binding a grammar slot but > a Helios input structure slot. Sorry I don't remember the exact syntax > now... If nobody else (Dan?) replies with more precision, I'll find it > out and email that later in the week. In any case, you might be able to > get what you want (the text of the input) without modifying anything at > all but your task specification (grammar mappings)... > > Otherwise, the second approach you are proposing (basically adding a > second "parser") sounds definitely much better than the first one, in > the sense that it preserves the integrity of the input line (which is > important for many things in the DM like grounding, reference to > previous turns....). > > I hope Dan will come up with a more precise answer soon :) > > antoine > > Svetlana Stenchikova wrote: > > > I just thought of another way to implement this without changing > > RavenClaw: > > > > Add another server ProcessRawData which receives the raw data input > > (in parallel with the parser) > > ProcessRawData outputs the input text in the format of the Parser: > > > > [RawText] ( the typed in text ) > > > > The output is passed to the DialogManager.process_parse message > > > > The the text will bind to all concept that are expected in the > > [RawText] grammar mapping. > > > > Please let me know your thoughts on this. > > > > thank you > > Svetlana > > > > > > On 11/5/06, *Svetlana Stenchikova* > > wrote: > > > > Dan, thank you for your answer. I have another question. > > > > We are interested in the functionality where a free text can be > > entered through TTY (or through free speech recognition). > > > > For example, a user can be asked to describe the event. The > > grammar for this description is not available, so we want to store > > all of the typed ( or recognized) text, and/or a URL of the > > recorded speech into the concept in a dialog manager. > > > > We need to bypass the parser. I would need to make changes to the > > RavenClaw to accommodate this, right? > > > > Currently the user input from the TTY (or ASR) is processed by > > the parser. > > > > The control flow is determined by the .pgm file. This entry > > causes it to call process_parse when an output from the parser is > > available: > > > > > > > > ;; If we have a confidence annotated hypothesis, but > > ;; no parse, call phoenix to parse it > > ;; --- > > RULE: :confhyps & !:parses --> phoenix.phoenixparse > > IN: :confhyps :npow :pow :clip_info > > OUT: :total_numparses :parses :input_source > > LOG_OUT: :parses > > > > > > > > So I plan to add another entry: > > > > ;; If we have a confidence annotated hypothesis, but > > ;; no parse, call DM directly > > ;; --- > > RULE: :confhyps & !:parses --> DM.process_raw > > IN: :confhyps :npow :pow :clip_info > > OUT: > > LOG_OUT: > > > > > > > > Add process_raw() to > > > > Libraries/RavenClaw/DMInterfaces/GalaxyInterfaceFunctions.h& > > implement in GalaxyInterface.cpp > > > > > > process_raw() function will be called whenever an output from the > > TTY (or ASR) is received. So, this output will go to both parser > > and DialogManager.process_raw. > > process_raw() assigns the text to a concept (but only in the case > > when the concept is currently expected): > > For every expected concept as compiled by > > CDMCoreAgent::compileExpectationAgenda(): > > If a concept is a "raw speech" concept, set its value to the raw > > text received in an input frame to process_raw > > I would have to add a type to a concept to indicate that it allows > > "raw text" input. > > > > > > Does this implementation plan make sense? > > > > Could you please make any suggestions? > > > > Jahanzeb must have done something similar because his application > > can record user's input. Do you know how that was implemented? > > > > > > > > Thank you, > > Svetlana > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061109/44938464/attachment.html From antoine at cs.cmu.edu Thu Nov 9 10:19:11 2006 From: antoine at cs.cmu.edu (Antoine Raux) Date: Thu, 9 Nov 2006 10:19:11 -0500 Subject: [RavenclawDev 189] Re: grounding_manager_configuration setting inconfig file of DM In-Reply-To: <31cecd6b0611090635q3d078099t9e560a57eb5390d2@mail.gmail.com> Message-ID: <000601c70412$7a68cf00$3ad80280@LaptopAntoine> Okay, let me reclarify this ;) There is no "hard" assumption about getting a single parse in Helios. What happens is that from the data it has been trained on, Helios *learned* that when you have more than one top-level node, it is more likely that your recognition hypothesis has errors (same thing for when you have words not covered by the parse). So rather than a requirement of Helios, this is an artifact of the training data we used in it. This means that once you get data for your system, you can retrain Helios to estimate confidence *given* your grammar. This answers your last question: the full system, including Helios and RavenClaw could work with a keyword spotting like grammar. Whether this would work better or worse than a full fledge NL grammar needs to be seen empirically (this is quite an interesting question actually). Of course the problem is that you need a system to get data in the first place. One approach to get to that is to start by basically ignoring confidence altogether in RavenClaw. This is done by first setting the rejection threshold to 0 in your dialogue specification (sorry I don't have access to my source code here but I think you need to add something like DEFAULT_REJECTION_THRESHOLD(0) to the CORE_CONFIGURATION section). The second thing you need to do is to use a heuristic-based grounding policy (this is what we do in Let's Go). This requires a little more changes and potentially getting the latest version of RavenClaw (I'm not sure which one you have). I'll get back to you on that later. antoine _____ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Thursday, November 09, 2006 9:36 AM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 184] Re: grounding_manager_configuration setting inconfig file of DM Thank you Antoine, this clarifies the things. So, currently Helios assumes that there should be a single parse for an utterance. I can make these changes. Do you think that if Helios did not have this requirement, the recognition would deteriorate? I though that since in RavenClaw you can set the nodes of the dialog tree where a concept is recognized, this limits the number of concepts that are in focus at each point and recognition of a full sentence may not be necessary to disambiguate between hypotheses. I think it would be interesting to investigate if RavenClaw would work with simple grammars that pick out concepts from an utterance. (Assuming Helios is modified to allow this). Or do you think there is a flaw in this idea? thanks, Svetlana On 11/8/06, Antoine Raux wrote: I see. I assume [Question] and [EventTypeSpec] are top-level slots in your grammar right? The "IS" in "WHERE IS CLASS" doesn't get covered by any parse? This looks like a key-phrase-spotting-like approach. While this is perfectly okay in the sense that Phoenix will return the right semantic structure, we try in general to get our grammars to cover as much as possible of the input. The idea is that the fact that we get a full parse or a fragmented parse gives an indication on how confident we should be in the recognizer output (assuming your user speaks natural English, I'd be more confident in an ASR result like "WHERE IS THE CLASS" than in "WHERE CLASS" or "WHERE BLAHBLAH CLASS"). So in your case, you can either extend your [where_event] rules to include an (optional) IS: [where_event] (where *is) (at which location *is) (what is the location *is) (which location *is) (what location *is) ; or (even better) you can create a top-level rule like: [Question] ([where_event] *is [EventType]) ([whattime_event] *is [EventType]) ([describe_event] [EventType]) ; This way you can still bind [where_event] and [EventType] to different concepts (in different agents) but your parse will cover more of your input, under a single top-level slot (which means that you can be pretty confident that your input was correctly recognized). Of course it requires more work to actually cover most of what users might say (all the variants of "WHERE IS THE MEETING", "IN WHICH ROOM IS THE MEETING", "WHERE WILL THE MEETING BE HELD AT".) but the best you can do that, the better your system will behave, both in terms of understanding what the user means and in terms of engaging in confirmations at the right time. Does that clarify things a bit? antoine _____ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Wednesday, November 08, 2006 9:17 PM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 184] Re: grounding_manager_configuration setting inconfig file of DM Thanks Antoine, Helios does not have access to the Phoenix grammar does it? It only gets the string and the output parse, right? Here is my grammar excerpt. (the full grammar file is attached) I am binding Question and EventTypeSpec concepts in separate agents, so phoenix outputs: (Maybe this is a problem?) [Question] ( [where_event] ( WHERE ) ) Features [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) Question and EventTypeSpec are declared in form [Question] ([whattime_event]) ([describe_event]) ([where_event]) ; [where_event] (where) (at which location) (what is the location) (which location) (what location) ; # SS: I used to have this, but it did not matter: # (where *is *the *EVENT) # (*at which location *is *the *EVENT) [EventTypeSpec] ([event_type]) ; [event_type] ([event_type_singular]) ([event_type_plural]) ; On 11/8/06, Antoine Raux wrote: I think the problem is either in the grammar or in Helios. Helios uses features such as "how many top level slots does the parse contain", "how many words are covered by the parse", . and combines them (along with more typical speech recognition features) to produce the confidence score. What seems to be happening here is that the grammar is such that the parse for "WHERE IS CLASS", while producing the right concepts, leads to low confidence. Three ways to solve this: 1) hack Helios to always give perfect confidence for parsed input (that is *not* a long term solution, I wouldn't recommend it), 2) fix your grammar so that it produces parses that are more like what Helios expect, 3) fix the Helios model to match your grammar. The problem with 3) is that the confidence model in Helios is trained on data so you'd need data for your system before being able to retrain Helios. So for now I'd try to go for 2). Could you send the parse for these two utterances and you grammar? antoine _____ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Wednesday, November 08, 2006 5:45 PM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 182] Re: grounding_manager_configuration setting inconfig file of DM In the case of WHERE CLASS, the conversation continues correctly, but in WHERE IS CLASS it it goes into default explicit confirmation for CLASS (I just added it - thanks for the hint) I don't think it is the desirable behavior though. These typed (or perfectly recognized) concepts. Why should it need to confirm them? thanks Svetlana On 11/8/06, Antoine Raux < antoine at cs.cmu.edu > wrote: What exactly is happening after you type either of those? In the second case (WHERE IS CLASS) RC is trying to perform an explicit confirmation "I think you said you wanted information about a class. Is this correct?", whereas in the first case it's just trying to do an implicit confirmation "Information about a class. ". For any of these to work properly, you need to have the ExplicitConfirm.pm and ImplicitConfirm.pm updated in the NLG. They should map the concept name to the proper confirmation prompt (these Rosetta modeuls work the same as Inform.pm or Request.pm). Maybe the problem is that you're missing those? It wouldn't be a major problem in the implicit confirmation case because the dialog moves on anyway but it would in the explicit confirmation case because RC waits for a user answer (it creates a new request agent and puts it on top of the stack). So the thing is, when you start dealing with speech, you need to have all those grounding models and their associated prompts in the NLG set correctly. It takes a little while but then RC does the whole grounding job for you. I guess we really need some documentation about this (Dan??). Hope this helps. antoine _____ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Wednesday, November 08, 2006 4:51 PM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 179] grounding_manager_configuration setting inconfig file of DM I am not sure which entry to look at. Could this be because I am binding multiple concepts? It is not a problem for me now as I can fix it by taking out grounding_manager_configuration from config. I am just concerned that I will deal with these issues with speech. >From a first glance at the helios log: WHERE CLASS confidence 0.62288 WHERE IS CLASS confidence 0.28883 In the dialog.log I get entries: I think this is for WHERE CLASS: **************************************** [GMX at 16:11:10.083] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.622880 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED:0.00 Action values (dumped below): ACCEPT:2.4576 EXPL_CONF:0.6568 Suggested action: ACCEPT [GMX at 16:11:10.083] Concept /RavenCalendar/the_event.event_type dumped below: CLASS|0.62 [GMX at 16:11:10.083] Grounding model /RavenCalendar/the_event.event_type [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.622880 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED:0.00 Action values (dumped below): ACCEPT:-2.1822 EXPL_CONF:0.6568 IMPL_CONF:1.8856 Suggested action: IMPL_CONF ***************************************** and this for WHERE IS CLASS [GMX at 16:11:13.314] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.000000 updated = false Belief state: INACTIVE:1.00 CONFIDENT:0.00 UNCONFIDENT:0.00 GROUNDED:0.00 Action values (dumped below): ACCEPT:10.0000 EXPL_CONF: - Suggested action: ACCEPT [GMX at 16:11:13.314] Concept /RavenCalendar/the_event.event_type dumped below: [GMX at 16:11:13.315] Grounding model /RavenCalendar/the_event.event_type [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.000000 [GMX at 16:11:37.467] Concept /RavenCalendar/question_type_string dumped below: where|0.29 [GMX at 16:11:37.467] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.288830 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.29 UNCONFIDENT:0.71 GROUNDED:0.00 Action values (dumped below): ACCEPT:-4.2234 EXPL_CONF:5.6675 Suggested action: EXPL_CONF On 11/8/06, Antoine Raux wrote: Hi Svetlana, Have you looked at the confidence value for the input in both cases? This is one of the features that Helios sends to the DM for each input. It is possible that "where class" has a low confidence for some reason (e.g. segmented parse.) and RC rejects it. Normally if your grammar gives a full parse for "where class" that shouldn't happen (at least on typed input) but. antoine _____ From: ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto:ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] On Behalf Of Svetlana Stenchikova Sent: Wednesday, November 08, 2006 2:50 PM To: ravenclaw-developers at cs.cmu.edu Subject: [RavenclawDev 179] grounding_manager_configuration setting inconfig file of DM I have encountered an issue with binding a concept in DM depending on a user utterance. in my DM I have a grammar mapping. "@(...)[Question.where_event]>where," The user can ask a question in multiple ways: where class where is class where is the class what is the location the class .. for all of these entries PHOENIX produces identical parse [Question] ( [where_event] ( WHERE ) ) Features [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) " } ) When a configuration file passed to the DM in -config has an entry: grounding_manager_configuration = turns:default;concepts:default;beliefupdatingmodel:npu if a user types: "where is the class" DM does not bind the concepts, but if the user types "where class" DM binds the concepts. When the grounding_manager_configuration is taken out of DM, the binding succeeds for both utterances. Can this be explained? Do you think there may be a problem with my grammar or something in my DM? The grammar of the Phoenix parser should not matter because in all cases the parses are identical, right? thank you Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061109/c168e890/attachment-0001.html From svetastenchikova at gmail.com Fri Nov 10 16:43:28 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Fri, 10 Nov 2006 16:43:28 -0500 Subject: [RavenclawDev 190] issue with phoenix grammar Message-ID: <31cecd6b0611101343tf366b82oa1386cdd4e573337@mail.gmail.com> Hi, I am encountering a strange issue with phoenix grammar. I am adding new locations: [LocationSpec] (at [location]) (in [location]) ([location]) ; [location] (LOCS) LOCS (sac) (student activities) (humanities) (newell simon) (newell simon hall) (nsh) (wean) (wean hall) ; Some of the locations are old - from roomline , the 3 new locations in the example above are "sac", "humanities", and "student activities". The only one that gets parsed is "sac", no matter the order that they appear. I checked that "SAC" occurs in all of the same files as "HUMANITIES". I have removed all pof the files that get generated before compiling. I tried another way to add it in grammar: [LocationSpec] (at [location]) (in [location]) ([location]) ; [location] (sac) (student activities) (humanities) (newell simon) (newell simon hall) (nsh) (wean) (wean hall) ; ********************* [TestLocationSpec] (at LOCS) (in LOCS) (LOCS) LOCS (sac) (hum) (student activities) (newell simon) (newell simon hall) (nsh) (wean) (wean hall) ; Actually out of your entries "NSH" does not get parsed either. Does anyone have any idea why this happens? Thank you Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061110/b31aaa17/attachment.html From svetastenchikova at gmail.com Sat Nov 11 00:34:01 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Sat, 11 Nov 2006 00:34:01 -0500 Subject: [RavenclawDev 191] issue with phoenix grammar Message-ID: <31cecd6b0611102134h258cd1bau2929f43501734c49@mail.gmail.com> Hi, I found what caused my problem. Earlier I was playing with the grammar and added an incorrect entry which was messing it up: [LocationFreeSpeech] (location is *) ; thanks Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061111/e1132041/attachment.html From svetastenchikova at gmail.com Sat Nov 11 23:27:10 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Sat, 11 Nov 2006 23:27:10 -0500 Subject: [RavenclawDev 192] Re: problems with resetting an agent In-Reply-To: <31cecd6b0610272205j6b261775v4b5d816d4a3876aa@mail.gmail.com> References: <31cecd6b0610272205j6b261775v4b5d816d4a3876aa@mail.gmail.com> Message-ID: <31cecd6b0611112027p73477d0dg8fae8fd27f72f549@mail.gmail.com> Hi, this is regarding the same issue. Could someone please elaborate on how the looping (as in main loop AskAnythingElse) is achieved? I assumed that RESET_AGENT is the macro that does it, but I am failing to achieve this so far. Please see more description on what I did in the previous message. I am also tried REOPEN_AGENT and reopening all children of the agent. Also tried reopening concepts (although I think that concepts should not have anything to do with looping, right?) Thank you, I appreciate all of your help. Svetlana On 10/28/06, Svetlana Stenchikova < svetastenchikova at gmail.com> wrote: > > Hi, > > I am trying to createan agent (CIdSingleFilter defined below) which > identifies a single event in a database. > It asks user to specify the type, date, time criterion until only one > event matches. > > eventlistresult is an array concept which is declared in an agency above > and gets filled in by the GetEvents > > NoMatch, RequestClarify, AskWhich are the inform agents which get > activated depending on the content of eventlistresult > > In order to simulate a loop I call ResetThisAgent, which calls RESET_AGENT > on IdSingleFilter. > > If SIZE(eventlistresult)== 1, the agency successfully completes. > > I would expect that if SIZE(eventlistresult)!= 1, then ResetThisAgent, > would place all of the subagents of IdSingleFilter back on stack and > continue the loop.Instead I get an exception: > > > *************************************************************************** > [COR at 00:52:53.563] Executing dialog agent > /RavenCalendar/Task/TaskRemoveEvents/IdSingleFilter ... > [WAR at 00:52:53.566] Failure information for Agency > /RavenCalendar/Task/TaskRemoveEvents/IdSingleFilter > Agent /RavenCalendar/Task/TaskRemoveEvents/IdSingleFilter/NoMatch has not > completed, does not have preconditions satisfied, and is not blocked. > Agent /RavenCalendar/Task/TaskRemoveEvents/IdSingleFilter/RequestClarify > has completed, does have preconditions satisfied, and is not blocked. > Agent /RavenCalendar/Task/TaskRemoveEvents/IdSingleFilter/AskWhich has not > completed, does not have preconditions satisfied, and is not blocked. > Agent /RavenCalendar/Task/TaskRemoveEvents/IdSingleFilter/QueryFilter has > completed, does have preconditions satisfied, and is not blocked. > Agent /RavenCalendar/Task/TaskRemoveEvents/IdSingleFilter/DBGetEvents has > completed, does have preconditions satisfied, and is not blocked. > Agent /RavenCalendar/Task/TaskRemoveEvents/IdSingleFilter/ResetThisAgent > has completed, does have preconditions satisfied, and is not blocked. > documents\work\calendar\ravencalendar\libraries\ravenclaw\dmcore\agents\dialogagents\basicagents\dialogagency.cpp, > line 183>. > [FATAL_ERROR at 00:52:53.566] Agency > /RavenCalendar/Task/TaskRemoveEvents/IdSingleFilter failed to complete. > documents\work\calendar\ravencalendar\libraries\ravenclaw\dmcore\agents\dialogagents\basicagents\dialogagency.cpp, > line 149>. > > *************************************************************************** > > > > Can you tell if something is wrong with my logic? > > thank you > > > > //----------------------------------------------------------------------------- > // /RavenCalendar/Task/IdSingleFilter > > //----------------------------------------------------------------------------- > DEFINE_AGENCY( CIdSingleFilter, > IS_MAIN_TOPIC() > > > // BOOL_SYSTEM_CONCEPT(backend_error)) > DEFINE_SUBAGENTS( > SUBAGENT(NoMatch, CNoMatch, "") > SUBAGENT(RequestClarify, CRequestClarify, "") > SUBAGENT(AskWhich, CAskWhich, "") //undefined num > SUBAGENT(QueryFilter, CQueryFilter, "") //agency getting concepts > SUBAGENT(DBGetEvents, CDBGetEvents, "") // get the list > SUBAGENT(ResetThisAgent, CResetThisAgent, "") > ) > > SUCCEEDS_WHEN(AVAILABLE(eventlistresult) && SIZE(eventlistresult)==1) > > ) > > > > //--------------------------------------------------------------------------- > // > //-------------------------------------------------------------------- > DEFINE_EXECUTE_AGENT( CResetThisAgent, > RESET_AGENT(..) > ) > > > also tried > > //--------------------------------------------------------------------------- > // > //-------------------------------------------------------------------- > DEFINE_EXECUTE_AGENT( CResetThisAgent, > > RESET_AGENT(/RavenCalendar/Task/TaskRemoveEvents/IdSingleFilter) > ) > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061111/7fe5e5a3/attachment.html From svetastenchikova at gmail.com Mon Nov 20 11:10:08 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Mon, 20 Nov 2006 11:10:08 -0500 Subject: [RavenclawDev 193] testing dialog system Message-ID: <31cecd6b0611200810i3e343e04y15015b78e50f5b79@mail.gmail.com> Hi, how do you usually test your dialog systems? Do you do any batch testing? I heard that it may be possible to do a "replay" from the hub log. Does anyone have information about it? thank you, Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061120/a3eaf871/attachment.html From antoine at cs.cmu.edu Mon Nov 20 11:39:31 2006 From: antoine at cs.cmu.edu (Antoine Raux) Date: Mon, 20 Nov 2006 11:39:31 -0500 Subject: [RavenclawDev 194] Re: testing dialog system In-Reply-To: <31cecd6b0611200810i3e343e04y15015b78e50f5b79@mail.gmail.com> Message-ID: <00ae01c70cc2$74372bf0$03bd0280@sp.cs.cmu.edu> Hi Svetlana, It depends what you want to test. If it's the DM or the whole system, at this point we don't have a batch testing set up. What I do is first (during development/debugging phase) use the TTY interface to explore as many dialogue paths as possible (or specifically the one that crashed your system). Then, on major updates, we have a stress test on the telephone system: we get everyone from the Let's Go team to call as much as possible during one hour. That invariably uncovers more bugs. Then of course, we still have bugs after that, which we only catch by monitoring the live system (that's one advantage of Let's Go, we get enough calls per day that we are usually able to catch a fatal bug in a couple of days). As you see, it's not very automated. There have been talks of implementing some kind of regression testing (i.e. batch from previous hub logs or something like that) but at this point it's still at the stage of future project. Now, if you want to test specific components like your ASR or parser, we have a way to run the input side of the system in batch. That is, we resend all the utterances from a corpus of past dialogues, recognize them, parse them and annotate them with Helios. That's where it ends though (because of course if we change the behavior of the system then the rest of the logs becomes irrelevant. for that to work, we'd need a simulated user kind of setup, but that's another story). We use that to compute our ASR when we update language or acoustic models, or retrain our confidence annotator on new data. Hope this helps. antoine _____ From: ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto:ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] On Behalf Of Svetlana Stenchikova Sent: Monday, November 20, 2006 11:10 AM To: ravenclaw-developers at cs.cmu.edu Subject: [RavenclawDev 193] testing dialog system Hi, how do you usually test your dialog systems? Do you do any batch testing? I heard that it may be possible to do a "replay" from the hub log. Does anyone have information about it? thank you, Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061120/fc0fc2a7/attachment-0001.html From svetastenchikova at gmail.com Thu Nov 23 12:50:37 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Thu, 23 Nov 2006 12:50:37 -0500 Subject: [RavenclawDev 195] Re: testing dialog system In-Reply-To: <00ae01c70cc2$74372bf0$03bd0280@sp.cs.cmu.edu> References: <31cecd6b0611200810i3e343e04y15015b78e50f5b79@mail.gmail.com> <00ae01c70cc2$74372bf0$03bd0280@sp.cs.cmu.edu> Message-ID: <31cecd6b0611230950l69c004d4nf2ac5b8e20ec7d7a@mail.gmail.com> Thank you Antoine, It seems like testing ASR in a batch is really essential. We are now just starting to add ASR to our system and we will definitely do something similar for the ASR testing. We are also making TTY capable of running in a batch mode in order to test parser, DM, backend, and NLG, so that when something changes we could quickly verify that existing functionality is not broken. Thanks, Svetlana On 11/20/06, Antoine Raux wrote: > > Hi Svetlana, > > > > It depends what you want to test. If it's the DM or the whole system, at > this point we don't have a batch testing set up. > > What I do is first (during development/debugging phase) use the TTY > interface to explore as many dialogue paths as possible (or specifically the > one that crashed your system). Then, on major updates, we have a stress test > on the telephone system: we get everyone from the Let's Go team to call as > much as possible during one hour. That invariably uncovers more bugs? Then > of course, we still have bugs after that, which we only catch by monitoring > the live system (that's one advantage of Let's Go, we get enough calls per > day that we are usually able to catch a fatal bug in a couple of days)? As > you see, it's not very automated? There have been talks of implementing some > kind of regression testing (i.e. batch from previous hub logs or something > like that) but at this point it's still at the stage of future project? > > > > Now, if you want to test specific components like your ASR or parser, we > have a way to run the input side of the system in batch. That is, we resend > all the utterances from a corpus of past dialogues, recognize them, parse > them and annotate them with Helios. That's where it ends though (because of > course if we change the behavior of the system then the rest of the logs > becomes irrelevant? for that to work, we'd need a simulated user kind of > setup, but that's another story). We use that to compute our ASR when we > update language or acoustic models, or retrain our confidence annotator on > new data. > > > > Hope this helps? > > > > antoine > > > ------------------------------ > > *From:* ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto: > ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] *On Behalf Of *Svetlana > Stenchikova > *Sent:* Monday, November 20, 2006 11:10 AM > *To:* ravenclaw-developers at cs.cmu.edu > *Subject:* [RavenclawDev 193] testing dialog system > > > > Hi, > > how do you usually test your dialog systems? Do you do any batch testing? > I heard that it may be possible to do a "replay" from the hub log. Does > anyone have information about it? > > thank you, > Svetlana > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061123/76eec707/attachment.html From dbohus at cs.cmu.edu Sat Nov 25 15:21:48 2006 From: dbohus at cs.cmu.edu (Dan Bohus) Date: Sat, 25 Nov 2006 15:21:48 -0500 Subject: [RavenclawDev 196] Re: grounding_manager_configuration settinginconfig file of DM In-Reply-To: <000601c70412$7a68cf00$3ad80280@LaptopAntoine> Message-ID: <1546B8BE23C96940B5B4929DB7CC2343C940E7@e2k3.srv.cs.cmu.edu> Hi Antoine, Svetlana, Antoine, thanks a lot for answering all these questions that Svetlana had about RavenClaw. Sorry I haven't been able to be more responsive, I've been really busy preparing job talks, etc. In any case, Svetlana, if you have more questions or need more help regarding helios and the confidence annotation model it uses, please let us know. I will try to be better at responding :) Hope things are well, Cheers, Dan, ________________________________ From: ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto:ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] On Behalf Of Antoine Raux Sent: Thursday, November 09, 2006 10:19 AM To: ravenclaw-developers at cs.cmu.edu Subject: [RavenclawDev 189] Re: grounding_manager_configuration settinginconfig file of DM Okay, let me reclarify this ;) There is no "hard" assumption about getting a single parse in Helios. What happens is that from the data it has been trained on, Helios *learned* that when you have more than one top-level node, it is more likely that your recognition hypothesis has errors (same thing for when you have words not covered by the parse). So rather than a requirement of Helios, this is an artifact of the training data we used in it. This means that once you get data for your system, you can retrain Helios to estimate confidence *given* your grammar. This answers your last question: the full system, including Helios and RavenClaw could work with a keyword spotting like grammar. Whether this would work better or worse than a full fledge NL grammar needs to be seen empirically (this is quite an interesting question actually)... Of course the problem is that you need a system to get data in the first place... One approach to get to that is to start by basically ignoring confidence altogether in RavenClaw. This is done by first setting the rejection threshold to 0 in your dialogue specification (sorry I don't have access to my source code here but I think you need to add something like DEFAULT_REJECTION_THRESHOLD(0) to the CORE_CONFIGURATION section). The second thing you need to do is to use a heuristic-based grounding policy (this is what we do in Let's Go). This requires a little more changes and potentially getting the latest version of RavenClaw (I'm not sure which one you have). I'll get back to you on that later. antoine ________________________________ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Thursday, November 09, 2006 9:36 AM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 184] Re: grounding_manager_configuration setting inconfig file of DM Thank you Antoine, this clarifies the things. So, currently Helios assumes that there should be a single parse for an utterance. I can make these changes. Do you think that if Helios did not have this requirement, the recognition would deteriorate? I though that since in RavenClaw you can set the nodes of the dialog tree where a concept is recognized, this limits the number of concepts that are in focus at each point and recognition of a full sentence may not be necessary to disambiguate between hypotheses. I think it would be interesting to investigate if RavenClaw would work with simple grammars that pick out concepts from an utterance. (Assuming Helios is modified to allow this). Or do you think there is a flaw in this idea? thanks, Svetlana On 11/8/06, Antoine Raux wrote: I see. I assume [Question] and [EventTypeSpec] are top-level slots in your grammar right? The "IS" in "WHERE IS CLASS" doesn't get covered by any parse? This looks like a key-phrase-spotting-like approach. While this is perfectly okay in the sense that Phoenix will return the right semantic structure, we try in general to get our grammars to cover as much as possible of the input. The idea is that the fact that we get a full parse or a fragmented parse gives an indication on how confident we should be in the recognizer output (assuming your user speaks natural English, I'd be more confident in an ASR result like "WHERE IS THE CLASS" than in "WHERE CLASS" or "WHERE BLAHBLAH CLASS"). So in your case, you can either extend your [where_event] rules to include an (optional) IS: [where_event] (where *is) (at which location *is) (what is the location *is) (which location *is) (what location *is) ; or (even better) you can create a top-level rule like: [Question] ([where_event] *is [EventType]) ([whattime_event] *is [EventType]) ([describe_event] [EventType]) ; This way you can still bind [where_event] and [EventType] to different concepts (in different agents) but your parse will cover more of your input, under a single top-level slot (which means that you can be pretty confident that your input was correctly recognized). Of course it requires more work to actually cover most of what users might say (all the variants of "WHERE IS THE MEETING", "IN WHICH ROOM IS THE MEETING", "WHERE WILL THE MEETING BE HELD AT"...) but the best you can do that, the better your system will behave, both in terms of understanding what the user means and in terms of engaging in confirmations at the right time. Does that clarify things a bit? antoine ________________________________ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Wednesday, November 08, 2006 9:17 PM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 184] Re: grounding_manager_configuration setting inconfig file of DM Thanks Antoine, Helios does not have access to the Phoenix grammar does it? It only gets the string and the output parse, right? Here is my grammar excerpt. (the full grammar file is attached) I am binding Question and EventTypeSpec concepts in separate agents, so phoenix outputs: (Maybe this is a problem?) [Question] ( [where_event] ( WHERE ) ) Features [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) Question and EventTypeSpec are declared in form [Question] ([whattime_event]) ([describe_event]) ([where_event]) ; [where_event] (where) (at which location) (what is the location) (which location) (what location) ; # SS: I used to have this, but it did not matter: # (where *is *the *EVENT) # (*at which location *is *the *EVENT) [EventTypeSpec] ([event_type]) ; [event_type] ([event_type_singular]) ([event_type_plural]) ; On 11/8/06, Antoine Raux wrote: I think the problem is either in the grammar or in Helios. Helios uses features such as "how many top level slots does the parse contain", "how many words are covered by the parse", ... and combines them (along with more typical speech recognition features) to produce the confidence score. What seems to be happening here is that the grammar is such that the parse for "WHERE IS CLASS", while producing the right concepts, leads to low confidence. Three ways to solve this: 1) hack Helios to always give perfect confidence for parsed input (that is *not* a long term solution, I wouldn't recommend it), 2) fix your grammar so that it produces parses that are more like what Helios expect, 3) fix the Helios model to match your grammar. The problem with 3) is that the confidence model in Helios is trained on data so you'd need data for your system before being able to retrain Helios. So for now I'd try to go for 2). Could you send the parse for these two utterances and you grammar? antoine ________________________________ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Wednesday, November 08, 2006 5:45 PM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 182] Re: grounding_manager_configuration setting inconfig file of DM In the case of WHERE CLASS, the conversation continues correctly, but in WHERE IS CLASS it it goes into default explicit confirmation for CLASS (I just added it - thanks for the hint) I don't think it is the desirable behavior though. These typed (or perfectly recognized) concepts. Why should it need to confirm them? thanks Svetlana On 11/8/06, Antoine Raux < antoine at cs.cmu.edu > wrote: What exactly is happening after you type either of those? In the second case (WHERE IS CLASS) RC is trying to perform an explicit confirmation "I think you said you wanted information about a class. Is this correct?", whereas in the first case it's just trying to do an implicit confirmation "Information about a class... ". For any of these to work properly, you need to have the ExplicitConfirm.pm and ImplicitConfirm.pm updated in the NLG. They should map the concept name to the proper confirmation prompt (these Rosetta modeuls work the same as Inform.pm or Request.pm). Maybe the problem is that you're missing those? It wouldn't be a major problem in the implicit confirmation case because the dialog moves on anyway but it would in the explicit confirmation case because RC waits for a user answer (it creates a new request agent and puts it on top of the stack). So the thing is, when you start dealing with speech, you need to have all those grounding models and their associated prompts in the NLG set correctly. It takes a little while but then RC does the whole grounding job for you... I guess we really need some documentation about this (Dan??)... Hope this helps... antoine ________________________________ From: Svetlana Stenchikova [mailto:svetastenchikova at gmail.com] Sent: Wednesday, November 08, 2006 4:51 PM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: Re: [RavenclawDev 179] grounding_manager_configuration setting inconfig file of DM I am not sure which entry to look at. Could this be because I am binding multiple concepts? It is not a problem for me now as I can fix it by taking out grounding_manager_configuration from config. I am just concerned that I will deal with these issues with speech. >From a first glance at the helios log: WHERE CLASS confidence 0.62288 WHERE IS CLASS confidence 0.28883 In the dialog.log I get entries: I think this is for WHERE CLASS: **************************************** [GMX at 16:11:10.083] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.622880 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED:0.00 Action values (dumped below): ACCEPT:2.4576 EXPL_CONF:0.6568 Suggested action: ACCEPT [GMX at 16:11:10.083] Concept /RavenCalendar/the_event.event_type dumped below: CLASS|0.62 [GMX at 16:11:10.083] Grounding model /RavenCalendar/the_event.event_type [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.622880 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.62 UNCONFIDENT:0.38 GROUNDED:0.00 Action values (dumped below): ACCEPT:-2.1822 EXPL_CONF:0.6568 IMPL_CONF:1.8856 Suggested action: IMPL_CONF ***************************************** and this for WHERE IS CLASS [GMX at 16:11:13.314] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.000000 updated = false Belief state: INACTIVE:1.00 CONFIDENT:0.00 UNCONFIDENT:0.00 GROUNDED:0.00 Action values (dumped below): ACCEPT:10.0000 EXPL_CONF: - Suggested action: ACCEPT [GMX at 16:11:13.314] Concept /RavenCalendar/the_event.event_type dumped below: [GMX at 16:11:13.315] Grounding model /RavenCalendar/the_event.event_type [TYPE=concept_default;POLICY=expl_impl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.000000 [GMX at 16:11:37.467] Concept /RavenCalendar/question_type_string dumped below: where|0.29 [GMX at 16:11:37.467] Grounding model /RavenCalendar/question_type_string [TYPE=concept_default;POLICY=expl;EM=greedy;EP=0.20]: Full state: grounded = false top_confidence = 0.288830 updated = true Belief state: INACTIVE:0.00 CONFIDENT:0.29 UNCONFIDENT:0.71 GROUNDED:0.00 Action values (dumped below): ACCEPT:-4.2234 EXPL_CONF:5.6675 Suggested action: EXPL_CONF On 11/8/06, Antoine Raux wrote: Hi Svetlana, Have you looked at the confidence value for the input in both cases? This is one of the features that Helios sends to the DM for each input. It is possible that "where class" has a low confidence for some reason (e.g. segmented parse...) and RC rejects it. Normally if your grammar gives a full parse for "where class" that shouldn't happen (at least on typed input) but... antoine ________________________________ From: ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto:ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] On Behalf Of Svetlana Stenchikova Sent: Wednesday, November 08, 2006 2:50 PM To: ravenclaw-developers at cs.cmu.edu Subject: [RavenclawDev 179] grounding_manager_configuration setting inconfig file of DM I have encountered an issue with binding a concept in DM depending on a user utterance. in my DM I have a grammar mapping. "@(...)[Question.where_event]>where," The user can ask a question in multiple ways: where class where is class where is the class what is the location the class .. for all of these entries PHOENIX produces identical parse [Question] ( [where_event] ( WHERE ) ) Features [EventTypeSpec] ( [event_type] ( [event_type_singular] ( MEETING ) ) ) " } ) When a configuration file passed to the DM in -config has an entry: grounding_manager_configuration = turns:default;concepts:default;beliefupdatingmodel:npu if a user types: "where is the class" DM does not bind the concepts, but if the user types "where class" DM binds the concepts. When the grounding_manager_configuration is taken out of DM, the binding succeeds for both utterances. Can this be explained? Do you think there may be a problem with my grammar or something in my DM? The grammar of the Phoenix parser should not matter because in all cases the parses are identical, right? thank you Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061125/759dbfed/attachment-0001.html From dbohus at cs.cmu.edu Sat Nov 25 15:25:54 2006 From: dbohus at cs.cmu.edu (Dan Bohus) Date: Sat, 25 Nov 2006 15:25:54 -0500 Subject: [RavenclawDev 197] Re: bypassing the parser In-Reply-To: <31cecd6b0611090639i1b59bbdbsc33970514afb4b6b@mail.gmail.com> Message-ID: <1546B8BE23C96940B5B4929DB7CC2343C940E8@e2k3.srv.cs.cmu.edu> Hi Svetlana, RavenClaw can bind on any of the attributes in the input. For instance, if you look at the "new input arrived" line in a dialog log, below it you will have a dump of all the attributes of the input and their values. You will notice that there are things there which represent the various parse fragments such as [Yes] or [Query.location] (which you normally bind on the dialog task). At the same time, there are a number of other attributes starting with h3_ ... these are the various features that helios3 in fact uses in the confidence annotation process. They are passed all the way to the DM. Among them, h3_hypothesis contains the actual text of the recognition and h3_parse_str contains the full parse of the current decoded result. You can therefore bind on [h3_hypothesis] to get the text of the recognition. Hope this helps, but let me know if you have other questions. Dan ________________________________ From: ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto:ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] On Behalf Of Svetlana Stenchikova Sent: Thursday, November 09, 2006 9:40 AM To: Antoine Raux Cc: ravenclaw-developers at cs.cmu.edu Subject: [RavenclawDev 188] Re: bypassing the parser Hi Antoine (or Dan), did you, by any chance, remember the syntax for the grammar mapping to get text instead of a parse in a request/expect agent? Thank you Svetlana On 11/5/06, Antoine Raux wrote: Hi Svetlana, RavenClaw actually has access to the (recognized) text corresponding to each input, whether it's parsable or not. It's part of the set of features that Helios passes to the DM. I forgot exactly how to access these features but I believe it's basically a grammar mapping with a special prefix that indicates that you're not binding a grammar slot but a Helios input structure slot. Sorry I don't remember the exact syntax now... If nobody else (Dan?) replies with more precision, I'll find it out and email that later in the week. In any case, you might be able to get what you want (the text of the input) without modifying anything at all but your task specification (grammar mappings)... Otherwise, the second approach you are proposing (basically adding a second "parser") sounds definitely much better than the first one, in the sense that it preserves the integrity of the input line (which is important for many things in the DM like grounding, reference to previous turns....). I hope Dan will come up with a more precise answer soon :) antoine Svetlana Stenchikova wrote: > I just thought of another way to implement this without changing > RavenClaw: > > Add another server ProcessRawData which receives the raw data input > (in parallel with the parser) > ProcessRawData outputs the input text in the format of the Parser: > > [RawText] ( the typed in text ) > > The output is passed to the DialogManager.process_parse message > > The the text will bind to all concept that are expected in the > [RawText] grammar mapping. > > Please let me know your thoughts on this. > > thank you > Svetlana > > > On 11/5/06, *Svetlana Stenchikova* > wrote: > > Dan, thank you for your answer. I have another question. > > We are interested in the functionality where a free text can be > entered through TTY (or through free speech recognition). > > For example, a user can be asked to describe the event. The > grammar for this description is not available, so we want to store > all of the typed ( or recognized) text, and/or a URL of the > recorded speech into the concept in a dialog manager. > > We need to bypass the parser. I would need to make changes to the > RavenClaw to accommodate this, right? > > Currently the user input from the TTY (or ASR) is processed by > the parser. > > The control flow is determined by the .pgm file. This entry > causes it to call process_parse when an output from the parser is > available: > > > > ;; If we have a confidence annotated hypothesis, but > ;; no parse, call phoenix to parse it > ;; --- > RULE: :confhyps & !:parses --> phoenix.phoenixparse > IN: :confhyps :npow :pow :clip_info > OUT: :total_numparses :parses :input_source > LOG_OUT: :parses > > > > So I plan to add another entry: > > ;; If we have a confidence annotated hypothesis, but > ;; no parse, call DM directly > ;; --- > RULE: :confhyps & !:parses --> DM.process_raw > IN: :confhyps :npow :pow :clip_info > OUT: > LOG_OUT: > > > > Add process_raw() to > > Libraries/RavenClaw/DMInterfaces/GalaxyInterfaceFunctions.h& > implement in GalaxyInterface.cpp > > > process_raw() function will be called whenever an output from the > TTY (or ASR) is received. So, this output will go to both parser > and DialogManager.process_raw. > process_raw() assigns the text to a concept (but only in the case > when the concept is currently expected): > For every expected concept as compiled by > CDMCoreAgent::compileExpectationAgenda(): > If a concept is a "raw speech" concept, set its value to the raw > text received in an input frame to process_raw > I would have to add a type to a concept to indicate that it allows > "raw text" input. > > > Does this implementation plan make sense? > > Could you please make any suggestions? > > Jahanzeb must have done something similar because his application > can record user's input. Do you know how that was implemented? > > > > Thank you, > Svetlana > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061125/95953544/attachment.html From svetastenchikova at gmail.com Sun Nov 26 22:24:08 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Sun, 26 Nov 2006 22:24:08 -0500 Subject: [RavenclawDev 198] Re: bypassing the parser In-Reply-To: <1546B8BE23C96940B5B4929DB7CC2343C940E8@e2k3.srv.cs.cmu.edu> References: <31cecd6b0611090639i1b59bbdbsc33970514afb4b6b@mail.gmail.com> <1546B8BE23C96940B5B4929DB7CC2343C940E8@e2k3.srv.cs.cmu.edu> Message-ID: <31cecd6b0611261924k6e44cabeva325c927e4c1450d@mail.gmail.com> Thank you Dan, binding to the h3_hypothesis worked. That's good because it saves me from doing more work :) Good luck on your interviews! Svetlana On 11/25/06, Dan Bohus wrote: > > Hi Svetlana, > > > > RavenClaw can bind on any of the attributes in the input. For instance, if > you look at the "new input arrived" line in a dialog log, below it you will > have a dump of all the attributes of the input and their values. You will > notice that there are things there which represent the various parse > fragments such as [Yes] or [Query.location] (which you normally bind on > the dialog task). At the same time, there are a number of other attributes > starting with h3_ ? these are the various features that helios3 in fact uses > in the confidence annotation process. They are passed all the way to the DM. > Among them, h3_hypothesis contains the actual text of the recognition and > h3_parse_str contains the full parse of the current decoded result. You can > therefore bind on [h3_hypothesis] to get the text of the recognition. > > > > Hope this helps, but let me know if you have other questions. > > Dan > > > ------------------------------ > > *From:* ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto: > ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] * On Behalf Of *Svetlana > Stenchikova > *Sent:* Thursday, November 09, 2006 9:40 AM > *To:* Antoine Raux > *Cc:* ravenclaw-developers at cs.cmu.edu > *Subject:* [RavenclawDev 188] Re: bypassing the parser > > > > Hi Antoine (or Dan), > > did you, by any chance, remember the syntax for the grammar mapping to get > text instead of a parse in a request/expect agent? > > Thank you > Svetlana > > On 11/5/06, *Antoine Raux* wrote: > > Hi Svetlana, > > RavenClaw actually has access to the (recognized) text corresponding to > each input, whether it's parsable or not. It's part of the set of > features that Helios passes to the DM. I forgot exactly how to access > these features but I believe it's basically a grammar mapping with a > special prefix that indicates that you're not binding a grammar slot but > a Helios input structure slot. Sorry I don't remember the exact syntax > now... If nobody else (Dan?) replies with more precision, I'll find it > out and email that later in the week. In any case, you might be able to > get what you want (the text of the input) without modifying anything at > all but your task specification (grammar mappings)... > > Otherwise, the second approach you are proposing (basically adding a > second "parser") sounds definitely much better than the first one, in > the sense that it preserves the integrity of the input line (which is > important for many things in the DM like grounding, reference to > previous turns....). > > I hope Dan will come up with a more precise answer soon :) > > antoine > > Svetlana Stenchikova wrote: > > > I just thought of another way to implement this without changing > > RavenClaw: > > > > Add another server ProcessRawData which receives the raw data input > > (in parallel with the parser) > > ProcessRawData outputs the input text in the format of the Parser: > > > > [RawText] ( the typed in text ) > > > > The output is passed to the DialogManager.process_parse message > > > > The the text will bind to all concept that are expected in the > > [RawText] grammar mapping. > > > > Please let me know your thoughts on this. > > > > thank you > > Svetlana > > > > > > On 11/5/06, *Svetlana Stenchikova* > > wrote: > > > > Dan, thank you for your answer. I have another question. > > > > We are interested in the functionality where a free text can be > > entered through TTY (or through free speech recognition). > > > > For example, a user can be asked to describe the event. The > > grammar for this description is not available, so we want to store > > all of the typed ( or recognized) text, and/or a URL of the > > recorded speech into the concept in a dialog manager. > > > > We need to bypass the parser. I would need to make changes to the > > RavenClaw to accommodate this, right? > > > > Currently the user input from the TTY (or ASR) is processed by > > the parser. > > > > The control flow is determined by the .pgm file. This entry > > causes it to call process_parse when an output from the parser is > > available: > > > > > > > > ;; If we have a confidence annotated hypothesis, but > > ;; no parse, call phoenix to parse it > > ;; --- > > RULE: :confhyps & !:parses --> phoenix.phoenixparse > > IN: :confhyps :npow :pow :clip_info > > OUT: :total_numparses :parses :input_source > > LOG_OUT: :parses > > > > > > > > So I plan to add another entry: > > > > ;; If we have a confidence annotated hypothesis, but > > ;; no parse, call DM directly > > ;; --- > > RULE: :confhyps & !:parses --> DM.process_raw > > IN: :confhyps :npow :pow :clip_info > > OUT: > > LOG_OUT: > > > > > > > > Add process_raw() to > > > > Libraries/RavenClaw/DMInterfaces/GalaxyInterfaceFunctions.h& > > implement in GalaxyInterface.cpp > > > > > > process_raw() function will be called whenever an output from the > > TTY (or ASR) is received. So, this output will go to both parser > > and DialogManager.process_raw. > > process_raw() assigns the text to a concept (but only in the case > > when the concept is currently expected): > > For every expected concept as compiled by > > CDMCoreAgent::compileExpectationAgenda(): > > If a concept is a "raw speech" concept, set its value to the raw > > text received in an input frame to process_raw > > I would have to add a type to a concept to indicate that it allows > > "raw text" input. > > > > > > Does this implementation plan make sense? > > > > Could you please make any suggestions? > > > > Jahanzeb must have done something similar because his application > > can record user's input. Do you know how that was implemented? > > > > > > > > Thank you, > > Svetlana > > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061126/a30c55c7/attachment-0001.html From svetastenchikova at gmail.com Mon Nov 27 13:25:49 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Mon, 27 Nov 2006 13:25:49 -0500 Subject: [RavenclawDev 199] Re: adding ASR Message-ID: <31cecd6b0611271025h20e32e6ah4f961ac8139a362b@mail.gmail.com> Hi, we are now starting to add sphinx ASR to our calendar system. We had some general questions about it. The roomline version is set to work with sphinx 2, and there are 3 servers that are starting for it. { title SPHINX_AUDIO dir . path ..\..\Bin\x86-nt\Audio_Server.exe -engine_list sphinx_engines.txt server_name sphinx } { title SPHINX_MALE dir ..\..\Resources\DecoderConfig\ path ..\..\Bin\x86-nt\Sphinx_Engine -name male -argfn male-8khz.arg-port 9990 server_name sphinx_male } { title SPHINX_FEMALE dir ..\..\Resources\DecoderConfig\ path ..\..\Bin\x86-nt\Sphinx_Engine -name female -argfn female-8khz.arg-port 9991 server_name sphinx_female } Is Audio_Server.exe the Galaxy wrapper? Where is the code for it? We plan to use sphinx 4, so we may want to write a java wrapper for it. If I understand correctly, we could just use the TTY wrapper that we have and replace the calls to output and input text with calls to sphinx. Is this correct? thank you. Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061127/d0fe7ad3/attachment.html From antoine at cs.cmu.edu Mon Nov 27 14:16:10 2006 From: antoine at cs.cmu.edu (Antoine Raux) Date: Mon, 27 Nov 2006 14:16:10 -0500 Subject: [RavenclawDev 200] Re: adding ASR In-Reply-To: <31cecd6b0611271025h20e32e6ah4f961ac8139a362b@mail.gmail.com> Message-ID: <005801c71258$7f9a9e50$03bd0280@sp.cs.cmu.edu> Hi Svetlana, Yes AudioServer is the Galaxy server for ASR. It is however a little more than just a wrapper since it runs as an interface to audio inputs and can stream the audio to several engines in parallel (hence the SPHINX_MALE and SPHINX_FEMALE processes). AudioServer then collects the hypotheses and sends them to the hub (all the hypotheses are then parsed and Helios picks one for RavenClaw). The code for AudioServer is in Agents/MultiDecoder/Audio_Server To answer your other question, yes you can use TTY-server as a template for your ASR agent. Don't forget to handle confidence scores (potentially dummy ones if Sphinx4 doesn't give you real ones). Hope this helps, antoine _____ From: ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto:ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] On Behalf Of Svetlana Stenchikova Sent: Monday, November 27, 2006 1:26 PM To: Dan Bohus Cc: Sarah Hoffman; ravenclaw-developers at cs.cmu.edu Subject: [RavenclawDev 199] Re: adding ASR Hi, we are now starting to add sphinx ASR to our calendar system. We had some general questions about it. The roomline version is set to work with sphinx 2, and there are 3 servers that are starting for it. { title SPHINX_AUDIO dir . path ..\..\Bin\x86-nt\Audio_Server.exe -engine_list sphinx_engines.txt server_name sphinx } { title SPHINX_MALE dir ..\..\Resources\DecoderConfig\ path ..\..\Bin\x86-nt\Sphinx_Engine -name male -argfn male-8khz.arg -port 9990 server_name sphinx_male } { title SPHINX_FEMALE dir ..\..\Resources\DecoderConfig\ path ..\..\Bin\x86-nt\Sphinx_Engine -name female -argfn female-8khz.arg -port 9991 server_name sphinx_female } Is Audio_Server.exe the Galaxy wrapper? Where is the code for it? We plan to use sphinx 4, so we may want to write a java wrapper for it. If I understand correctly, we could just use the TTY wrapper that we have and replace the calls to output and input text with calls to sphinx. Is this correct? thank you. Svetlana -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061127/8bd092f1/attachment.html From svetastenchikova at gmail.com Mon Nov 27 16:07:38 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Mon, 27 Nov 2006 16:07:38 -0500 Subject: [RavenclawDev 201] Re: adding ASR In-Reply-To: <005801c71258$7f9a9e50$03bd0280@sp.cs.cmu.edu> References: <31cecd6b0611271025h20e32e6ah4f961ac8139a362b@mail.gmail.com> <005801c71258$7f9a9e50$03bd0280@sp.cs.cmu.edu> Message-ID: <31cecd6b0611271307v120a9b49q48bfec08c81e93f3@mail.gmail.com> Thank you Antoine. If there is a documentation for the AudioServer somewhere, could you please point to it. Does AudioServer communicates with Sphinx male and female via sockets? Did sphinx 2 provide a socket interface or did you write a wrapper for it? It seems that running 3 separate servers for ASR adds complexity to the application (unless sphinx already provides socked api). Alternatively, we could (as we did with the back end) have a single java (or c) server which runs as a wrapper to the sphinx. However, if we are running multiple versions of sphinx(male/female) we would still need a single server to combine output from both, right? Could you please comment on this design issue. thank you, Svetlana On 11/27/06, Antoine Raux wrote: > > Hi Svetlana, > > > > Yes AudioServer is the Galaxy server for ASR. It is however a little more > than just a wrapper since it runs as an interface to audio inputs and can > stream the audio to several engines in parallel (hence the SPHINX_MALE and > SPHINX_FEMALE processes). AudioServer then collects the hypotheses and sends > them to the hub (all the hypotheses are then parsed and Helios picks one for > RavenClaw). > > The code for AudioServer is in Agents/MultiDecoder/Audio_Server > > > > To answer your other question, yes you can use TTY-server as a template > for your ASR agent. Don't forget to handle confidence scores (potentially > dummy ones if Sphinx4 doesn't give you real ones). > > > > Hope this helps, > > antoine > > > ------------------------------ > > *From:* ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto: > ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] *On Behalf Of *Svetlana > Stenchikova > *Sent:* Monday, November 27, 2006 1:26 PM > *To:* Dan Bohus > *Cc:* Sarah Hoffman; ravenclaw-developers at cs.cmu.edu > *Subject:* [RavenclawDev 199] Re: adding ASR > > > > Hi, > we are now starting to add sphinx ASR to our calendar system. > > We had some general questions about it. > > The roomline version is set to work with sphinx 2, and there are 3 servers > that are starting for it. > > > > { > title SPHINX_AUDIO > dir . > path ..\..\Bin\x86-nt\Audio_Server.exe -engine_list sphinx_engines.txt > server_name sphinx > } > > { > title SPHINX_MALE > dir ..\..\Resources\DecoderConfig\ > path ..\..\Bin\x86-nt\Sphinx_Engine -name male -argfn male-8khz.arg-port 9990 > server_name sphinx_male > } > > { > title SPHINX_FEMALE > dir ..\..\Resources\DecoderConfig\ > path ..\..\Bin\x86-nt\Sphinx_Engine -name female -argfn > female-8khz.arg -port 9991 > server_name sphinx_female > } > > > Is Audio_Server.exe the Galaxy wrapper? Where is the code for it? > > We plan to use sphinx 4, so we may want to write a java wrapper for it. If > I understand correctly, we could just use the TTY wrapper that we have and > replace the calls to output and input text with calls to sphinx. Is this > correct? > > thank you. > > Svetlana > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061127/b5f51ed8/attachment.html From svetastenchikova at gmail.com Tue Nov 28 17:02:19 2006 From: svetastenchikova at gmail.com (Svetlana Stenchikova) Date: Tue, 28 Nov 2006 17:02:19 -0500 Subject: [RavenclawDev 202] Re: adding ASR In-Reply-To: <31cecd6b0611271307v120a9b49q48bfec08c81e93f3@mail.gmail.com> References: <31cecd6b0611271025h20e32e6ah4f961ac8139a362b@mail.gmail.com> <005801c71258$7f9a9e50$03bd0280@sp.cs.cmu.edu> <31cecd6b0611271307v120a9b49q48bfec08c81e93f3@mail.gmail.com> Message-ID: <31cecd6b0611281402l6e283669ndc6741ee447b8b94@mail.gmail.com> We are thinking that the easiest way would be to wrap the sphinx 4 engine with the socket interface which would accept the connection from the AudioServer and send the result the same way as sphinx 2 does now. Then we could reuse your AudioServer without changes. Does this sound reasonable to you? We need to know the format of what is sent and received via sockets from AudioServer to Sphinx and back. It looks like AudioServer sends raw audio data to the Sphinx, is that right? And what is the format of the information returned from the ASR engine? Is it a frame encoded as a string? Which is the code that implements the sphinx 2 wrappers? Thank you, Svetlana On 11/27/06, Svetlana Stenchikova wrote: > > Thank you Antoine. > > If there is a documentation for the AudioServer somewhere, could you > please point to it. > > Does AudioServer communicates with Sphinx male and female via sockets? > > Did sphinx 2 provide a socket interface or did you write a wrapper for it? > > It seems that running 3 separate servers for ASR adds complexity to the > application (unless sphinx already provides socked api). > Alternatively, we could (as we did with the back end) have a single java > (or c) server which runs as a wrapper to the sphinx. > However, if we are running multiple versions of sphinx(male/female) we > would still need a single server to combine output from both, right? > > Could you please comment on this design issue. > > > thank you, > Svetlana > > > On 11/27/06, Antoine Raux wrote: > > > > Hi Svetlana, > > > > > > > > Yes AudioServer is the Galaxy server for ASR. It is however a little > > more than just a wrapper since it runs as an interface to audio inputs and > > can stream the audio to several engines in parallel (hence the SPHINX_MALE > > and SPHINX_FEMALE processes). AudioServer then collects the hypotheses and > > sends them to the hub (all the hypotheses are then parsed and Helios picks > > one for RavenClaw). > > > > The code for AudioServer is in Agents/MultiDecoder/Audio_Server > > > > > > > > To answer your other question, yes you can use TTY-server as a template > > for your ASR agent. Don't forget to handle confidence scores (potentially > > dummy ones if Sphinx4 doesn't give you real ones). > > > > > > > > Hope this helps, > > > > antoine > > > > > > ------------------------------ > > > > *From:* ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu [mailto: > > ravenclaw-developers-bounces at LOGANBERRY.srv.cs.cmu.edu] * On Behalf Of *Svetlana > > Stenchikova > > *Sent:* Monday, November 27, 2006 1:26 PM > > *To:* Dan Bohus > > *Cc:* Sarah Hoffman; ravenclaw-developers at cs.cmu.edu > > *Subject:* [RavenclawDev 199] Re: adding ASR > > > > > > > > Hi, > > we are now starting to add sphinx ASR to our calendar system. > > > > We had some general questions about it. > > > > The roomline version is set to work with sphinx 2, and there are 3 > > servers that are starting for it. > > > > > > > > { > > title SPHINX_AUDIO > > dir . > > path ..\..\Bin\x86-nt\Audio_Server.exe -engine_list > > sphinx_engines.txt > > server_name sphinx > > } > > > > { > > title SPHINX_MALE > > dir ..\..\Resources\DecoderConfig\ > > path ..\..\Bin\x86-nt\Sphinx_Engine -name male -argfn male-8khz.arg-port 9990 > > server_name sphinx_male > > } > > > > { > > title SPHINX_FEMALE > > dir ..\..\Resources\DecoderConfig\ > > path ..\..\Bin\x86-nt\Sphinx_Engine -name female -argfn > > female-8khz.arg -port 9991 > > server_name sphinx_female > > } > > > > > > Is Audio_Server.exe the Galaxy wrapper? Where is the code for it? > > > > We plan to use sphinx 4, so we may want to write a java wrapper for it. > > If I understand correctly, we could just use the TTY wrapper that we have > > and replace the calls to output and input text with calls to sphinx. Is this > > correct? > > > > thank you. > > > > Svetlana > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.srv.cs.cmu.edu/pipermail/ravenclaw-developers/attachments/20061128/1ff9086a/attachment-0001.html