From bfrisch at cs.cmu.edu  Sat Apr  3 16:58:14 2010
From: bfrisch at cs.cmu.edu (Benjamin Frisch)
Date: Sat, 3 Apr 2010 14:58:14 -0600
Subject: [Olympus developers 216]: Re: Olympus and LetsGoPublic
In-Reply-To: <B8D8C631-BAED-477B-BE2C-6EA747659BF4@gmail.com>
References: <4B967A4E.70208@supelec.fr> <4B9A7079.8050404@polytechnique.org> 
	<4BA4EC76.2060608@supelec.fr>
	<B8D8C631-BAED-477B-BE2C-6EA747659BF4@gmail.com>
Message-ID: <y2wbd80926e1004031358raeb918cfl8954fb9a6d94b3ed@mail.gmail.com>

Sorry for taking so long to get back to you.  I've been really busy the past
few weeks.  If you are looking for an example of GUI frames, take a look at
the RoomLineGUI in the RoomLine Olympus 2.5 example system.  It gives an
example of how to use the new in 2.5 jLibOlympusUtility to send GUI frames
in Java, and a similar thing could be done in C as well.

Thanks,
Ben

On Sat, Mar 20, 2010 at 5:03 PM, Thomas Harris <tkharris at gmail.com> wrote:

> Two approaches. 1) emulate what the ttyrecognitionserver does.  You will
> need a grammar fir it, since that stuff is parsed. Another approach 2) is to
> send "GUI" frames. There should be a listener agent in Apollo that processes
> those frames as concepts and passes them to Ravenclaw. That method is not
> well proven though.
>
> Thanks,
> -Thomas
>
>
> On Mar 20, 2010, at 11:40 AM, Guillaume Hazebroucq <
> guillaume.hazebroucq at supelec.fr> wrote:
>
>  Bonjour Antoine, et merci de votre r?ponse.
>>
>> First, sorry for being that less specific. What I meant back then was that
>> our "user" was complete as a stand-alone project, and that we had troubles
>> connecting it.
>> Its entries are the "act/object" (nlg_prompt) couple sent to the NLG. We
>> did build a NLGServer-based server for our user, modified the rules, and it
>> seems that this part is functionning.
>> As for the output, I think it must be directly processed to the DM, but I
>> don't know the communication rule used in that case, nor what should the
>> sent frame look like.
>> I'm currently trying to figure that out, but I'm running out of time
>> (sic), so if anybody has a clue or a hint, I'll take it !
>>
>> Merci d'avance,
>> Guillaume
>>
>> Antoine Raux a ?crit :
>>
>>> Bonjour Guillaume,
>>>
>>> I'm sure you can get some help from the developers who read this mailing
>>> list. However, you need to be more specific about your situation and the
>>> problems you are facing.
>>> E.g. when you say "Our agent is complete", does that mean it is
>>> integrated as an Olympus agent or just a stand-alone program? Also what are
>>> the specific issues you have: is it about compiling, crashing at runtime,
>>> not providing the expected behavior...
>>>
>>> The best thing is for you to send to this mailing list whatever source
>>> code you have written so far (if it's too big, upload it to some web page
>>> and post the link) and a precise description of the problems (if possible
>>> attach log files or screen dumps).
>>>
>>> In case you weren't aware of it yet, you can find some (incomplete)
>>> documentation at
>>> http://wiki.speech.cs.cmu.edu/olympus/index.php/Documentation. For
>>> example,
>>> http://wiki.speech.cs.cmu.edu/olympus/index.php/Service_Providerprovides you a list of the most important existing Olympus components and
>>> some information on each of them (like their input and output frames).
>>>
>>> I'm sure you'll find the answer to your questions once they are asked
>>> more precisely.
>>>
>>> Bonne chance!
>>>
>>> antoine
>>>
>>>
>>>
>>> Guillaume Hazebroucq wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm currently a student in Sup?lec, a french engineering school, and my
>>>> final project deals with the Olympus system, and the Let'sGoPBus application
>>>> that uses it.
>>>> Our aim is to create a simulated user that could interact with the
>>>> LetsGoBus interface by himslef. It has no speech capacity, we are working on
>>>> text-based answers for now.(No need to interact with the Speech Recognition)
>>>> Our agent is now complete, but we have some issues regarding its
>>>> integration in the Olympus/LetsGoPublic application.We would like to know if
>>>> you had some advice to give us, on how to create an Olympus agent that could
>>>> integrate perfectly, or maybe if a skeleton is available to add a new
>>>> module.
>>>>
>>>> Thank you in advance,
>>>> Guillaume Hazebroucq
>>>>
>>>>
>>>> Bonjour,
>>>> Je suis actuellement en 3?me ann?e d'?cole d'ing?nieurs ?  Sup?lec, et
>>>> mon projet de fin d'?tudes porte sur la cr?ation d'un agent simul? pour le
>>>> systeme LetsGoBus, afin d'effectuer leurs tests.
>>>> Il ne r?pond au syst?me que par du texte, il n'y a pas de capacit?
>>>> vocale pour le moment.(Pas besoin d'interagir avec le Speech Recognition)
>>>> Cependant, nous rencontrons des difficult?s pour int?grer notre agent
>>>> dans le syst?me Olympus/LetsGoBus. Nous ne savons pas vraiment comment les
>>>> processus communiquent, et nous aimerions savoir si vous aviez quelques
>>>> conseils ? nous donner, ou un squelette disponible pour un module
>>>> suppl?mentaire.
>>>>
>>>> D'avance merci,
>>>> Guillaume Hazebroucq
>>>>
>>>>
>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.srv.cs.cmu.edu/pipermail/olympus-developers/attachments/20100403/fdd434e6/attachment.html

From bfrisch at cs.cmu.edu  Sat Apr  3 16:55:17 2010
From: bfrisch at cs.cmu.edu (Benjamin Frisch)
Date: Sat, 3 Apr 2010 14:55:17 -0600
Subject: [Olympus developers 217]: Re: N-best lists for PocketSphinx /
	Olympus
In-Reply-To: <76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>
References: <4BA9CFD6.1040804@cam.ac.uk>
	<76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>
Message-ID: <s2ibd80926e1004031355x874cd14do24244c3607f67e0f@mail.gmail.com>

Hi Blaise,
  I think I actually fixed a similar bug in what became Olympus 2.5.  Are
you using that?  E-mail the list if you have questions about upgrading to
Olympus 2.5 if you are interested.  Either way, can you send me your
changes?  I'll try to get them committed if necessary.

Thanks,
Ben

On Wed, Mar 24, 2010 at 11:04 AM, Thomas Harris <tkharris at gmail.com> wrote:

> Hi Blaise,
>
> Thanks for looking into this. I hope we can include your bugfixes. I've
> been looking into this as well, and there's a more fundamental issue. It
> seems like you can't get word confidence metrics from the PocketSphinx
> segment iterators when you've gotten the sement iterators from the n_best
> hypothisis iterator. It smells like a PocketSphinx bug, but I haven't seen
> any reference implementation of PocketSphinx that makes use of those
> confidence metrics in an n_best setting, so I'm not sure that it isn't a
> problem with how the PocketSphinx api is used. Until that issue is resolved
> n_best lists won't work in Olympus, too many downhill processes depend on
> those confidence metrics.
>
> Thanks,
> -Thomas
>
> On Wed, Mar 24, 2010 at 4:39 AM, Blaise Thomson <brmt2 at cam.ac.uk> wrote:
>
>> Dear Olympus developers,
>>
>> I am trying to get the Olympus LetsGo! system to provide an N-best list of
>> speech recognition hypotheses. I found the -n_best switch which can be
>> passed to the PocketSphinxEngine which is supposed to enable this but when I
>> set the switch to anything other than 0 the system crashes immediately on
>> any audio input. I remember you said that the system had been build to
>> provide N-best lists so I was wondering if you could give any advice on why
>> it is not working. Do you have a working N-best list system that you could
>> send me to see how things are configured?
>>
>> In trying to solve the problem I took a look at the PocketSphinxEngine
>> source code and have noticed some possible memory access bugs which may be
>> contributing to this. These were related to the way the iHypsGenerated
>> variable was used. I've fixed these and can send them if you would like (I
>> tried attaching them but the mailing list won't let me). The resulting code
>> still crashes but at a later stage. After the fix, the log file generates a
>> WARNING: "ngram_search.c", line 1000:. I don't know if this might be the
>> cause of the problem. There is also a possibility that I simply have to add
>> a configuration variable to PocketSphinx itself. At the moment I have only
>> used the n_best switch on PocketSphinxEngine.
>>
>> Please do let me know if you have any ideas of how to get this working or
>> who else to contact.
>>
>> Thanks for all you help,
>>
>> Blaise
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.srv.cs.cmu.edu/pipermail/olympus-developers/attachments/20100403/c72a2924/attachment.html

From antoine.raux at gmail.com  Tue Apr 13 11:58:14 2010
From: antoine.raux at gmail.com (Antoine Raux)
Date: Tue, 13 Apr 2010 08:58:14 -0700
Subject: [Olympus developers 218]: Re: N-best lists for PocketSphinx /
	Olympus
In-Reply-To: <4BC48EDE.9090001@cam.ac.uk>
References: <4BA9CFD6.1040804@cam.ac.uk>
	<76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>
	<4BC48EDE.9090001@cam.ac.uk>
Message-ID: <4BC49496.2020208@gmail.com>

Hi all,

What exactly is the confidence computation problem? Is it that we cannot 
compute the LM backoff type-based word confidence (see hyp_conf_slm in 
PocketsphinxEngine's main.cpp)?
If that is the problem, one way to fix this might be to modify 
hyp_conf_slm to accept a ps_seg_t as an argument (instead of always 
getting seg_iter from ps_seg_iter):

float* hyp_conf_slm (bool useFixedScore = false, ps_seg_t *seg_iter = NULL)
{
    const int MAX_TYPE_SIZE = 4096;
    int32 score, type[MAX_TYPE_SIZE];
    int32 k = 0;

    // (antoine) no seg_iter was given, get the top segment iterator from ps
    if (seg_iter == NULL)
        seg_iter = ps_seg_iter(psd, &score);

        type[k++] = 3;                      // use the trigram dummy for 
first word

    if (seg_iter != NULL) {
        while (seg_iter = ps_seg_next(seg_iter)) {
            if (k == MAX_TYPE_SIZE) return NULL;

            int32 lscr, ascr;
            ps_seg_prob(seg_iter, &ascr, &lscr, &type[k++]);
        }
    }
    type[k++] = 3; // (tk) dummy trigram after utterance
    type[k++] = 3; // (tk) sometimes there's no end token, in which case
                       // the list one was for the end token and this 
one is the dummy

    // (antoine) allocate the array of confidence scores
    float* conf = (float*)malloc(k*sizeof(float));

    for (int32 i = 1; i < k-2; i++) {
        if(!useFixedScore) {
            int32 t = type[i-1] + type[i] + ((type[i+1] + 
type[i+2])<<1); // (tk) wtf?
            conf[i-1] = (float)((double)(t-6)/12.0);
        } else {
            conf[i-1] = 0.7f;
        }
    }

    return conf;
}

Then further down, you can modify the third version of 
fillPartialHypStruct by just adding the argument when it calls hyp_conf_slm:

// [2008-02-19] (antoine): this function takes a partial hypothesis and 
a reference to a
//                        THypStruct and fills in the hyp struct
void fillPartialHypStruct(ps_seg_t* curr_seg_iter, THypStruct* phs, int 
fromNBest) {

    Log(STD_STREAM, "Filling partial hyp struct\n");

    size_t h_len, ch_len;
    int n_words = 0, n_validwords, has_oov;
    char tmp[16384];
    float *lm_conf = NULL;

    // Fill in confidence values for words in result and build filtered 
hypothesis
    if (slm)
        lm_conf = hyp_conf_slm(curr_seg_iter);
    else
        lm_conf = hyp_conf_slm(curr_seg_iter, true);

(...)

I don't really have any setup to test this but if someone who has could 
give it a shot and post the result to the mailing list...
Now it might be that I misunderstood what the problem was altogether (in 
which case I apologize for the spam)...

On a side note, the big commented out block in getHypStructs (as sent by 
Blaise) is from my Cactus code (which I had sent to Blaise as an 
example), so it's irrelevant to Olympus and should be deleted (for 
clarity's sake).

antoine

Blaise Thomson wrote:
> Hi Thomas / Alan,
>
> I've now got some preliminary N-best list code to work with 
> PocketSphinx. With the help of  some example code from Antoine I've 
> modified the pocketsphinx engine to produce a 1-best list for partial 
> recognition results but an N-best list upon completion. I've also 
> modified the AudioServer to be able to receive multiple N-best lists 
> from each of the recognizer (the number for each decoder specified by 
> an optional ":N" after the decoder definition in the config file). In 
> case this may be something you want to include in future versions of 
> Olympus I've attached my modified files.
>
> Note, however, that the code still doesn't produce any confidence 
> score information for the N-best list. For this reason we will still 
> probably be unable to use Olympus for our version of the LetsGo! 
> system. If the PocketSphinx bugs you mentioned are fixed any time soon 
> or if anyone finds out how to get confidence scores with the N-best 
> list would you please let us know?
>
> Many thanks,
> Blaise
>
>
>
> Thomas Harris wrote:
>> Hi Blaise,
>>
>> Thanks for looking into this. I hope we can include your bugfixes. 
>> I've been looking into this as well, and there's a more fundamental 
>> issue. It seems like you can't get word confidence metrics from the 
>> PocketSphinx segment iterators when you've gotten the sement 
>> iterators from the n_best hypothisis iterator. It smells like a 
>> PocketSphinx bug, but I haven't seen any reference implementation of 
>> PocketSphinx that makes use of those confidence metrics in an n_best 
>> setting, so I'm not sure that it isn't a problem with how the 
>> PocketSphinx api is used. Until that issue is resolved n_best lists 
>> won't work in Olympus, too many downhill processes depend on those 
>> confidence metrics.
>>
>> Thanks,
>> -Thomas
>>
>> On Wed, Mar 24, 2010 at 4:39 AM, Blaise Thomson <brmt2 at cam.ac.uk 
>> <mailto:brmt2 at cam.ac.uk>> wrote:
>>
>>     Dear Olympus developers,
>>
>>     I am trying to get the Olympus LetsGo! system to provide an N-best
>>     list of speech recognition hypotheses. I found the -n_best switch
>>     which can be passed to the PocketSphinxEngine which is supposed to
>>     enable this but when I set the switch to anything other than 0 the
>>     system crashes immediately on any audio input. I remember you said
>>     that the system had been build to provide N-best lists so I was
>>     wondering if you could give any advice on why it is not working.
>>     Do you have a working N-best list system that you could send me to
>>     see how things are configured?
>>
>>     In trying to solve the problem I took a look at the
>>     PocketSphinxEngine source code and have noticed some possible
>>     memory access bugs which may be contributing to this. These were
>>     related to the way the iHypsGenerated variable was used. I've
>>     fixed these and can send them if you would like (I tried attaching
>>     them but the mailing list won't let me). The resulting code still
>>     crashes but at a later stage. After the fix, the log file
>>     generates a WARNING: "ngram_search.c", line 1000:. I don't know if
>>     this might be the cause of the problem. There is also a
>>     possibility that I simply have to add a configuration variable to
>>     PocketSphinx itself. At the moment I have only used the n_best
>>     switch on PocketSphinxEngine.
>>
>>     Please do let me know if you have any ideas of how to get this
>>     working or who else to contact.
>>
>>     Thanks for all you help,
>>
>>     Blaise
>>
>>
>>
>


From tkharris at gmail.com  Tue Apr 13 12:37:47 2010
From: tkharris at gmail.com (Thomas Harris)
Date: Tue, 13 Apr 2010 12:37:47 -0400
Subject: [Olympus developers 219]: Re: N-best lists for PocketSphinx /
	Olympus
In-Reply-To: <4BC49496.2020208@gmail.com>
References: <4BA9CFD6.1040804@cam.ac.uk>
	<76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>
	<4BC48EDE.9090001@cam.ac.uk> <4BC49496.2020208@gmail.com>
Message-ID: <l2z76c228b1004130937hf3804b7h32021c3ae4ee9a3f@mail.gmail.com>

Hi Antoine,

Yes, that was/is a problem and I tried something like this. But even more
fundamental is the problem is that the p_seg_t* segment iterator that you
get from pocketsphinx doesn't correctly implement ps_seg_prob when the
segment iterator comes from the hypothesis iterator even though it works
fine if you get the segment iterator from the best_hyp function (or whatever
that's called). I've sent David the code segment that illustrates this bug.
I don't know that there's any kind of work around. For the most part we've
gotten mutiple hypotheses by running multiple recognizers, I guess.

Thanks,
-Thomas

On Tue, Apr 13, 2010 at 11:58 AM, Antoine Raux <antoine.raux at gmail.com>wrote:

> Hi all,
>
> What exactly is the confidence computation problem? Is it that we cannot
> compute the LM backoff type-based word confidence (see hyp_conf_slm in
> PocketsphinxEngine's main.cpp)?
> If that is the problem, one way to fix this might be to modify hyp_conf_slm
> to accept a ps_seg_t as an argument (instead of always getting seg_iter from
> ps_seg_iter):
>
> float* hyp_conf_slm (bool useFixedScore = false, ps_seg_t *seg_iter = NULL)
> {
>   const int MAX_TYPE_SIZE = 4096;
>   int32 score, type[MAX_TYPE_SIZE];
>   int32 k = 0;
>
>   // (antoine) no seg_iter was given, get the top segment iterator from ps
>   if (seg_iter == NULL)
>       seg_iter = ps_seg_iter(psd, &score);
>
>       type[k++] = 3;                      // use the trigram dummy for
> first word
>
>   if (seg_iter != NULL) {
>       while (seg_iter = ps_seg_next(seg_iter)) {
>           if (k == MAX_TYPE_SIZE) return NULL;
>
>           int32 lscr, ascr;
>           ps_seg_prob(seg_iter, &ascr, &lscr, &type[k++]);
>       }
>   }
>   type[k++] = 3; // (tk) dummy trigram after utterance
>   type[k++] = 3; // (tk) sometimes there's no end token, in which case
>                      // the list one was for the end token and this one is
> the dummy
>
>   // (antoine) allocate the array of confidence scores
>   float* conf = (float*)malloc(k*sizeof(float));
>
>   for (int32 i = 1; i < k-2; i++) {
>       if(!useFixedScore) {
>           int32 t = type[i-1] + type[i] + ((type[i+1] + type[i+2])<<1); //
> (tk) wtf?
>           conf[i-1] = (float)((double)(t-6)/12.0);
>       } else {
>           conf[i-1] = 0.7f;
>       }
>   }
>
>   return conf;
> }
>
> Then further down, you can modify the third version of fillPartialHypStruct
> by just adding the argument when it calls hyp_conf_slm:
>
> // [2008-02-19] (antoine): this function takes a partial hypothesis and a
> reference to a
> //                        THypStruct and fills in the hyp struct
> void fillPartialHypStruct(ps_seg_t* curr_seg_iter, THypStruct* phs, int
> fromNBest) {
>
>   Log(STD_STREAM, "Filling partial hyp struct\n");
>
>   size_t h_len, ch_len;
>   int n_words = 0, n_validwords, has_oov;
>   char tmp[16384];
>   float *lm_conf = NULL;
>
>   // Fill in confidence values for words in result and build filtered
> hypothesis
>   if (slm)
>       lm_conf = hyp_conf_slm(curr_seg_iter);
>   else
>       lm_conf = hyp_conf_slm(curr_seg_iter, true);
>
> (...)
>
> I don't really have any setup to test this but if someone who has could
> give it a shot and post the result to the mailing list...
> Now it might be that I misunderstood what the problem was altogether (in
> which case I apologize for the spam)...
>
> On a side note, the big commented out block in getHypStructs (as sent by
> Blaise) is from my Cactus code (which I had sent to Blaise as an example),
> so it's irrelevant to Olympus and should be deleted (for clarity's sake).
>
> antoine
>
> Blaise Thomson wrote:
>
>> Hi Thomas / Alan,
>>
>> I've now got some preliminary N-best list code to work with PocketSphinx.
>> With the help of  some example code from Antoine I've modified the
>> pocketsphinx engine to produce a 1-best list for partial recognition results
>> but an N-best list upon completion. I've also modified the AudioServer to be
>> able to receive multiple N-best lists from each of the recognizer (the
>> number for each decoder specified by an optional ":N" after the decoder
>> definition in the config file). In case this may be something you want to
>> include in future versions of Olympus I've attached my modified files.
>>
>> Note, however, that the code still doesn't produce any confidence score
>> information for the N-best list. For this reason we will still probably be
>> unable to use Olympus for our version of the LetsGo! system. If the
>> PocketSphinx bugs you mentioned are fixed any time soon or if anyone finds
>> out how to get confidence scores with the N-best list would you please let
>> us know?
>>
>> Many thanks,
>> Blaise
>>
>>
>>
>> Thomas Harris wrote:
>>
>>> Hi Blaise,
>>>
>>> Thanks for looking into this. I hope we can include your bugfixes. I've
>>> been looking into this as well, and there's a more fundamental issue. It
>>> seems like you can't get word confidence metrics from the PocketSphinx
>>> segment iterators when you've gotten the sement iterators from the n_best
>>> hypothisis iterator. It smells like a PocketSphinx bug, but I haven't seen
>>> any reference implementation of PocketSphinx that makes use of those
>>> confidence metrics in an n_best setting, so I'm not sure that it isn't a
>>> problem with how the PocketSphinx api is used. Until that issue is resolved
>>> n_best lists won't work in Olympus, too many downhill processes depend on
>>> those confidence metrics.
>>>
>>> Thanks,
>>> -Thomas
>>>
>>> On Wed, Mar 24, 2010 at 4:39 AM, Blaise Thomson <brmt2 at cam.ac.uk<mailto:
>>> brmt2 at cam.ac.uk>> wrote:
>>>
>>>    Dear Olympus developers,
>>>
>>>    I am trying to get the Olympus LetsGo! system to provide an N-best
>>>    list of speech recognition hypotheses. I found the -n_best switch
>>>    which can be passed to the PocketSphinxEngine which is supposed to
>>>    enable this but when I set the switch to anything other than 0 the
>>>    system crashes immediately on any audio input. I remember you said
>>>    that the system had been build to provide N-best lists so I was
>>>    wondering if you could give any advice on why it is not working.
>>>    Do you have a working N-best list system that you could send me to
>>>    see how things are configured?
>>>
>>>    In trying to solve the problem I took a look at the
>>>    PocketSphinxEngine source code and have noticed some possible
>>>    memory access bugs which may be contributing to this. These were
>>>    related to the way the iHypsGenerated variable was used. I've
>>>    fixed these and can send them if you would like (I tried attaching
>>>    them but the mailing list won't let me). The resulting code still
>>>    crashes but at a later stage. After the fix, the log file
>>>    generates a WARNING: "ngram_search.c", line 1000:. I don't know if
>>>    this might be the cause of the problem. There is also a
>>>    possibility that I simply have to add a configuration variable to
>>>    PocketSphinx itself. At the moment I have only used the n_best
>>>    switch on PocketSphinxEngine.
>>>
>>>    Please do let me know if you have any ideas of how to get this
>>>    working or who else to contact.
>>>
>>>    Thanks for all you help,
>>>
>>>    Blaise
>>>
>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.srv.cs.cmu.edu/pipermail/olympus-developers/attachments/20100413/d045b3e1/attachment.html

From antoine.raux at gmail.com  Tue Apr 13 12:53:43 2010
From: antoine.raux at gmail.com (Antoine Raux)
Date: Tue, 13 Apr 2010 09:53:43 -0700
Subject: [Olympus developers 220]: Re: N-best lists for PocketSphinx /
 Olympus
In-Reply-To: <l2z76c228b1004130937hf3804b7h32021c3ae4ee9a3f@mail.gmail.com>
References: <4BA9CFD6.1040804@cam.ac.uk>	
	<76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>	
	<4BC48EDE.9090001@cam.ac.uk> <4BC49496.2020208@gmail.com>
	<l2z76c228b1004130937hf3804b7h32021c3ae4ee9a3f@mail.gmail.com>
Message-ID: <4BC4A197.80600@gmail.com>

Okay I feared there might be such a deeper problem... It could be that 
ps does not keep track of all the necessary information for non-top 
hypotheses (for lighter weight book keeping)... A work around I kind 
think of (which is kind of heavyweight), would be for PocketsphinxEnging 
to recompute the backoff type from the word string instead of getting it 
from the recognizer (a bunch of ifs to test if the trigram exists in the 
LM, or the bigram, etc... should do the trick so it shouldn't be that 
hard...). If there's no short term solution in sight from ps, I would 
recommend doing it until ps gets fixed.

antoine

Thomas Harris wrote:
> Hi Antoine,
>
> Yes, that was/is a problem and I tried something like this. But even 
> more fundamental is the problem is that the p_seg_t* segment iterator 
> that you get from pocketsphinx doesn't correctly implement ps_seg_prob 
> when the segment iterator comes from the hypothesis iterator even 
> though it works fine if you get the segment iterator from the best_hyp 
> function (or whatever that's called). I've sent David the code segment 
> that illustrates this bug. I don't know that there's any kind of work 
> around. For the most part we've gotten mutiple hypotheses by running 
> multiple recognizers, I guess.
>
> Thanks,
> -Thomas
>
> On Tue, Apr 13, 2010 at 11:58 AM, Antoine Raux <antoine.raux at gmail.com 
> <mailto:antoine.raux at gmail.com>> wrote:
>
>     Hi all,
>
>     What exactly is the confidence computation problem? Is it that we
>     cannot compute the LM backoff type-based word confidence (see
>     hyp_conf_slm in PocketsphinxEngine's main.cpp)?
>     If that is the problem, one way to fix this might be to modify
>     hyp_conf_slm to accept a ps_seg_t as an argument (instead of
>     always getting seg_iter from ps_seg_iter):
>
>     float* hyp_conf_slm (bool useFixedScore = false, ps_seg_t
>     *seg_iter = NULL)
>     {
>       const int MAX_TYPE_SIZE = 4096;
>       int32 score, type[MAX_TYPE_SIZE];
>       int32 k = 0;
>
>       // (antoine) no seg_iter was given, get the top segment iterator
>     from ps
>       if (seg_iter == NULL)
>           seg_iter = ps_seg_iter(psd, &score);
>
>           type[k++] = 3;                      // use the trigram dummy
>     for first word
>
>       if (seg_iter != NULL) {
>           while (seg_iter = ps_seg_next(seg_iter)) {
>               if (k == MAX_TYPE_SIZE) return NULL;
>
>               int32 lscr, ascr;
>               ps_seg_prob(seg_iter, &ascr, &lscr, &type[k++]);
>           }
>       }
>       type[k++] = 3; // (tk) dummy trigram after utterance
>       type[k++] = 3; // (tk) sometimes there's no end token, in which case
>                          // the list one was for the end token and
>     this one is the dummy
>
>       // (antoine) allocate the array of confidence scores
>       float* conf = (float*)malloc(k*sizeof(float));
>
>       for (int32 i = 1; i < k-2; i++) {
>           if(!useFixedScore) {
>               int32 t = type[i-1] + type[i] + ((type[i+1] +
>     type[i+2])<<1); // (tk) wtf?
>               conf[i-1] = (float)((double)(t-6)/12.0);
>           } else {
>               conf[i-1] = 0.7f;
>           }
>       }
>
>       return conf;
>     }
>
>     Then further down, you can modify the third version of
>     fillPartialHypStruct by just adding the argument when it calls
>     hyp_conf_slm:
>
>     // [2008-02-19] (antoine): this function takes a partial
>     hypothesis and a reference to a
>     //                        THypStruct and fills in the hyp struct
>     void fillPartialHypStruct(ps_seg_t* curr_seg_iter, THypStruct*
>     phs, int fromNBest) {
>
>       Log(STD_STREAM, "Filling partial hyp struct\n");
>
>       size_t h_len, ch_len;
>       int n_words = 0, n_validwords, has_oov;
>       char tmp[16384];
>       float *lm_conf = NULL;
>
>       // Fill in confidence values for words in result and build
>     filtered hypothesis
>       if (slm)
>           lm_conf = hyp_conf_slm(curr_seg_iter);
>       else
>           lm_conf = hyp_conf_slm(curr_seg_iter, true);
>
>     (...)
>
>     I don't really have any setup to test this but if someone who has
>     could give it a shot and post the result to the mailing list...
>     Now it might be that I misunderstood what the problem was
>     altogether (in which case I apologize for the spam)...
>
>     On a side note, the big commented out block in getHypStructs (as
>     sent by Blaise) is from my Cactus code (which I had sent to Blaise
>     as an example), so it's irrelevant to Olympus and should be
>     deleted (for clarity's sake).
>
>     antoine
>
>     Blaise Thomson wrote:
>
>         Hi Thomas / Alan,
>
>         I've now got some preliminary N-best list code to work with
>         PocketSphinx. With the help of  some example code from Antoine
>         I've modified the pocketsphinx engine to produce a 1-best list
>         for partial recognition results but an N-best list upon
>         completion. I've also modified the AudioServer to be able to
>         receive multiple N-best lists from each of the recognizer (the
>         number for each decoder specified by an optional ":N" after
>         the decoder definition in the config file). In case this may
>         be something you want to include in future versions of Olympus
>         I've attached my modified files.
>
>         Note, however, that the code still doesn't produce any
>         confidence score information for the N-best list. For this
>         reason we will still probably be unable to use Olympus for our
>         version of the LetsGo! system. If the PocketSphinx bugs you
>         mentioned are fixed any time soon or if anyone finds out how
>         to get confidence scores with the N-best list would you please
>         let us know?
>
>         Many thanks,
>         Blaise
>
>
>
>         Thomas Harris wrote:
>
>             Hi Blaise,
>
>             Thanks for looking into this. I hope we can include your
>             bugfixes. I've been looking into this as well, and there's
>             a more fundamental issue. It seems like you can't get word
>             confidence metrics from the PocketSphinx segment iterators
>             when you've gotten the sement iterators from the n_best
>             hypothisis iterator. It smells like a PocketSphinx bug,
>             but I haven't seen any reference implementation of
>             PocketSphinx that makes use of those confidence metrics in
>             an n_best setting, so I'm not sure that it isn't a problem
>             with how the PocketSphinx api is used. Until that issue is
>             resolved n_best lists won't work in Olympus, too many
>             downhill processes depend on those confidence metrics.
>
>             Thanks,
>             -Thomas
>
>             On Wed, Mar 24, 2010 at 4:39 AM, Blaise Thomson
>             <brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>             <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>> wrote:
>
>                Dear Olympus developers,
>
>                I am trying to get the Olympus LetsGo! system to
>             provide an N-best
>                list of speech recognition hypotheses. I found the
>             -n_best switch
>                which can be passed to the PocketSphinxEngine which is
>             supposed to
>                enable this but when I set the switch to anything other
>             than 0 the
>                system crashes immediately on any audio input. I
>             remember you said
>                that the system had been build to provide N-best lists
>             so I was
>                wondering if you could give any advice on why it is not
>             working.
>                Do you have a working N-best list system that you could
>             send me to
>                see how things are configured?
>
>                In trying to solve the problem I took a look at the
>                PocketSphinxEngine source code and have noticed some
>             possible
>                memory access bugs which may be contributing to this.
>             These were
>                related to the way the iHypsGenerated variable was
>             used. I've
>                fixed these and can send them if you would like (I
>             tried attaching
>                them but the mailing list won't let me). The resulting
>             code still
>                crashes but at a later stage. After the fix, the log file
>                generates a WARNING: "ngram_search.c", line 1000:. I
>             don't know if
>                this might be the cause of the problem. There is also a
>                possibility that I simply have to add a configuration
>             variable to
>                PocketSphinx itself. At the moment I have only used the
>             n_best
>                switch on PocketSphinxEngine.
>
>                Please do let me know if you have any ideas of how to
>             get this
>                working or who else to contact.
>
>                Thanks for all you help,
>
>                Blaise
>
>
>
>
>
>


From antoine.raux at gmail.com  Tue Apr 13 13:12:22 2010
From: antoine.raux at gmail.com (Antoine Raux)
Date: Tue, 13 Apr 2010 10:12:22 -0700
Subject: [Olympus developers 221]: Re: N-best lists for PocketSphinx /
 Olympus
In-Reply-To: <l2z76c228b1004130937hf3804b7h32021c3ae4ee9a3f@mail.gmail.com>
References: <4BA9CFD6.1040804@cam.ac.uk>	
	<76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>	
	<4BC48EDE.9090001@cam.ac.uk> <4BC49496.2020208@gmail.com>
	<l2z76c228b1004130937hf3804b7h32021c3ae4ee9a3f@mail.gmail.com>
Message-ID: <4BC4A5F6.2030201@gmail.com>

Actually, rather than a bunch of ifs as I wrote, the (still temporary) 
solution is to get the ngram_model_t object from ps, and then use the 
sphinxbase functions (such as ngram_tg_score) to compute the backoff 
type (which is exactly what ps does at decoding time).

antoine

Thomas Harris wrote:
> Hi Antoine,
>
> Yes, that was/is a problem and I tried something like this. But even 
> more fundamental is the problem is that the p_seg_t* segment iterator 
> that you get from pocketsphinx doesn't correctly implement ps_seg_prob 
> when the segment iterator comes from the hypothesis iterator even 
> though it works fine if you get the segment iterator from the best_hyp 
> function (or whatever that's called). I've sent David the code segment 
> that illustrates this bug. I don't know that there's any kind of work 
> around. For the most part we've gotten mutiple hypotheses by running 
> multiple recognizers, I guess.
>
> Thanks,
> -Thomas
>
> On Tue, Apr 13, 2010 at 11:58 AM, Antoine Raux <antoine.raux at gmail.com 
> <mailto:antoine.raux at gmail.com>> wrote:
>
>     Hi all,
>
>     What exactly is the confidence computation problem? Is it that we
>     cannot compute the LM backoff type-based word confidence (see
>     hyp_conf_slm in PocketsphinxEngine's main.cpp)?
>     If that is the problem, one way to fix this might be to modify
>     hyp_conf_slm to accept a ps_seg_t as an argument (instead of
>     always getting seg_iter from ps_seg_iter):
>
>     float* hyp_conf_slm (bool useFixedScore = false, ps_seg_t
>     *seg_iter = NULL)
>     {
>       const int MAX_TYPE_SIZE = 4096;
>       int32 score, type[MAX_TYPE_SIZE];
>       int32 k = 0;
>
>       // (antoine) no seg_iter was given, get the top segment iterator
>     from ps
>       if (seg_iter == NULL)
>           seg_iter = ps_seg_iter(psd, &score);
>
>           type[k++] = 3;                      // use the trigram dummy
>     for first word
>
>       if (seg_iter != NULL) {
>           while (seg_iter = ps_seg_next(seg_iter)) {
>               if (k == MAX_TYPE_SIZE) return NULL;
>
>               int32 lscr, ascr;
>               ps_seg_prob(seg_iter, &ascr, &lscr, &type[k++]);
>           }
>       }
>       type[k++] = 3; // (tk) dummy trigram after utterance
>       type[k++] = 3; // (tk) sometimes there's no end token, in which case
>                          // the list one was for the end token and
>     this one is the dummy
>
>       // (antoine) allocate the array of confidence scores
>       float* conf = (float*)malloc(k*sizeof(float));
>
>       for (int32 i = 1; i < k-2; i++) {
>           if(!useFixedScore) {
>               int32 t = type[i-1] + type[i] + ((type[i+1] +
>     type[i+2])<<1); // (tk) wtf?
>               conf[i-1] = (float)((double)(t-6)/12.0);
>           } else {
>               conf[i-1] = 0.7f;
>           }
>       }
>
>       return conf;
>     }
>
>     Then further down, you can modify the third version of
>     fillPartialHypStruct by just adding the argument when it calls
>     hyp_conf_slm:
>
>     // [2008-02-19] (antoine): this function takes a partial
>     hypothesis and a reference to a
>     //                        THypStruct and fills in the hyp struct
>     void fillPartialHypStruct(ps_seg_t* curr_seg_iter, THypStruct*
>     phs, int fromNBest) {
>
>       Log(STD_STREAM, "Filling partial hyp struct\n");
>
>       size_t h_len, ch_len;
>       int n_words = 0, n_validwords, has_oov;
>       char tmp[16384];
>       float *lm_conf = NULL;
>
>       // Fill in confidence values for words in result and build
>     filtered hypothesis
>       if (slm)
>           lm_conf = hyp_conf_slm(curr_seg_iter);
>       else
>           lm_conf = hyp_conf_slm(curr_seg_iter, true);
>
>     (...)
>
>     I don't really have any setup to test this but if someone who has
>     could give it a shot and post the result to the mailing list...
>     Now it might be that I misunderstood what the problem was
>     altogether (in which case I apologize for the spam)...
>
>     On a side note, the big commented out block in getHypStructs (as
>     sent by Blaise) is from my Cactus code (which I had sent to Blaise
>     as an example), so it's irrelevant to Olympus and should be
>     deleted (for clarity's sake).
>
>     antoine
>
>     Blaise Thomson wrote:
>
>         Hi Thomas / Alan,
>
>         I've now got some preliminary N-best list code to work with
>         PocketSphinx. With the help of  some example code from Antoine
>         I've modified the pocketsphinx engine to produce a 1-best list
>         for partial recognition results but an N-best list upon
>         completion. I've also modified the AudioServer to be able to
>         receive multiple N-best lists from each of the recognizer (the
>         number for each decoder specified by an optional ":N" after
>         the decoder definition in the config file). In case this may
>         be something you want to include in future versions of Olympus
>         I've attached my modified files.
>
>         Note, however, that the code still doesn't produce any
>         confidence score information for the N-best list. For this
>         reason we will still probably be unable to use Olympus for our
>         version of the LetsGo! system. If the PocketSphinx bugs you
>         mentioned are fixed any time soon or if anyone finds out how
>         to get confidence scores with the N-best list would you please
>         let us know?
>
>         Many thanks,
>         Blaise
>
>
>
>         Thomas Harris wrote:
>
>             Hi Blaise,
>
>             Thanks for looking into this. I hope we can include your
>             bugfixes. I've been looking into this as well, and there's
>             a more fundamental issue. It seems like you can't get word
>             confidence metrics from the PocketSphinx segment iterators
>             when you've gotten the sement iterators from the n_best
>             hypothisis iterator. It smells like a PocketSphinx bug,
>             but I haven't seen any reference implementation of
>             PocketSphinx that makes use of those confidence metrics in
>             an n_best setting, so I'm not sure that it isn't a problem
>             with how the PocketSphinx api is used. Until that issue is
>             resolved n_best lists won't work in Olympus, too many
>             downhill processes depend on those confidence metrics.
>
>             Thanks,
>             -Thomas
>
>             On Wed, Mar 24, 2010 at 4:39 AM, Blaise Thomson
>             <brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>             <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>> wrote:
>
>                Dear Olympus developers,
>
>                I am trying to get the Olympus LetsGo! system to
>             provide an N-best
>                list of speech recognition hypotheses. I found the
>             -n_best switch
>                which can be passed to the PocketSphinxEngine which is
>             supposed to
>                enable this but when I set the switch to anything other
>             than 0 the
>                system crashes immediately on any audio input. I
>             remember you said
>                that the system had been build to provide N-best lists
>             so I was
>                wondering if you could give any advice on why it is not
>             working.
>                Do you have a working N-best list system that you could
>             send me to
>                see how things are configured?
>
>                In trying to solve the problem I took a look at the
>                PocketSphinxEngine source code and have noticed some
>             possible
>                memory access bugs which may be contributing to this.
>             These were
>                related to the way the iHypsGenerated variable was
>             used. I've
>                fixed these and can send them if you would like (I
>             tried attaching
>                them but the mailing list won't let me). The resulting
>             code still
>                crashes but at a later stage. After the fix, the log file
>                generates a WARNING: "ngram_search.c", line 1000:. I
>             don't know if
>                this might be the cause of the problem. There is also a
>                possibility that I simply have to add a configuration
>             variable to
>                PocketSphinx itself. At the moment I have only used the
>             n_best
>                switch on PocketSphinxEngine.
>
>                Please do let me know if you have any ideas of how to
>             get this
>                working or who else to contact.
>
>                Thanks for all you help,
>
>                Blaise
>
>
>
>
>
>


From tkharris at gmail.com  Tue Apr 13 13:18:22 2010
From: tkharris at gmail.com (Thomas Harris)
Date: Tue, 13 Apr 2010 13:18:22 -0400
Subject: [Olympus developers 222]: Re: N-best lists for PocketSphinx /
	Olympus
In-Reply-To: <4BC4A5F6.2030201@gmail.com>
References: <4BA9CFD6.1040804@cam.ac.uk>
	<76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>
	<4BC48EDE.9090001@cam.ac.uk> <4BC49496.2020208@gmail.com>
	<l2z76c228b1004130937hf3804b7h32021c3ae4ee9a3f@mail.gmail.com>
	<4BC4A5F6.2030201@gmail.com>
Message-ID: <w2k76c228b1004131018o6fea570fi4f3e9ca8643f2445@mail.gmail.com>

Good idea. That sounds doable. But we're also not getting acoustic model
scores. I don't know for sure, do latter stages of Olympus (like Helios)
depend on word-level acoustic scores?

-Thomas

On Tue, Apr 13, 2010 at 1:12 PM, Antoine Raux <antoine.raux at gmail.com>wrote:

> Actually, rather than a bunch of ifs as I wrote, the (still temporary)
> solution is to get the ngram_model_t object from ps, and then use the
> sphinxbase functions (such as ngram_tg_score) to compute the backoff type
> (which is exactly what ps does at decoding time).
>
> antoine
>
> Thomas Harris wrote:
>
>> Hi Antoine,
>>
>> Yes, that was/is a problem and I tried something like this. But even more
>> fundamental is the problem is that the p_seg_t* segment iterator that you
>> get from pocketsphinx doesn't correctly implement ps_seg_prob when the
>> segment iterator comes from the hypothesis iterator even though it works
>> fine if you get the segment iterator from the best_hyp function (or whatever
>> that's called). I've sent David the code segment that illustrates this bug.
>> I don't know that there's any kind of work around. For the most part we've
>> gotten mutiple hypotheses by running multiple recognizers, I guess.
>>
>> Thanks,
>> -Thomas
>>
>> On Tue, Apr 13, 2010 at 11:58 AM, Antoine Raux <antoine.raux at gmail.com<mailto:
>> antoine.raux at gmail.com>> wrote:
>>
>>    Hi all,
>>
>>    What exactly is the confidence computation problem? Is it that we
>>    cannot compute the LM backoff type-based word confidence (see
>>    hyp_conf_slm in PocketsphinxEngine's main.cpp)?
>>    If that is the problem, one way to fix this might be to modify
>>    hyp_conf_slm to accept a ps_seg_t as an argument (instead of
>>    always getting seg_iter from ps_seg_iter):
>>
>>    float* hyp_conf_slm (bool useFixedScore = false, ps_seg_t
>>    *seg_iter = NULL)
>>    {
>>      const int MAX_TYPE_SIZE = 4096;
>>      int32 score, type[MAX_TYPE_SIZE];
>>      int32 k = 0;
>>
>>      // (antoine) no seg_iter was given, get the top segment iterator
>>    from ps
>>      if (seg_iter == NULL)
>>          seg_iter = ps_seg_iter(psd, &score);
>>
>>          type[k++] = 3;                      // use the trigram dummy
>>    for first word
>>
>>      if (seg_iter != NULL) {
>>          while (seg_iter = ps_seg_next(seg_iter)) {
>>              if (k == MAX_TYPE_SIZE) return NULL;
>>
>>              int32 lscr, ascr;
>>              ps_seg_prob(seg_iter, &ascr, &lscr, &type[k++]);
>>          }
>>      }
>>      type[k++] = 3; // (tk) dummy trigram after utterance
>>      type[k++] = 3; // (tk) sometimes there's no end token, in which case
>>                         // the list one was for the end token and
>>    this one is the dummy
>>
>>      // (antoine) allocate the array of confidence scores
>>      float* conf = (float*)malloc(k*sizeof(float));
>>
>>      for (int32 i = 1; i < k-2; i++) {
>>          if(!useFixedScore) {
>>              int32 t = type[i-1] + type[i] + ((type[i+1] +
>>    type[i+2])<<1); // (tk) wtf?
>>              conf[i-1] = (float)((double)(t-6)/12.0);
>>          } else {
>>              conf[i-1] = 0.7f;
>>          }
>>      }
>>
>>      return conf;
>>    }
>>
>>    Then further down, you can modify the third version of
>>    fillPartialHypStruct by just adding the argument when it calls
>>    hyp_conf_slm:
>>
>>    // [2008-02-19] (antoine): this function takes a partial
>>    hypothesis and a reference to a
>>    //                        THypStruct and fills in the hyp struct
>>    void fillPartialHypStruct(ps_seg_t* curr_seg_iter, THypStruct*
>>    phs, int fromNBest) {
>>
>>      Log(STD_STREAM, "Filling partial hyp struct\n");
>>
>>      size_t h_len, ch_len;
>>      int n_words = 0, n_validwords, has_oov;
>>      char tmp[16384];
>>      float *lm_conf = NULL;
>>
>>      // Fill in confidence values for words in result and build
>>    filtered hypothesis
>>      if (slm)
>>          lm_conf = hyp_conf_slm(curr_seg_iter);
>>      else
>>          lm_conf = hyp_conf_slm(curr_seg_iter, true);
>>
>>    (...)
>>
>>    I don't really have any setup to test this but if someone who has
>>    could give it a shot and post the result to the mailing list...
>>    Now it might be that I misunderstood what the problem was
>>    altogether (in which case I apologize for the spam)...
>>
>>    On a side note, the big commented out block in getHypStructs (as
>>    sent by Blaise) is from my Cactus code (which I had sent to Blaise
>>    as an example), so it's irrelevant to Olympus and should be
>>    deleted (for clarity's sake).
>>
>>    antoine
>>
>>    Blaise Thomson wrote:
>>
>>        Hi Thomas / Alan,
>>
>>        I've now got some preliminary N-best list code to work with
>>        PocketSphinx. With the help of  some example code from Antoine
>>        I've modified the pocketsphinx engine to produce a 1-best list
>>        for partial recognition results but an N-best list upon
>>        completion. I've also modified the AudioServer to be able to
>>        receive multiple N-best lists from each of the recognizer (the
>>        number for each decoder specified by an optional ":N" after
>>        the decoder definition in the config file). In case this may
>>        be something you want to include in future versions of Olympus
>>        I've attached my modified files.
>>
>>        Note, however, that the code still doesn't produce any
>>        confidence score information for the N-best list. For this
>>        reason we will still probably be unable to use Olympus for our
>>        version of the LetsGo! system. If the PocketSphinx bugs you
>>        mentioned are fixed any time soon or if anyone finds out how
>>        to get confidence scores with the N-best list would you please
>>        let us know?
>>
>>        Many thanks,
>>        Blaise
>>
>>
>>
>>        Thomas Harris wrote:
>>
>>            Hi Blaise,
>>
>>            Thanks for looking into this. I hope we can include your
>>            bugfixes. I've been looking into this as well, and there's
>>            a more fundamental issue. It seems like you can't get word
>>            confidence metrics from the PocketSphinx segment iterators
>>            when you've gotten the sement iterators from the n_best
>>            hypothisis iterator. It smells like a PocketSphinx bug,
>>            but I haven't seen any reference implementation of
>>            PocketSphinx that makes use of those confidence metrics in
>>            an n_best setting, so I'm not sure that it isn't a problem
>>            with how the PocketSphinx api is used. Until that issue is
>>            resolved n_best lists won't work in Olympus, too many
>>            downhill processes depend on those confidence metrics.
>>
>>            Thanks,
>>            -Thomas
>>
>>            On Wed, Mar 24, 2010 at 4:39 AM, Blaise Thomson
>>            <brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>>            <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>> wrote:
>>
>>               Dear Olympus developers,
>>
>>               I am trying to get the Olympus LetsGo! system to
>>            provide an N-best
>>               list of speech recognition hypotheses. I found the
>>            -n_best switch
>>               which can be passed to the PocketSphinxEngine which is
>>            supposed to
>>               enable this but when I set the switch to anything other
>>            than 0 the
>>               system crashes immediately on any audio input. I
>>            remember you said
>>               that the system had been build to provide N-best lists
>>            so I was
>>               wondering if you could give any advice on why it is not
>>            working.
>>               Do you have a working N-best list system that you could
>>            send me to
>>               see how things are configured?
>>
>>               In trying to solve the problem I took a look at the
>>               PocketSphinxEngine source code and have noticed some
>>            possible
>>               memory access bugs which may be contributing to this.
>>            These were
>>               related to the way the iHypsGenerated variable was
>>            used. I've
>>               fixed these and can send them if you would like (I
>>            tried attaching
>>               them but the mailing list won't let me). The resulting
>>            code still
>>               crashes but at a later stage. After the fix, the log file
>>               generates a WARNING: "ngram_search.c", line 1000:. I
>>            don't know if
>>               this might be the cause of the problem. There is also a
>>               possibility that I simply have to add a configuration
>>            variable to
>>               PocketSphinx itself. At the moment I have only used the
>>            n_best
>>               switch on PocketSphinxEngine.
>>
>>               Please do let me know if you have any ideas of how to
>>            get this
>>               working or who else to contact.
>>
>>               Thanks for all you help,
>>
>>               Blaise
>>
>>
>>
>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.srv.cs.cmu.edu/pipermail/olympus-developers/attachments/20100413/27fdf769/attachment-0001.html

From antoine.raux at gmail.com  Tue Apr 13 13:20:12 2010
From: antoine.raux at gmail.com (Antoine Raux)
Date: Tue, 13 Apr 2010 10:20:12 -0700
Subject: [Olympus developers 223]: Re: N-best lists for PocketSphinx /
 Olympus
In-Reply-To: <w2k76c228b1004131018o6fea570fi4f3e9ca8643f2445@mail.gmail.com>
References: <4BA9CFD6.1040804@cam.ac.uk>	
	<76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>	
	<4BC48EDE.9090001@cam.ac.uk> <4BC49496.2020208@gmail.com>	
	<l2z76c228b1004130937hf3804b7h32021c3ae4ee9a3f@mail.gmail.com>	
	<4BC4A5F6.2030201@gmail.com>
	<w2k76c228b1004131018o6fea570fi4f3e9ca8643f2445@mail.gmail.com>
Message-ID: <4BC4A7CC.8020908@gmail.com>

Hmmm... I don't know about word-level acoustic scores. I don't remember 
ever using these. Are these actually stored in the frame that 
PocketsphinxEngine and/or AudioServer send?

antoine

Thomas Harris wrote:
> Good idea. That sounds doable. But we're also not getting acoustic 
> model scores. I don't know for sure, do latter stages of Olympus (like 
> Helios) depend on word-level acoustic scores?
>
> -Thomas
>
> On Tue, Apr 13, 2010 at 1:12 PM, Antoine Raux <antoine.raux at gmail.com 
> <mailto:antoine.raux at gmail.com>> wrote:
>
>     Actually, rather than a bunch of ifs as I wrote, the (still
>     temporary) solution is to get the ngram_model_t object from ps,
>     and then use the sphinxbase functions (such as ngram_tg_score) to
>     compute the backoff type (which is exactly what ps does at
>     decoding time).
>
>     antoine
>
>     Thomas Harris wrote:
>
>         Hi Antoine,
>
>         Yes, that was/is a problem and I tried something like this.
>         But even more fundamental is the problem is that the p_seg_t*
>         segment iterator that you get from pocketsphinx doesn't
>         correctly implement ps_seg_prob when the segment iterator
>         comes from the hypothesis iterator even though it works fine
>         if you get the segment iterator from the best_hyp function (or
>         whatever that's called). I've sent David the code segment that
>         illustrates this bug. I don't know that there's any kind of
>         work around. For the most part we've gotten mutiple hypotheses
>         by running multiple recognizers, I guess.
>
>         Thanks,
>         -Thomas
>
>         On Tue, Apr 13, 2010 at 11:58 AM, Antoine Raux
>         <antoine.raux at gmail.com <mailto:antoine.raux at gmail.com>
>         <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com>>> wrote:
>
>            Hi all,
>
>            What exactly is the confidence computation problem? Is it
>         that we
>            cannot compute the LM backoff type-based word confidence (see
>            hyp_conf_slm in PocketsphinxEngine's main.cpp)?
>            If that is the problem, one way to fix this might be to modify
>            hyp_conf_slm to accept a ps_seg_t as an argument (instead of
>            always getting seg_iter from ps_seg_iter):
>
>            float* hyp_conf_slm (bool useFixedScore = false, ps_seg_t
>            *seg_iter = NULL)
>            {
>              const int MAX_TYPE_SIZE = 4096;
>              int32 score, type[MAX_TYPE_SIZE];
>              int32 k = 0;
>
>              // (antoine) no seg_iter was given, get the top segment
>         iterator
>            from ps
>              if (seg_iter == NULL)
>                  seg_iter = ps_seg_iter(psd, &score);
>
>                  type[k++] = 3;                      // use the
>         trigram dummy
>            for first word
>
>              if (seg_iter != NULL) {
>                  while (seg_iter = ps_seg_next(seg_iter)) {
>                      if (k == MAX_TYPE_SIZE) return NULL;
>
>                      int32 lscr, ascr;
>                      ps_seg_prob(seg_iter, &ascr, &lscr, &type[k++]);
>                  }
>              }
>              type[k++] = 3; // (tk) dummy trigram after utterance
>              type[k++] = 3; // (tk) sometimes there's no end token, in
>         which case
>                                 // the list one was for the end token and
>            this one is the dummy
>
>              // (antoine) allocate the array of confidence scores
>              float* conf = (float*)malloc(k*sizeof(float));
>
>              for (int32 i = 1; i < k-2; i++) {
>                  if(!useFixedScore) {
>                      int32 t = type[i-1] + type[i] + ((type[i+1] +
>            type[i+2])<<1); // (tk) wtf?
>                      conf[i-1] = (float)((double)(t-6)/12.0);
>                  } else {
>                      conf[i-1] = 0.7f;
>                  }
>              }
>
>              return conf;
>            }
>
>            Then further down, you can modify the third version of
>            fillPartialHypStruct by just adding the argument when it calls
>            hyp_conf_slm:
>
>            // [2008-02-19] (antoine): this function takes a partial
>            hypothesis and a reference to a
>            //                        THypStruct and fills in the hyp
>         struct
>            void fillPartialHypStruct(ps_seg_t* curr_seg_iter, THypStruct*
>            phs, int fromNBest) {
>
>              Log(STD_STREAM, "Filling partial hyp struct\n");
>
>              size_t h_len, ch_len;
>              int n_words = 0, n_validwords, has_oov;
>              char tmp[16384];
>              float *lm_conf = NULL;
>
>              // Fill in confidence values for words in result and build
>            filtered hypothesis
>              if (slm)
>                  lm_conf = hyp_conf_slm(curr_seg_iter);
>              else
>                  lm_conf = hyp_conf_slm(curr_seg_iter, true);
>
>            (...)
>
>            I don't really have any setup to test this but if someone
>         who has
>            could give it a shot and post the result to the mailing list...
>            Now it might be that I misunderstood what the problem was
>            altogether (in which case I apologize for the spam)...
>
>            On a side note, the big commented out block in
>         getHypStructs (as
>            sent by Blaise) is from my Cactus code (which I had sent to
>         Blaise
>            as an example), so it's irrelevant to Olympus and should be
>            deleted (for clarity's sake).
>
>            antoine
>
>            Blaise Thomson wrote:
>
>                Hi Thomas / Alan,
>
>                I've now got some preliminary N-best list code to work with
>                PocketSphinx. With the help of  some example code from
>         Antoine
>                I've modified the pocketsphinx engine to produce a
>         1-best list
>                for partial recognition results but an N-best list upon
>                completion. I've also modified the AudioServer to be
>         able to
>                receive multiple N-best lists from each of the
>         recognizer (the
>                number for each decoder specified by an optional ":N" after
>                the decoder definition in the config file). In case
>         this may
>                be something you want to include in future versions of
>         Olympus
>                I've attached my modified files.
>
>                Note, however, that the code still doesn't produce any
>                confidence score information for the N-best list. For this
>                reason we will still probably be unable to use Olympus
>         for our
>                version of the LetsGo! system. If the PocketSphinx bugs you
>                mentioned are fixed any time soon or if anyone finds
>         out how
>                to get confidence scores with the N-best list would you
>         please
>                let us know?
>
>                Many thanks,
>                Blaise
>
>
>
>                Thomas Harris wrote:
>
>                    Hi Blaise,
>
>                    Thanks for looking into this. I hope we can include
>         your
>                    bugfixes. I've been looking into this as well, and
>         there's
>                    a more fundamental issue. It seems like you can't
>         get word
>                    confidence metrics from the PocketSphinx segment
>         iterators
>                    when you've gotten the sement iterators from the n_best
>                    hypothisis iterator. It smells like a PocketSphinx bug,
>                    but I haven't seen any reference implementation of
>                    PocketSphinx that makes use of those confidence
>         metrics in
>                    an n_best setting, so I'm not sure that it isn't a
>         problem
>                    with how the PocketSphinx api is used. Until that
>         issue is
>                    resolved n_best lists won't work in Olympus, too many
>                    downhill processes depend on those confidence metrics.
>
>                    Thanks,
>                    -Thomas
>
>                    On Wed, Mar 24, 2010 at 4:39 AM, Blaise Thomson
>                    <brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>         <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>
>                    <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>         <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>>> wrote:
>
>                       Dear Olympus developers,
>
>                       I am trying to get the Olympus LetsGo! system to
>                    provide an N-best
>                       list of speech recognition hypotheses. I found the
>                    -n_best switch
>                       which can be passed to the PocketSphinxEngine
>         which is
>                    supposed to
>                       enable this but when I set the switch to
>         anything other
>                    than 0 the
>                       system crashes immediately on any audio input. I
>                    remember you said
>                       that the system had been build to provide N-best
>         lists
>                    so I was
>                       wondering if you could give any advice on why it
>         is not
>                    working.
>                       Do you have a working N-best list system that
>         you could
>                    send me to
>                       see how things are configured?
>
>                       In trying to solve the problem I took a look at the
>                       PocketSphinxEngine source code and have noticed some
>                    possible
>                       memory access bugs which may be contributing to
>         this.
>                    These were
>                       related to the way the iHypsGenerated variable was
>                    used. I've
>                       fixed these and can send them if you would like (I
>                    tried attaching
>                       them but the mailing list won't let me). The
>         resulting
>                    code still
>                       crashes but at a later stage. After the fix, the
>         log file
>                       generates a WARNING: "ngram_search.c", line 1000:. I
>                    don't know if
>                       this might be the cause of the problem. There is
>         also a
>                       possibility that I simply have to add a
>         configuration
>                    variable to
>                       PocketSphinx itself. At the moment I have only
>         used the
>                    n_best
>                       switch on PocketSphinxEngine.
>
>                       Please do let me know if you have any ideas of
>         how to
>                    get this
>                       working or who else to contact.
>
>                       Thanks for all you help,
>
>                       Blaise
>
>
>
>
>
>
>
>


From tkharris at gmail.com  Tue Apr 13 13:31:09 2010
From: tkharris at gmail.com (Thomas Harris)
Date: Tue, 13 Apr 2010 13:31:09 -0400
Subject: [Olympus developers 224]: Re: N-best lists for PocketSphinx /
	Olympus
In-Reply-To: <4BC4A7CC.8020908@gmail.com>
References: <4BA9CFD6.1040804@cam.ac.uk>
	<76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>
	<4BC48EDE.9090001@cam.ac.uk> <4BC49496.2020208@gmail.com>
	<l2z76c228b1004130937hf3804b7h32021c3ae4ee9a3f@mail.gmail.com>
	<4BC4A5F6.2030201@gmail.com>
	<w2k76c228b1004131018o6fea570fi4f3e9ca8643f2445@mail.gmail.com>
	<4BC4A7CC.8020908@gmail.com>
Message-ID: <o2i76c228b1004131031z3f242007q4896ed7837853930@mail.gmail.com>

I have seen in the frame some statistics like #words>p where p is some kind
of confidence metric. It would seem that those statistics can only be
derived from word-level confidence scores in the recognizer. But that's just
my recollection right now. Someone should investigate, I guess.

Thanks,
-Thomas

On Tue, Apr 13, 2010 at 1:20 PM, Antoine Raux <antoine.raux at gmail.com>wrote:

> Hmmm... I don't know about word-level acoustic scores. I don't remember
> ever using these. Are these actually stored in the frame that
> PocketsphinxEngine and/or AudioServer send?
>
> antoine
>
> Thomas Harris wrote:
>
>> Good idea. That sounds doable. But we're also not getting acoustic model
>> scores. I don't know for sure, do latter stages of Olympus (like Helios)
>> depend on word-level acoustic scores?
>>
>> -Thomas
>>
>> On Tue, Apr 13, 2010 at 1:12 PM, Antoine Raux <antoine.raux at gmail.com<mailto:
>> antoine.raux at gmail.com>> wrote:
>>
>>    Actually, rather than a bunch of ifs as I wrote, the (still
>>    temporary) solution is to get the ngram_model_t object from ps,
>>    and then use the sphinxbase functions (such as ngram_tg_score) to
>>    compute the backoff type (which is exactly what ps does at
>>    decoding time).
>>
>>    antoine
>>
>>    Thomas Harris wrote:
>>
>>        Hi Antoine,
>>
>>        Yes, that was/is a problem and I tried something like this.
>>        But even more fundamental is the problem is that the p_seg_t*
>>        segment iterator that you get from pocketsphinx doesn't
>>        correctly implement ps_seg_prob when the segment iterator
>>        comes from the hypothesis iterator even though it works fine
>>        if you get the segment iterator from the best_hyp function (or
>>        whatever that's called). I've sent David the code segment that
>>        illustrates this bug. I don't know that there's any kind of
>>        work around. For the most part we've gotten mutiple hypotheses
>>        by running multiple recognizers, I guess.
>>
>>        Thanks,
>>        -Thomas
>>
>>        On Tue, Apr 13, 2010 at 11:58 AM, Antoine Raux
>>        <antoine.raux at gmail.com <mailto:antoine.raux at gmail.com>
>>        <mailto:antoine.raux at gmail.com
>>        <mailto:antoine.raux at gmail.com>>> wrote:
>>
>>           Hi all,
>>
>>           What exactly is the confidence computation problem? Is it
>>        that we
>>           cannot compute the LM backoff type-based word confidence (see
>>           hyp_conf_slm in PocketsphinxEngine's main.cpp)?
>>           If that is the problem, one way to fix this might be to modify
>>           hyp_conf_slm to accept a ps_seg_t as an argument (instead of
>>           always getting seg_iter from ps_seg_iter):
>>
>>           float* hyp_conf_slm (bool useFixedScore = false, ps_seg_t
>>           *seg_iter = NULL)
>>           {
>>             const int MAX_TYPE_SIZE = 4096;
>>             int32 score, type[MAX_TYPE_SIZE];
>>             int32 k = 0;
>>
>>             // (antoine) no seg_iter was given, get the top segment
>>        iterator
>>           from ps
>>             if (seg_iter == NULL)
>>                 seg_iter = ps_seg_iter(psd, &score);
>>
>>                 type[k++] = 3;                      // use the
>>        trigram dummy
>>           for first word
>>
>>             if (seg_iter != NULL) {
>>                 while (seg_iter = ps_seg_next(seg_iter)) {
>>                     if (k == MAX_TYPE_SIZE) return NULL;
>>
>>                     int32 lscr, ascr;
>>                     ps_seg_prob(seg_iter, &ascr, &lscr, &type[k++]);
>>                 }
>>             }
>>             type[k++] = 3; // (tk) dummy trigram after utterance
>>             type[k++] = 3; // (tk) sometimes there's no end token, in
>>        which case
>>                                // the list one was for the end token and
>>           this one is the dummy
>>
>>             // (antoine) allocate the array of confidence scores
>>             float* conf = (float*)malloc(k*sizeof(float));
>>
>>             for (int32 i = 1; i < k-2; i++) {
>>                 if(!useFixedScore) {
>>                     int32 t = type[i-1] + type[i] + ((type[i+1] +
>>           type[i+2])<<1); // (tk) wtf?
>>                     conf[i-1] = (float)((double)(t-6)/12.0);
>>                 } else {
>>                     conf[i-1] = 0.7f;
>>                 }
>>             }
>>
>>             return conf;
>>           }
>>
>>           Then further down, you can modify the third version of
>>           fillPartialHypStruct by just adding the argument when it calls
>>           hyp_conf_slm:
>>
>>           // [2008-02-19] (antoine): this function takes a partial
>>           hypothesis and a reference to a
>>           //                        THypStruct and fills in the hyp
>>        struct
>>           void fillPartialHypStruct(ps_seg_t* curr_seg_iter, THypStruct*
>>           phs, int fromNBest) {
>>
>>             Log(STD_STREAM, "Filling partial hyp struct\n");
>>
>>             size_t h_len, ch_len;
>>             int n_words = 0, n_validwords, has_oov;
>>             char tmp[16384];
>>             float *lm_conf = NULL;
>>
>>             // Fill in confidence values for words in result and build
>>           filtered hypothesis
>>             if (slm)
>>                 lm_conf = hyp_conf_slm(curr_seg_iter);
>>             else
>>                 lm_conf = hyp_conf_slm(curr_seg_iter, true);
>>
>>           (...)
>>
>>           I don't really have any setup to test this but if someone
>>        who has
>>           could give it a shot and post the result to the mailing list...
>>           Now it might be that I misunderstood what the problem was
>>           altogether (in which case I apologize for the spam)...
>>
>>           On a side note, the big commented out block in
>>        getHypStructs (as
>>           sent by Blaise) is from my Cactus code (which I had sent to
>>        Blaise
>>           as an example), so it's irrelevant to Olympus and should be
>>           deleted (for clarity's sake).
>>
>>           antoine
>>
>>           Blaise Thomson wrote:
>>
>>               Hi Thomas / Alan,
>>
>>               I've now got some preliminary N-best list code to work with
>>               PocketSphinx. With the help of  some example code from
>>        Antoine
>>               I've modified the pocketsphinx engine to produce a
>>        1-best list
>>               for partial recognition results but an N-best list upon
>>               completion. I've also modified the AudioServer to be
>>        able to
>>               receive multiple N-best lists from each of the
>>        recognizer (the
>>               number for each decoder specified by an optional ":N" after
>>               the decoder definition in the config file). In case
>>        this may
>>               be something you want to include in future versions of
>>        Olympus
>>               I've attached my modified files.
>>
>>               Note, however, that the code still doesn't produce any
>>               confidence score information for the N-best list. For this
>>               reason we will still probably be unable to use Olympus
>>        for our
>>               version of the LetsGo! system. If the PocketSphinx bugs you
>>               mentioned are fixed any time soon or if anyone finds
>>        out how
>>               to get confidence scores with the N-best list would you
>>        please
>>               let us know?
>>
>>               Many thanks,
>>               Blaise
>>
>>
>>
>>               Thomas Harris wrote:
>>
>>                   Hi Blaise,
>>
>>                   Thanks for looking into this. I hope we can include
>>        your
>>                   bugfixes. I've been looking into this as well, and
>>        there's
>>                   a more fundamental issue. It seems like you can't
>>        get word
>>                   confidence metrics from the PocketSphinx segment
>>        iterators
>>                   when you've gotten the sement iterators from the n_best
>>                   hypothisis iterator. It smells like a PocketSphinx bug,
>>                   but I haven't seen any reference implementation of
>>                   PocketSphinx that makes use of those confidence
>>        metrics in
>>                   an n_best setting, so I'm not sure that it isn't a
>>        problem
>>                   with how the PocketSphinx api is used. Until that
>>        issue is
>>                   resolved n_best lists won't work in Olympus, too many
>>                   downhill processes depend on those confidence metrics.
>>
>>                   Thanks,
>>                   -Thomas
>>
>>                   On Wed, Mar 24, 2010 at 4:39 AM, Blaise Thomson
>>                   <brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>>        <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>
>>                   <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>>        <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>>> wrote:
>>
>>                      Dear Olympus developers,
>>
>>                      I am trying to get the Olympus LetsGo! system to
>>                   provide an N-best
>>                      list of speech recognition hypotheses. I found the
>>                   -n_best switch
>>                      which can be passed to the PocketSphinxEngine
>>        which is
>>                   supposed to
>>                      enable this but when I set the switch to
>>        anything other
>>                   than 0 the
>>                      system crashes immediately on any audio input. I
>>                   remember you said
>>                      that the system had been build to provide N-best
>>        lists
>>                   so I was
>>                      wondering if you could give any advice on why it
>>        is not
>>                   working.
>>                      Do you have a working N-best list system that
>>        you could
>>                   send me to
>>                      see how things are configured?
>>
>>                      In trying to solve the problem I took a look at the
>>                      PocketSphinxEngine source code and have noticed some
>>                   possible
>>                      memory access bugs which may be contributing to
>>        this.
>>                   These were
>>                      related to the way the iHypsGenerated variable was
>>                   used. I've
>>                      fixed these and can send them if you would like (I
>>                   tried attaching
>>                      them but the mailing list won't let me). The
>>        resulting
>>                   code still
>>                      crashes but at a later stage. After the fix, the
>>        log file
>>                      generates a WARNING: "ngram_search.c", line 1000:. I
>>                   don't know if
>>                      this might be the cause of the problem. There is
>>        also a
>>                      possibility that I simply have to add a
>>        configuration
>>                   variable to
>>                      PocketSphinx itself. At the moment I have only
>>        used the
>>                   n_best
>>                      switch on PocketSphinxEngine.
>>
>>                      Please do let me know if you have any ideas of
>>        how to
>>                   get this
>>                      working or who else to contact.
>>
>>                      Thanks for all you help,
>>
>>                      Blaise
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.srv.cs.cmu.edu/pipermail/olympus-developers/attachments/20100413/103faedb/attachment-0001.html

From antoine.raux at gmail.com  Tue Apr 13 16:41:17 2010
From: antoine.raux at gmail.com (Antoine Raux)
Date: Tue, 13 Apr 2010 13:41:17 -0700
Subject: [Olympus developers 225]: Re: N-best lists for PocketSphinx /
 Olympus
In-Reply-To: <4437_1271179884_o3DHVNwd020691_o2i76c228b1004131031z3f242007q4896ed7837853930@mail.gmail.com>
References: <4BA9CFD6.1040804@cam.ac.uk>	<76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>	<4BC48EDE.9090001@cam.ac.uk>
	<4BC49496.2020208@gmail.com>	<l2z76c228b1004130937hf3804b7h32021c3ae4ee9a3f@mail.gmail.com>	<4BC4A5F6.2030201@gmail.com>	<w2k76c228b1004131018o6fea570fi4f3e9ca8643f2445@mail.gmail.com>	<4BC4A7CC.8020908@gmail.com>
	<4437_1271179884_o3DHVNwd020691_o2i76c228b1004131031z3f242007q4896ed7837853930@mail.gmail.com>
Message-ID: <4BC4D6ED.6050504@gmail.com>

As far as I can tell by looking at the code, PocketsphinxEngine does not 
send any word-level confidence information to AudioServer. That's good 
news. The bad news is that it computes utterance-level AM and LM scores 
by summing those of the words (so it needs word-level scores for 
that...). This seems the only way to get separate AM and LM score from 
Pocketsphinx (someone, correct me if I'm wrong). You can get the overall 
score though (the sum of the the AM and LM scores), using ps_nbest_hyp. 
So we could recompute the LM score of the i-th hypothesis using the 
Sphinxbase's ngram_tg_score routine (at the same time as getting the LM 
backoff type) and subtract the LM score from the total score to get the 
AM score (after potentially weighting by the LM weight, although the LM 
score might already include that, I'm not sure)...

All in all, it's not that hard but I'm not even sure the overall AM 
score is used anywhere (I think it can be used when computing confidence 
with Helios but that depends on your model. E.g., the LetsGoPublic model 
does not use that feature).
The only question is: is there someone motivated enough to do this? ;)

antoine


Thomas Harris wrote:
> I have seen in the frame some statistics like #words>p where p is some 
> kind of confidence metric. It would seem that those statistics can 
> only be derived from word-level confidence scores in the recognizer. 
> But that's just my recollection right now. Someone should investigate, 
> I guess.
>
> Thanks,
> -Thomas
>
> On Tue, Apr 13, 2010 at 1:20 PM, Antoine Raux <antoine.raux at gmail.com 
> <mailto:antoine.raux at gmail.com>> wrote:
>
>     Hmmm... I don't know about word-level acoustic scores. I don't
>     remember ever using these. Are these actually stored in the frame
>     that PocketsphinxEngine and/or AudioServer send?
>
>     antoine
>
>     Thomas Harris wrote:
>
>         Good idea. That sounds doable. But we're also not getting
>         acoustic model scores. I don't know for sure, do latter stages
>         of Olympus (like Helios) depend on word-level acoustic scores?
>
>         -Thomas
>
>         On Tue, Apr 13, 2010 at 1:12 PM, Antoine Raux
>         <antoine.raux at gmail.com <mailto:antoine.raux at gmail.com>
>         <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com>>> wrote:
>
>            Actually, rather than a bunch of ifs as I wrote, the (still
>            temporary) solution is to get the ngram_model_t object from ps,
>            and then use the sphinxbase functions (such as
>         ngram_tg_score) to
>            compute the backoff type (which is exactly what ps does at
>            decoding time).
>
>            antoine
>
>            Thomas Harris wrote:
>
>                Hi Antoine,
>
>                Yes, that was/is a problem and I tried something like this.
>                But even more fundamental is the problem is that the
>         p_seg_t*
>                segment iterator that you get from pocketsphinx doesn't
>                correctly implement ps_seg_prob when the segment iterator
>                comes from the hypothesis iterator even though it works
>         fine
>                if you get the segment iterator from the best_hyp
>         function (or
>                whatever that's called). I've sent David the code
>         segment that
>                illustrates this bug. I don't know that there's any kind of
>                work around. For the most part we've gotten mutiple
>         hypotheses
>                by running multiple recognizers, I guess.
>
>                Thanks,
>                -Thomas
>
>                On Tue, Apr 13, 2010 at 11:58 AM, Antoine Raux
>                <antoine.raux at gmail.com <mailto:antoine.raux at gmail.com>
>         <mailto:antoine.raux at gmail.com <mailto:antoine.raux at gmail.com>>
>                <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com>
>                <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com>>>> wrote:
>
>                   Hi all,
>
>                   What exactly is the confidence computation problem?
>         Is it
>                that we
>                   cannot compute the LM backoff type-based word
>         confidence (see
>                   hyp_conf_slm in PocketsphinxEngine's main.cpp)?
>                   If that is the problem, one way to fix this might be
>         to modify
>                   hyp_conf_slm to accept a ps_seg_t as an argument
>         (instead of
>                   always getting seg_iter from ps_seg_iter):
>
>                   float* hyp_conf_slm (bool useFixedScore = false,
>         ps_seg_t
>                   *seg_iter = NULL)
>                   {
>                     const int MAX_TYPE_SIZE = 4096;
>                     int32 score, type[MAX_TYPE_SIZE];
>                     int32 k = 0;
>
>                     // (antoine) no seg_iter was given, get the top
>         segment
>                iterator
>                   from ps
>                     if (seg_iter == NULL)
>                         seg_iter = ps_seg_iter(psd, &score);
>
>                         type[k++] = 3;                      // use the
>                trigram dummy
>                   for first word
>
>                     if (seg_iter != NULL) {
>                         while (seg_iter = ps_seg_next(seg_iter)) {
>                             if (k == MAX_TYPE_SIZE) return NULL;
>
>                             int32 lscr, ascr;
>                             ps_seg_prob(seg_iter, &ascr, &lscr,
>         &type[k++]);
>                         }
>                     }
>                     type[k++] = 3; // (tk) dummy trigram after utterance
>                     type[k++] = 3; // (tk) sometimes there's no end
>         token, in
>                which case
>                                        // the list one was for the end
>         token and
>                   this one is the dummy
>
>                     // (antoine) allocate the array of confidence scores
>                     float* conf = (float*)malloc(k*sizeof(float));
>
>                     for (int32 i = 1; i < k-2; i++) {
>                         if(!useFixedScore) {
>                             int32 t = type[i-1] + type[i] + ((type[i+1] +
>                   type[i+2])<<1); // (tk) wtf?
>                             conf[i-1] = (float)((double)(t-6)/12.0);
>                         } else {
>                             conf[i-1] = 0.7f;
>                         }
>                     }
>
>                     return conf;
>                   }
>
>                   Then further down, you can modify the third version of
>                   fillPartialHypStruct by just adding the argument
>         when it calls
>                   hyp_conf_slm:
>
>                   // [2008-02-19] (antoine): this function takes a partial
>                   hypothesis and a reference to a
>                   //                        THypStruct and fills in
>         the hyp
>                struct
>                   void fillPartialHypStruct(ps_seg_t* curr_seg_iter,
>         THypStruct*
>                   phs, int fromNBest) {
>
>                     Log(STD_STREAM, "Filling partial hyp struct\n");
>
>                     size_t h_len, ch_len;
>                     int n_words = 0, n_validwords, has_oov;
>                     char tmp[16384];
>                     float *lm_conf = NULL;
>
>                     // Fill in confidence values for words in result
>         and build
>                   filtered hypothesis
>                     if (slm)
>                         lm_conf = hyp_conf_slm(curr_seg_iter);
>                     else
>                         lm_conf = hyp_conf_slm(curr_seg_iter, true);
>
>                   (...)
>
>                   I don't really have any setup to test this but if
>         someone
>                who has
>                   could give it a shot and post the result to the
>         mailing list...
>                   Now it might be that I misunderstood what the
>         problem was
>                   altogether (in which case I apologize for the spam)...
>
>                   On a side note, the big commented out block in
>                getHypStructs (as
>                   sent by Blaise) is from my Cactus code (which I had
>         sent to
>                Blaise
>                   as an example), so it's irrelevant to Olympus and
>         should be
>                   deleted (for clarity's sake).
>
>                   antoine
>
>                   Blaise Thomson wrote:
>
>                       Hi Thomas / Alan,
>
>                       I've now got some preliminary N-best list code
>         to work with
>                       PocketSphinx. With the help of  some example
>         code from
>                Antoine
>                       I've modified the pocketsphinx engine to produce a
>                1-best list
>                       for partial recognition results but an N-best
>         list upon
>                       completion. I've also modified the AudioServer to be
>                able to
>                       receive multiple N-best lists from each of the
>                recognizer (the
>                       number for each decoder specified by an optional
>         ":N" after
>                       the decoder definition in the config file). In case
>                this may
>                       be something you want to include in future
>         versions of
>                Olympus
>                       I've attached my modified files.
>
>                       Note, however, that the code still doesn't
>         produce any
>                       confidence score information for the N-best
>         list. For this
>                       reason we will still probably be unable to use
>         Olympus
>                for our
>                       version of the LetsGo! system. If the
>         PocketSphinx bugs you
>                       mentioned are fixed any time soon or if anyone finds
>                out how
>                       to get confidence scores with the N-best list
>         would you
>                please
>                       let us know?
>
>                       Many thanks,
>                       Blaise
>
>
>
>                       Thomas Harris wrote:
>
>                           Hi Blaise,
>
>                           Thanks for looking into this. I hope we can
>         include
>                your
>                           bugfixes. I've been looking into this as
>         well, and
>                there's
>                           a more fundamental issue. It seems like you
>         can't
>                get word
>                           confidence metrics from the PocketSphinx segment
>                iterators
>                           when you've gotten the sement iterators from
>         the n_best
>                           hypothisis iterator. It smells like a
>         PocketSphinx bug,
>                           but I haven't seen any reference
>         implementation of
>                           PocketSphinx that makes use of those confidence
>                metrics in
>                           an n_best setting, so I'm not sure that it
>         isn't a
>                problem
>                           with how the PocketSphinx api is used. Until
>         that
>                issue is
>                           resolved n_best lists won't work in Olympus,
>         too many
>                           downhill processes depend on those
>         confidence metrics.
>
>                           Thanks,
>                           -Thomas
>
>                           On Wed, Mar 24, 2010 at 4:39 AM, Blaise Thomson
>                           <brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>         <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>
>                <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>         <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>>
>                           <mailto:brmt2 at cam.ac.uk
>         <mailto:brmt2 at cam.ac.uk> <mailto:brmt2 at cam.ac.uk
>         <mailto:brmt2 at cam.ac.uk>>
>                <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>         <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>>>> wrote:
>
>                              Dear Olympus developers,
>
>                              I am trying to get the Olympus LetsGo!
>         system to
>                           provide an N-best
>                              list of speech recognition hypotheses. I
>         found the
>                           -n_best switch
>                              which can be passed to the PocketSphinxEngine
>                which is
>                           supposed to
>                              enable this but when I set the switch to
>                anything other
>                           than 0 the
>                              system crashes immediately on any audio
>         input. I
>                           remember you said
>                              that the system had been build to provide
>         N-best
>                lists
>                           so I was
>                              wondering if you could give any advice on
>         why it
>                is not
>                           working.
>                              Do you have a working N-best list system that
>                you could
>                           send me to
>                              see how things are configured?
>
>                              In trying to solve the problem I took a
>         look at the
>                              PocketSphinxEngine source code and have
>         noticed some
>                           possible
>                              memory access bugs which may be
>         contributing to
>                this.
>                           These were
>                              related to the way the iHypsGenerated
>         variable was
>                           used. I've
>                              fixed these and can send them if you
>         would like (I
>                           tried attaching
>                              them but the mailing list won't let me). The
>                resulting
>                           code still
>                              crashes but at a later stage. After the
>         fix, the
>                log file
>                              generates a WARNING: "ngram_search.c",
>         line 1000:. I
>                           don't know if
>                              this might be the cause of the problem.
>         There is
>                also a
>                              possibility that I simply have to add a
>                configuration
>                           variable to
>                              PocketSphinx itself. At the moment I have
>         only
>                used the
>                           n_best
>                              switch on PocketSphinxEngine.
>
>                              Please do let me know if you have any
>         ideas of
>                how to
>                           get this
>                              working or who else to contact.
>
>                              Thanks for all you help,
>
>                              Blaise
>
>
>
>
>
>
>
>
>
>


From tkharris at gmail.com  Tue Apr 13 16:53:06 2010
From: tkharris at gmail.com (Thomas Harris)
Date: Tue, 13 Apr 2010 16:53:06 -0400
Subject: [Olympus developers 226]: Re: N-best lists for PocketSphinx /
	Olympus
In-Reply-To: <4BC4D6ED.6050504@gmail.com>
References: <4BA9CFD6.1040804@cam.ac.uk>
	<76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>
	<4BC48EDE.9090001@cam.ac.uk> <4BC49496.2020208@gmail.com>
	<l2z76c228b1004130937hf3804b7h32021c3ae4ee9a3f@mail.gmail.com>
	<4BC4A5F6.2030201@gmail.com>
	<w2k76c228b1004131018o6fea570fi4f3e9ca8643f2445@mail.gmail.com>
	<4BC4A7CC.8020908@gmail.com>
	<4437_1271179884_o3DHVNwd020691_o2i76c228b1004131031z3f242007q4896ed7837853930@mail.gmail.com>
	<4BC4D6ED.6050504@gmail.com>
Message-ID: <l2l76c228b1004131353g8e528e73hef43dfa1f03fdc18@mail.gmail.com>

These fixes, as described, could be applied directly to the PocketSphinx
API, and could be a benefit to the larger community of PocketSphinx users,
so I think that the issue of motivation could be raised there as well. This
is especially the case since it's unclear whether the current PocketSphinx
API accurately reflects the actually functionality of the system.

Thanks,
-Thomas

On Tue, Apr 13, 2010 at 4:41 PM, Antoine Raux <antoine.raux at gmail.com>wrote:

> As far as I can tell by looking at the code, PocketsphinxEngine does not
> send any word-level confidence information to AudioServer. That's good news.
> The bad news is that it computes utterance-level AM and LM scores by summing
> those of the words (so it needs word-level scores for that...). This seems
> the only way to get separate AM and LM score from Pocketsphinx (someone,
> correct me if I'm wrong). You can get the overall score though (the sum of
> the the AM and LM scores), using ps_nbest_hyp. So we could recompute the LM
> score of the i-th hypothesis using the Sphinxbase's ngram_tg_score routine
> (at the same time as getting the LM backoff type) and subtract the LM score
> from the total score to get the AM score (after potentially weighting by the
> LM weight, although the LM score might already include that, I'm not
> sure)...
>
> All in all, it's not that hard but I'm not even sure the overall AM score
> is used anywhere (I think it can be used when computing confidence with
> Helios but that depends on your model. E.g., the LetsGoPublic model does not
> use that feature).
> The only question is: is there someone motivated enough to do this? ;)
>
> antoine
>
>
> Thomas Harris wrote:
>
>> I have seen in the frame some statistics like #words>p where p is some
>> kind of confidence metric. It would seem that those statistics can only be
>> derived from word-level confidence scores in the recognizer. But that's just
>> my recollection right now. Someone should investigate, I guess.
>>
>> Thanks,
>> -Thomas
>>
>>
>> On Tue, Apr 13, 2010 at 1:20 PM, Antoine Raux <antoine.raux at gmail.com<mailto:
>> antoine.raux at gmail.com>> wrote:
>>
>>    Hmmm... I don't know about word-level acoustic scores. I don't
>>    remember ever using these. Are these actually stored in the frame
>>    that PocketsphinxEngine and/or AudioServer send?
>>
>>    antoine
>>
>>    Thomas Harris wrote:
>>
>>        Good idea. That sounds doable. But we're also not getting
>>        acoustic model scores. I don't know for sure, do latter stages
>>        of Olympus (like Helios) depend on word-level acoustic scores?
>>
>>        -Thomas
>>
>>        On Tue, Apr 13, 2010 at 1:12 PM, Antoine Raux
>>        <antoine.raux at gmail.com <mailto:antoine.raux at gmail.com>
>>        <mailto:antoine.raux at gmail.com
>>        <mailto:antoine.raux at gmail.com>>> wrote:
>>
>>           Actually, rather than a bunch of ifs as I wrote, the (still
>>           temporary) solution is to get the ngram_model_t object from ps,
>>           and then use the sphinxbase functions (such as
>>        ngram_tg_score) to
>>           compute the backoff type (which is exactly what ps does at
>>           decoding time).
>>
>>           antoine
>>
>>           Thomas Harris wrote:
>>
>>               Hi Antoine,
>>
>>               Yes, that was/is a problem and I tried something like this.
>>               But even more fundamental is the problem is that the
>>        p_seg_t*
>>               segment iterator that you get from pocketsphinx doesn't
>>               correctly implement ps_seg_prob when the segment iterator
>>               comes from the hypothesis iterator even though it works
>>        fine
>>               if you get the segment iterator from the best_hyp
>>        function (or
>>               whatever that's called). I've sent David the code
>>        segment that
>>               illustrates this bug. I don't know that there's any kind of
>>               work around. For the most part we've gotten mutiple
>>        hypotheses
>>               by running multiple recognizers, I guess.
>>
>>               Thanks,
>>               -Thomas
>>
>>               On Tue, Apr 13, 2010 at 11:58 AM, Antoine Raux
>>               <antoine.raux at gmail.com <mailto:antoine.raux at gmail.com>
>>        <mailto:antoine.raux at gmail.com <mailto:antoine.raux at gmail.com>>
>>               <mailto:antoine.raux at gmail.com
>>        <mailto:antoine.raux at gmail.com>
>>               <mailto:antoine.raux at gmail.com
>>        <mailto:antoine.raux at gmail.com>>>> wrote:
>>
>>                  Hi all,
>>
>>                  What exactly is the confidence computation problem?
>>        Is it
>>               that we
>>                  cannot compute the LM backoff type-based word
>>        confidence (see
>>                  hyp_conf_slm in PocketsphinxEngine's main.cpp)?
>>                  If that is the problem, one way to fix this might be
>>        to modify
>>                  hyp_conf_slm to accept a ps_seg_t as an argument
>>        (instead of
>>                  always getting seg_iter from ps_seg_iter):
>>
>>                  float* hyp_conf_slm (bool useFixedScore = false,
>>        ps_seg_t
>>                  *seg_iter = NULL)
>>                  {
>>                    const int MAX_TYPE_SIZE = 4096;
>>                    int32 score, type[MAX_TYPE_SIZE];
>>                    int32 k = 0;
>>
>>                    // (antoine) no seg_iter was given, get the top
>>        segment
>>               iterator
>>                  from ps
>>                    if (seg_iter == NULL)
>>                        seg_iter = ps_seg_iter(psd, &score);
>>
>>                        type[k++] = 3;                      // use the
>>               trigram dummy
>>                  for first word
>>
>>                    if (seg_iter != NULL) {
>>                        while (seg_iter = ps_seg_next(seg_iter)) {
>>                            if (k == MAX_TYPE_SIZE) return NULL;
>>
>>                            int32 lscr, ascr;
>>                            ps_seg_prob(seg_iter, &ascr, &lscr,
>>        &type[k++]);
>>                        }
>>                    }
>>                    type[k++] = 3; // (tk) dummy trigram after utterance
>>                    type[k++] = 3; // (tk) sometimes there's no end
>>        token, in
>>               which case
>>                                       // the list one was for the end
>>        token and
>>                  this one is the dummy
>>
>>                    // (antoine) allocate the array of confidence scores
>>                    float* conf = (float*)malloc(k*sizeof(float));
>>
>>                    for (int32 i = 1; i < k-2; i++) {
>>                        if(!useFixedScore) {
>>                            int32 t = type[i-1] + type[i] + ((type[i+1] +
>>                  type[i+2])<<1); // (tk) wtf?
>>                            conf[i-1] = (float)((double)(t-6)/12.0);
>>                        } else {
>>                            conf[i-1] = 0.7f;
>>                        }
>>                    }
>>
>>                    return conf;
>>                  }
>>
>>                  Then further down, you can modify the third version of
>>                  fillPartialHypStruct by just adding the argument
>>        when it calls
>>                  hyp_conf_slm:
>>
>>                  // [2008-02-19] (antoine): this function takes a partial
>>                  hypothesis and a reference to a
>>                  //                        THypStruct and fills in
>>        the hyp
>>               struct
>>                  void fillPartialHypStruct(ps_seg_t* curr_seg_iter,
>>        THypStruct*
>>                  phs, int fromNBest) {
>>
>>                    Log(STD_STREAM, "Filling partial hyp struct\n");
>>
>>                    size_t h_len, ch_len;
>>                    int n_words = 0, n_validwords, has_oov;
>>                    char tmp[16384];
>>                    float *lm_conf = NULL;
>>
>>                    // Fill in confidence values for words in result
>>        and build
>>                  filtered hypothesis
>>                    if (slm)
>>                        lm_conf = hyp_conf_slm(curr_seg_iter);
>>                    else
>>                        lm_conf = hyp_conf_slm(curr_seg_iter, true);
>>
>>                  (...)
>>
>>                  I don't really have any setup to test this but if
>>        someone
>>               who has
>>                  could give it a shot and post the result to the
>>        mailing list...
>>                  Now it might be that I misunderstood what the
>>        problem was
>>                  altogether (in which case I apologize for the spam)...
>>
>>                  On a side note, the big commented out block in
>>               getHypStructs (as
>>                  sent by Blaise) is from my Cactus code (which I had
>>        sent to
>>               Blaise
>>                  as an example), so it's irrelevant to Olympus and
>>        should be
>>                  deleted (for clarity's sake).
>>
>>                  antoine
>>
>>                  Blaise Thomson wrote:
>>
>>                      Hi Thomas / Alan,
>>
>>                      I've now got some preliminary N-best list code
>>        to work with
>>                      PocketSphinx. With the help of  some example
>>        code from
>>               Antoine
>>                      I've modified the pocketsphinx engine to produce a
>>               1-best list
>>                      for partial recognition results but an N-best
>>        list upon
>>                      completion. I've also modified the AudioServer to be
>>               able to
>>                      receive multiple N-best lists from each of the
>>               recognizer (the
>>                      number for each decoder specified by an optional
>>        ":N" after
>>                      the decoder definition in the config file). In case
>>               this may
>>                      be something you want to include in future
>>        versions of
>>               Olympus
>>                      I've attached my modified files.
>>
>>                      Note, however, that the code still doesn't
>>        produce any
>>                      confidence score information for the N-best
>>        list. For this
>>                      reason we will still probably be unable to use
>>        Olympus
>>               for our
>>                      version of the LetsGo! system. If the
>>        PocketSphinx bugs you
>>                      mentioned are fixed any time soon or if anyone finds
>>               out how
>>                      to get confidence scores with the N-best list
>>        would you
>>               please
>>                      let us know?
>>
>>                      Many thanks,
>>                      Blaise
>>
>>
>>
>>                      Thomas Harris wrote:
>>
>>                          Hi Blaise,
>>
>>                          Thanks for looking into this. I hope we can
>>        include
>>               your
>>                          bugfixes. I've been looking into this as
>>        well, and
>>               there's
>>                          a more fundamental issue. It seems like you
>>        can't
>>               get word
>>                          confidence metrics from the PocketSphinx segment
>>               iterators
>>                          when you've gotten the sement iterators from
>>        the n_best
>>                          hypothisis iterator. It smells like a
>>        PocketSphinx bug,
>>                          but I haven't seen any reference
>>        implementation of
>>                          PocketSphinx that makes use of those confidence
>>               metrics in
>>                          an n_best setting, so I'm not sure that it
>>        isn't a
>>               problem
>>                          with how the PocketSphinx api is used. Until
>>        that
>>               issue is
>>                          resolved n_best lists won't work in Olympus,
>>        too many
>>                          downhill processes depend on those
>>        confidence metrics.
>>
>>                          Thanks,
>>                          -Thomas
>>
>>                          On Wed, Mar 24, 2010 at 4:39 AM, Blaise Thomson
>>                          <brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>>        <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>
>>               <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>>        <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>>
>>                          <mailto:brmt2 at cam.ac.uk
>>        <mailto:brmt2 at cam.ac.uk> <mailto:brmt2 at cam.ac.uk
>>        <mailto:brmt2 at cam.ac.uk>>
>>               <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>>        <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>>>> wrote:
>>
>>                             Dear Olympus developers,
>>
>>                             I am trying to get the Olympus LetsGo!
>>        system to
>>                          provide an N-best
>>                             list of speech recognition hypotheses. I
>>        found the
>>                          -n_best switch
>>                             which can be passed to the PocketSphinxEngine
>>               which is
>>                          supposed to
>>                             enable this but when I set the switch to
>>               anything other
>>                          than 0 the
>>                             system crashes immediately on any audio
>>        input. I
>>                          remember you said
>>                             that the system had been build to provide
>>        N-best
>>               lists
>>                          so I was
>>                             wondering if you could give any advice on
>>        why it
>>               is not
>>                          working.
>>                             Do you have a working N-best list system that
>>               you could
>>                          send me to
>>                             see how things are configured?
>>
>>                             In trying to solve the problem I took a
>>        look at the
>>                             PocketSphinxEngine source code and have
>>        noticed some
>>                          possible
>>                             memory access bugs which may be
>>        contributing to
>>               this.
>>                          These were
>>                             related to the way the iHypsGenerated
>>        variable was
>>                          used. I've
>>                             fixed these and can send them if you
>>        would like (I
>>                          tried attaching
>>                             them but the mailing list won't let me). The
>>               resulting
>>                          code still
>>                             crashes but at a later stage. After the
>>        fix, the
>>               log file
>>                             generates a WARNING: "ngram_search.c",
>>        line 1000:. I
>>                          don't know if
>>                             this might be the cause of the problem.
>>        There is
>>               also a
>>                             possibility that I simply have to add a
>>               configuration
>>                          variable to
>>                             PocketSphinx itself. At the moment I have
>>        only
>>               used the
>>                          n_best
>>                             switch on PocketSphinxEngine.
>>
>>                             Please do let me know if you have any
>>        ideas of
>>               how to
>>                          get this
>>                             working or who else to contact.
>>
>>                             Thanks for all you help,
>>
>>                             Blaise
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.srv.cs.cmu.edu/pipermail/olympus-developers/attachments/20100413/d415bbaf/attachment-0001.html

From antoine.raux at gmail.com  Tue Apr 13 19:42:04 2010
From: antoine.raux at gmail.com (Antoine Raux)
Date: Tue, 13 Apr 2010 16:42:04 -0700
Subject: [Olympus developers 227]: Re: N-best lists for PocketSphinx /
 Olympus
In-Reply-To: <l2l76c228b1004131353g8e528e73hef43dfa1f03fdc18@mail.gmail.com>
References: <4BA9CFD6.1040804@cam.ac.uk>	
	<76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>	
	<4BC48EDE.9090001@cam.ac.uk> <4BC49496.2020208@gmail.com>	
	<l2z76c228b1004130937hf3804b7h32021c3ae4ee9a3f@mail.gmail.com>	
	<4BC4A5F6.2030201@gmail.com>	
	<w2k76c228b1004131018o6fea570fi4f3e9ca8643f2445@mail.gmail.com>	
	<4BC4A7CC.8020908@gmail.com>	
	<4437_1271179884_o3DHVNwd020691_o2i76c228b1004131031z3f242007q4896ed7837853930@mail.gmail.com>	
	<4BC4D6ED.6050504@gmail.com>
	<l2l76c228b1004131353g8e528e73hef43dfa1f03fdc18@mail.gmail.com>
Message-ID: <4BC5014C.7030009@gmail.com>

Point taken. I posted a message about this on the Sphinx Help forum...

antoine

Thomas Harris wrote:
> These fixes, as described, could be applied directly to the 
> PocketSphinx API, and could be a benefit to the larger community of 
> PocketSphinx users, so I think that the issue of motivation could be 
> raised there as well. This is especially the case since it's unclear 
> whether the current PocketSphinx API accurately reflects the actually 
> functionality of the system.
>
> Thanks,
> -Thomas
>
> On Tue, Apr 13, 2010 at 4:41 PM, Antoine Raux <antoine.raux at gmail.com 
> <mailto:antoine.raux at gmail.com>> wrote:
>
>     As far as I can tell by looking at the code, PocketsphinxEngine
>     does not send any word-level confidence information to
>     AudioServer. That's good news. The bad news is that it computes
>     utterance-level AM and LM scores by summing those of the words (so
>     it needs word-level scores for that...). This seems the only way
>     to get separate AM and LM score from Pocketsphinx (someone,
>     correct me if I'm wrong). You can get the overall score though
>     (the sum of the the AM and LM scores), using ps_nbest_hyp. So we
>     could recompute the LM score of the i-th hypothesis using the
>     Sphinxbase's ngram_tg_score routine (at the same time as getting
>     the LM backoff type) and subtract the LM score from the total
>     score to get the AM score (after potentially weighting by the LM
>     weight, although the LM score might already include that, I'm not
>     sure)...
>
>     All in all, it's not that hard but I'm not even sure the overall
>     AM score is used anywhere (I think it can be used when computing
>     confidence with Helios but that depends on your model. E.g., the
>     LetsGoPublic model does not use that feature).
>     The only question is: is there someone motivated enough to do this? ;)
>
>     antoine
>
>
>     Thomas Harris wrote:
>
>         I have seen in the frame some statistics like #words>p where p
>         is some kind of confidence metric. It would seem that those
>         statistics can only be derived from word-level confidence
>         scores in the recognizer. But that's just my recollection
>         right now. Someone should investigate, I guess.
>
>         Thanks,
>         -Thomas
>
>
>         On Tue, Apr 13, 2010 at 1:20 PM, Antoine Raux
>         <antoine.raux at gmail.com <mailto:antoine.raux at gmail.com>
>         <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com>>> wrote:
>
>            Hmmm... I don't know about word-level acoustic scores. I don't
>            remember ever using these. Are these actually stored in the
>         frame
>            that PocketsphinxEngine and/or AudioServer send?
>
>            antoine
>
>            Thomas Harris wrote:
>
>                Good idea. That sounds doable. But we're also not getting
>                acoustic model scores. I don't know for sure, do latter
>         stages
>                of Olympus (like Helios) depend on word-level acoustic
>         scores?
>
>                -Thomas
>
>                On Tue, Apr 13, 2010 at 1:12 PM, Antoine Raux
>                <antoine.raux at gmail.com <mailto:antoine.raux at gmail.com>
>         <mailto:antoine.raux at gmail.com <mailto:antoine.raux at gmail.com>>
>                <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com>
>                <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com>>>> wrote:
>
>                   Actually, rather than a bunch of ifs as I wrote, the
>         (still
>                   temporary) solution is to get the ngram_model_t
>         object from ps,
>                   and then use the sphinxbase functions (such as
>                ngram_tg_score) to
>                   compute the backoff type (which is exactly what ps
>         does at
>                   decoding time).
>
>                   antoine
>
>                   Thomas Harris wrote:
>
>                       Hi Antoine,
>
>                       Yes, that was/is a problem and I tried something
>         like this.
>                       But even more fundamental is the problem is that the
>                p_seg_t*
>                       segment iterator that you get from pocketsphinx
>         doesn't
>                       correctly implement ps_seg_prob when the segment
>         iterator
>                       comes from the hypothesis iterator even though
>         it works
>                fine
>                       if you get the segment iterator from the best_hyp
>                function (or
>                       whatever that's called). I've sent David the code
>                segment that
>                       illustrates this bug. I don't know that there's
>         any kind of
>                       work around. For the most part we've gotten mutiple
>                hypotheses
>                       by running multiple recognizers, I guess.
>
>                       Thanks,
>                       -Thomas
>
>                       On Tue, Apr 13, 2010 at 11:58 AM, Antoine Raux
>                       <antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com> <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com>>
>                <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com> <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com>>>
>                       <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com>
>                <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com>>
>                       <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com>
>                <mailto:antoine.raux at gmail.com
>         <mailto:antoine.raux at gmail.com>>>>> wrote:
>
>                          Hi all,
>
>                          What exactly is the confidence computation
>         problem?
>                Is it
>                       that we
>                          cannot compute the LM backoff type-based word
>                confidence (see
>                          hyp_conf_slm in PocketsphinxEngine's main.cpp)?
>                          If that is the problem, one way to fix this
>         might be
>                to modify
>                          hyp_conf_slm to accept a ps_seg_t as an argument
>                (instead of
>                          always getting seg_iter from ps_seg_iter):
>
>                          float* hyp_conf_slm (bool useFixedScore = false,
>                ps_seg_t
>                          *seg_iter = NULL)
>                          {
>                            const int MAX_TYPE_SIZE = 4096;
>                            int32 score, type[MAX_TYPE_SIZE];
>                            int32 k = 0;
>
>                            // (antoine) no seg_iter was given, get the top
>                segment
>                       iterator
>                          from ps
>                            if (seg_iter == NULL)
>                                seg_iter = ps_seg_iter(psd, &score);
>
>                                type[k++] = 3;                      //
>         use the
>                       trigram dummy
>                          for first word
>
>                            if (seg_iter != NULL) {
>                                while (seg_iter = ps_seg_next(seg_iter)) {
>                                    if (k == MAX_TYPE_SIZE) return NULL;
>
>                                    int32 lscr, ascr;
>                                    ps_seg_prob(seg_iter, &ascr, &lscr,
>                &type[k++]);
>                                }
>                            }
>                            type[k++] = 3; // (tk) dummy trigram after
>         utterance
>                            type[k++] = 3; // (tk) sometimes there's no end
>                token, in
>                       which case
>                                               // the list one was for
>         the end
>                token and
>                          this one is the dummy
>
>                            // (antoine) allocate the array of
>         confidence scores
>                            float* conf = (float*)malloc(k*sizeof(float));
>
>                            for (int32 i = 1; i < k-2; i++) {
>                                if(!useFixedScore) {
>                                    int32 t = type[i-1] + type[i] +
>         ((type[i+1] +
>                          type[i+2])<<1); // (tk) wtf?
>                                    conf[i-1] =
>         (float)((double)(t-6)/12.0);
>                                } else {
>                                    conf[i-1] = 0.7f;
>                                }
>                            }
>
>                            return conf;
>                          }
>
>                          Then further down, you can modify the third
>         version of
>                          fillPartialHypStruct by just adding the argument
>                when it calls
>                          hyp_conf_slm:
>
>                          // [2008-02-19] (antoine): this function
>         takes a partial
>                          hypothesis and a reference to a
>                          //                        THypStruct and fills in
>                the hyp
>                       struct
>                          void fillPartialHypStruct(ps_seg_t*
>         curr_seg_iter,
>                THypStruct*
>                          phs, int fromNBest) {
>
>                            Log(STD_STREAM, "Filling partial hyp
>         struct\n");
>
>                            size_t h_len, ch_len;
>                            int n_words = 0, n_validwords, has_oov;
>                            char tmp[16384];
>                            float *lm_conf = NULL;
>
>                            // Fill in confidence values for words in
>         result
>                and build
>                          filtered hypothesis
>                            if (slm)
>                                lm_conf = hyp_conf_slm(curr_seg_iter);
>                            else
>                                lm_conf = hyp_conf_slm(curr_seg_iter,
>         true);
>
>                          (...)
>
>                          I don't really have any setup to test this but if
>                someone
>                       who has
>                          could give it a shot and post the result to the
>                mailing list...
>                          Now it might be that I misunderstood what the
>                problem was
>                          altogether (in which case I apologize for the
>         spam)...
>
>                          On a side note, the big commented out block in
>                       getHypStructs (as
>                          sent by Blaise) is from my Cactus code (which
>         I had
>                sent to
>                       Blaise
>                          as an example), so it's irrelevant to Olympus and
>                should be
>                          deleted (for clarity's sake).
>
>                          antoine
>
>                          Blaise Thomson wrote:
>
>                              Hi Thomas / Alan,
>
>                              I've now got some preliminary N-best list
>         code
>                to work with
>                              PocketSphinx. With the help of  some example
>                code from
>                       Antoine
>                              I've modified the pocketsphinx engine to
>         produce a
>                       1-best list
>                              for partial recognition results but an N-best
>                list upon
>                              completion. I've also modified the
>         AudioServer to be
>                       able to
>                              receive multiple N-best lists from each
>         of the
>                       recognizer (the
>                              number for each decoder specified by an
>         optional
>                ":N" after
>                              the decoder definition in the config
>         file). In case
>                       this may
>                              be something you want to include in future
>                versions of
>                       Olympus
>                              I've attached my modified files.
>
>                              Note, however, that the code still doesn't
>                produce any
>                              confidence score information for the N-best
>                list. For this
>                              reason we will still probably be unable
>         to use
>                Olympus
>                       for our
>                              version of the LetsGo! system. If the
>                PocketSphinx bugs you
>                              mentioned are fixed any time soon or if
>         anyone finds
>                       out how
>                              to get confidence scores with the N-best list
>                would you
>                       please
>                              let us know?
>
>                              Many thanks,
>                              Blaise
>
>
>
>                              Thomas Harris wrote:
>
>                                  Hi Blaise,
>
>                                  Thanks for looking into this. I hope
>         we can
>                include
>                       your
>                                  bugfixes. I've been looking into this as
>                well, and
>                       there's
>                                  a more fundamental issue. It seems
>         like you
>                can't
>                       get word
>                                  confidence metrics from the
>         PocketSphinx segment
>                       iterators
>                                  when you've gotten the sement
>         iterators from
>                the n_best
>                                  hypothisis iterator. It smells like a
>                PocketSphinx bug,
>                                  but I haven't seen any reference
>                implementation of
>                                  PocketSphinx that makes use of those
>         confidence
>                       metrics in
>                                  an n_best setting, so I'm not sure
>         that it
>                isn't a
>                       problem
>                                  with how the PocketSphinx api is
>         used. Until
>                that
>                       issue is
>                                  resolved n_best lists won't work in
>         Olympus,
>                too many
>                                  downhill processes depend on those
>                confidence metrics.
>
>                                  Thanks,
>                                  -Thomas
>
>                                  On Wed, Mar 24, 2010 at 4:39 AM,
>         Blaise Thomson
>                                  <brmt2 at cam.ac.uk
>         <mailto:brmt2 at cam.ac.uk> <mailto:brmt2 at cam.ac.uk
>         <mailto:brmt2 at cam.ac.uk>>
>                <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>         <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>>
>                       <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>         <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>
>                <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>         <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>>>
>                                  <mailto:brmt2 at cam.ac.uk
>         <mailto:brmt2 at cam.ac.uk>
>                <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>
>         <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>                <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>>
>                       <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>         <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>
>                <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>
>         <mailto:brmt2 at cam.ac.uk <mailto:brmt2 at cam.ac.uk>>>>>> wrote:
>
>                                     Dear Olympus developers,
>
>                                     I am trying to get the Olympus LetsGo!
>                system to
>                                  provide an N-best
>                                     list of speech recognition
>         hypotheses. I
>                found the
>                                  -n_best switch
>                                     which can be passed to the
>         PocketSphinxEngine
>                       which is
>                                  supposed to
>                                     enable this but when I set the
>         switch to
>                       anything other
>                                  than 0 the
>                                     system crashes immediately on any
>         audio
>                input. I
>                                  remember you said
>                                     that the system had been build to
>         provide
>                N-best
>                       lists
>                                  so I was
>                                     wondering if you could give any
>         advice on
>                why it
>                       is not
>                                  working.
>                                     Do you have a working N-best list
>         system that
>                       you could
>                                  send me to
>                                     see how things are configured?
>
>                                     In trying to solve the problem I
>         took a
>                look at the
>                                     PocketSphinxEngine source code and
>         have
>                noticed some
>                                  possible
>                                     memory access bugs which may be
>                contributing to
>                       this.
>                                  These were
>                                     related to the way the iHypsGenerated
>                variable was
>                                  used. I've
>                                     fixed these and can send them if you
>                would like (I
>                                  tried attaching
>                                     them but the mailing list won't
>         let me). The
>                       resulting
>                                  code still
>                                     crashes but at a later stage.
>         After the
>                fix, the
>                       log file
>                                     generates a WARNING: "ngram_search.c",
>                line 1000:. I
>                                  don't know if
>                                     this might be the cause of the
>         problem.
>                There is
>                       also a
>                                     possibility that I simply have to
>         add a
>                       configuration
>                                  variable to
>                                     PocketSphinx itself. At the moment
>         I have
>                only
>                       used the
>                                  n_best
>                                     switch on PocketSphinxEngine.
>
>                                     Please do let me know if you have any
>                ideas of
>                       how to
>                                  get this
>                                     working or who else to contact.
>
>                                     Thanks for all you help,
>
>                                     Blaise
>
>
>
>
>
>
>
>
>
>
>
>


From LA0001BO at e.ntu.edu.sg  Wed Apr 14 01:31:31 2010
From: LA0001BO at e.ntu.edu.sg (#LAI DANBO#)
Date: Wed, 14 Apr 2010 05:31:31 +0000
Subject: [Olympus developers 228]:  problem of building
Message-ID: <7AF24D1F33D31B49925319EDE39D2D2E105ED4F3@SINPRD0103MB037.apcprd01.prod.exchangelabs.com>

Hi,

I download your zip archive of source code for Olympus from your website.
But, when I run the perl script  OlympusBuild.pl     it said: Can't locate Tools\logios\scripts\Logios.pm
And I serched the whole files that I download, there is no Logios.pm

Could you kindly tell me what shall I do to build it successfully?
Thank you very much.

Best regards,
Lai Danbo

School of Computer Engineering
Nanyang Technological University, Singapore
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.srv.cs.cmu.edu/pipermail/olympus-developers/attachments/20100414/51dcec4c/attachment.html

From tkharris at gmail.com  Wed Apr 14 08:21:26 2010
From: tkharris at gmail.com (Thomas Harris)
Date: Wed, 14 Apr 2010 08:21:26 -0400
Subject: [Olympus developers 229]: Re: problem of building
In-Reply-To: <7AF24D1F33D31B49925319EDE39D2D2E105ED4F3@SINPRD0103MB037.apcprd01.prod.exchangelabs.com>
References: <7AF24D1F33D31B49925319EDE39D2D2E105ED4F3@SINPRD0103MB037.apcprd01.prod.exchangelabs.com>
Message-ID: <y2x76c228b1004140521m195ce821q7d8d3da2a638ca95@mail.gmail.com>

I see that's a problem with the trac zip archives. They apparently don't
include externals, which makes sense. There are 3 external sources: logios,
sphinxbase, and pocketsphinx. If you download using a subversion update then
you get them automatically as subversion externals.

All three of these externals come from the sphinx repository at sourceforge.
Logios goes in the Tools directory and Sphinxbase and Pocketsphinx go in the
Libraries directory.

-Thomas

2010/4/14 #LAI DANBO# <LA0001BO at e.ntu.edu.sg>

>  Hi,
>
> I download your zip archive of source code for Olympus from your website.
> But, when I run the perl script  OlympusBuild.pl     it said: Can't locate
> Tools\logios\scripts\Logios.pm
> And I serched the whole files that I download, there is no Logios.pm
>
> Could you kindly tell me what shall I do to build it successfully?
> Thank you very much.
>
> Best regards,
> Lai Danbo
>
> School of Computer Engineering
> Nanyang Technological University, Singapore
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.srv.cs.cmu.edu/pipermail/olympus-developers/attachments/20100414/bb2f5fad/attachment-0001.html

From bfrisch at cs.cmu.edu  Sun Apr 18 18:39:58 2010
From: bfrisch at cs.cmu.edu (Benjamin Frisch)
Date: Sun, 18 Apr 2010 17:39:58 -0500
Subject: [Olympus developers 230]:  Short speech.cs Web Server Downtime
Message-ID: <n2sbd80926e1004181539o559151b4m1e8c8c3dea87686d@mail.gmail.com>

Hi All,
  I wanted to let you know that www.speech.cs.cmu.edu and associated web
services like trac.speech and wiki.speech are going through a short 30
minute to 1 hour downtime for feature and security upgrades to almost all of
the services.

Thanks,
Ben Frisch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.srv.cs.cmu.edu/pipermail/olympus-developers/attachments/20100418/d5f3bc05/attachment.html

From brmt2 at cam.ac.uk  Tue Apr 13 11:35:39 2010
From: brmt2 at cam.ac.uk (Blaise Thomson)
Date: Tue, 13 Apr 2010 15:35:39 -0000
Subject: [Olympus developers 295]: Re: N-best lists for PocketSphinx /
	Olympus
In-Reply-To: <76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>
References: <4BA9CFD6.1040804@cam.ac.uk>
	<76c228b1003241004i62f6b7bbnd105faeb0e7aea0f@mail.gmail.com>
Message-ID: <4BC48EDE.9090001@cam.ac.uk>

Hi Thomas / Alan,

I've now got some preliminary N-best list code to work with 
PocketSphinx. With the help of  some example code from Antoine I've 
modified the pocketsphinx engine to produce a 1-best list for partial 
recognition results but an N-best list upon completion. I've also 
modified the AudioServer to be able to receive multiple N-best lists 
from each of the recognizer (the number for each decoder specified by an 
optional ":N" after the decoder definition in the config file). In case 
this may be something you want to include in future versions of Olympus 
I've attached my modified files.

Note, however, that the code still doesn't produce any confidence score 
information for the N-best list. For this reason we will still probably 
be unable to use Olympus for our version of the LetsGo! system. If the 
PocketSphinx bugs you mentioned are fixed any time soon or if anyone 
finds out how to get confidence scores with the N-best list would you 
please let us know?

Many thanks,
Blaise



Thomas Harris wrote:
> Hi Blaise,
>
> Thanks for looking into this. I hope we can include your bugfixes. 
> I've been looking into this as well, and there's a more fundamental 
> issue. It seems like you can't get word confidence metrics from the 
> PocketSphinx segment iterators when you've gotten the sement iterators 
> from the n_best hypothisis iterator. It smells like a PocketSphinx 
> bug, but I haven't seen any reference implementation of PocketSphinx 
> that makes use of those confidence metrics in an n_best setting, so 
> I'm not sure that it isn't a problem with how the PocketSphinx api is 
> used. Until that issue is resolved n_best lists won't work in Olympus, 
> too many downhill processes depend on those confidence metrics.
>
> Thanks,
> -Thomas
>
> On Wed, Mar 24, 2010 at 4:39 AM, Blaise Thomson <brmt2 at cam.ac.uk 
> <mailto:brmt2 at cam.ac.uk>> wrote:
>
>     Dear Olympus developers,
>
>     I am trying to get the Olympus LetsGo! system to provide an N-best
>     list of speech recognition hypotheses. I found the -n_best switch
>     which can be passed to the PocketSphinxEngine which is supposed to
>     enable this but when I set the switch to anything other than 0 the
>     system crashes immediately on any audio input. I remember you said
>     that the system had been build to provide N-best lists so I was
>     wondering if you could give any advice on why it is not working.
>     Do you have a working N-best list system that you could send me to
>     see how things are configured?
>
>     In trying to solve the problem I took a look at the
>     PocketSphinxEngine source code and have noticed some possible
>     memory access bugs which may be contributing to this. These were
>     related to the way the iHypsGenerated variable was used. I've
>     fixed these and can send them if you would like (I tried attaching
>     them but the mailing list won't let me). The resulting code still
>     crashes but at a later stage. After the fix, the log file
>     generates a WARNING: "ngram_search.c", line 1000:. I don't know if
>     this might be the cause of the problem. There is also a
>     possibility that I simply have to add a configuration variable to
>     PocketSphinx itself. At the moment I have only used the n_best
>     switch on PocketSphinxEngine.
>
>     Please do let me know if you have any ideas of how to get this
>     working or who else to contact.
>
>     Thanks for all you help,
>
>     Blaise
>
>
>

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: main.cpp
Url: http://mailman.srv.cs.cmu.edu/pipermail/olympus-developers/attachments/20100413/b832667b/main-0001.ksh
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: LiveMode.cpp
Url: http://mailman.srv.cs.cmu.edu/pipermail/olympus-developers/attachments/20100413/b832667b/LiveMode-0001.ksh
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: EngineInterface.cpp
Url: http://mailman.srv.cs.cmu.edu/pipermail/olympus-developers/attachments/20100413/b832667b/EngineInterface-0001.ksh
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: EngineInterface.h
Url: http://mailman.srv.cs.cmu.edu/pipermail/olympus-developers/attachments/20100413/b832667b/EngineInterface-0001.h