[ACT-R-users] Models of listening to continuous speech?

[Bonnie John]

This helps a LOT.
Thanks,
Bonnie


Martin Greaves wrote:
> Hi Bonnie
>
> Further to Richard's comments, one of the key issues we were interested 
> in was rehearsal of auditory items.  We attempted to model the word 
> length effect in ACT-R using the audicon to represent a phonological 
> loop-like structure.  Whereas the Huss & Byrne model used declarative 
> memory for item storage and decay, and encoded list position within the 
> item chunk, we wanted to see whether a phonological loop could be 
> achieved using the intrinsic mechanisms of the audicon to account for 
> order and decay.  However, we found that maintaining any form of 
> rehearsal in the audicon to be difficult to implement.
>
> One of the key problems is determining when the end of a list of items 
> has been achieved, since ACT-R places 'rehearsed' items back in the 
> audicon as unattended items.  Hence, once an item has been rehearsed, it 
> is added to the end of the existing list of items with nothing to 
> discriminate it from items in the current (unrehearsed) list of items.  
> Consequently, the audicon does not allow discrimination of lists.  In 
> order to get around this problem and determine when the end of the list 
> had been reached we added an 'end' item.  As Richard said, this was 
> argued as being analogous to a drawing of breath or a pause prior to any 
> further rehearsal of items.
>
> A further problem with the model was that it failed to reproduce serial 
> order effects.  Although we obtained a good fit for word length, 
> particularly for one-syllable words, we found that serial position 
> curves were 'm' shaped rather than displaying extended primacy and 
> single-item recency.  This was because items simply dropped out of 
> memory if they were not rehearsed within the decay period, with no 
> possibility of recovery due to the all-or-nothing decay in the audicon.  
> Without intending to be in any way critical, this was not really 
> surprising given the operation of the audicon.  However, rehearsal would 
> not appear to be an issue in your case, so recall from the audicon 
> should simply represent an extended recency effect.
>
> In the running memory span models we took a slightly different approach 
> to the model.  We assumed that storage might utilise a range of 
> available short-term memory structures, with recall from whichever 
> structures still contained items at recall.  This allowed continued use 
> of the audicon but brings declarative  memory into play, since a 
> spin-off from sub-vocal rehearsal is an item chunk in DM.  The audicon 
> was considered it to operate simply as an extended buffer containing the 
> last 2-3 items presented.  Thus, we recalled whatever was available from 
> the audicon at the point of recall, with order implicit in recall, prior 
> to recall of earlier items from declarative memory.  Using this approach 
> we were able to account for a number of possible rehearsal strategies 
> and what we termed rehearsal 'micro-strategies' (cf. Gray & Boehm-Davis, 
> 2000) reproducing a range of patterns of recall observed empirically.  
> Were were also able to account for patterns of delayed recall when items 
> in the audicon had long since decayed through recall from declarative 
> memory.
>
> Hope this is of help,
>
> Martin
>
>
>
>
>
>
> Richard M Young wrote:
>   
>> Hello Bonnie,
>>
>> Some years ago, following on from David Huss's work with Mike Byrne, 
>> Martin Greaves and I played around with simple models using the 
>> audicon to deal with lists of isolated words in a short-term memory 
>> experiment.  At least in our hands, the audicon was assumed to hold 
>> "words" in some unspecified phonemic/phonological/articulatory code, 
>> which therefore required an access to DM/LTM in order to retrieve the 
>> word as a lexical item.
>>
>> As I remember it -- which is not well -- we found it a bit clunky to 
>> use the audicon to deal with ephemeral material spread out in time. 
>> With no disrespect to those who had worked on it, I think it's fair to 
>> say that not much thought had been given to the underlying design of a 
>> transducer dealing with speech sounds.  For example, I *think* we had 
>> to add an END signal to indicate the silence following the last item 
>> in a list.
>>
>> Martin later build partial models of the running memory span task 
>> using a similar approach, and included them in his PhD thesis.  He may 
>> be able to add to what I'm saying here.
>>
>> Word recognition in speech is a well-studied area in cognitive 
>> psychology, and there are some good (ad hoc) models around, and I 
>> believe a reasonable amount of consensus.  It had crossed my mind from 
>> time to time that it would be an interesting area to bring into 
>> contact with Act-R.  There are of course Act-R models of the lexical 
>> retrieval stage itself.
>>
>> Good luck!
>>
>> ~ Richard
>>
>> At 11:18 -0400 11/8/09, Bonnie John wrote:
>>     
>>> Folks,
>>>
>>> Anyone have an models using ACT-R's Audition Module to listen to and
>>> comprehend continuous speech?
>>> If not continuous speech, how about short phrases or anything other than
>>> tones?
>>> Any experience using the Audition Module at all?
>>>
>>> I'm asking because I'd like to make models of ACT-R using the JAWS
>>> screen reader to navigate a web site and compare it to ACT-R models of
>>> visually navigating a web site. I'd like to read papers or speak with
>>> someone who has had experience with the Audition Module to get a
>>> heads-up on peaks and pitfalls.
>>>
>>> I did browse the Publications page, but could only find papers that use
>>> vision (but I could have missed the listening ones - apologies if I 
>>> did).
>>>
>>> Thanks,
>>> Bonnie
>>>       
> _______________________________________________
> ACT-R-users mailing list
> ACT-R-users at act-r.psy.cmu.edu
> http://act-r.psy.cmu.edu/mailman/listinfo/act-r-users
>
>