Integration of Natural Language and Vision Processing
Paul Mc Kevitt
P.McKevitt at dcs.shef.ac.uk
Tue May 17 15:01:52 EDT 1994
**** VISION AND LANGUAGE AND VISION AND LANGUAGE AND VISION AND LANGUAGE ****
**** VISION AND LANGUAGE AND VISION AND LANGUAGE AND VISION AND LANGUAGE ****
PROGRAMME AND CALL FOR PARTICIPATION
AAAI-94 Workshop on
Integration of Natural Language and Vision Processing
Twelfth National Conference on Artificial Intelligence (AAAI-94)
Seattle, Washington, USA
Tuesday/Wednesday, August 2nd/3rd, 1994
Chair:
Paul Mc Kevitt
Department of Computer Science
University of Sheffield, ENGLAND, EU
WORKSHOP COMMITTEE:
Prof. Mike Brady (Oxford, England)
Prof. Jerry Feldman (ICSI, Berkeley, USA)
Prof. John Frisby (Sheffield, England)
Prof. Frank Harary (CRL, New Mexico, USA)
Dr. Eduard Hovy (USC ISI, Los Angeles, USA)
Dr. Mark Maybury (MITRE, Cambridge, USA)
Dr. Ryuichi Oka (RWC P, Tsukuba, Japan)
Prof. Derek Partridge (Exeter, England)
Dr. Terry Regier (ICSI, Berkeley, USA)
Prof. Roger Schank (ILS, Illinois, USA)
Prof. Noel Sharkey (Sheffield, England)
Dr. Oliviero Stock (IRST, Italy)
Prof. Dr. Wolfgang Wahlster (DFKI, Germany)
Prof. Yorick Wilks (Sheffield, England)
WORKSHOP DESCRIPTION:
There has been a recent move towards considering the integration of
perception sources in Artificial Intelligence (AI) (see Dennett 1991
and Mc Kevitt (Guest Ed.) 1994). This workshop will focus on research
involved in the integration of Natural Language Processing (NLP) and
Vision Processing (VP).
Although there has been much progress in developing theories, models
and systems in the areas of NLP and VP there has been little progress
on integrating these two subareas of Artificial Intelligence (AI). It
is not clear why there has not already been much activity in
integrating these two areas. Is it because of the long-time reductionist
trend in science up until the recent emphasis on chaos theory,
nonlinear systems, and emergent behaviour? Or, is it because the
people who have tended to work on NLP tend to be in other Departments,
or of a different ilk, from those who have worked on VP?
We believe it is high time to bring together NLP and VP. Already we
have advertised a call for papers for a special volume of the Journal
of AI Review to focus on their integration and we have had
a tremendous response. There will be three special issues focussing
on theory and applications of NLP and VP and intelligent multimedia
systems.
The workshop is of particular interest at this time because research
in NLP and VP has advanced to the stage that they can each benefit
from integrated approaches. Also, such integration is important as
people in NLP and VP can gain insight from each others' work.
References
Dennett, Daniel (1991)
Consciousness explained
Harmondsworth: Penguin
Mc Kevitt, Paul (1994) (Guest Editor)
Integration of Natural Language and Vision Processing
Special Volume 8(1,2,3) of AI Review Journal
Dordrecht: Kluwer (forthcoming)
WORKSHOP TOPICS:
The workshop will focus on these themes:
* Multimedia retrieval
* Multimedia document processing
* Speech, gesture and gaze
* Theory
* Multimedia presentation
* Spatial relations
* Multimedia interfaces
* Reference
PROGRAMME:
Tuesday, August 2nd, 1994
*************************
INTRODUCTION I:
8.45 `Introduction'
Paul Mc Kevitt
MULTIMEDIA RETRIEVAL:
(Chair: Neil C. Rowe)
9.00 `Domain-independent rules relating captions and pictures'
Neil C. Rowe
Computer Science, U.S. Naval Postgraduate School, Monterey CA, USA
9.30 `An image retrieval system that accepts natural language'
Hiromasa NAKATANI and Yukihiro ITOH
Department of Information and Knowledge Engineering,
Shizuoka University, Hamamatsu, Japan
10.00 Break
MULTIMEDIA DOCUMENT PROCESSING:
(Chair: Rohini Srihari)
10.30 `Integrating text and graphical input to a knowledge base'
Raman Rajagopalan
Dept. of Computer Sciences, University of Texas at Austin, USA
11.00 `Photo understanding using visual constraints generated'
from accompanying text
Rohini Srihari
Center of Excellence for Document Analysis and Recognition (CEDAR),
SUNY Buffalo, NY, USA
11.30 Discussion
SPEECH, GESTURE AND GAZE:
(Chair: Jordi Robert-Ribes)
12.00 `Audiovisual recognition of speech units: a tentative functional
model compatible with psychological data'
Jordi Robert-Ribes, Michel Piquemal, Jean-Luc Schwartz &
Pierre Escudier
Institut de la Communication Parlee (ICP)
Grenoble, France, EU
12.30 Discussion
12.45 LUNCH
SITE DESCRIPTION (VIDEO):
(Chair: Arnold G. Smith)
2.00 `The spoken image system: on the visual interpretation of verbal
scene descriptions'
Sean O Nuallain, Benoit Farley & Arnold G. Smith
Dublin City University, Dublin, Ireland, EU &
NRC, Ottawa, Canada
THEORY:
2.20 `Behavioural descriptions from image sequences'
Hilary Buxton and Richard Howarth
School of Cognitive and Computing Sciences, University of Sussex &
Department of Computing Science, QMW, University of London
2.50 `Visions of language'
Paul Mc Kevitt
Department of Computer Science, University of Sheffield, England, EU
3.15 Discussion
3.30 Break
4.00 `Language animation'
A. Narayanan, L. Ford, D. Manuel, D. Tallis, and M. Yazdani
Media Laboratory, Department of Computer Science,
University of Exeter, England, EU
4.30 Discussion
MULTIMEDIA PRESENTATION:
(Chair: Arnold G. Smith)
4.45 `Assembly plan generation by integrating pictorial and textual
information in an assembly illustration'
Shoujie He, Norihiro Abe and Tadahiro Kitahashi
Dept of Information Systems and Computer Science,
National Univ. of Singapore, Singapore,
Faculty of Computer Science and Systems Engineering,
Kyushu Institute of Technology, Iizuka-shi, Japan &
The Institute of Scientific and Industrial Research
Osaka University, Osaka, Japan
5.15 `Multimedia presentation of interpreted visual data'
Elisabeth Andre, Gerd Herzog & Thomas Rist
DFKI & Universitaet des Saarlandes, Saarbruecken, Germany, EU
5.45 Discussion
6.00 OICHE MHAITH
Wednesday, August 3rd, 1994
***************************
INTRODUCTION:
8.45 `Introduction'
Paul Mc Kevitt
SPATIAL RELATIONS I:
(Chair: Jeffrey Mark Siskind)
9.00 `Propositional semantics in the WIP system'
Patrick Olivier & Jun-ichi Tsujii
Centre for Intelligent Systems
University of Wales at Aberystwyth, Penglais, Wales, EU &
Centre for Computational Linguistics, UMIST, Manchester, England, EU
9.30 `Spatial layout identification and incremental descriptions'
Klaus-Peter Gapp & Wolfgang Maass
Cognitive Science Program, Saarbruecken, Germany, EU
10.00 Break
10.30 `Axiomatic support for event perception'
Jeffrey Mark Siskind
Department of Computer Science, University of Toronto, Canada
11.00 Discussion
SPATIAL RELATIONS II:
(Chair: Stephan Kerpedjiev)
11.30 `A cognitive approach to an interlingua representation of
spatial descriptions'
Irina Reyero-Sans & Jun-ichi Tsujii
Centre for Computational Linguistics, UMIST, Manchester, England, EU
12.00 `Describing spatial relations in weather reports through prepositions'
Stephan Kerpedjiev,
NOAA/ERL/Forecast Systems Laboratory, Boulder, Colorado, USA
12.30 Discussion
12.45 LUNCH
MULTIMEDIA INTERFACES:
(Chair: Yuri A. TIJERINO)
2.00 `Talking pictures: an empirical study into the usefulness of
natural language output in a graphical interface'
Carla Huls, Edwin Bos & Alice Dijkstra
NICI, Nijmegen University, Nijmegen, The Netherlands &
Unit of Experimental and Theoretical Psychology, Leiden University,
The Netherlands
2.30 `From verbal and gestural input to 3-D visual feedback'
Yuri A. TIJERINO, Tsutomu MIYASATO & Fumio KISHINO
ATR Communication Systems Research Laboratories, Kyoto, Japan
3.00 Discussion
3.30 Break
4.00 `An integration of natural language and vision processing
towards an agent-based future TV system'
Yeun-Bae Kim, Masahiro Shibata & Masaki Hayashi
NHK (Japan Broadcasting Corporation)
Science & Technical Research Laboratories, Tokyo, Japan
4.30 Discussion
REFERENCE:
(Chair: Lawrence D. Roberts)
4.45 `An AI module for reference based on perception'
John Moulton, Hartwick College, Oneonta, N.Y. USA
and Lawrence D. Roberts, SUNY, Binghamton, N.Y. USA
5.15 `Instruction use by a vision-based mobile robot'
Tomohiro Shibata, M. Inaba, & H. Inoue
Department of Mechano Informatics, The University of Tokyo, Japan
5.45 Discussion
6.00 OICHE MHAITH
PUBLICATION:
Workshop notes/preprints will be published by AAAI. If there is
sufficient interest we will publish a book on the workshop with AAAI
Press.
WORKSHOP CHAIR:
Paul Mc Kevitt
Department of Computer Science
Regent Court
University of Sheffield
211 Portobello Street
GB- S1 4DP, Sheffield
England, UK, EU.
e-mail: p.mckevitt at dcs.shef.ac.uk
fax: +44 742 780972
phone: +44 742 825572 (office)
825590 (secretary)
ATTENDANCE:
We hope to have an attendance between 30-50 people at the workshop.
If you are interested in attending then please send the following
form to p.mckevitt at dcs.shef.ac.uk as soon as possible:
cut---------------------------------------------------------------------------
Name:
Affiliation:
Full Address:
E-mail:
cut----------------------------------------------------------------------------
REGISTRATION ENQUIRIES FOR AAAI CAN BE MADE TO:
NCAI at aaai.org
REGISTRATION FEE:
Incorporated into the technical registration fee except for
those who are workshop attendees only.
**** VISION AND LANGUAGE AND VISION AND LANGUAGE AND VISION AND LANGUAGE ****
**** VISION AND LANGUAGE AND VISION AND LANGUAGE AND VISION AND LANGUAGE ****
More information about the Connectionists
mailing list