INRIA, Evaluation of Theme COGC 1. Personnel

INRIA, Evaluation of Theme COGC
Project-team MErLIn
May 17-18, 2006
Project-team title: MErLIn, Méthodes pour l’ergonomie des logiciels interactifs
Scientific Leader: Dominique L. Scapin
Scientific Co-Leader: Noëlle Carbonell
Research centres: INRIA Research Units Rocquencourt & Lorraine
Common project-team with: University Henri Poincaré, Nancy 1
1. Personnel
Personnel (October 2001)
Misc.
INRIA
1
DR (1) / Professors
CR (2) / Assistant Professors
Permanent Engineers (3)
Temporary Engineers
Ph.D. Students
Post-Doc.
Total
External Collaborators
Visitors (> 1 month
CNRS
University
2
2
Total
3
2
0,5
3
4
7
4,5
1
8
12,5
1
0,5
(1) “Senior Research Scientist (Directeur de Recherche)”
(2) “Junior Research Scientist (Chargé de Recherche) ”
(3) “Civil Servant (CNRS, INRIA, …)”
(4) “Associated with a contract (Ingénieur Expert or Ingénieur Associé) ”
1
Personnel (May 2006)
Misc.
INRIA
1
DR (1) / Professors
CR (2) / Assistant Professors
Permanent Engineers (3)
Temporary Engineers
Ph.D. Students
Post-Doc.
Total
External Collaborators
Visitors (> 1 month
CNRS
University
1
1
3
2
7
1
1
Total
2
1
3
2
8
1
Changes in staff
DR / Professors
Misc.
INRIA
CNRS
University
Total
3
3
CR / Assistant Professors
Arrival
Leaving
Comments:
Academic staff at University René Descartes (Paris 5), that is one Professor and two Assistant
Professors, had to leave the MErLIn project-team due to the reorganisation of the Paris 5 Lab.
in ergonomics together with the retirement of Pr. J.C. Sperandio.
Current composition of the project-team (May 17-18, 2006):
•
Dominique L. Scapin, Research Director, INRIA, UR Rocquencourt
•
Noëlle Carbonell, Professor, University Henri Poincaré, LORIA
•
Suzanne Kieffer, Lecturer, University Henri Poincaré, LORIA
•
Mickaël Baron, Post-Doc., INRIA, UR Rocquencourt
•
Marius Hategan, Temporary Engineer, INRIA, LORIA
•
Syrine Charfi, Ph.D. Student, Tunisian gov. grant, UR Rocquencourt
•
Olivier Christmann, Ph.D. Student, MENESR grant, LORIA
•
Jerôme Simonin, Ph.D. Student, DGA-CNRS grant, LORIA
•
Daniel Gepner, External Collaborator (Dedale SA, Paris), LORIA
Current position of former project-team members (including students) during the
2002-2005 period (4 years):
•
Delphine Autard (former Technical Staff, Rocquencourt): Research Engineer
(Computer Science) at INSERM (since 2006)
•
Cédric Bach (former Ph.D. Student, Rocquencourt): Post-Doc. (Ergonomics) at IRIT,
Toulouse (since 2005)
•
Tristan Blanc-Brude (former Post-Doc.,
(Ergonomics) Candidate (since 2006)
2
Rocquencourt):
Assistant
Professor
•
Antonio Capobianco (former Ph.D. Student, LORIA): Assistant Professor (Computer
Science) at University Louis Pasteur, Strasbourg (since 2004)
•
Noelly Grondin (former Ph.D. Student, Rocquencourt): Consultant (Ergonomics) (since
2004)
•
Corinne Leulier (former Ph.D. Student, Rocquencourt): Consultant (Ergonomics),
currently Kleebox, Paris (since 2002)
•
Vincent Lucquiaud (former Ph.D. Student, Rocquencourt): Assistant Professor
(Computer Science) Candidate (since 2006)
•
Wendy Mackay (former Invited Researcher, Rocquencourt) : Research Director at
INRIA Futurs, project-team In-Situ (since 2003)
•
Lionel Medini (former Ph.D. Student, Rocquencourt): Assistant Professor (Computer
Science) at Avignon University (since 2002)
•
Rémy Taillefer (former Technical Staff, Rocquencourt): created a start-up company, RT
Consultants (since 2003)
•
Charles Tison (former Ph.D. Student, Rocquencourt): current occupational activity
unknown (since 2005)
•
Zhaowu Luo (former Post-Doc., LORIA): in the USA for personal reasons, looking for
a research position (since September 2005)
Last INRIA enlistments: None
Other comments:
Two candidates presented by MErLIn at Rocquencourt were not successful in 2002, but were
successful in 2003 at Futurs (CR1 J.-D. Fekete; DR2 W. Mackay who went on to create the InSitu project).
2 Work progress
2.1 Keywords
User interface design and evaluation methods:
Formal task description, ergonomic criteria, ergonomic quality of interactive software, user
assistance (online help), adaptive user interfaces, user modelling
New forms of human-computer interaction:
Hypermedia, multimodal interaction (speech, gestures, gaze), 3D visualisation and interaction,
mixed-reality, visual search
2.2 Content and overall goal of the project
The main goal of the MErLIn project-team is to contribute to the improvement of the
Ergonomic Quality of Interactive Software. Three complementary sub-goals contribute to this
general goal:
•
To improve the utility and usability of interactive software-based systems through empirical
or experimental studies of users’ interactions with such systems. These studies are meant to
increase available knowledge on users’ activities, perceptual and cognitive behaviours.
•
To provide software designers with sound, ergonomic design and evaluation methods. The
aim is to advance the integration of user-centred concerns throughout the design process
3
life-cycle by increasing available knowledge on these processes, proposing and
experimenting new methods or enhancing existing ones.
•
To contribute to the dissemination of best practice, especially by contributing to
standardisation activities and developing collaborations with the industry
2.3 Objectives for the evaluation period
Objectives of the MErLIn project-team (October 2001 Evaluation Seminar, translation):
•
Elaboration, validation and implementation of ergonomic methods for user-centred software
design and evaluation, namely: design and development of assistance tools for task analysis,
definition and ergonomic evaluation of contextual online help strategies.
•
Investigation of usability issues raised by new software applications and areas, new user
communities and interaction facilities. Main research topics concern Web ergonomics,
multimodal human-computer interaction, clothing design, Web accessibility (especially for
users with visual deficiencies).
Scientific activities and contributions of the team have been grouped along three objectives: the
first objective presented in 2001 has been broken in two sub-objectives in order to account for
the actual project-team’s activity during the evaluation period more accurately:
i. Task-based studies.
ii. Usability dimensions and methods.
iii. Enhanced multimodal interaction with virtual and mixed reality environments.
2.4 Objective i: Executive summary
For task-based studies, the goal is to gain a better knowledge of users’ activities, which is
certainly a major goal of any user centred approach, and to provide ways of improving this
knowledge with appropriate models and tools. The research work has concerned:
(a) Identifying and specifying mental workload factors.
(b) Task modelling and tools.
(c) Ease of learning of a new interface for air-traffic control.
(d) Analysis of fashion designers’ activities (stylists and “modélistes”).
(e) Management of Personal Information Systems (PIMS).
(f) Adaptive online help for the general public.
2.4.1 Personnel
(a)
(b)
(c)
(d)
(e)
(f)
N. Grondin (2001-2004), D. L. Scapin
V. Lucquiaud (2001-2005), D. L. Scapin
J.-M. C. Bastien (2001-2003)
R. Taillefer (2002-2004), D. L. Scapin
T. Blanc-Brude (2005), D. L. Scapin
A. Capobianco, (until 2003), M. Hategan (01/10/05-30/10/06), J. Simonin (since 01/10/03),
N. Carbonell
2.4.2 Project-team positioning
(a) Mental workload has been mainly looked at with quantitative methods and scales. Our
approach has been to run field studies and to attempt modelling the observed activities. It
proved to be rich in data, but very lengthy to collect.
4
(b) Research on task modelling and tools has been initially stimulated by our previous work at
INRIA (project-team PsychoErgo). Since, several models have been offered (e.g., IMAD,
Euterpe, CTTE, Tamot). A detailed analysis of the pros and cons of these models led to the
definition of a kernel model, with the ambition of making it as formal as possible, which
helps in providing real usable software tools, as well as potential model querying. It also
makes it possible to compare this kernel model to other models and provide links with
formal description languages (e.g., Abrial’s B method).
(c) The design of new interfaces for air-traffic control has motivated numerous studies. The
study referred to here is an ad hoc study which was not followed up by further
investigations.
(d) This applied research study was initiated following an industrial request.
(e) The conclusions of this study offer new prospects on PIMS management: it is the first study
that investigates how users organize the storage of personal or professional electronic
information and retrieve specific data files in an “ecological” situation context, that is
through interviews with actual users who describe the organisation of their own information
system and recall its content at their own workstation.
(f) Adaptive user interaction which requires the generation and implementation of dynamic
user models is a research area which is fast developing. Learning environments, online help
included, count among the most promising future application domains. While software
issues about dynamic user modelling have been investigated extensively, ergonomic issues
about adaptive interaction have motivated but a few studies (see Jameson 20031). We are
currently assessing the effectiveness and usability of adaptive online help.
2.4.3 Scientific achievements
(a) Various parameters explaining mental work over- and under-load have been proposed.
However, it appears quite difficult to find clear-cut cases of underload and overload:
workload varies a lot from one person to another (levels of experience); and there are
differences between the different sites observed (work organisation and roles). 2 reports
have been published (state-of-the-art review, analysis of field data).
(b) Production of a tool which is quite modular and is being widely distributed, as well as a
Ph.D. thesis, and 2 papers at IHM. The availability of such a tool is a prerequisite for further
research on model-based design and evaluation, and for further studies on computer-aided
user-centred design and ergonomic evaluation.
(c) This study had the specificity of concentrating on training issues. It demonstrated the
primary role of problem-solving by level-change instructions. It has been presented at the
21st Digital Avionics Systems Conference, Air Traffic Management for Commercial and
Military Systems, and was judged ‘Best in Track’.
(d) Outcomes of this project include a State-of-the-art review of the various ergonomics studies
in the domain of clothes design and production; analyses of clothes design activities; a
survey of relevant Web sites. (3 contract Reports).
(e) Results of this study have been summed up in a research report. Two conference papers
have been submitted, as well as a journal article.
(f) During the first half of the evaluation period, previous research on expert contextual online
help strategies has been further developed. Based on the experience gained on help
strategies and system architecture, we are currently preparing an experimental study meant
1
A. Jameson (2003). Adaptive Interfaces and Agents. In J. Jacko, A. Sears, Eds., Human-Computer
Interaction Handbook, Mahwah,NJ: Lawrence Erlbaum Associates.
5
to assess the influence of adaptive (compared to “static”) online help on the performances
and subjective satisfaction of novices in the general public. Main publications (on
contextual help only): 1 Ph.D. thesis, 1 article in a journal, 1 book chapter, 4
communications at national and international conferences: IHM’02, ERGO-IA’02, HCI
International’03, INTERACT’03 (long paper).
2.4.4 Collaborations
(a)
(b)
(c)
(d)
(e)
(f)
SNCF (French Railways) and RATP (Paris Subway System)
SNCF, RATP, and the Universities of Poitiers (LISI) and Grenoble (IMAG)
EuroControl
LECTRA systèmes
Project-team In-Situ (Micromégas project)
University Nancy 2 (LABPSYLOR), IMASSA-CERMA, Brétigny sur Orge
2.4.5 External support
(a)
(b)
(c)
(d)
(e)
PREDIT contract (Ministry of Equipment and transports)
PREDIT contract
EuroControl
Eureka contract, COMEDIA project
1 Post-Doc. INRIA, ACI Masses de données, project Micromégas
(f) 1 Ph.D. grant (DGA/CNRS)
2.4.6 Self assessment
(a) Lots of data has been obtained. One concern is that, while very useful, such data is not
publishable material. Besides, the collaboration with the involved partners has been difficult
due to timing issues as well as difficulties in deciphering “incident” log databases. If further
conditions of collaboration and financing are met, it would be useful in the future, with the
K-MADe tool now available, to set up further testing in the field, in domains were
regulation activities are complex. Unfortunately, the Ph.D. student involved resigned to
accept a consulting job.
(b) A stable kernel model has been proposed and published; and a tool (25000 lines of code)
has been produced and made available (a demonstration has been presented at IHM 2006
and an article is being prepared). This tool is very strongly in demand in the research
community, and in the industry as well, especially for in-house or external consultants in
charge of preparing task requirements. However, the tool has not been evaluated yet
(including from an ergonomics point of view!); lots of additions are needed, for instance on
how to query the model, how to incorporate task-based recommendations for evaluation
purposes; and how to link the model with interaction/dialogue models and software design
processes.
(c) While interesting, that study was limited in terms of participants (students rather than airtraffic controllers).
(d) Very interesting data, but not followed by further testing of new user interaction solutions.
(e) The set of data obtained and the recommendations extracted from that study are very
promising for the design of better file management systems. This is an area worth
investigating further, together with visualization studies, as the amount of desktop files are
often making users lost.
6
(f) The design of effective online help is a crucial issue for the success of Internet services.
Companies are aware of that. Research work on contextual help being completed, we are
now considering online help as a useful application area for testing and assessing,
experimentally, hypotheses on the utility and usability of adaptive interaction and embodied
conversational agents (see, section 4, collaboration with France Télécom R&D).
2.5 Objective ii: Executive summary
For overall usability studies, the goal is to design and assess usability dimensions and methods.
The research work has concerned:
(a) Comparison of ergonomics evaluation methods for the Web.
(b) Co-inspection for evaluation and Ergo-Monitor.
(c) Paired-user testing.
(d) Usability Net.
(e) Usability dimensions for Virtual reality.
In addition, with the aim of covering and classifying usability methods, a book chapter has been
written. The attempt has been to introduce various human issues in the design and evaluation of
information systems, with an ergonomic perspective. The goal was not to achieve a thorough
review of the state-of-the-art, but to provide a survey of the main issues. After a few definitions
and a contextual description of the main problems, the chapter presents the major ergonomics
requirements to be taken into account. Then, along a classification of methods, a brief
description of current ergonomic methods and standards is provided, followed by a set of issues
concerning the choice of ergonomic methods depending on the system lifecycle and other
parameters relating to the information system context.
Also, while participating to the European project COST294-MAUSE, new classification and
description templates have been designed as a support to the critical analysis of usability
methods.
Finally, on the topic of ergonomics methods, recommendations, and dimensions, the project is
strongly involved in standardization, specifically:
− At European level: CEN/TC 122/WG5
− At ISO level, ISO/TC 159/SC 4/WG 5, WG6, WG9, SC1/WG1. Work in progress:
ISO/CD 9241-151 - Ergonomic design of World Wide Web User Interfaces.
ISO 9241-171- Ergonomics of human-system interaction -Guidance on accessibility
for human-computer interfaces.
NP ISO 9241-100- Dialogue principles.
NP Ergonomic Guidance for Laptop Telephone Communications.
New ISO 9241-Part 1 (reorganization of 9241).
NPI haptic Devices.
ISO CD 9241/20: Ergonomics of human-system interaction - Part 20: Guidance for
the accessibility of information communication equipments and services.
New CIF (Common Industry Format).
Revision of ISO 6385.
etc.
A questionnaire-based study was also performed for AFNOR, on the knowledge and use of
software ergonomics standards.
7
2.5.1 Personnel
(a)
(b)
(c)
(d)
(e)
J.-M. C. Bastien (2001-2003), C. Leulier (until 2002), D. L. Scapin
W. Cybis (invited Professor, 1 month), D. L. Scapin
N. Grondin (2001-2004), J.-M. C. Bastien (2001-2003)
J.-M. C. Bastien (2001-2003), D. L. Scapin
C. Bach (2001-2004), D. L. Scapin
2.5.2 Project-team positioning
(a) Web studies are very numerous. This research is one of the few studies that explicitly
attempted to compare several evaluation methods. Its objective is twofold: investigation of
the ergonomics difficulties that users of web sites (3 different sites) are faced with, and
identification of the contributions of major methods to the ergonomic evaluation of web
sites, namely, user testing, questionnaires, and ergonomic inspection. Analysis of results
made it possible to determine the scope of each method (i.e., the problems it can detect,
especially those undetected by other methods).
(b) This approach is quite original. It synthesizes recommendation-based inspection methods
(using Ergonomic Criteria and Interaction Components) with log analyses.
(c) Most user testing studies concern individual participants. The study set-up compared user
evaluation by pairs versus individual sessions. Results show that individual testing leads to
detect a larger scope of problems, but that testing by pairs leads to a better understanding of
usability problems.
(d) The first attempt by the IST program to provide assistance on usability methods to the
European research and industry community.
(e) First definition and testing of ergonomic dimensions for virtual reality environment design
and evaluation.
2.5.3 Scientific achievements
(a)
(b)
(c)
(d)
(e)
1 paper at IHM.
2 reports and 2 conference papers (WEBIST & CLIHC).
1 paper at IHM.
An extensive web site on usability methods, regularly used.
2 conference papers (IHM’03 and INTERACT’03); one workshop paper (CADUIMIXER’04); 1 Ph.D. thesis.
2.5.4 Collaborations
(a)
(b)
(c)
(d)
(e)
Galeries Lafayette and Sears
UFSC (Universidade Federal de Santa Catarina, Departamento de Informàtica)
Local
Over 20 European partners
University of Metz (LAPSILOR)
2.5.5 External support
(a) Follow-up on the COMEDIA project, Eureka contract
(b) France-Brazil cooperation (CNPq-INRIA)
(c) None
8
(d) IST Programme
(e) Follow-up on the COMEDIA project, Eureka contract
2.5.6 Self assessment
(a) Only one paper at IHM. The Ph.D. student has moved to a consulting job.
(b) Very interesting subject. Quite successful output … but, for further industrialization,
stronger collaboration is needed.
(c) While interesting, the topic was not further followed.
(d) Excellent dissemination, useful for acquiring initial knowledge.
(e) Good results. Opportunities for applying these results further (Post-Doc.). 1 journal paper in
preparation.
2.6 Objective iii: Executive summary
The project-team has a longstanding expertise in oral human-computer dialogue and multimodal
(speech + hand-gestures) interaction. Current research focuses on the study of new input-output
modalities and interaction metaphors. The main objective is to propose user interfaces that meet
new usability demands stemming from both the emergence of new contexts of use, namely
mobility and virtual or mixed reality environments, and the development of interactive
visualisations of very large data sets which raises new navigation and visual search issues.
Speech combined with gaze or head movements as a substitute for pointing hand gestures
appears as an attractive solution for interacting with large screens or while on the move. The
presentation is organised as follows:
−
New input modalities:
(a) Head movements.
(b) Gaze.
(c) User centred design and evaluation of mixed systems.
−
Assistance to visual search (target detection) in collections of pictures:
(d) Contribution of oral indications on the location of the target on the display.
(e) Influence of the spatial display layout on visual search efficiency and comfort.
(f) 3D interaction and navigation metaphors.
2.6.1 Personnel
(a)
(b)
(c)
(d)
(e)
(f)
(g)
C. Tison (2001-2004), D. L. Scapin, N. Carbonell, L. Kalaidji (3 months, 2004)
D. Gepner (since 2002), J. Simonin (since 2003), N. Carbonell
S. Charfi (since November 2005), D. L. Scapin
S. Kieffer (since 2001), N. Carbonell
S. Kieffer (since 2001), J. Simonin (since 2003), N. Carbonell
O. Christmann (since 2004), N. Carbonell
D. Gepner (since 2002), N. Carbonell, L. Zhaowu (2005)
2.6.2 Project-team positioning
(a) In order to achieve 3D objects manipulation, especially in clothe design contexts, classical
interaction techniques and devices (mouse and keyboard) are not necessarily most suitable.
The research is a very initial attempt to combine several modalities such as speech and head
movements for such control purposes.
9
(b) Research on gaze as an interaction modality2 is quite original in France where, to our
knowledge, eye trackers are exclusively used for analysing gaze activity, by psychologists
and ergonomists mostly. At the international level, scientific activity in this emerging field
is developing fast along two directions: (i) implementation of gaze as a supplementary
modality to speech for interacting with large displays (e.g., reality centres and caves) or
while on the move; (ii) design and development of gaze controlled 2D displays for visual
exploration tasks (e.g., gaze-contingent displays whose resolution varies according to gaze
points)3. The project-team investigates both directions.
(c) The goal is to contribute to the facilitation of joined manipulation of usual physical objects
(cup, pen, lancet, etc.) together with computer-based objects, and to accommodate naturally
the capabilities of physical and digital worlds. Several topics are being investigated,
including a major one: modelling such systems in order to facilitate design and development
of hardware and software components. The approach is usage-centred, particularly in terms
of users’ tasks in mixed systems.
(d) Input multimodality, especially speech combined with pointing gestures has been
extensively studied from both software and ergonomics points of view, while speech
associated with graphics has motivated fewer research studies, especially regarding the
utility and usability of speech as a supplementary modality to graphics. Our aim is to
contribute to fill in this gap.
(e) 2D arrays are currently used for displaying small collections of graphical objects (i.e. a few
dozens of icons, photographs, …) although the actual superiority of this standard layout
over other possible spatial structures has never been established from an ergonomic point of
view. To address this issue we performed an experimental study which attempts to evaluate
the influence of spatial layout on visual search efficiency and comfort by comparing
participants’ scan paths for different display layouts including 2 D arrays.
(f) Both 2D and 3D representations are used by techniques for visualising large data sets; for
instance, treemaps and hyperbolic trees represent data in two dimensions while data
mountains and Card’s wall implement perspective views. Usability issues regarding
navigation and search in such visualisations have motivated only a few studies, most of
them focused on comparing the usability of 2D versus 3D presentations. To our knowledge,
the usability of basic 3D interaction concepts, such as immersion of the user in a 3D space
versus manipulation by the user of a 3D object, has never been assessed. We performed an
experimental study to investigate this issue.
2.6.3 Scientific achievements
(a) Taking into account the absence of any tactilo-kinesthetic interaction, the most suitable
input modalities in this context appear to be vocal commands synergistically combined to
gaze or head direction and hand gestural deictic or mimetic movements. Publication: 1
Ph.D. thesis.
(b) (i) To acquire the knowledge (on temporal correlations between eye movements and oral
commands) necessary for designing a ‘natural’ multimodal (speech+gaze) command
language, we collected a corpus of spontaneous oral interactions with various 3D
2
which could replace pointing hand gestures and virtual object selection through manual devices.
e.g., for (i) and (ii), see respectively:
Cournia, N., Smith, J.D., Duchowski, A.T. (2003). Gaze vs. Hand-Based Pointing in Virtual
Environments. CHI'2003, Ft. Lauderdale (FL), April 5-10 2003, Short Talks and Interactive Posters.
L.C. Loschky, G.W. McConkie (2000). User Performance with Gaze Contingent Multiresolutional
Displays. In Eye Tracking Research and Applications Symposium (ETRA), Palm Beach Gardens, FL,
November 6-8, pp. 97-103.
3
10
applications using the W. of Oz technique. Participants eye movements, speech utterances
and interactions together with 3D displays were recorded. This corpus is being analysed.
(ii) For implementing gaze as an input modality, a robust algorithm for computing gaze
fixations in real time has been developed. A gaze-contingent prototype is now operational.
It has been used for controlling a directional camera (remote electronic surveillance
application). Current research focuses on improving the fluidity of resolution variations.
(c) Research work only started 5 months ago.
(d) Results of the two studies performed to assess the efficiency of oral assistance to visual
search (familiar target detection) indicate that coarse spatial information on the location of
the target on the display reduces both selection times and errors significantly. Publications:
1 Ph.D. thesis, 1 book chapter, 1 article in a journal (RIHM), 3 communications at
conferences (CLASS’02, IHM’03, AVI’06).
(e) Analysis of gaze trajectories during visual search (familiar target, small picture collections)
indicates that participants’ performances were worse for 2D array layouts than for some of
the other spatial structures tested. Publications: 1 Ph.D. thesis (the same as in d), 2
communications at conferences (INTERACT’05, IHM’05).
(f) Results of the comparisons between the two 3D views: analysis of participants’
performances and subjective judgements suggest that both views should be proposed to
future users due to the great diversity of individual performances and preferences.
Publication: 1 communication at a conference (AVI’06).
2.6.4 Collaborations
(a) LECTRA Systèmes and project-team MIRAGES
(b) None
(c) University of Toulouse (IRIT) and Métapages (Toulouse)
(d) Project-team In-Situ (Micromégas Project)
(e) IMASSA-CERMA, Brétigny sur Orge (Cognitive Ergonomics Dpt)
(f) Project-team In-Situ and LPM (Micromégas Project),
Centre virtuel sur la connaissance de l’Europe (CVCE), Luxembourg
(g) CRAN (Centre de Recherche en Automatique de Nancy)
2.6.5 External support
(a) Eureka contract, COMEDIA project
(b) 1 junior engineer (AI) supported by INRIA
(c) 1 Ph.D. grant
(d) 1. INRIA-Région-Lorraine Ph.D. grant
(e) 1. INRIA-Région-Lorraine Ph.D. grant (same as in d)
(f) 1 MENRT Ph.D. grant
(g) None
2.6.6 Self assessment
(a) Very interesting experiment results which indicate that this new style of interaction and low
level task-modalities assignment are efficient. They also indicate that the verbal group used
in the command (in French) does not determine the operation to be performed from a
11
system point of view. Such a determination can be obtained through a combination of
several terms of the provided sentences. This led to a Ph.D. thesis, but only 2 reports were
produced. A potential journal submission is only starting.
(b) The software provided by eye tracker manufacturers for computing fixations only operates
on files of recorded points of gaze samples; it was then necessary to develop a real time
algorithm. To analyse semi-automatically gaze fixations during human-computer
interaction, it is necessary to develop software tools for recording rich interaction logs
including speech utterances, screen copies, gaze samples (up to 240 Hz sampling rate) and
application events. Collecting these data and re-playing them raise complex synchronisation
problems to achieve accurate real time operation. In the absence of permanent engineering
staff, these developments were slow.
(c) Research work only started 5 months ago.
(d) Results of this experimental study may have an impact on the design of 2D presentations of
graphical objects, since they indicate that coarse spatial information on the location of a
graphical object on the screen divides search times by three. However, to convince
designers of the interest of oral assistance to visual search tasks, the benefits that can be
gained from the substitution of such oral messages for standard visual enhancements should
be assessed.
(e) Results of this study are important since they demonstrate that 2D array layouts currently
used for presenting graphical objects are less efficient and less comfortable (visual comfort)
for visual search tasks than other possible spatial layouts. However, before disseminating
this result towards designers of picture browsers and Web sites, it is necessary to assess
experimentally its scope, especially its stability with respect to the number of graphical
objects displayed simultaneously.
(f) Public demonstrations4 of the two 3D views and associated navigation facilities suggest that
picture browsers implementing these visualisation and interaction metaphors will raise easy
acceptance from the general public. The next step is to compare both 3D views with 2D
presentations currently used by picture browser designers. An appropriate experimental
protocol is in the process of being defined.
4
for instance at University Henri Poincaré ‘Fête de la Science’ in Nancy.
12
3 Knowledge dissemination
3.0 Publications
Year 1
Year 2
Year 3
Year 4
2
2
Ph.D. Thesis
1
H.D.R. (*)
-
Journal
1
1
Conference proceedings (**)
13
7
2
9
Book chapter
1
1
2
2
2
Book (written)
1
Book (edited)
Patent
1
Technical report
4
4
2
3
Deliverable
6
3
1
3
(*) H.D.R.: Habilitation à Diriger des Recherches
(**) Conference with a Program Committee
Major journals in the field and, for each, the number of papers co-authored by members of the
project-team that have been accepted during the evaluation period (see the list of publications
where the four articles referred to here are preceded by “†”):
− 3 International Journal on Universal Access in the Information Society (UAIS)
− 1 Revue d’Interaction Homme-Machine (RIHM)
Major conferences in the field and, for each, number of papers co-authored by members of the
project-team that have been accepted during the evaluation period (see the list of publications
where communications at the first three conferences are preceded by “†”):
− 1 ACM Int. Conference on Human Factors in Computing Systems (CHI)
− 4 IFIP TC13 Int. Conference on Human-Computer Interaction (INTERACT)
− 1 (+ 2 in 20065), ACM Working Conference on Advanced Visual Interfaces (AVI)
− 1 Int. Conference on Web Information Systems and Technologies (WEBIST)
− 3 International Conference on Human-Computer Interaction (HCI International)
− 1 ACM Symposium on User Interface Software and Technology (UIST)
− 1 Annual Conference of the British HCI Group (BCS HCI)
− 1 Int. Conference on Working with Display Units (WWDU)
− 1 Digital Avionics Systems Conference (DASC)
− 1 European Academy of Design Conference (EAD)
−
−
−
5
7
4
1
Conférence Francophone sur l’Interaction Homme-Machine (IHM)
Conférence Francophone ‘Ergonomie et Informatique Avancée’ (ERGO-IA)
Conférence annuelle de la Société d’Ergonomie de Langue Française (SELF)
S. Kieffer, N. Carbonell. Oral messages improve visual search.
O. Christmann, N. Carbonell. Browsing through 3D representations of unstructured picture collections:
an empirical study.
(Submission deadlines: December 2005)
13
3.1 Software
1
Design and development of a software platform in Java (Windows) with the following
functionalities:
a. Capture and storage of rich logs of users’ interactions with any Windows
application, that is: system and user events, screen copies, user spoken utterances,
user gaze samples.
b. “Playback” of stored interactions (i.e., standard functionalities of video recorders).
c. Implementation of the Wizard of Oz paradigm including:
− real time transfer, towards the Wizard’s distant work station, of the subject’s
speech and gaze movements, together with the current display and mouse
cursor position on the subject’s work station;
− real time transfer, towards the subject’s workstation, of text, graphics,
animations and speech, on the Wizard’s initiative.
d. Integration of embodied conversational agents with speech capabilities into the
Wizard of Oz platform (i.e., into the partly simulated user interface).
This software has been developed within the framework of a scientific collaboration with
France Télécom R&D (Contrat de Recherche Externalisé, 2005-2006). It is the property of
France Télécom (FT). Embodied conversational agents have been provided by FT R&D.
2
Implementation of a real time algorithm (C++, Linux) for computing eye fixations from
point of gaze samples. This software component meant for ASL head-mounted eye trackers
takes account of head movements. It is the property of ASL.
3
K-MADe (Kernel of Model for Activity Description environment): K-MADe is a tool
which aims at facilitating task and activity analysis, and contributes to the incorporation of
ergonomic knowledge into the design process of interactive systems. The originality of this
tool is to be based on a model whose expressive power results from the formal description
of its semantics. This tool facilitates not only task description and analysis, but also
querying the model and navigating within models and software lifecycle steps.
The K-MADe tool was developed with version 1.5 of the JDKJava (25000 lines of code),
using the Swing toolbox for programming the graphical user interface. Different API were
also used for graph manipulation: JGraph (http://www.jgraph.com) for modelling part of the
tree of tasks; L2Fprod (http://common.l2fprod.com) for additional graphical components to
the basic API Swing; JGoodies (http://www.jgoodies.com) for the “look and feel” of the
tool. The Express language was used for describing the task model and objects in the
current state of the world. Express models were implementing using a “custom” API which
provides all the necessary services for creating and handling objects.
Finally the language XML was used to describe the grammars associated with the
expression of preconditions, postconditions and iterations. The handling of XML files
required the use of an additional API: DTDParser (http://www.wutka.com/dtdparser.html).
This API makes it possible to analyze and interpret an XML file according to its DTD.
3.1.1 “Valorisation” and technology transfer
Dissemination and transfer of MErLIn’s research results towards the software industry is
ensured through scientific collaboration contracts with industrial R&D partners in large
companies (e.g., SNCF, FT) and a strong involvement in standardisation activities. During the
last four years, our transfer activities have been mainly focused on user-centred design methods
and ergonomic evaluation criteria, usability testing of new interaction modalities, new forms of
user support and adaptive user interfaces.
For instance, the multimodal software platform described in subsection 3.1 (see software 1) is
currently used by several research groups at FT R&D (Lannion) for testing the usability of
future services.
14
The project-team has also contributed to the elaboration of a code of practice which proposes
solutions to the main challenges of Design for All principles for the design of interactive
products and services in the domain of Health Telematics; see, in subsection 6.3, our
contribution to LNCS 3040, an outcome of the IS4ALL network activity.
3.2 Teaching
Dominique Scapin
Research staff.
− Ph.D. supervision (5): C. Tison (2000-2004), C. Bach (2001-2004), N. Grondin (20012004), V. Lucquiaud (200162005), S. Charfi (2005-…)
−
Courses at graduate level:
Institut Supérieur de Management (ISTM): Human-Computer Interaction (27h)
Noëlle Carbonell
Academic staff. Full time researcher at INRIA for 2 academic years (2003-2004, 2004-2005).
− Ph.D. supervision (5): C. Tison (2000-2004, co-supervision with D. L. Scapin), S. Kieffer
(2001-2005), J. Simonin (2003-…), O. Christmann (2004-…), D. Gepner (2005-…)
−
Teaching activities, main courses (academic year 2005-2006):
− At undergraduate level:
Algorithms on trees and graphs (3rd year students, 60h)
− At graduate level:
User-interface design and ergonomic evaluation (4th year students, 30h)
Artificial Intelligence (4th year students, 30h)
User modelling and adaptive user interfaces (5th year students, 15h)
−
Main pedagogical responsibility:
Coordination of the AI research curriculum in the Master6 of Computer Science
3.3 Visibility
3.3.1 Editorial Boards of Journals
Behaviour and Information Technology: D. L. Scapin
Interacting with Computers: D. L. Scapin
International Journal of Human-Computer Interactions: D. L. Scapin
International Journal of Human-Computer Studies: D. L. Scapin
International Journal of Universal Access in the Information Society: N. Carbonell, D. L. Scapin
Revue d’Interaction Homme-Machine : D. L. Scapin
Le Travail Humain: N. Carbonell, D. L. Scapin
Information, Interaction, Intelligence (I3) : N. Carbonell
3.3.2 Major Conference Programme Committees
ASSETS
CADUI
CUU
HCII
ICMI
Int. ACM Conference on Assistive Technologies: N. Carbonell
Int. Conference on Computer-Aided Design of User Interfaces: D. L. Scapin
Int. ACM Conference on Universal Usability: N. Carbonell
HCI International: D. L. Scapin
Int. ACM Conference on Multimodal Interfaces: N. Carbonell (Area Chair, 2005)
6
Responsabilité de la spécialité recherche ‘Perception, raisonnement et interaction multimodale’ du
Master Informatique des Universités de Nancy.
15
INTERACT IFIP TC13 Int. Conference on Human-Computer Interaction: D. L. Scapin
OzCHI
Conference of the Australasian Computer-Interaction Special Interest Group:
D. L. Scapin
UAHCI
Universal Access in Human-Computer Interaction, HCI International:
N. Carbonell
UM
Int. Conference on User Modeling: N. Carbonell
WWCS
Int. Conference on Work with Computer Systems: D. L. Scapin
Ergo-IA
IHM
Colloque ‘Ergonomie et Informatique Avancée’: N. Carbonell, D. L. Scapin
Conférence francophone annuelle sur l’Interaction Homme-Machine:
N. Carbonell, D. L. Scapin
SELF Conf. Annual Meeting of the French Ergonmics Society: D. L. Scapin
Major international Workshops:
CWUAAT Workshop Series on Universal Access and Assistive Technologies: N. Carbonell
IWIPS
Annual Int. Workshop on Internalisation of Products and Systems: D. L. Scapin
MIXER
Int. Workshop on Design and Engineering of Mixed Reality Systems:
D. L. Scapin
MTWAS
Int. Workshop on Modern Technologies for Web-based Adaptive Systems:
N. Carbonell
RAAWS
Int. Workshop on Recommender Agents and Adaptive Web-based Systems:
N. Carbonell
TAMODIA Int. Workshop on TAsk MOdels and DIAgrams for user interface design:
D. L. Scapin
UI4ALL
ERCIM Workshop Series on User Interfaces for All: N. Carbonell
WEB3D
Int. Workshop on Web 3D Technologies in Learning, Education and Training:
N. Carbonell
3.3.3 International Working Groups
−
Expert for Standard Bodies:
ISO/TC 159/SC4, CEN TC 122, AFNOR X35 (D. L. Scapin)
−
Participation in International Working Groups:
Steering Committee of the ERCIM WG on User Interfaces for All (N. Carbonell)
Advisory Board of Int. Conference on Work With Computer Systems (D. L. Scapin)
3.3.4 Organisation of Scientific Events
−
7th ERCIM Workshop on User Interfaces for All (UI4ALL), Paris, October 23-25, 2002.
(N. Carbonell, PC Chair and local organiser)
3rd IS4ALL Seminar and 1st IS4ALL thematic Workshop ‘IS4ALL in Health Telematics’,
Paris, October 22, 2002. (N. Carbonell, organiser)
−
Participation in the organisation of the MAUSE Workshop on User Quality Models
(COST294-MAUSE) at INTERACT’05, Rome, 12-13 September, 2005. (D. L. Scapin)
−
Participation in the organisation of a CESAME Workshop (Conception et Evaluation de
Systèmes interactifs Adaptables et/ou MixtEs), Groupe de Travail 4.6 du GDR I3, at
UbiMob’05, Deuxièmes Journées Francophones ‘Mobilité et Ubiquité’, Grenoble, May 31,
2005. (D. L. Scapin)
16
4 External Funding
(k euros)
2002
2003
2004
2005
National Research Initiatives
Ministère Equipement Transport (PREDIT)
148 (4 years)
ACI Masses de données: Micromégas project
71 (3 years)
AS-RTP16 MIXER
4 (2 years)
European Projects
th
5 PCRD, IST, Thematic Network: IS4ALL
129 (3 years)
Eureka, Comedia
808 (3 years)
th
246 (3 years)
th
4 (2 years)
5 PCRD, IST, INTERLIVING
5 PCRD, IST, Usability Net
COST, MAUSE
20 (4 years
Industrial Contracts
France Télécom R&D, CRE
92
Scholarships
Ph.D.*: 2
CNRS
MENESR°
Post Doc.*: 1
INRIA
+
AI : 1
INRIA (2 years)
Regional Research Initiatives
Télécommerce project
5
Télévis project (1 year junior engineer)
10
27
27
* other than those supported by one of the above projects
+
junior engineer supported by INRIA
° French Ministry of Education and Research
National Research Initiatives
Ministère de l’Equipement et du Transport (PREDIT)
PREDIT project, in collaboration with SNCF and RATP:
Identification of potential factors influencing mental workload of human operators in charge of
controlling railways traffic. Research work has focused both on workload analysis (survey of
the literature and field analysis of the controllers’ activities), and on the enhancement of the task
analysis model MAD (Méthode Analytique de description): design of the K-MAD model and
development of the associated software tool (K-MADe).
ACI Masses de données
Micromégas project, in collaboration with the In-Situ project-team (INRIA-Futurs, Orsay) and
the LPM (Laboratoire de la Perception et du Mouvement, Marseille).
Overall scientific goal: Design, implementation and evaluation of software tools for multi-scale
navigation in visualisations of large sets of familiar data both professional and personal.
Contribution: Firstly, analysis of actual strategies for storing and retrieving personal and
professional electronic data. Secondly, design and ergonomic evaluation of two 3D metaphors
for visualising, and interacting with, unstructured collections of photographs.
17
AS-RTP16 − Méthodes et outils pour les systèmes mixtes.
This AS from CNRS-STIC, RTP 16, ‘Methods and tools for Human-Machine Interactions’,
grouped several research groups working on Mixed Systems, i.e., systems combining physical
and digital worlds. The goal has been to organise scientific exchanges, workshops,
collaborations on new methods and tools dedicated to the specification, design and evaluation of
such systems. Part of the work is currently being followed by a new GT, CESAME, ‘Design and
Evaluation of Adaptable and/or Mixed interactive systems’ (GDR-I3, GT 4.6, CNRS).
European Projects
5th PCRD, IST, IS4ALL Thematic Network
Partners: CNR-IROE (It.), FhG-IAO (G), GMD (G), ICS-FORTH (Gr.), European Health
Telematics Association, Microsoft Healthcare Users Group Europe.
Objective: Collect and disseminate methods and techniques for implementing Universal Access
in the Information Society, especially through Universal Design principles, in the Healthcare
domain.
Contribution: Dissemination of best practice knowledge; in particular, organisation of dedicated
international scientific workshops, tutorials and parallel sessions at international conferences.
Eureka Comedia
Project Comedia, in collaboration with LECTRA-Systèmes and project-team MIRAGES
(INRIA-Rocquencourt) was focused on the study of HCI techniques that would improve the
ergonomic quality of graphical design environments such as in the clothes industry. Research
work includes an initial survey of design activities in the field of clothing, a survey of novel
available input techniques and devices, experimental testing of vocal commands synergistically
combined with head direction and hand deictic or mimetic gestures.
5th PCRD, IST, INTERLIVING
Ethnographic study of distributed, inter generational families. Design and implementation of a
set of prototypes to be installed in volunteer families’ homes. Evaluation of the long-term use of
these prototypes by family members. Dissemination of results in the research community.
5th PCRD, IST, Usability Net
A European Union project that provides usability and user centred design resources to
practitioners, managers and EU projects. The contributions were related to the structuring and
delivery of usability methods, best practices, etc.
COST, MAUSE
COST294-MAUSE is a usability research community. Over 20 countries are represented. Its
ultimate goal is to bring more science to bear on Usability Evaluation Methods (UEMs). This
goal is being realized through scientific activities of four Working Groups:
− WG1, Critical Review and Analysis of Individual UEMs (MErLIn is in charge of this WG);
− WG2, Comparing UEMs: Strategies and Implementation;
− WG3, Refining and Validating Classification Schemes for Usability Problems;
− WG4, Review on the Computational and Definitional Approaches in Usability Evaluation.
Industrial contracts
France Télécom R&D (CRE)
Scientific goal: Assessment of the possible effects of the presence of an embodied
conversational agent on the efficiency of interactions with online help systems, and user
satisfaction.
18
Contribution: Design and realisation of an experimental study, involving the development of a
software platform for storing and “re-playing” multimodal user interactions with any Windows
application, and implementing enhanced Wizard of Oz functionalities (see section 3.1).
Regional Research Initiatives
Télécommerce project
In collaboration with CRAN (Centre de Recherche en Automatique de Nancy). Design of an
interactive remote CAD service for carving forms on request (in sheets of metal). A feasibility
study was realised which proposed a suitable multimodal user interface (speech and gestures)
with acceptable response times (i.e., Quality of Service), namely, “continuous” 3D feedback to
users’ commands and actions over the Internet.
Télévis project
In collaboration with CRAN (Centre de Recherche en Automatique de Nancy). Remote control
of an electronic surveillance robot by gaze and gestures (or voice).
The equipment includes a mobile robot equipped with a swivelling camera. The current
prototype operates thus: the video stream generated by the camera is sent to the distant human
operator via a heterogeneous network, and displayed on his/her workstation. Orientation of the
camera is controlled by the operator’s gaze, while the moves of the robot are controlled with a
joystick, and, in the near future, by speech commands. Considered application domains include
electronic surveillance and mobile computing.
5 Objectives for the next four years
The implementation of objectives will be dependent on the capability of the project to recruit (or
otherwise said, the capability of INRIA to support its research groups in novel and
pluridisciplinary domains such as HCI), or to find funding to support its activities. However, in
any case, the strategy will also be to consolidate the various research results, especially
methodological, obtained through the years, and to make them accessible widely through
common tools. That consolidation will also mean more paper submissions. Then, the main
research aim of the project-team during the next four years will be to consolidate results on
design and evaluation methods, as well as novel interaction modalities, multimodality and new
interaction paradigms.
Basically, the scientific goals of the project will remain stable, while domains of involvement
will evolve (and have evolved since 2001, e.g., mixed systems), and the focus (if remaining a
small research team) will be reduced. Objectives of the project-team for the next four years can
be summed up as follows.
5.1 Design and evaluation methods
5.1.1 Task-based studies
The main focus will be put on disseminating, assessing, and improving the K-MAD model and
K-MADe tool in various fields. In addition to utility and usability improvements, the idea is to:
•
Incorporate in the tool query systems allowing to facilitate design and evaluation of HCI
(e.g., task-based recommendations; visualization of data, etc.). The tool is being distributed
with requests for usage feedback; assessment studies are on-going; task-based
recommendations are available. A candidate for a research position at INRIA is hoped to be
successful, as well as a post-doc application.
•
Integrate the task-based approach and the usability dimensions (e.g., ergonomic criteria).
The idea is to provide design and evaluation help based on applying specific
recommendations from our recommendations database as well as from existing standards.
19
•
Establish links with software design methods (e.g., through parallel task and dialogue
modelling; by establishing links with UML, RUP, etc.). A Ph.D. grant is being considered
on this topic (with University of Poitiers). Another one is being considered with support
from DGA.
5.1.2 Study of Personal Information Systems (PIMs)
The study of PIM management activities will be carried on, and analysis results will be
implemented into PIMs tools which will be experimented and assessed in real contexts of use.
This area of research seems very promising: large companies such as Google and Microsoft are
recruiting in that area. This endeavour will require additional funding and personnel.
5.1.3 Study of Mixed Reality Systems
This work is just starting with a Ph.D. student, through a collaboration with University of
Toulouse (IRIT).
5.2 New interaction modalities and multimodality
The efforts of the project-team during the next four years will focus on:
− gaze as an input modality, especially as a supplementary modality to speech and gestures,
− interactive 3D visualisations of large collections of visual information items (graphical
objects, text with or without graphics, photographs, pictures),
− embodied conversational agents (ECAs) with speech and emotion expression capabilities.
5.2.1 Gaze as an input modality
Research activities of the project-team will focus on the following objectives:
− Acquiring and implementing appropriate knowledge on visual perception with a view to
improving the usability of the current gaze-contingent prototype. Collaborations with
psychologists (specialists in visual perception) will be developed.
−
Designing and implementing usable multimodal command languages for interacting with
3D environments and virtual reality applications. Gaze will be combined with speech, head
movements or manual devices (with or without force or haptic feedback). The usefulness of
each couple (or triplet) of modalities implemented will be assessed thoroughly, through
experimental studies focused on determining the user activities, computer environments
(especially displays) and application domains for which this form of multimodality is most
appropriate. The reality centre and the cave soon available at INRIA-Lorraine will be used
in this context.
Collaboration with the SIAMES7 project-team at IRISA will be considered.
5.2.2 Interactive 3D visualisations of large collections of visual information items
The two 3D views experimented until now are appropriate for visualising unstructured
collections of pictures. The scientific challenge now is to design, implement and experiment
appropriate 3D views for visualising large hierarchical collections of graphical or multimedia
information items. However, access to large collections of indexed pictures or graphical objects
is not easy; in addition, visual material indexation is often poor and inconsistent. At present, the
small multimedia database realised by the CVCE (Centre Virtuel de la Connaissnce sur
l’Europe) is used. Another database will be necessary for experimentations with potential users.
7
on Computer generated images, animation, modeling and simulation.
20
5.2.3 Embodied conversational agents
The knowledge and experience gained in the area of Embodied Conversational Agents (ECAs)
thanks to the present research contract with France Télécom R&D will be used to define an
appropriate methodology for assessing the impact of ECAs on users’ efficiency and subjective
satisfaction. In particular, the considered methodology will include recommendations on eye
tracking data analysis and interpretation of physiological indicator measures, as well as
recommendations on the design of experimental protocols. Guidelines for interpreting objective
measurements and subjective judgement elicitation tools (e.g., SAM) will also be proposed.
Collaboration with University Paris VIII (C. Pélachaud) will be developed.
5.2.4 Comments
The three research objectives mentioned above raise difficult research issues which are major
obstacles to the development of new, powerful and attractive human-computer interaction
paradigms, hence new promising application domains. They lie within emerging research areas
which are developing fast following recent major technological advances.
During the evaluation period, the project-team has obtained some original scientific results in
these areas. However, these results have been insufficiently disseminated in the international
research community, due to insufficient permanent research staff mainly. In addition, research
in these areas requires specific software engineering expertise. The absence of any support from
the permanent engineering personnel is a serious obstacle to keeping up with the international
competition pace and getting increased scientific visibility. In addition, INRIA support, in the
form of part-time involvement of a permanent research engineer in specific software
development and maintenance, would prevent INRIA from going on being deprived of the
ownership of software developed by the project-team.
6 Bibliography of the project-team
6.1 Books and Monographs
N. Carbonell, Guest Editor, Special issue ‘Multimodality, a step towards universal access’,
Universal Access in the Information Society Int. Journal (UAIS), 2/2, June 2003.
N. Carbonell, C. Stephanidis Eds. (2003). Universal Access: Theoretical perspectives, practice
and experience (7th ERCIM UI4ALL Workshop, October 2002, Paris, France − Selected
papers), LNCS 2615, Springer-Verlag Berlin Heidelberg, 2003.
6.2 Doctoral Dissertations and “Habilitation” theses
Antonio Capobianco (2002). Stratégies d’aide en ligne contextuelles : acquisition d’expertises,
modélisation et évaluation expérimentale. Thèse de l’Université Henri Poincaré Nancy 1,
spécialité Informatique, octobre 2002.
Cédric Bach (2004). Elaboration et validation de critères ergonomiques pour les interactions
homme-environnements virtuels. Thèse de l’Université de Metz, spécialité Ergonomie,
novembre 2004.
Charles Tison (2004). Mise en œuvre et évaluation d’interactions multimodales orientées
communication appliquées au déplacement d’objets graphiques 3D. Thèse de l’Université Henri
Poincaré Nancy 1, spécialité Informatique, novembre 2004.
Suzanne Kieffer (2005). Assistance multimodale à l’exploration de visualisation 2D
interactives. Thèse de l’Université Henri Poincaré Nancy 1, spécialité Informatique, juillet
2005.
21
Vincent Lucquiaud (2005). Sémantique et outil pour la modélisation des tâches utilisateur : NMDA. Thèse de l’Université de Poitiers, spécialité Informatique, décembre 2005.
6.3 Articles in refereed journals and book chapters
J.-M. C. Bastien, D.L. Scapin (2002). La conception de logiciels interactifs centrée sur
l’utilisateur : étapes et méthodes. In P. Falzon Ed., Ergonomie, Paris : Presses Universitaires de
France, 2004.
A. Capobianco, N. Carbonell (2002). Contextual Online Help: a Contribution to the
Implementation of Universal Access. In S. Keates, P. J. Clarkson, P. Robinson Eds., Universal
Access and Assistive Technology, London: Springer-Verlag, pp. 131-140.
† N. Carbonell (2003). Recommendations for the design of usable multimodal command
languages. Universal Access in the Information Society Int. Journal (UAIS), Special issue
‘Multimodality, a step towards universal access’, 2/2, 143:159.
† A. Capobianco, N. Carbonell (2003). Online help for the general public: specific design issues
and recommendations. Universal Access in the Information Society Int. Journal (UAIS), Special
issue ‘Countering design exclusion’, 2/3:265-279.
J.-M. C. Bastien, D.L. Scapin (2004). Les méthodes ergonomiques : de l’analyse à la
conception et à l’évaluation. In P. Falzon Ed., Ergonomie, Paris : Presses Universitaires de
France, 2004.
N. Carbonell (2005). Multimodal Interfaces – A Generic Design Approach. In C. Stephanidis
Ed., Universal Access in Health Telematics – A Design Consortium Code of Practice, LNCS
3040, Springer-Verlag Berlin Heidelberg, pp. 209-223.
† N. Carbonell (2005). Ambient Multimodality: towards Advancing Computer Accessibility and
Assisted Living. Universal Access in the Information Society Int. Journal (UAIS), 18 pp.
(accepted for publication)
† S. Kieffer, N. Carbonell (2005). Apport d’indications spatiales orales à la recherche de cibles
dans des affichages complexes. Revue d’Interaction Homme-Machine (RIHM), 29 pp.
(accepted for publication)
N. Carbonell, S. Kieffer (2005). Do oral messages help visual search? In J. van Kuppevelt,
L. Dybkjaer, N.O. Bernsen Eds., Advances in Natural Multimodal Dialogue Systems, Dordrecht
(NL): Springer, Vol. 30 ‘Text, Speech and Language Technology’, pp. 131-157.
D.L. Scapin (2005). Exigences ergonomiques, méthodes et normes pour la conception de
Systèmes d’Information centrés humain. In Encyclopédie des Systèmes d’Information, Paris :
Vuibert. (to appear)
6.4 Publications in Conferences and Workshops
2002
J.-M. C. Bastien (2002). Panorama des outils d'aide au recueil d'information. In A. Drouin,
G. Eude, J.-M. Robert Eds., Actes 8ème Colloque Francophone Ergonomie et Informatique
Avancée (ERGO-IA'2002), Biarritz, 8-10 Octobre 2002, Bidart (64210) : ESTIA & ESTIA
Innovation.
J.-M. C. Bastien, D. L. Scapin (2002). Les méthodes ergonomiques : de l'analyse à la
conception et à l'évaluation. In A. Drouin, G. Eude, J.-M. Robert Eds., Actes 8ème Colloque
Francophone Ergonomie et Informatique Avancée (ERGO-IA'2002), Biarritz, 8-10 Octobre
2002, Bidart (64210) : ESTIA & ESTIA Innovation.
22
A. Capobianco (2002). Demandes d’aide procédurale : spécification des besoins d’utilisateurs
novices. In Actes 14ème Conférence Francophone sur l’Interaction Homme-Machine
(IHM’2002), Poitiers, 26-29 novembre 2002, New York : ACM Press, pp. 57-64.
A. Capobianco, N. Carbonell (2002). Conception d'aides en ligne pour le grand public : défis et
propositions. In A. Drouin, G. Eude, J.-M. Robert Eds., Actes 8ème Colloque francophone
Ergonomie et Informatique Avancée (ERGO-IA'2002), Biarritz, 8-10 Octobre 2002, Bidart
(64210) : ESTIA & ESTIA Innovation, pp. 309-335.
N. Carbonell, S. Kieffer (2002). Do oral messages help visual exploration? In J. van Kuppevelt,
L. Dybkjaer, N. Bernsen Eds., Proc. Int. CLASS Workshop on Natural, Intelligent and Effective
Interaction in Multimodal Dialogue Systems, Copenhagen, DK, June 28-29, 2002, pp. 27-36.
N. Carbonell, D.L. Scapin (2002). Multimodality: Contributions to the Universal Access
Research Agenda. In H. Luczak, A. Cakir, G. Cakir Eds., Proc. 6th Int. Scientific Conference on
Work With Display Units (WWDU'02), Berchtesgaden, Germany, May 23-25 2002, Berlin:
Ergonomic Institut für Arbeits- und Sozialforschung Forschungsgesellschaft mbH, pp. 628-630.
H. David, J.-M. C. Bastien (2002). Evaluation of a radically revised air traffic management
interface. In Proc. 21st Digital Avionics Systems Conference (DASC), ‘Air Traffic
Management for Commercial and Military Systems’, Irvine, California, USA, 27-31 October,
2002.
N. Grondin, J.-M. C. Bastien, B. Agopian (2002). Les tests utilisateurs : avantages et
inconvénients des passations individuelles et par paires. In Actes 14ème Conférence Francophone
sur l’Interaction Homme-Machine (IHM’2002), Poitiers, 26-29 novembre 2002, New York :
ACM Press, pp. 121-128.
V. Lucquiaud, F. Jambon, D. L. Scapin (2002). Outils de modélisation des tâches utilisateurs :
exigences du point de vue utilisation. In Actes 14ème Conférence Francophone sur l’Interaction
Homme-Machine (IHM’2002), Poitiers, 26-29 novembre 2002, New York : ACM Press,
pp. 243-246.
W. E. Mackay, C. Letondal, G. Pothier, K. Bøegh, H. E. Sørensen (2002). The Missing Link:
Augmenting Biologist's Laboratory Notebooks. In Proc. 15th ACM Symposium on User
Interface Software and Technology (UIST'2002), Paris, October 27-30, 2002, New York: ACM
Press, CHI Letters 4(2): 41-50. W. E. Mackay (2002). Integrating Multiple Perspectives on
Participatory Design: Interliving. Workshop on Domestic Technologies, at CHI’2002,
Minneapolis, Minn, April 20-25, 2002.
† W. E. Mackay (2002). Which Interaction Technique Works When? Floating Palettes, Marking
Menus and Toolglasses Support Different Task Strategies. In Proc. Int. Working Conference on
Advanced Visual Interfaces, Trento, It. May 22-24, 2002, pp. 203-208.
2003
C. Bach, D. L. Scapin (2003. Adaptation des Critères Ergonomiques aux Interactions Homme
Environment Virtuel. In Actes 15ème Conférence Francophone sur l'Interaction HommeMachine (IHM'03), Caen, 26-28 novembre 2003, New York : ACM Press, pp. 24-31.
† C. Bach, D. L. Scapin (2003). Adaptation of Ergonomic Criteria to Human Virtual
Environment. In Proc. 9th IFIP TC13 Int. Conference on Human-Computer Interaction
(INTERACT’03), Zürich, CH, September 1-5 2003, Amsterdam: IOS Press, pp. 880-883.
J.-M. C. Bastien (2003). Approches et méthodes ergonomiques pour la conception de systèmes
interactifs. In École d'été du CNRS, ‘Théories et méthodes pour la conception, l'évaluation et
l'usage d'environnements informatiques pour l'apprentissage humain’, Autrans, France, 4-9
juillet 2003.
23
J.-M. C. Bastien (2003). Les nouvelles technologies de l'information et de la communication
(NTIC) : nos méthodes d'analyse du travail sont-elles toujours adaptées ? In G. Vallery, Ed.,
Actes 18ème Congrès de la Société d'Ergonomie de Langue Française (SELF), Paris, France, 2426 septembre 2003, pp. 85-86.
J.-M. C. Bastien (2003). Étapes et méthodes clés pour une conception ergonomique des logiciels
interactifs. In Actes 4ème Colloque sur l'Enseignement des Technologies et des Sciences de
l'Information et des Systèmes (CETSIS-EEA), Toulouse, France, 12-14 novembre 2003.
A. Capobianco (2003). Experimental Evaluation of the Effectiveness of Expert Online Help
Strategies. In Proc. 10th Int. Conference on Human-Computer Interaction (HCI International
2003), Crête, Greece, June 22-27, 2003, Mahwah, NJ, Human-Computer Interaction – Theory
and Practice (Part 1), Vol. 1, London: Lawrence Erlbaum Associates, pp. 626-630.
A Capobianco (2003). Contextual online help design: questioning research practice. Workshop
‘Selecting best practice research methods for HCI: beyond fashion statements’ at HCI
International 2003, Crete, Greece, June 2003.
† A. Capobianco (2003). Questioning the effectiveness of contextual online help: some alternative
propositions. In Proc. 9th IFIP TC13 Int. Conference on Human-Computer Interaction
(INTERACT’03), Zürich, CH, September 1-5 2003, Amsterdam: IOS Press, pp. 65-72.
† H. Hutchinson, W. Mackay, B. Westerlund, B. B. Bederson, A. Druin, C. Plaisant, M.
Beaudouin-Lafon, S. Conversy, H. Evans, H. Hansen, N. Roussel, B. Eiderback, S. Lindquist,
Y. Sundblad (2003). Technology Probes: Inspiring Design for and with Families. In Proc. ACM
Int. Conference on Human Factors in Computing Systems (CHI 2003), Fort Lauderdale, FL,
April 5-10, 2003, New York: ACM Press, CHI Letters 5(1): pp. 17-24.
S. Kieffer, N. Carbonell (2003). Assistance orale à la recherche visuelle dans des affichages
complexes. In Actes 15ème Conférence Francophone sur l'Interaction Homme-Machine
(IHM'03), Caen, 26-28 novembre 2003, New York : ACM Press, pp. 126-133.
B. Westerlund, S. Lindquist, W. E. Mackay, Y. Sundblad (2003). Co-designing methods for
designing with and for families. In 5th European Academy of Design Conference (EAD'03),
Barcelona, Spain, April 2003.
2004
C. Bach, D. L. Scapin (2004). Obstacles and Perspectives for Evaluating Mixed Reality Systems
Usability. In Proc. Workshop MIXER ‘Exploring the Design and Engineering of MR systems’
at IUI-CADUI 2004, Funchal, Portugal, January 2004.
E. Dubois, B. Mansoux, C. Bach, D. L. Scapin, G. Masserey, J. Viola. (2004). Un modèle
préliminaire des systèmes mixtes. In Actes 16ème Conférence Francophone sur l'Interaction
Homme Machine (IHM’04), Namur, 30 août-3 septembre 2004, New York : ACM Press, pp.
43-54.
2005
N. Carbonell (2005). Ambient Multimodality: an Asset for Developing Universal Access into
the Information Society. In Proc. 3rd Int. Conference on Universal Access in Human-Computer
Interaction, Las Vegas, July 22-27, 2005, Universal Access in HCI: Towards an Information
Society for All, Vol. 3, Mahwah (NJ), London: Lawrence Erlbaum Associates, pp. 266-270.
W. Cybis, D. L. Scapin, M. Morandini (2005). ERGOMANAGER: a UIMS for monitoring and
revising user interfaces for web sites. In Int. Conference on Web Information Systems and
Technologies (WEBIST), Miami, USA, May 26-28 2005.
W. Cybis, D. L. Scapin, M. Morandini, (2005). ERGOMANAGER: um sistema gerenciador de
interfaces com o usuârio para sites web transacionais. In Congresso Latino-americano de
Interacao Humano-Computador (CLIHC’05), Cuernavaca, Mexico, October 23-26.
24
E. Law, E. Hvannberg, J. Vanderdonckt, C.Stary, M. Springett, D. L. Scapin, P. Palanque,
G. Cockton (2005). MAUSE: Network of Usability Excellence. In 19th Annual Conference of
the British HCI Group (BCS HCI), Edinburgh, Scotland, 5-9 September 2005 Poster, SpringerVerlag.
† E. Law, E. Hvannberg, J. Vanderdonckt, C. Stary, M. Springett, D. L. Scapin, P. Palanque, G.
Cockton (2005). Towards the Maturation of IT Usability Evaluation (MAUSE). In: M.-F.
Costabile, F. Paterno Eds., Proc. 10th IFIP TC13 Int. Conference on Human-Computer
Interaction (INTERACT’05), Rome, September 12-16, 2005, LNCS Series n° 3585, Springer.
V. Lucquiaud (2005). Proposition d’un noyau et d’une structure pour les modèles de tâches
orientés utilisateurs. In Actes 17ème Conférence Francophone sur l’Interaction Homme-Machine,
Toulouse, 27-30 septembre 2005, New York : ACM Press, pp. 83-90.
† J. Simonin, S. Kieffer, N. Carbonell (2005). Effects of display layout on gaze activity during
visual search. In M.-F. Costabile, F. Paterno Eds., Proc. 10th IFIP TC13 Int. Conference on
Human-Computer Interaction (INTERACT’05), Rome, September 12-16, 2005, LNCS Series
n° 3585, Springer, pp. 1054-1058.
J. Simonin, S. Kieffer, N. Carbonell (2005). Influence de l’organisation spatiale des affichages
sur l’efficacité de la recherche visuelle. In Actes 17ème Conférence Francophone sur l’Interaction
Homme-Machine, Toulouse, 27-30 septembre 2005, New York : ACM Press, pp. 35-42.
6.5 Internal Reports
Deliverables
2002
C. Bach, D. L. Scapin (2002). Recommandations pour l’inspection ergonomique
d’environnements virtuels. Rapport, Contrat Comedia, Rocquencourt : INRIA.
N. Carbonell (2002). Interview guidelines: European SMEs' current design practice for
implementing Universal Access. Deliverable 2.2.2, Thematic Network (IST) ‘Information
Society for All’ (IS4ALL), 10 p., Nancy: LORIA.
W. Cybis, D. L. Scapin, D. P. Andres, M. Morandini (2002). ErgoCoIn: a usability inspection
tool for web site evaluation. Rapport, Convention CNPq-INRIA, Project TVU-CECI,
Rocquencourt : INRIA..
N. Grondin (2002). Rapport d’analyse : questionnaire X35E sur les normes en ergonomie de
l’informatique. Rapport, AFNOR, Rocquencourt : INRIA.
W. E. Mackay, G. Pothier (2002). Dispositif et procédé de gestion de données entre
équipements de communication en vue de l’obtention d’un service. N-AFF 61, brevet fr.02
13387.
R. Taillefer (2002). Recensement de quelques sites web de vente en ligne dans les industries de
l’habillement. Rapport, Contrat Comedia, Rocquencourt : INRIA.
C. Tison, D. L. Scapin, N. Carbonell (2002). Techniques et modalités d’interaction hommemachine avec des environnements et des objets 3D : revue de questions. Rapport, Contrat
Comedia, Rocquencourt : INRIA.
2003
N. Carbonell (2003). Multimodal interfaces − A generic design approach. Deliverable
3.3, Thematic Network (IST) ‘Information Society for All’ (IS4ALL), 12 p., Nancy: LORIA.
25
N. Carbonell, F. Lepage (2003). Télé-commerce : bilan et perspectives. Rapport,
Contrat de Plan Etat-Région, Pôle Intelligence Logicielle, Thème Télé-opération et
Assistants Intelligents, Projet Télé-commerce, 18 p., Nancy : LORIA.
N. Grondin, D. L. Scapin (2003). Revue de questions sur la charge mentale de travail : notions,
mesures et prédictions. Rapport, Convention de recherche INRIA-SNCF-RATP (projet
PREDIT), Rocquencourt : INRIA.
D. L. Scapin (2003). Rapport final : convention de recherche INRIA-LECTRA. Rapport,
Contrat Comedia, Rocquencourt : INRIA.
2004
N. Carbonell (2004). Commande multimodale, parole+regard. Rapport, Contrat de Plan Etat-
Région, Pôle Intelligence Logicielle, Thème Télé-opération et Assistants Intelligents,
Projet TéléVis, 28 p., Nancy : LORIA.
N. Carbonell (2004). Contribution – Rapport d’avancement à mi-parcours. Contrat ACI
‘Masses de Données’, Projet Micromégas, 4 p., Nancy : LORIA.
N. Grondin, D. L. Scapin (2004). Prédiction de la charge mentale de travail des opérateurs des
postes d’aiguillage ferroviaire : premiers résultats. Rapport, Convention de recherche INRIASNCF-RATP (projet PREDIT), Rocquencourt : INRIA.
2005
N. Carbonell (2005). Livrable Phase 1 : Revue de l’état de l’art sceintifique et technique.
Rapport, Contrat de Prestation de Recherche Externalisée (CRE, France Télécom), Nancy :
LORIA, 24 p.
N. Carbonell (2005). Livrable Phase 2 : Conception du protocole expérimental. Rapport,
Contrat de Prestation de Recherche Externalisée (CRE, France Télécom), Nancy : LORIA, 13 p.
V. Lucquiaud (2005). Modèles et outils pour la description de l’activité orientée tâche. Rapport,
Convention de recherche INRIA-SNCF-RATP (projet PREDIT), Rocquencourt : INRIA.
Research Reports
2002
W. E. Mackay (2002). ATC’01: Future Air Traffic Control User Interfaces. Rapport de
Recherche, Rocquencourt : INRIA. DVD for EuroControl Workshop, Brétigny sur Orge.
W. E. Mackay (2002). Using Video to Support Interaction Design. DVD Tutorial distributed at
CHI’02, Rocquencourt: INRIA.
M. Negri (2002). Liens entre performance, apprentissage et préférence dans l’évaluation de sites
commerciaux sur Internet. Mémoire de Maîtrise, Paris : Université René Descartes, Paris V.
B. Renaudin-Humbert (2002). Rôle du regard en tant que mode d’expression dans un
environnement d’interaction multimodale. Mémoire, DEA Informatique, Ecole Doctorale
IAEM-Lorraine, 44 p., Nancy : LORIA.
2003
M.-L. Belin (2003). Utilisabilité des sites web : liens entre performance, apprentissage et
préférence, et apport de l’investigation oculométrique. Mémoire de Maîtrise, Paris : Université
René Descartes, Paris V.
C. Mennessier (2003). Performance, apprentissage et exploration visuelle : une approche des
déterminants de la préférence dans l’évaluation comparative des sites web. Mémoire de
Maîtrise, Paris : Université René Descartes, Paris V.
26
J. Simonin (2003). Présentation de documents procéduraux : apports de la multimodalité à la
mise en œuvre de consignes. Mémoire, DEA Informatique, Ecole Doctorale IAEM-Lorraine,
59 p., Nancy : LORIA.
E. Wiederkhier (2003). L’analyse de l’interaction vocale homme-machine : étude comparative
exploratoire de l’analyse des contingences et de l’analyse conversationnelle. Mémoire de
Maîtrise, Paris : Université René Descartes, Paris V.
2004
C. Bach, D. L. Scapin (2004). Critères ergonomiques pour les interactions hommeenvironnements virtuels : définitions, justifications et exemples. Rapport de recherche,
Rocquencourt : INRIA.
O. Christmann (2004). Navigation multi-échelle dans des visualisations de grands ensembles
d’informations. Mémoire, DEA Informatique, Ecole Doctorale IAEM-Lorraine, 45 p., Nancy :
LORIA.
2005
T. Blanc-Brude, D. L. Scapin (2005). Etude de la mémoire des documents chez l’utilisateur
pour la conception des systèmes d’aide à leur récupération. Rapport de recherche,
Rocquencourt : INRIA.
W. Cybis (2005). ErgoManager: a UIMS for monitoring and revising user interfaces for web
sites. Rapport de recherche, Rocquencourt : INRIA.
27