robot teams - JNRR 2013, Annecy

Interaction homme-robot
Tour d’horizon et défis
Rachid Alami
LAAS-CNRS
Toulouse - France
JNRR – Annecy – 16 Octobre 2013
A shift in the perspective:
From a single robot to humanrobot teams
2
1
Autonomous Robotics
n 
a dream and a challenge
n 
The robot was alone
n 
n 
n 
n 
n 
Very capable
Very intelligent
You give it a goal/task
a
It does the job
… end it is ready for a new adventure
hero
The collaborative robot
n 
New abilities
n 
Far more challenging and subtle …
2
n 
n 
Robots will be among us
Robots will have to be able to do
something useful
3
n 
Robots will have to be safe
Robots will have to plan (and adapt)
their activities with us or simply in our
presence
n  Robots will need models and algorithms
n  Models of us
n 
4
When it comes to Human-machine
interaction
We have a precursor
10
5
« Modern Times »
n 
n 
n 
n 
n 
n 
n 
n 
Shot between 1932 and August 1935
Première : Februray 5, 1936
Proximal Interaction
Initiative Taking
Adaptation to human
Pertinence of choices / decisions
Acceptability issues
Reliability – Fault tolerance
An added parameter: the human
n 
n 
…
Besides design choices, it is necessary to
endow the robot with the ability to take
explicitly into account the presence of humans
Re-visit the robot decisional capabilities
12
6
Questions for a robot which collaborates
with humans: what, who, where, when,
how?
Do, Observe, Decide/plan, Communicate
Integrative approach for a robot that acts
in interaction with humans
n 
Work on Collaborative / Interactive task
achievement
n 
n 
n 
n 
based on a study of human-robot interaction
inspired from Joint activity / Teamwork
concretized as a set of robot decisional abilities
Work: Inspiration and Collaboration
n 
n 
n 
n 
n 
Cohen P. R., Levesque H. J. (1991), Tambe (Teamwork), H.
Clark (Joint Activities, Dialogue)�
Bratman (1992). Shared cooperative activity.
Tomasello M. Warneken F. et al (2005 - ..)
Knoblich G. Sebanz N. et al. (2009 - ..)
Pacherie, E. (2012 - ..)
7
Topics
n 
n 
n 
n 
n 
n 
n 
n 
Social signal processing / Situation assessment
Production de signaux sociaux (boucles sensori-motrices
multi-modales, rythmes ..)
HR Physical interaction
Situated dialogue
Theory of Mind / Belief Management
Proactivity and turn taking
Acceptability issues
Intentional actions and motions
Integrative approach for a robot that acts
in interaction with humans
n 
Work on Collaborative / Interactive task
achievement
n 
n 
n 
n 
based on a study of human-robot interaction
inspired from Joint activity / Teamwork
concretized as a set of robot decisional abilities
is progressively producing a coherent basis for
Joint Human-Robot Activity
8
List of co-authors
Rachid Alami, Samir Alili, Gérard Bailly, Michael Beetz, Ludovic
Brethes, Maxime Cottret, Raja Chatila, Aurélie Clodic, Patrick
Danes, Kerstin Dautenhan, Xavier Dollat, Peter Dominey,
Frédéric Elisei, Isabelle Ferrane, Sarah Fleury, Martin Haegele,
Matthieu Herrb, Mamoun Gharbi, Mokhtar Gharbi, Guillaume
Infantes, Felix Ingrand, Ken Koay, Madhava Krishna, Thibaut
Kruse, Jens Kubacki, Jean-Paul Laumond, Christian Lemaire,
Frédéric Lerasle, Séverin Lemaignan, Efrain Lopez Damian,
Jerome Manhes, Philippe Marcoul, Luis Marin, Jim Mainprice,
Paulo Menezes, Vincent Montreuil, Christopher Nehaniv,
Christopher Parlitz, Amit Pandey, Raquel Ros Espinoza, Daniel
Sidobre, Thierry Siméon, Akin Sisbot, Mick Walters, Felix
Warneken, Matthieu Warnier, Britta Wrede, Sara Woods.
A task-oriented architecture for an interactive
robot
�   Task-Oriented: How to perform
a task, in presence or in
interaction with humans, in the
best possible way
� 
� 
� 
� 
H&R Sharing Task and Space
Efficiency
Safety
Acceptability
Intentionality
�   Plan-Based: Planning and OnLine Deliberation
� 
� 
� 
� 
Reasoning
Anticipation
Pro-active behaviour
Theory of Mind – Predicting and
reasoning about human activity and
mental state
9
Decisional ingredients for an Interactive Autonomous Assistants
A Control Architecture
for a Cognitive Assistant
Human-Robot Joint Action
Affordances
Perspective Taking
Human Aware Navigation
Shared Plans:
Human Aware Task Planning
Human Aware Manipulation
RIS
Adapting motion dynamics
“Soft” Motion
19
Who can do what with which object with which effort.
Put Video
(Manipulability_Graph_Validation_Short)
Put Video
(Manipulability_Graph_Validation_short_sce_2)
10
Now let us clean the table (or put the tools in the box)
+
Who can do what with which object with which
effort.
Who can do what for whom, with which effort
and where.
Put Video
(PR2_clean_table_affordance_graph_shorter_2.)
Affordance Graph
Robot dealing with HR shared activity
n 
Robot maintains a (symbolic) state:
n 
n 
n 
n 
n 
Robot elaborates plans involving H & R
n 
n 
n 
n 
tracking of objects visible to it
Tracking of human posture and hands
visibility and reachability of objects w/r human
Reachability of objects wiith respect to iself
Computes a plan involving (potentially) both itself and the
human
Human action is selected when necessary
Robot computes also grasps, placements and motions
Robot executes plans and monitors H activity
n 
n 
n 
By performing its own actions
Asking human to achieve some actions
Re-plans when necessary: state different from what is expected
11
Robot dealing with HR shared activity
n 
Robot maintains a (symbolic) state:
n 
n 
n 
n 
n 
Robot elaborates plans involving H & R
n 
n 
n 
n 
Tracking of objects visible to it
Tracking of human posture and hands
Visibility and reachability of objects w/r human
Reachability of objects wiith respect to iself
Computes a plan involving (potentially) both itself and the
human
Human action is selected when necessary
Robot computes also grasps, placements and motions
Robot executes plans and monitors H activity
n 
n 
n 
By performing its own actions
Asking human to achieve some actions
Re-plans when necessary: state different from what is expected
JOINT GOAL: CLEAN THE TABLE
Kinect
Camera + Pan/Tilt platfrom
12
Review Meeting Y1
26
13
Merci …
27
14