Follow us with InsideScience and take a look over the shoulders of
researchers.
In this episode of Intention Recognition Dirk Gehrig from the
Karlsruher Institute of Technology explains what a robot needs for
intention recognition.
Hello, I'm sure you've wished for someone to do the housework for you.
Imagine you have a house as a robot.
Of course, you don't want to explain every little thing to it, but you
want it to read every wish from your eyes.
And now I'm going to explain how we do that with our robot.
Imagine there is a table with a person behind it.
The person has a tablet in their hand.
And on the tablet are objects and on the table is also an object.
Now we want to find out what the person is doing right now or what
intentions they have.
If we observe someone as a human being, we automatically notice what
their intention is.
As a robot, it has to look at things more closely.
For example, it can look at what time of day it is right now.
Or what activity the person is doing right now.
In our case, objects are moved back and forth.
The whole thing on a table.
If we now know what activity is being done right now, we still don't
know exactly what the person is doing.
That means we're still looking at the movement of the person.
For example, I'm putting an object on the table right now.
Now we know that the person is moving objects, but we still don't know
which objects.
That's why we still have to look at which objects are on the table.
For example, is there a bowl?
Or a spoon?
Or a muesli package?
So if the robot knows what time of day it is right now, what activity
is being done, it can deduce what the intention is.
But first it has to find out what the current activity or movement is.
To do this, the robot stands in front of the person.
It looks at what the person is doing with its camera.
And using the camera images, it can recognize the activity, it can
recognize the movement, and it can recognize the objects.
Assuming it's quarter to seven, the activity is moving objects back
and forth, the movement is putting an object on the table, and we have
the following objects, a bowl, a muesli package, and a spoon.
What does the person want to do?
She's having breakfast.
So if the robot knows that the person is having breakfast, then it can
clean the dishwasher, so that the person can put her dirty dishes in
it.
Now I'm going to show you what it looks like in practice.
Here we see the live demo of the explanations I just gave.
Our assistant will walk into the room, perform an activity on the
table, and at the same time be observed by our robot.
Timo, please.
You can see the robot's head here.
It's observing what's happening on the table.
Timo is now covering the table, putting a plate and a cup on it.
And at the same time, there's a live demonstration.
The demonstration shows us which movement is being performed.
An object is being put from the middle to the outside.
As an activity, it recognizes that the table is being covered.
And these two demonstrations are connected to the intention
demonstration.
The intention demonstration recognizes that a table is being covered.
We just saw in the explanation that there are four components that are
part of the intention demonstration.
The activity demonstration and the movement demonstration have just
been shown live.
At the same time, we have the daytime.
It's currently 7.15 a.m.
And we have object knowledge.
For example, there's a high probability that there's a plate or a cup
on the table.
And in total, the intention demonstration recognizes that the table is
being covered.
The robot can now start and set things for the day.
Or it can continue to observe what Timo is doing.
Timo is sitting at the table.
And he's eating the breakfast that he just prepared.
And if the robot now observes what he's doing, it recognizes that Timo
is having breakfast with eggs and can react accordingly.
And the future?
I think that the intention demonstration will play a big role in the
future.
If we want humanoid robots to interact with humans in everyday life,
if we want humanoid robots to help and support humans, that's a very
important thing.
And we will continue to implement the intention demonstration on the
basis of actions.
That means we will analyze the actions of humans and on that basis,
the robot will intervene accordingly.
There are other techniques.
You could proceed more invasively.
You could, for example, intercept the brainwaves of humans, etc.
But that would be too invasive.
So we really want to investigate on the basis of actions what humans
would like.
We will use more sensors than usual, not just visual sensors, but we
will also use distance sensors, deep sensors like the Microsoft Kinect
and on that basis try to read the needs of humans.
For the intention recognition, the robot basically needs the
connections between the actions it sees and the intention of the
human.
We would like to learn that automatically.
So far, we have extracted it from expert knowledge.
Experts have brought their knowledge in, we have partly learned it
from data, and of course we would like to have that much easier.
An alternative would be to use telepresence now.
The telepresence that you see here in the background.
And that the user can now log in to his robot.
He can move into the robot, can demonstrate the environment through
the eyes of the robot and thus the robot has a much easier way to
learn these essential connections and then use them for the subsequent
intention recognition.
The KIT series InsideScience looks over the shoulders of the
scientists of the special research area.
See in further posts how a robot is constructed, how it learns, how it
interacts with people and which social aspects are discussed.