In the end of july we had our first factory run.

It served as a test for the bigger run in October which will be done in conjunction with the EU project TAPAS.

The setup was slightly smaller scale to not disturbe production in case something goes wrong: We used mock-ups for the feeders and for the magazin.

The experiment was carried out in the factory hall next to the production line.

The experiment extended the previously published video in two ways: 1) while the video shows the training part only, the new experiment obviously also contains the programing part where the robot is able to play back the program. 2) the experiment is done under real condiditions.

The experiment went very well during a sunday but the programming part using the gestures essentially failed during the monday. 

We observed so far two errors

1) It seems that the Kinect camera that we are using as main input gets disturbed by some of the running machines.

2) Illumintation from the roof might have been a problem so that we might have to request robot programming at special locations, but this is not clear.

We will further investigate this in August.

In the past reporting period we have mainly consilidated our demo in order to prepare for our large scale experiment at Grundfos in October. In addition, we have started investigating the use of an iPad for skill and task programming.

In the last reporting period we have finished our first complete demo.

In this demo, which will be visible under youtube, the user first specifies that the robot should do a feeding task.

Then, the human specifies the necessary parameters through gestures: the human operator points at the SLCs containing the parts for the feeding task. At this point, the robot actually grasps the corresponding SLC. Then, the human shows the robot the feeder into which these parts should go. Finally, the robot empties the SLC into the right feeder.

Once the library of different task programs is provided, the robot can be use for different tasks without difficult re-programing.

Even though the present demo was tested so far only in the lab, most of the components of the system have already been tested at Grundfos in the real scenario. The only system component that was not tested so far is the Kinect-based human-robot interaction. Given that the Kinect is a well-tested Microsoft product, we do not expect any difficulties when finally testing the system on the shop floor.

In December we have finished our demo: A human operator is able to tell a robot what object to take by simply pointing at it from some distance. The video of that demo is available under YouTube http://youtu.be/H6kX494AsaE. For this demo, we have included a completely automatic calibration procedure that calibrates the robot arm with the camera. The human pointing gesture is perceived by the camera, the pointing direction is recognized. The system then identifies the object that is in the pointing direction and directs the robot arm to that object in order to pinpoint the object the robot thinks the human pointed to. In this reporting period we have done an evaluation of how good the pointing recognition works. The evaluation was done with 8 different and un-trained individuals who pointed at the three different boxes from 2m, 3m and 4m distance. It can be expected that recognition rate degrades with distance.  Recognition rates were  

In this demo, we have used the hand and arm for pointing. This works well, if the distance between the robot and the human is large enough. In the next reporting period we are going to report experimental results in which we only use the hand for pointing. This is useful when the robot is very close to the human.