Call2 Monitoring GISA

Public Summary Month 7/2012

Fri, 27 Jul 2012 03:39:34 GMT

In the end of july we had our first factory run.

It served as a test for the bigger run in October which will be done in conjunction with the EU project TAPAS.

The setup was slightly smaller scale to not disturbe production in case something goes wrong: We used mock-ups for the feeders and for the magazin.

The experiment was carried out in the factory hall next to the production line.

The experiment extended the previously published video in two ways: 1) while the video shows the training part only, the new experiment obviously also contains the programing part where the robot is able to play back the program. 2) the experiment is done under real condiditions.

The experiment went very well during a sunday but the programming part using the gestures essentially failed during the monday.

We observed so far two errors

1) It seems that the Kinect camera that we are using as main input gets disturbed by some of the running machines.

2) Illumintation from the roof might have been a problem so that we might have to request robot programming at special locations, but this is not clear.

We will further investigate this in August.

Public Summary Month 5/2012

Fri, 25 May 2012 05:56:04 GMT

In the past reporting period we have mainly consilidated our demo in order to prepare for our large scale experiment at Grundfos in October. In addition, we have started investigating the use of an iPad for skill and task programming.

Public Summary Month 3/2012

Fri, 23 Mar 2012 09:56:53 GMT

In the last reporting period we have finished our first complete demo.

In this demo, which will be visible under youtube, the user first specifies that the robot should do a feeding task.

Then, the human specifies the necessary parameters through gestures: the human operator points at the SLCs containing the parts for the feeding task. At this point, the robot actually grasps the corresponding SLC. Then, the human shows the robot the feeder into which these parts should go. Finally, the robot empties the SLC into the right feeder.

Once the library of different task programs is provided, the robot can be use for different tasks without difficult re-programing.

Even though the present demo was tested so far only in the lab, most of the components of the system have already been tested at Grundfos in the real scenario. The only system component that was not tested so far is the Kinect-based human-robot interaction. Given that the Kinect is a well-tested Microsoft product, we do not expect any difficulties when finally testing the system on the shop floor.

Public Summary Month 1/2012

Thu, 26 Jan 2012 18:53:07 GMT

In December we have finished our demo: A human operator is able to tell a robot what object to take by simply pointing at it from some distance. The video of that demo is available under YouTube http://youtu.be/H6kX494AsaE. For this demo, we have included a completely automatic calibration procedure that calibrates the robot arm with the camera. The human pointing gesture is perceived by the camera, the pointing direction is recognized. The system then identifies the object that is in the pointing direction and directs the robot arm to that object in order to pinpoint the object the robot thinks the human pointed to. In this reporting period we have done an evaluation of how good the pointing recognition works. The evaluation was done with 8 different and un-trained individuals who pointed at the three different boxes from 2m, 3m and 4m distance. It can be expected that recognition rate degrades with distance. Recognition rates were

2m	0.8442
3m	0.9535
4m	0.6923

In this demo, we have used the hand and arm for pointing. This works well, if the distance between the robot and the human is large enough. In the next reporting period we are going to report experimental results in which we only use the hand for pointing. This is useful when the robot is very close to the human.

Public Summary Month 11/2011

Fri, 25 Nov 2011 14:02:22 GMT

An ad-hoc pointing detection and parameter extraction methods has been implemented on ROS, running real-time and on-line. We have implemented an ad-hoc version to be able to focus on implementational issues. The adhoc poiting recognition will be replaced by a parametric HMM within the next reporting period.

Steps during the next reporting period:
Next upcoming even it the deliverable in M10 where we will demonstrate the first integrated version of our system that is able to track and recognize human pointing gestures. For this, it is necessary to be able to 1) use the Kinect camera to track the human (head and hand in particular), 2) to identify the pointing gesture and to where the human is pointing, and 3) to calibrate the robot arm with the Kinect camera so that the robot is able to move its endeffector to the object at which the human was pointing. In the demo-derliverable, the robot will be able to touch the object at which the human was pointing.
The M10 deliverable will be a first demo running on the Little Helper Plus. It will be built directly on the present setup of our robot.

Public Summary Month 9/2011

Tue, 27 Sep 2011 15:03:52 GMT

We have carried out an experiment to test the accuracy and precision of the Kinect. We are currently analysing the data from this experiment.

We have demonstrated basic robot interaction at the Scandinavian Industry Expo 2011 in Herning, Denmark. A Kinect camera was placed on the robot, so people visiting the expo could come to our booth and try having the robot arm imitate their own arm movements. This is the first step in the development of software that can recognize human gestures and translate them into commands for an industrial robot.

The video from the Herning Expo can be seen under http://www.youtube.com/watch?v=xja_oNAd5Mg

Public Summary Month 8/2011

Wed, 10 Aug 2011 14:22:40 GMT

Production machines in factory halls are often equiped with feeders that contain the parts that are used during the production process. These feeders need to be refilled once in a while. The aim of this project is to develop a small, mobile robot that is able to assist humans in feeding task of production machines in a factory hall: A human who is not trained to work with robots should be able to show the mobile robot what it should do, i.e., by pointing at corresponding boxes and feeders in order to clarify with parts belong into which feeder. The aim is to become able to communicate with such a robot in a way that does not require complex programming or long-term training. Instead, robot programming should be possible 'on-the-fly' with a minimal time consumption for the actual programming task.

Present progress in this project contains a) communication of the mobile robot with the production line of a collaborating company, b) installation of a state-of-art robot arm on a mobile platform, c) use of the Microsoft KINECT camera for human tracking.