Call2 Monitoring BRACOG

Public Summary Month 11/2012

Thu, 03 Jan 2013 09:08:48 GMT

We tested our system with a couple of natural objects (telephone headset, cup, tea box, and ball) which are more or less good-natured but which can neither be called artificial nor simplified. Although there are big gaps in the surface reconstruction no problematic artifacts could be observed that prevented the successful grasp of the tested objects when the object was standing upright.

In our experiment BRACOG we could successfully show that it is possible to initiate grasps only by voluntary modulation of brain waves. We conclude that we finished the experiment successfully.

Public Summary Month 9/2012

Thu, 18 Oct 2012 14:50:38 GMT

We started to implement an automated error correction. The detection of an error we aim to decode from brain signals that are evoked after an unexpected feedback.

We succeeded to complete the steps from object recognition to a successful grasp of the robotic gripper. In the following we present the underlying tool-chain.

The scene scan consists of recording two stereoscopic images. We extract point clouds that circumscribe the objects.

The grasp-planner iteratively moves the robot in the force-field.

The robot is moved synchronously with its virtual model.

Public Summary Month 7/2012

Fri, 03 Aug 2012 08:03:45 GMT

TASK 3 + TASK 4:

We finished the verification of object selection paradigms for grasp initiation. The P300 and SSVEP paradigms differ in the number of alternatives as well as trial durations, but reveal comparable detection rates and information transfer rates (bit/min, see Figure 1). We prefer the P300 over the SSVEP paradigm for object selection for three reasons: the feasibility of gaze independence, the flexibility regarding the number of selectable objects and the more convenient task experience of subjects.

Figure 1: Direct comparison of bit-rates between SSVEP as well as the P300 experiment (each square one subject). Blue data points are scaling in bit/trial, red data points in bit/min. Circles represent averages over subjects.

TASK 9:

After calibration, matching pixels in the left and right camera pictures can be used to calculate their spatial depth. To this end, all resulting 3D points are defined in the camera coordinate system. The transformation to robot coordinates is calculated by using calibration markers in the robot framework. Furthermore, we finished the image-based segmentation algorithm.

Figure 3: Markers on the robot framework

Figure 4: Recognized grasp targets

In order to deal with artifacts, we added an algorithm to analyze the neighborhood of each 3D-point. Figure 5 shows the result of the algorithm.

Figure 5: Overlay of original and artifact-cleansed scene. Orange points are identified as artifacts and deleted.

TASK 10:

We realized a need to fine-tune our strategy to deal with huge holes in the grasp targets resulting from object recognition. Our current strategy is to put the point-poles used for the force-based grasp planning on a bounding ellipsoid containing the object (see Figure 6).

Figure 6: Distributed point-charges for force-based grasp planning of 3D-recognized objects

Public Summary Month 5/2012

Fri, 22 Jun 2012 08:41:25 GMT

TASK 3:

We further investigated the motor imagery (MI) paradigm in more detail by evaluating the ability of 17 subjects to control the grasp initiation of a virtual arm. Our results indicate that good MI control in our BCI critically depends on the presence of µ‑rhythms Subjects who did not exhibit the rhythm could not achieve control.

We now extended the object selection paradigm such that a virtual robot performs a grasp of the object as feedback rather than displaying colored rings. Moreover, we successfully tested the system with the real robot moving simultaneously with the virtual robot.

TASK 4:

We improved our P300 based object selection setup. Specifically, we increased the number of selectable objects to six. The current paradigm also will serve to derive error signals from brain activity when erroneous feedback was provided.

TASK 9:

We realized that lighting conditions are critical for optimal stereo object recognition.

Therefore, we extended our setup by a dimmable light source (see Figure 1 and Figure 2 )

Figure 1: Object recognition system. The new system consists of two grayscale cameras and a dimmable lamp (2x55W).

Figure 2: Current demonstrator setup

With improved image quality we succeeded in scanning different objects placed on the table in front of the robot (see Figure 3 and Figure 4).

Figure 3: Different test objects for optical recognition

Figure 4: Point clouds obtained from the 3D object recognition system with improved illumination

TASK 11:

We submitted a multimedia report draft to ECHORD which is now embedded in our websites.

Public Summary Month 3/2012

Fri, 13 Apr 2012 12:03:53 GMT

TASK 3:

We finished the test of the first version of the SSVEP based object selection task with 19 participants. The maximum recognition rate was 91.7% (guessing level 25%). So far the SSVEP-paradigm provided the best results among the tested algorithms.

TASK 4:

We found a significant improvement to using the object’s flicker frequency alone when we included the 1st and 2 nd harmonic of the flicker frequency. No further improvement was found with different approaches of preprocessing, noise cancellation, frequency decomposition, channel selection and different classifier algorithms.

Figure 1: Demonstrator and virtual model

TASK 9:

A stereo-vision based object-recognition-system brings along a major additional complication compared to geometry imported from a 3D modeler: Only parts of the object surface visible to the cameras can be recognized. We realized during the experiment, the robot has a quite limited workspace. Consequently, an eye-in-hand-setup in fact makes no sense: Therefore we constructed a mechanical carrier holding the cameras in one specific position (see Figure 2).

Figure 2: Current demonstator setup

TASK 11:

We finished the implementation of the new RESI communication methods. Now we can couple the robot with the MEG system over the Internet.

Public Summary Month 1/2012

Fri, 10 Feb 2012 10:10:10 GMT

TASK 3:

Figure 1 depicts the achieved rates of correctly detected objects in the 4 target SSVEP selection task for all eleven participants so far. This shows the successful application of the task in the majority of subjects.

Fig. 1 Average detection rate of single subjects. Error bars indicate the standard error across sessions (4 to 6 sessions). The guessing level for a balanced 4 target session is at 25% (dotted line).

TASK 4:

Based on simultaneous MEG and ECG recordings of 3 additional subjects we currently test algorithms to remove ECG artifacts from the MEG signals without removing information in brain signals.

TASK 10:

Objects with strong convex surfaces prevent the virtual grasper to get a force closure grasp. Therefore, we temporarily neglect repulsive forces driving the grasper away from the target center point. The calculation time for a force closure grasp is reduced to less than 5 s for the majority of CAD-generated grasping targets.
For fine concave structures (e.g. bristles) the BSP-tree collision detection algorithm requires too much time and is not parallelizable. We developed a new collision detection strategy which we will call shriveled bounds. Although the bristles of the toothbrush (see Fig. 2) do not prevent the object from being grasped further optimization is required.

Fig. 2: A force-closure grasp of a toothbrush

TASK 11:

Recently, a poster was accepted at the CogSys Conference 2012 in Vienna, Austria.

Public Summary Month 11/2011

Tue, 20 Dec 2011 15:23:33 GMT

TASK 3, 4

So far our tests in a movement imagery task revealed huge performance differences between subjects. In our MEG setup we found that magnetic activity of the heart is the source of another important artefact. The main work in this project phase will be the adjustment of the algorithms to allow for online removal of these artefacts.

TASK 5:

We integrated the functionality of the selection prototype, the virtual robot and grasp planner to build the virtual prototype.

Fig. 1 Integrated Scenario

TASK 7:

The grasp planning is still very time-consuming. To reduce the waiting period until completion of the planned grasp we optimized the process in the following way: The transformation calculation from attractive or repulsive force at the fingertip of the grasper to a torque in the joints of the robot is parallelized.

Currently the system comprises the parts depicted in Fig. 2.

Fig. 2 System Structure

Public Summary Month 09/2011

Thu, 20 Oct 2011 09:56:48 GMT

In the last bi-monthly report period we were able to assemble the Mitsubishi RV-E2 with the SCHUNK SDH Gripper (T8) and transferred the final CAD-geometry models to our VR-tool.

Fig. 1 Final geometry model of the manipulator and first test scenario

Fig. 3: Generated grasps in the new scenario all classified as force closure

As the grasp planner moves the robot only very close to the target and cannot assure predefined grasping forces, the final step to get a firm hold of an object has to be done only by the robot controller without VR-support. This functionality is realized as a service.

We tested three subjects in a target selection task based on steady state visual evoked potentials (SSVEP) while we recorded the magnetoencephalogram (MEG). In the online processing cycle we implemented the possibility to continuously retrain a classifier within a BMI-session to compensate non stationarities. This approach fosters mutual adaptation of classifier and human: the classifier learns the subject’s brain patterns while the subject learns to control the actuator using the classifier.

Public Summary Month 7/2011

Fri, 19 Aug 2011 14:45:22 GMT

In Brain-Computer-Interface settings the task of the subjects is to deliberately modify brain activation to communicate. The imagery of motor actions and to focus attention to different flickering objects are two communication strategies implemented in the first period.

We intend to select an object to be grasped by presenting flickering lights at different frequencies, while the subjects have to focus their attention to one object. From brain waves we decode the frequency of the attended object.

Fig. 1 Stimulus setup. Top: Objects are presented on discs, which can flicker at a defined frequency. Bottom: The flicker frequency is physiologically represented in the MEG signal

A first version of a new algorithm for grasp and trajectory planning is developed. The algorithm generates point poles on each surface triangle of the grasping target. Between these poles and the ones placed on the manipulator a force field is assumed. The impact of the forces on the manipulator is simulated and used as trajectory.

Fig. 2 Robotic manipulator used in this experiment