392270 ISY Project: Robotic Tool Manipulation by Using Eye Tracking Virtual Reality and Robotic Multi-modality Feedback (Pj) (SoSe 2018)

- Short Description
Virtual reality (VR) is a computer technology which makes users wearing virtual reality equipment be able to "look around" the artificial world. It has been widely used in game development, movie and entertainment domain. Recent year, it was also used in robotic grasping tasks.

In this project, we propose to use eye tracking and virtual reality technology to help robot using a tool in a semi-autonomous way. To this end, (1) Human intention will be detected and tracked by the eye tracking system. (2) Rendered virtual environment will serve as feedback to guide user to move a tool. (3) Because of the partly occluded scenario in the virtual world, Vision and force perception in robotic system will quickly adapt robot behaviour while the tool is physically interacting with environment in an online fashion.
Team members will:
(1) Reconstructing the scenario by data fusion method. e.g. combining data from HTC vive and RGB-D camera.
(2) Goal detector by eye tracking system in virtual really environment.
(3) Controling simulated/real Kuka LWR to manipulate a rigid or an articulated tool.

Please note that the teams will be selected by the supervisors on the basis of short applications that students are expected to send to them. Registering to the project in the ekVV will only be regarded as expression of interest; it will not secure a team membership.
Please get in touch with the supervisors for information on the application procedure.

Requirements for participation, required level

Required skills:
The analysis and programming will be done in C/C++,C#,Unity 3D and Matlab/Python.