392177 User Interfaces for Aumented Reality and Smartglasses (PjS) (WiSe 2016/2017)

Introduction

With the recent hype in Virtual Reality, also Augmented Reality comes in to focus - think about Pokemon Go for example. We are going one step further and work on AR smart glasses. For example, we use smart glasses for assisiting elderly people with maintaining a coffee machine or we support the assembly of birdhouses with AR visualizations.

In this project, we like to focus on the interaction and design, implement and test ideas on how to interact with AR smart glasses properly. The idea is to design different types of user interfaces for a number of input modalities (gaze, eye gaze, gestures). These can finally be evaluated in a user study.

Project Goals

The goal is to transfer standard UI elements knwon from 2D desktop environments to 2D and 3D AR. In 2D, the UI is presented on a Head-Up-Display (HUD) which moves with the user's head movements. In 3D, the UI is anchored in the real world. These elements should be designed in a way that the user can intuitively use them without having to be further instructed. As a hardware basis, the Epson Moverio BT-200 and the Microsoft HoloLens can be used. On the software side, we use the Unity 3D game engine.

For interacting with the UI, different input modalities can be considered: Simulating a cursor which moves with the head direction (gaze), using eye movements (eye gaze), or using the hand as input (Microsoft Hololens or Leap Motion).

Finally, the implemented UI elements and interactions shall be evaluated to find out about positive and negative sides of the ideas.

Project Timeline

Milestone 1: November 7-11
• Concept for different UI elements for the input modalities
• Presentation of Mock-Ups (either in AR glasses or on the screen)

Milestone 2: December 5-9
• Leap motion integrated into the software
○ Click and slide
• Eye tracking algorithms (e.g. fixation detection implemented)
• Simple interaction concepts implemented for the 2D HUD

Milestone 3: January 16-20
• User interfaces and interaction methods implemented
• For the 2D HUD and in 3D

Final Milestone: February 28
• Evaluation and Paper