392249 ISY Project: Variational Autoencoders for Multisensory Data (Pj) (SoSe 2019)

Variational autoencoders (VAE) let us design generative models and derive compact representation of complex data. A VAE is a type of artificial neural network used to learn efficient data codings in an unsupervised manner [1, 2, 3], whereas the aim is to learn an ideally linear separable representation (encoding) for a dimensionality reduction. Along with the encoder network, a decoder network is learnt, which tries to generate the original data from the encoding, hence its name.

In this project, we will test different VAE models [4, 5] and conduct experiments in robotics simulated environments [6, 7]. In-hand manipulation of objects lends itself for testing compact representations of multisensory data (touch, vision, proprioception). The derivation of a compact representation is an important preprocessing step for deep reinforcement learning approaches, since it enables faster and more stable convergence to the optimal policy.

[1] https://en.wikipedia.org/wiki/Autoencoder#Variational_autoencoder_.28VAE.29
[2] https://jaan.io/what-is-variational-autoencoder-vae-tutorial
[3] https://www.youtube.com/results?search_query=variational+autoencoder
[4] http://geometry.cs.ucl.ac.uk/dl4g/slides/part7_3DdataGeometryPhysics.pdf
[5] http://geometry.cs.ucl.ac.uk/creativeai/slides/part7_3Ddomains_niloy.pdf
[6] https://drive.google.com/open?id=1J2H92AstGpcFYqjmVymMSRKI5xISdB1b
[7] http://gym.openai.com/envs/#robotics

Please note that the teams will be selected by the supervisors on the basis of short applications that students are expected to send to them. Registering to the project in the ekVV will only be regarded as expression of interest; it will not secure a team membership. Please get in touch with the supervisors for information on the application procedure.

Requirements for participation, required level

Required skills:

• Introduction to Neural Networks course or Advanced Neural Networks
• Python or C++ (>= 1 year)