392270 Learning to Walk with a Six-Legged Robot (Pj) (WiSe 2018/2019)

For more information about the project, please contact
Dr. Andrew Melnik anmelnik(at)techfak.uni-bielefeld.de
Office 3.308 (CITEC Building)
Consultation hours: Wednesdays 17:15-18:00

The project intends to gain practical experience with hierarchical deep reinforcement learning in a simulated and/or real six-legged robot. In this project, you will apply neural networks to learn and produce complex purposeful actions. The structural organization of the controller will reflect insights from biology. First, the overall structure is decentralized and the architecture consists of local control modules that interact only through local coordination influences as can be found in walking insects. Secondly, this leads to a hierarchical control approach which — in the formulation of Deep Reinforcement Learning — is realized as different policies (or sub-policies) with overlapping scopes.

The architecture will be setup in a simulated environment. The controlled system has a high number of degrees of freedom: it is a six-legged robot and each leg consists of three rotational joints. The architecture will be initially pre-trained using the current reactive Walknet system to come up with basic walking behavior. Further future work might include porting this to our robot.

References:
- Decentralized robot controller: Schilling, M., Hoinville, T., Schmitz, J., Cruse, H.: Walknet, a bio-inspired con- troller for hexapod walking. Biological Cybernetics 107(4), 397–419 (2013)
- Extended version including planning: Schilling, M., Hoinville, T., Schmitz, J., Cruse, H.: Walknet, a bio-inspired con- troller for hexapod walking. Biological Cybernetics 107(4), 397–419 (2013)
- Six-legged robot Hector: Schneider, A., Paskarbeit, J., Schilling, M. and Schmitz, J. (2014), HECTOR, A Bio-Inspired and Compliant Hexapod Robot. In: A. Duff, T. Prescott, P. Verschure, N. Lepora (eds.): Living Machines 2014, LNAI 8608, pp. 427–429.
- New robot platform: Phantom X, as discussed in
http://forums.trossenrobotics.com/showthread.php?6315-PhantomX-ROS-Robot-Project

Requirements for participation, required level

- Introduction to Neural Networks course or Advanced Neural Networks
- Python (>= 1 year)