See Open Bachelor Assignments

[M] Efficient Learning in Deep Learning through Neuroscience theories.

BACHELOR Assignment

Efficient Learning in Deep Learning through Neuroscience theories.

Type: Bachelor CS

Period: TBD

Student: (Unassigned)

If you are interested please contact :

Context of the work:

Deep Learning (DL) is a very important machine learning area nowadays and it has proven to be a successful tool for all machine learning paradigms, i.e. supervised learning, unsupervised learning, and reinforcement learning. Big companies such as Google, Apple, Microsoft already implement DL techniques in their products (e.g. Apple’s Siri virtual personal assistant, Google Maps). Yet, training deep learning models is still a very difficult and time consuming task and unfortunately it can end up in local minima. Also, traditional training methods for deep learning, e.g. BackProgation with Stochastic Gradient Descent (BP-SGD), are not biologically plausible. This project will try to overcome these limitations by exploring how neuroscience theories may be used to train DL.

Short description of the assignment:

The goal of this assignment is to find if/how the learning algorithms of DL models can be replaced with new algorithms inspired from neuroscience. The focus will be on a supervised learning model, i.e. Multi-Layer Perceptron (MLP), and an unsupervised learning model, e.g. Variational Autoencoders (VAE), following three phases:

  • Phase 1 – consists in exploring the possibilities of training MLP and VAE models with neuroscience learning theories, e.g. Hebbian rule, BCM theory [1].
  • Phase 2 - performs a thorough comparative analyze and interpretation of the performance of the methods proposed in phase one against widely used method for training MLP and VAE, e.g. BP-SGD.
  • Phase 3 – prepare the project output: the developed code and a report. 

Possible expected outcomes:

finding new insights on the relations between learning in biological brain and learning in artificial neural networks, advancing state-of-the-art in learning algorithms for artificial neural networks, obtaining publishable results.

Requirements

  • Basic Calculus, Probabilities, and Optimization
  • Good programming skills (preferably Python)
  • Basic understanding (or the willingness to learn) of artificial neural networks

Learning Objectives:

Upon successful completion of this project, the student will have learnt:

  • Get familiar with deep learning and basic neuroscience
  • Get familiar with widely used deep learning libraries, e.g. Keras, Tensorflow
  • Interpret the behavior of deep learning models
  • Implement from scratch artificial neural network models

References:

[1] EL Bienenstock, LN Cooper and PW Munro, Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex, Journal of Neuroscience 1 January 1982, 2 (1) 32-48