See Open Bachelor Assignments

[B] [M] Visual Semantic Role Labelling

BACHELOR Assignment

Visual Semantic Role Labelling

Type: Bachelor CS/EE/HMI 

Period: TBD

Student: (Unassigned)

If you are interested please contact :

Description:

Creating graphical resources, such as 3D scenes or animations requires a good level of knowledge of 3D modelling applications and/or programming. Despite the usefulness of such resources, as in advertising, theatrical productions or education, they (the graphical resources) are difficult to create for people with low levels of computers literacy.

This is we’re we need your help to design parts of a system able to read natural language descriptions and generate 3D or VR animations from them.

You can choose to work in one of two parts of this project:

  1. Sentence simplification using deep learning.
    That is to remove visually irrelevant tokens from a sentence and provide a shorter version easier for further processing.
  2. Visual semantic role labelling using deep learning
    That is to assign visual roles (actor, action, object, manner, etc.) to each token in a sentence.

Basic understanding of deep learning approaches is highly recommended, but most important is an interest in natural language processing and the willingness to learn its related concepts during the research project.

As a starter, you can look into the review available at :

Kaveh Hassani and Won-Sook Lee. 2016. Visualizing Natural Language Descriptions: A Survey. ACM Comput. Surv. 49, 1, Article 17 (July 2016), 34 pages.