Master Assignment
Visual place recognition
Type: Master CS
Period: TBD
Student: (Unassigned)
If you are interested please contact :
Background:
Visual place recognition consists of searching into a map or database of images an image similar to a given query. It is thus formalized as an image retrieval problem. Algorithms for visual place recognition are deployed in visual localization pipelines and algorithms for robot or autonomous car navigation. An autonomous vehicle, indeed, has the need of recognizing previously visited places, in order to correctly localize itself within a map [1].
Existing approaches in the literature are based on training Convolutional Networks using pairs or triplets of images for which a ground truth similarity is defined in terms of a binary label (similar place/different place) [3]. In practice, however, image similarity is a graded property: two images are indeed x% similar. This property was exploited to train more robust models (image descriptors) for place recognition using a generalization of the contrastive loss [3].
In this project, we aim at investigating robust place recognition algorithms exploiting the concept of graded similarity ground truth. Options to look into are:
- Sequence-based models: considering sequences of images to recognize a place
- Extension of the generalized contrastive loss to triplet learning
- Benchmarking of CNN backbones and state-of-the-art methods re-trained using graded similarity ground truth
- Augmenting the training with heuristics for similarity estimation for data sets and tasks for which a ground truth cannot be pre-computed
References:
[1] Lowry, S., Sunderhauf, N., Newman, P., Leonard, J.J., Cox, D., Corke, P., Milford, M.J.: Visual place recognition: A survey. IEEE Transactions on Robotics 32(1), 1–19 (2016)
[2] Hausler, S., Garg, S., Xu, M., Milford, M., Fischer, T.: Patch-netvlad: Multi-scale fusion of locally-global descriptors for place recognition. In: CVPR, pp. 14141–14152 (2021)
[3] Leyva-Vallina, M., Strisciuglio, N., & Petkov, N. (2021). Generalized Contrastive Optimization of Siamese Networks for Place Recognition. arXiv preprint arXiv:2103.06638