Pervasive Systems (PS)

Indoor / Outdoor / Transition Scene Recognition with Confidence for Mobile Robots

Indoor / Outdoor / Transition Scene Recognition with Confidence for Mobile Robots

AlFusIOn (RAAK-PRO) – WP4: AI & Data Management

Project Introduction

This assignment is part of AlFusIOn – AI-based robust indoor/outdoor localization for autonomous mobile robots – a RAAK-PRO project in which the Ambient Intelligence (AMI) research group leads Work Package 4 (WP4): AI & Data Management.

Mobile robots increasingly drive both outdoors and indoors – entering barns, tunnels, underpasses, parking garages or buildings. During these indoor/outdoor (IO) transitions, satellite positioning (GNSS) becomes unreliable and localization can drift. To stay accurate, the robot must first recognise its operational condition – indoor, outdoor or transition – and how confident it is, so the localization/fusion system can decide which sensors to trust.

In this assignment you build the first-iteration vision model for WP4: a camera-based classifier that labels each scene as indoor / outdoor / transition / unknown and outputs a calibrated confidence score. You pre-train on large open-source scene datasets and benchmark on robotics datasets that contain real indoor↔outdoor routes. The result is the AI signal that the AlFusIOn fusion stack (WP5) uses to weight or switch its localization modules.

Open-source datasets you will use

The assignment is deliberately grounded in existing open datasets, so you can start training before AlFusIOn field data is collected:

Dataset

Modalities / labels

Role in this assignment

Use

Places365

RGB images; 365 scene categories

Pre-train indoor vs. outdoor scene features at scale

Core

SUN RGB-D

RGB-D; indoor scene/room labels

Strengthen the indoor class (corridors, labs, rooms)

Core

Cityscapes / Mapillary Vistas

Street RGB; semantic labels, weather/lighting variety

Broaden the outdoor class and lighting robustness

Support

Oxford RobotCar

Camera, LiDAR, GPS/INS; long routes incl. tunnels

Benchmark on real robot routes; derive transition segments

Benchmark

NCLT

Camera, LiDAR, GPS/IMU; campus indoor + outdoor

Real indoor↔outdoor campus routes for the transition class

Benchmark

 

Student Tasks

1.         Literature & dataset scan

• Review scene-classification and indoor/outdoor detection methods (CNN backbones, lightweight models such as MobileNet, and confidence calibration).

• Map the open-source datasets above to the task and document their labels, licences and limitations.

2.         Data preparation & labeling

• Define the label taxonomy: indoor, outdoor, transition, unknown / low-confidence.

• Derive indoor/outdoor/transition labels for the robotics datasets from route metadata and GNSS availability; build clean train / validation / test splits.

• Implement augmentation for lighting changes, glare, motion blur and limited field-of-view.

3.         Model development

• Train a CNN classifier (e.g. MobileNet or a small ResNet) for the four classes, pre-trained on Places365/SUN RGB-D and fine-tuned on robot routes.

• Add a calibrated confidence output (softmax + temperature scaling) and an “unknown / low-confidence” rule.

4.         Evaluation

• Report accuracy, balanced accuracy, F1 and a confusion matrix, with attention to the harder transition class.

• Measure confidence calibration (Expected Calibration Error) and robustness across day/night and weather.

• Measure runtime (latency, FPS) to judge whether the model can run on a robot.

5.         Documentation & handover

• Provide reproducible code and a short report; explain dataset choices and limitations, and outline how the classifier output feeds the AlFusIOn ROS2 interface used by the fusion stack.


Practical information

Student profile: for HBO-ICT (Saxion) and University of Twente students interested in computer vision, deep learning, Python and robotics. Experience with PyTorch or TensorFlow is a plus; affinity with autonomous robots or localization is welcome. A proactive, independent attitude is key.

Scope: graduation or internship project (approx. one semester); individual or pair.


Contact

Alessandro Chiumento – Pervasive Systems, EEMCS, University of Twente (a.chiumento@utwente.nl)

Eyuel Ayele – Ambient Intelligence (AMI) research group, Saxion University of Applied Sciences (e.d.ayele@saxion.nl)

Javier Ferreira Gonzalez – Ambient Intelligence (AMI) research group, Saxion University of Applied Sciences (j.ferreiragonzalez@saxion.nl)

More info: www.saxion.nl/ami