Indoor / Outdoor / Transition Scene Recognition with Confidence for Mobile Robots
AlFusIOn (RAAK-PRO) – WP4: AI & Data Management
Project Introduction
This assignment is part of AlFusIOn – AI-based robust indoor/outdoor localization for autonomous mobile robots – a RAAK-PRO project in which the Ambient Intelligence (AMI) research group leads Work Package 4 (WP4): AI & Data Management.
Mobile robots increasingly drive both outdoors and indoors – entering barns, tunnels, underpasses, parking garages or buildings. During these indoor/outdoor (IO) transitions, satellite positioning (GNSS) becomes unreliable and localization can drift. To stay accurate, the robot must first recognise its operational condition – indoor, outdoor or transition – and how confident it is, so the localization/fusion system can decide which sensors to trust.
In this assignment you build the first-iteration vision model for WP4: a camera-based classifier that labels each scene as indoor / outdoor / transition / unknown and outputs a calibrated confidence score. You pre-train on large open-source scene datasets and benchmark on robotics datasets that contain real indoor↔outdoor routes. The result is the AI signal that the AlFusIOn fusion stack (WP5) uses to weight or switch its localization modules.
Open-source datasets you will use
The assignment is deliberately grounded in existing open datasets, so you can start training before AlFusIOn field data is collected:
Dataset | Modalities / labels | Role in this assignment | Use |
|---|---|---|---|
Places365 | RGB images; 365 scene categories | Pre-train indoor vs. outdoor scene features at scale | Core |
SUN RGB-D | RGB-D; indoor scene/room labels | Strengthen the indoor class (corridors, labs, rooms) | Core |
Cityscapes / Mapillary Vistas | Street RGB; semantic labels, weather/lighting variety | Broaden the outdoor class and lighting robustness | Support |
Oxford RobotCar | Camera, LiDAR, GPS/INS; long routes incl. tunnels | Benchmark on real robot routes; derive transition segments | Benchmark |
NCLT | Camera, LiDAR, GPS/IMU; campus indoor + outdoor | Real indoor↔outdoor campus routes for the transition class | Benchmark |
Student Tasks
1. Literature & dataset scan
• Review scene-classification and indoor/outdoor detection methods (CNN backbones, lightweight models such as MobileNet, and confidence calibration).
• Map the open-source datasets above to the task and document their labels, licences and limitations.
2. Data preparation & labeling
• Define the label taxonomy: indoor, outdoor, transition, unknown / low-confidence.
• Derive indoor/outdoor/transition labels for the robotics datasets from route metadata and GNSS availability; build clean train / validation / test splits.
• Implement augmentation for lighting changes, glare, motion blur and limited field-of-view.
3. Model development
• Train a CNN classifier (e.g. MobileNet or a small ResNet) for the four classes, pre-trained on Places365/SUN RGB-D and fine-tuned on robot routes.
• Add a calibrated confidence output (softmax + temperature scaling) and an “unknown / low-confidence” rule.
4. Evaluation
• Report accuracy, balanced accuracy, F1 and a confusion matrix, with attention to the harder transition class.
• Measure confidence calibration (Expected Calibration Error) and robustness across day/night and weather.
• Measure runtime (latency, FPS) to judge whether the model can run on a robot.
5. Documentation & handover
• Provide reproducible code and a short report; explain dataset choices and limitations, and outline how the classifier output feeds the AlFusIOn ROS2 interface used by the fusion stack.
Practical information
Student profile: for HBO-ICT (Saxion) and University of Twente students interested in computer vision, deep learning, Python and robotics. Experience with PyTorch or TensorFlow is a plus; affinity with autonomous robots or localization is welcome. A proactive, independent attitude is key.
Scope: graduation or internship project (approx. one semester); individual or pair.
Contact
Alessandro Chiumento – Pervasive Systems, EEMCS, University of Twente (a.chiumento@utwente.nl)
Eyuel Ayele – Ambient Intelligence (AMI) research group, Saxion University of Applied Sciences (e.d.ayele@saxion.nl)
Javier Ferreira Gonzalez – Ambient Intelligence (AMI) research group, Saxion University of Applied Sciences (j.ferreiragonzalez@saxion.nl)
More info: www.saxion.nl/ami