Research

none

Numerous African nations are currently undergoing a rapid process of urban transition, resulting in a significant surge in the proliferation of slum-like communities. This unprecedented urbanization presents considerable challenges for these countries, as they grapple with limited capacity to effectively address the associated problems. Consequently, socio-economic disparities are widening, and an unprecedented number of individuals find themselves in vulnerable circumstances. Regrettably, the availability of data on slums and their social attributes remains limited, hampering targeted efforts in prioritizing development and aid interventions. In this context, the integration of earth observation (EO), deep learning (DL) and citizen science (CS) emerges as a potential solution to effectively map and capture the intricacies of slums and their defining characteristics at large scales. 
The current paradigm of AI development has been model-centric, where the focus has been building new network architectures or loss functions or new ways to optimize hyperparameters. However, data-centric AI (DCAI), veers away from touching the models themselves and instead pushes for ensuring the quality of the input datasets in a systematic and semi- or fully-automated manner. We employ DCAI to potentially improve the spatial transferability of slum mapping models in a data-scarce environment. To achieve this, we will (1) develop a systematic data refinement method to reduce and "clean" original input data and then (2) train several models for different cities using synthetic imagery labelled using citizen science. From these models, we will (3) select the optimal model by quantifying the domain shift between all trained models and unseen datasets (from other cities). Lastly, throughout the research, we will, (4) with the local stakeholders and slum dwellers, co-develop communication and visualization strategies of the research outputs .

Topographic maps are abstractions of the real world showcasing the spatial arrangement of various objects and landscapes. In the Netherlands, many government agencies use Basisregisteratie Grootschalige Topografie (BGT) maps for urban planning, water management, and other purposes. However, these maps can quickly become outdated due to the constant environmental changes. At present, different agencies manually update the BGT maps, leading to prolonged processing periods. With recent developments in deep learning (DL), many methods have automated change detection using multi-temporal geospatial data to obtain "from-to" changes. In principle, existing digital maps can offer valuable ground-truth data. However, their potential as a baseline for detecting changes remains underexplored.
Figure 1: Flow of geospatial data through BGT maps from source to central facilities and finally to the users for real-world applications. (Source: Author and image from Geonovum)
In this context, the current research is part of an umbrella project aiming to automate updating BGT maps of the Netherlands. This Ph.D. research focuses on detecting semantically meaningful changes that provide "from-to" information using existing maps and newly acquired airborne data. Utilizing BGT maps and 2D-3D airborne data from the Dutch agencies Beeldmateriaal and Actueel Hoogtebestand Nederland (AHN) is central to this effort. Additionally, the research aims to examine the potential of applying this change detection methodology in disaster management scenarios. The goal is to refine the designed approach further to support automatic change detection across different international contexts in post-disaster evaluations.
Figure 2: Overall research framework illustrated in the context of three identified objectives represented individually and collectively concerning each other. Objectives I, II, and III workflows use green, pink, and purple colours.
Keywords: BGT map, airborne data, deep learning, multi-modal semantic segmentation, 2D-3D fusion, change detection

First responders (FRs) need to make rapid response efforts after emergency environments. It is a tough task that requires abundant professional knowledge and complex operations, particularly in urban post-earthquake scenarios, such as the recent earthquake in Morocco. To execute rescue operations promptly, it’s necessary to gain a comprehensive understanding of post-disaster terrain conditions, the location of individuals, and the extent of damages incurred. Nowadays, Unmanned Aerial Vehicles (UAVs) are increasingly being used to create situational awareness and support FRs in post-disaster rescue operations. UAVs offer several advantages over other types of robots, including lightweight design, compact size, high flexibility, and high spatial resolution. Due to the limited battery capacity of a single drone, it is often necessary to utilize multiple drones to cover larger areas effectively. These capabilities make multiple UAVs particularly well-suited for urban post-earthquake scenarios, as they can serve as effective tools for field surveys. To provide FRs with valid information about the on-site situations, including determining the real-time location of the multiple UAVs, creating 3D maps, and answering FRs’ questions regarding the post-disaster environment, these challenges need to be addressed with the help of simultaneous localization and mapping (SLAM), three-dimensional (3D) reconstruction, and visual question answering (VQA). Based on this, my PhD research will focus on three topics: 1) Multi-UAV SLAM, 2) 3D emergency mapping, and 3) VQA to support the rescue tasks.
The project aims to create an effective tool to establish situational awareness in emergency environments and facilitate the implementation of post-disaster rescue efforts. The expected outcome is to develop UAV swarm-based multi-SLAM and VQA algorithms for emergency environments for constructing 3D maps and delivering descriptive answers to FRs.

None

With climate change, disaster increases, and traffic load increase, bridges are experiencing accelerated aging and deterioration. Therefore, more bridges in the Netherlands and around the world need regular inspection and maintenance to extend sustainability and reduce the potential risks in regard to bridge damage.
Effective methods such as non-destructive evaluation (NDE) and structural health monitoring (SHM) are studied to improve structure reliability and prevent catastrophic failures. With technologies developing, Unmanned Aerial Vehicles (UAVs) arise coming along with many advanced algorithms to monitor bridges. The inspection process using UAVs is accelerated without traffic closure while also minimizing disruption to the targeted bridge structure.
Here we focus on cracks in concrete bridges because they are part of the most common and important parameters to evaluate a bridge's serviceability. The main objective of this research is to design and execute an innovative crack-detecting Unmanned Aerial Vehicle setup for three-dimensional modelling of concrete bridges. The primary purpose is to design an active path planning of autonomous crack detection using UAV and integrate crack information into dense 3D point cloud as well as the corresponding digital twin with the aid of advanced algorithms and technologies thereby improving the maintenance and safety protocols in bridges.

None

Plant-based proteins currently dominate the global protein supply (57%), with meat (18%), dairy products (10%), fish and shellfish (6%), and other animal products (9%) making up the rest (Diouf and Sheeran 2010). Cereal crops provide an important solution as one of the richest sources of plant-based dietary protein (Samaranayaka 2017; Shewry and Halford 2002; van der Spiegel et al. 2013). Cereal proteins not only meet the growing demand for essential amino acids, but also help reduce greenhouse gas emissions and protect the environment (Grasso et al. 2021).
Given the gravity of the situation, understanding and predicting the grain protein content (GPC) of cereals has become a critical factor. Furthermore, GPC is a fundamental indicator of flour quality, influencing gluten formation in bread production, determining the characteristics of baked goods, and determining the economic value of cereal commodities (Xue et al. 2019). Rapid and accurate estimates of GPC at regional and national scales are necessary to optimize agricultural management practices, to grade harvests for commercial purposes, to manage inventories efficiently, and to plan prices and policies in response to market fluctuations (Apan et al. 2006; Skerritt et al. 2002).
This project aims to develop cutting-edge deep learning algorithms for accurate estimation and prediction of GPC from multi-temporal very high-resolution imagery acquired using unmanned aerial vehicle (UAV) platforms. The algorithms will be optimized for various sensor types, ranging from high-end hyperspectral to low-cost multi-spectral cameras.
Furthermore, the developed deep learning algorithms will be embedded on edge devices and deployed on UAV platforms, enabling real-time georeferenced mapping of GPC and other relevant crop parameters. This approach will facilitate repeated mapping across different growth stages, providing insights into GPC variations and guiding strategies for optimized fertilizer application through multi-temporal analysis.
Additionally, the project will leverage the advantages of high precision and timeliness of UAV platform data in combination with the wide coverage of remote sensing satellite data. Innovative deep learning algorithms will be developed to scale up the information from UAV imagery to hyperspectral satellite images, utilizing high-resolution data to train lower-resolution images and improve the accuracy of the information obtained from satellite data.
References
Apan, A., Kelly, R., Phinn, S., Strong, W., Lester, D., Butler, D., & Robson, A. (2006). Predicting grain protein content in wheat using hyperspectral sensing of in-season crop canopies and partial least squares regression. International Journal of Geoinformatics, 2, 93-108
Diouf, J., & Sheeran, J. (2010). The State of Food Insecurity in the World: Addressing food insecurity in protracted crises. World Food Program (WFP) and Food and Agriculture Organization of the United Nations (FAO) Joint Report
Grasso, A.C., Olthof, M.R., van Dooren, C., Broekema, R., Visser, M., & Brouwer, I.A. (2021). Protein for a Healthy Future: How to Increase Protein Intake in an Environmentally Sustainable Way in Older Adults in the Netherlands. Journal of Nutrition, 151, 109-119
Samaranayaka, A. (2017). Lentil: revival of poor man’s meat. Sustainable protein sources (pp. 185-196): Elsevier
Shewry, P.R., & Halford, N.G. (2002). Cereal seed storage proteins: structures, properties and role in grain utilization. Journal of Experimental Botany, 53, 947-958
Skerritt, J.H., Adams, M.L., Cook, S.E., & Naglis, G. (2002). Within-field variation in wheat quality: implications for precision agricultural management. Australian Journal of Agricultural Research, 53, 1229-1242
van der Spiegel, M., Noordam, M.Y., & van der Fels-Klerx, H.J. (2013). Safety of Novel Protein Sources (Insects, Microalgae, Seaweed, Duckweed, and Rapeseed) and Legislative Aspects for Their Application in Food and Feed Production. Comprehensive Reviews in Food Science and Food Safety, 12, 662-678
Xue, C., Matros, A., Mock, H.P., & Mühling, K.H. (2019). Protein Composition and Baking Quality of Wheat Flour as Affected by Split Nitrogen Application. Frontiers in Plant Science, 10

Motivation
Firefighters operate in extremely hostile conditions, where visibility is severely compromised due to smoke and low lighting. They often enter unknown buildings, which requires on-the-spot improvisation and increases the risk of injury or death. Further challenges are coming from the risks related to new advanced technology, such as lithium-ion batteries from electric vehicles. While some responders have turned to unmanned vehicles equipped with cameras to enhance situational awareness, these solutions have proven inefficient due to the lack of peripheral vision. This "tunnel vision"-effect is further exacerbated by the inadequate performance of existing 3D-mapping technologies during fire incidents.
In addition to escalating threats, there is a serious shortage of personnel within fire departments. Hence, there is an urgent imperative for increased efficiency, effectiveness, and safety of the firefighters by bolstering their capabilities using state-of-the-art technologies.
Recent advancements in sensing technologies offer potential to increase situational awareness. Sensors like LiDAR and thermal cameras use a longer wavelength that is less affected by smoke and dust. Additionally, these techniques are not influenced by lighting conditions. By coupling LiDAR, thermal cameras, and potentially other sensors with advanced deep learning (computer vision) algorithms, emergency responders can achieve the global situational awareness they need with minimal effort during crisis situations, thus significantly improving their response efficiency and safety.  
Problem Statement
The lack of situational awareness can be split into three subproblems. The first is seeing through smoke. Thermal radiation, which consists of long-wave infrared waves, is emitted by objects with temperatures above absolute zero. Thermography enables us to see in environments regardless of visible light, making it useful for visualizing temperature differences. A smoke-filled environment is invisible to the naked eye, while the thermal cameras can perceive the environment due to the reduced impact of smoke on long-wave infrared waves. Similarly, LiDAR uses short-wave infrared beams to perceive smoke-filled environments. However, since LiDAR beams are closer to the visible spectrum than thermal radiation, their effectiveness decreases as smoke density increases. Other sensors like RADAR and SONAR also hold potential for use in smoke-filled environments.
The second challenge involves fusing multimodal data, akin to solving a Simultaneous Localization and Mapping (SLAM) problem. It entails using various sensor data to incrementally construct a 3D map while simultaneously pinpointing the robot's location. For firefighters to operate effectively, precision matters. Accurate localization and detailed 3D mapping are crucial for two reasons: (1) to enable remote robot navigation around obstacles while surveying and (2) to plan the most efficient and safe firefighting strategy. Unfortunately, these algorithms are ineffective in smoke-filled environments, directly impacting firefighters’ ability to navigate through complex environments and make informed decisions during fire incidents.
The third challenge pertains to scene understanding in high-pressure fire incidents. Time constraints prevent firefighters from analyzing extensive 3D models. Hence, the model should be pre-processed to pinpoint crucial points-of-interest (POI) like fire sources, victims, cars, and exits. Additionally, for increased robustness in thermally dynamic fire environments, it's vital to separate essential geometric information (e.g., surfaces) from thermal data.
Objective
This PhD-project aims to develop SAFE (Situational Awareness for Firefighters in Emergencies), an end-to-end 3D-mapping framework that performs Simultaneous-Localization-And-Mapping (SLAM) in smoke-filled environments to increase firefighters’ situational awareness.

The large scale (1:1000) topographic map BGT1 provides abundant geographic information of the Netherlands and is of great importance in many practical applications, such as environmental permits evaluation, public space maintenance, issue of licenses, as well as subsidy applications and approvals. With the rapid urbanization, spatial objects in the BGT change dramatically every year. A delay in updating the BGT undermines its potential usage. Decisions made upon outdated map data may be incomplete or incorrect, which may lead to severe consequences. It is thus crucial to keep the BGT accurate and updated. At the moment, however, updating the BGT is a labor intensive and time-consuming process, which is not only expensive but also not timely.
Therefore, it is of utmost significance to develop novel algorithms to automatically interpret newly acquired airborne sensor data to fasten the updating of the map. Of particular concern when updating the map is extracting precise object outlines in the vector data format, which are the key components of the BGT map. Unlike the raster data format of semantic or instance segmentations, the vector representation of object outline in the form of polygons is more compact, scalable and geolocation accurate, which provides desired topological information that favors many topological operations in Geographic Information Systems (GIS).
Hence, this research aims to design novel and fully end-to-end deep learning approaches to generate accurate object boundaries in vector format which can overcome the limitations of existing methods. The output object polygons should have not only precise pixel-level coverage, but also accurate instance-level detection, together with outstanding geometry-level properties and clean irredundant vertices. The networks will be trained on highly detailed and complete data in The Netherlands and cover all common object classes in the BGT map. The transfer ability will also be tested on other international datasets.
1The Basisregistrate Grootschalige Topography (BGT) is a high-resolution digital topographic map of the Netherlands on which buildings, roads, watercourses, terrains and railway lines are unambiguously recorded.
Figure BGT map data (top) and airborne sensor data (middle: 2D aerial image, bottom: 3D lidar point cloud)