PhD Defence Pinar Santemiz | Side-View Face Recognition

Side-View Face Recognition

Pinar Santemiz is a PhD student in the Department of Datamanagement & Biometrics. Promotors are Prof.dr.ir. R.N.J. Veldhuis and Dr.ir. L.J. Spreeuwers from the Faculty of Electrical Engineering, Mathematics and Computer Science.

Face recognition is a popular biometric method valued for its non-intrusive and passive nature. However, in real-world scenarios, factors like occlusion and pose variations degrade performance, with side-view faces presenting distinct challenges. This dissertation primarily focuses on side-view face recognition, a domain that offers unique advantages and opens new avenues for applications where frontal views are unavailable or impractical. Unlike most existing systems that depend on frontal images or frontalization, we propose a dedicated side-view-to-side-view recognition framework to address this critical gap. Such a system is vital for real-world applications, including surveillance, access control, and long-term re-identification.

We first survey state-of-the-art approaches, categorizing them into feature-based, image-based, and set-based methods, and identify a strong frontal bias in existing benchmarks that limits robustness in extreme poses. To address this, we introduce the UT-DOOR, a video-based face dataset featuring recordings of 98 individuals captured by four doorpost-mounted cameras. The dataset primarily comprises side-view poses and presents unique challenges, including significant face size and rotation variations.

Our analysis highlights that face detection and registration remain the most challenging tasks for side-view recognition. By focusing on reliable landmarks such as the eye center, mouth corner, and nose tip, we improve alignment and recognition. We further examine the effects of face size, rotation, and position on deep learning similarity scores and propose a registration method that enhances detection performance. Comparative evaluations show that while deep learning improves pose handling, it suffers from poor cross-dataset generalization due to feature sparsity. We benchmark deep learning against Local Binary Patterns (LBP) and show that LBP, as an untrained method, exhibits strong resilience to cross-dataset variations, achieving comparable performance to deep learning.

Overall, this work advances side-view-to-side-view face recognition by introducing a new dataset, proposing improved preprocessing methods, and benchmarking conventional and deep learning approaches, underscoring the need for improved generalization in real-world non-frontal recognition scenarios.