Master Assignment
[B] Disentangled Flows for Face editing
Type: Master EE/CS/ITC
Period: TBD
Student: (Unassigned)
If you are interested please contact :
Background:
Disentangled Flows for Face editing
Face editing can take place by navigating the latent space of StyleGAN, however it faces challenges in the case of complicated navigation due to the entanglement of latent factors. SDFlow has been proposed, for the semantic decomposition in original latent space using continuous conditional normalizing flows. SDFlow jointly optimizes: (i) a semantic encoder to estimate semantic variables from input faces and (ii) a flow-based trans formation module to map the latent code into a semantic-irrelevant variable in Gaussian distribution, conditioned on the learned semantic variables. A disentangled learning strategy under a mutual information framework is used, allowing for precise manipulation and SoA face editing. This thesis will examine the effects face editing can achieve and implement explainability techniques to understand the effect of disentangled latent spaces.
- Disentangled Flows for Face editing https://arxiv.org/pdf/2309.05314.pdf
Disentangled Flows for Anomaly Detection
This work will implement disentangled flows with SDFLow, as it separates semantic from visual concepts in latent space. Its focus will be changed from face editing to disentangling representations of other kinds of visual data, such as medical images. Other disentangling networks may also be examined in its place for comparison. The presence of anomalies will be investigated in the disentangled latent spaces. Real-world benchmarking datasets will be used, but first there will be experiments with artificially added noise, for a systematic examination of results.
- Disentangling Physical Parameters for Anomalous Sound Detection Under Domain Shifts https://arxiv.org/abs/2111.0653
- Rare Event Detection using Disentangled Representation Learning https://openaccess.thecvf.com/content_CVPR_2019/papers/Hamaguchi_Rare_Event_Detection_Using_Disentangled_Representation_Learning_CVPR_2019_paper.pdf
- SDFlow https://arxiv.org/pdf/2309.05314.pdf
Unsupervised Anomaly Detection Under Domain Shifts
Unsupervised Anomaly Detection is usually based on pre-trained networks trained on benchmarking datasets such as ImageNet. However, there is a domain shift between these datasets and real-world data such as industrial or medial imaging. Shifting to the target domain using contrastive-learning inspired training has been proposed in ReContrast. This thesis will examine models like ReContrast for artificially introduced anomalies, domain shifts and real-world medical image datasets, including from challenges such as the MICCAI 2023 OOD challenge.
- ReContrast: Domain-Specific Anomaly Detection via Contrastive Reconstruction
- https://arxiv.org/abs/2306.02602
- https://github.com/guojiajeremy/recontrast
- Disentangling physical parameters for anomalous sound detection under domain shifts
- https://eurasip.org/Proceedings/Eusipco/Eusipco2022/pdfs/0000279.pdf