Master Assignment
How do different down-sampling methods affect neural networks’ reaction towards different frequencies?
Type: Master CS
Period: TBD
Student: (Unassigned)
If you are interested please contact :
Background:
Empirically, down-sampling would not affect the frequency information of an image as long as it satisfies the Nyquist-Shannon sampling theorem. However, the down-sampling achieved by max-pooling, average pooling and other pooling methods do not satisfy the sampling theorem [1] and thus may induce aliasing frequency during training [2], which further influences the frequency response of neural network. In some experiments, it is observed that without further constraints, a neural network that is trained to predict images with high-frequency information only, may give the same predictions for images containing relatively lower frequencies only. Therefore, this project aims at investigating the underlying reasons of this phenomenon from the perspective of the frequency response of down-sampling methods. It is aimed to investigate further what kind of down-sampling methods can avoid neural networks from being influenced by the induced aliasing frequency across different datasets and architectures.
References:
[1] Zhang, R. (2019). Making Convolutional Networks Shift-Invariant Again. doi:10.48550/ARXIV.1904.11486
[2] Romero, D. W., Bruintjes, R.-J., Tomczak, J. M., Bekkers, E. J., Hoogendoorn, M., & van Gemert, J. C. (2021). FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes. doi:10.48550/ARXIV.2110.08059