Fast Fourier Transform based Method of Neural Network Training for Human Re-Rendering

Paweł Kowaleczko

supervisor: Przemysław Rokita



Novel view synthesis is one of the generative imaging issues in which Generative Adversarial Networks (GANs) can be applied. One of such tasks is also human re-rendering from a single image. The goal of such an algorithm is to generate a human picture in a target pose using only the source image of this person in a different pose. The current state of the art is presented in the paper „Neural Re-rendering of Humans from a Single Image” by Sarkar et al. We reimplemented this paper and identified its main drawback - low quality of rendered images in the areas of high-frequency details like hair, faces, hands etc. We slightly modified the architectures of baseline models and investigated the influence of operations on Fourier spectra of the images, which we believe may be the solution to the main issue of missing quality of high-frequency details. In particular, we proposed the Discrete Fourier Transform loss function (DFT loss) and investigated how this loss function influences the visual quality and evaluation metrics values for the rendered images.