Speaker: Ali Heydari
Title: How to Super Resolve your Face 101
Abstract: The emergence of Generative Adversarial Network (GAN)-based single-image super resolution (SISR) has allowed for finer textures in the super-resolved images, thus making them seem realistic to humans. However, GAN-based models may depend on very large high-quality data and are known to be very costly and unstable to train. On the other hand, Variational Autoencoders (VAE) have intuitive mathematical properties, and they are relatively cheap and stable to train; but VAEs produce blurry images which prevents them from being used for super resolution. In this paper, we propose a novel and the first of its kind SISR method that takes advantage of a self-evaluating VAE (IntroVAE) to judge the quality of the generated high-resolution (HR) images with the target images in an adversarial manner, which allows for high perceptual image generation. First, the encoder and the decoder of the IntroVAE learn the manifold of HR images, while the encoder and decoder are simultaneously learning the reconstruction of the low-resolution (LR) images. Second, LR images are inputted to the generator, then the output is fed to the encoder to compare the encoded LR to the encoded HR. This allows SRVAE to be a fast single-stream framework that generates photo-realistic images without requiring an additional discriminator. On one hand SRVAE has the same "nice" latent manifold structure of VAEs and a stable training, while playing a max-min adversarial game between the generator and the encoder like GANs. Our experiments show that our super-resolved images are comparable to the state-of-the-art GAN-based super resolution.
