Semantic-Aware Person Inpainting Using Generative Adversarial Networks

Abstract

The widespread use of street view imagery in various applications, such as urban planning, navigation, and real estate, has raised concerns about the privacy of individuals captured in these images. While anonymization methods, such as blurring or pixelation, are commonly used to address these concerns, they often result in a loss of important information and can be easily circumvented by determined individuals. This thesis aims to explore an alternative approach to protecting privacy in street view imagery by using image inpainting techniques. Image inpainting involves filling in missing or obscured regions of an image in a way that preserves the overall structure and context of the scene. The proposed approach will utilize conditional Generative Adversarial Networks (cGANs) for the purpose of image inpainting, specifically in the context of street view images where people have been removed. While current cGAN-based methods rely solely on incomplete images with missing areas, our approach incorporates an additional source of information in the form of a semantic segmentation map. The semantic segmentation map serves as a prior to guide the inpainting process, allowing the model to better understand the context and structure of the image. By utilizing both the incomplete image and the semantic segmentation map, we aim to improve the quality and realism of the inpainted results. We compared the performance of our two proposed models, SemGAN and SemGAN-GT, with the existing pix2pix method. SemGAN utilizes inpainted semantic segmentation maps as a prior in image inpainting, while SemGAN-GT uses ground truth semantic segmentation maps. We evaluated the performance of each method using established image quality metrics, such as L1, SSIM, and PSNR, as well as a qualitative analysis of the inpainted images. Our findings indicate that both SemGAN and SemGAN-GT outperform the pix2pix method in terms of image quality and realism. This suggests that semantic information improves the quality and realism of the inpainted results, based on our quantitative results and qualitative analysis.