Deep Learning-Driven Virtual Furniture Replacement Using GANs and Spatial Transformer Networks

Resmy Vijaykumar; Muneer Ahmad; Maizatul Akmar Ismail; Iftikhar Ahmad; Neelum Noreen

doi:10.3390/math12223513

Deep Learning-Driven Virtual Furniture Replacement Using GANs and Spatial Transformer Networks

Resmy Vijaykumar, Muneer Ahmad, Maizatul Akmar Ismail, Iftikhar Ahmad, Neelum Noreen

University of Roehampton

Research output: Contribution to journal › Article › peer-review

Abstract

This study proposes a Generative Adversarial Network (GAN)-based method for virtual furniture replacement within indoor scenes. The proposed method addresses the challenge of accurately positioning new furniture in an indoor space by combining image reconstruction with geometric matching through combining spatial transformer networks and GANs. The system leverages deep learning architectures like Mask R-CNN for executing image segmentation and generating masks, and it employs DeepLabv3+, EdgeConnect algorithms, and ST-GAN networks for carrying out virtual furniture replacement. With the proposed system, furniture shoppers can obtain a virtual shopping experience, providing an easier way to understand the aesthetic effects of furniture rearrangement without putting in effort to physically move furniture. The proposed system has practical applications in the furnishing industry and interior design practices, providing a cost-effective and efficient alternative to physical furniture replacement. The results indicate that the proposed method achieves accurate positioning of new furniture in indoor scenes with minimal distortion or displacement. The proposed system is limited to 2D front-view images of furniture and indoor scenes. Future work would involve synthesizing 3D scenes and expanding the system to replace furniture images photographed from different angles. This would enhance the efficiency and practicality of the proposed system for virtual furniture replacement in indoor scenes.

Original language	English
Pages (from-to)	3513
Number of pages	1
Journal	Mathematics
Volume	12
Issue number	22
DOIs	https://doi.org/10.3390/math12223513
Publication status	Accepted/In press - 29 Nov 2024

Access to Document

10.3390/math12223513

Cite this

@article{a223eb066ddf44aeabab3a96bef17f13,

title = "Deep Learning-Driven Virtual Furniture Replacement Using GANs and Spatial Transformer Networks",

abstract = "This study proposes a Generative Adversarial Network (GAN)-based method for virtual furniture replacement within indoor scenes. The proposed method addresses the challenge of accurately positioning new furniture in an indoor space by combining image reconstruction with geometric matching through combining spatial transformer networks and GANs. The system leverages deep learning architectures like Mask R-CNN for executing image segmentation and generating masks, and it employs DeepLabv3+, EdgeConnect algorithms, and ST-GAN networks for carrying out virtual furniture replacement. With the proposed system, furniture shoppers can obtain a virtual shopping experience, providing an easier way to understand the aesthetic effects of furniture rearrangement without putting in effort to physically move furniture. The proposed system has practical applications in the furnishing industry and interior design practices, providing a cost-effective and efficient alternative to physical furniture replacement. The results indicate that the proposed method achieves accurate positioning of new furniture in indoor scenes with minimal distortion or displacement. The proposed system is limited to 2D front-view images of furniture and indoor scenes. Future work would involve synthesizing 3D scenes and expanding the system to replace furniture images photographed from different angles. This would enhance the efficiency and practicality of the proposed system for virtual furniture replacement in indoor scenes.",

author = "Resmy Vijaykumar and Muneer Ahmad and Ismail, {Maizatul Akmar} and Iftikhar Ahmad and Neelum Noreen",

year = "2024",

month = nov,

day = "29",

doi = "10.3390/math12223513",

language = "English",

volume = "12",

pages = "3513",

journal = "Mathematics",

issn = "2227-7390",

publisher = "MDPI AG",

number = "22",

}

TY - JOUR

T1 - Deep Learning-Driven Virtual Furniture Replacement Using GANs and Spatial Transformer Networks

AU - Vijaykumar, Resmy

AU - Ahmad, Muneer

AU - Ismail, Maizatul Akmar

AU - Ahmad, Iftikhar

AU - Noreen, Neelum

PY - 2024/11/29

Y1 - 2024/11/29

N2 - This study proposes a Generative Adversarial Network (GAN)-based method for virtual furniture replacement within indoor scenes. The proposed method addresses the challenge of accurately positioning new furniture in an indoor space by combining image reconstruction with geometric matching through combining spatial transformer networks and GANs. The system leverages deep learning architectures like Mask R-CNN for executing image segmentation and generating masks, and it employs DeepLabv3+, EdgeConnect algorithms, and ST-GAN networks for carrying out virtual furniture replacement. With the proposed system, furniture shoppers can obtain a virtual shopping experience, providing an easier way to understand the aesthetic effects of furniture rearrangement without putting in effort to physically move furniture. The proposed system has practical applications in the furnishing industry and interior design practices, providing a cost-effective and efficient alternative to physical furniture replacement. The results indicate that the proposed method achieves accurate positioning of new furniture in indoor scenes with minimal distortion or displacement. The proposed system is limited to 2D front-view images of furniture and indoor scenes. Future work would involve synthesizing 3D scenes and expanding the system to replace furniture images photographed from different angles. This would enhance the efficiency and practicality of the proposed system for virtual furniture replacement in indoor scenes.

AB - This study proposes a Generative Adversarial Network (GAN)-based method for virtual furniture replacement within indoor scenes. The proposed method addresses the challenge of accurately positioning new furniture in an indoor space by combining image reconstruction with geometric matching through combining spatial transformer networks and GANs. The system leverages deep learning architectures like Mask R-CNN for executing image segmentation and generating masks, and it employs DeepLabv3+, EdgeConnect algorithms, and ST-GAN networks for carrying out virtual furniture replacement. With the proposed system, furniture shoppers can obtain a virtual shopping experience, providing an easier way to understand the aesthetic effects of furniture rearrangement without putting in effort to physically move furniture. The proposed system has practical applications in the furnishing industry and interior design practices, providing a cost-effective and efficient alternative to physical furniture replacement. The results indicate that the proposed method achieves accurate positioning of new furniture in indoor scenes with minimal distortion or displacement. The proposed system is limited to 2D front-view images of furniture and indoor scenes. Future work would involve synthesizing 3D scenes and expanding the system to replace furniture images photographed from different angles. This would enhance the efficiency and practicality of the proposed system for virtual furniture replacement in indoor scenes.

U2 - 10.3390/math12223513

DO - 10.3390/math12223513

M3 - Article

SN - 2227-7390

VL - 12

SP - 3513

JO - Mathematics

JF - Mathematics

IS - 22

ER -