IIT Jodhpur
Congratulations to Dr. Gaurav Bhatnagar et al. for the acceptance of their research article in the journal IEEE Transactions on Dependable and Secure Computing

Title: Knowledge driven Description Synthesis for Floor Plan Interpretation

Authors: S. Goyal, C. Chattopadhyay and G. Bhatnagar

Journal:  International Journal on Document Analysis and Recognition

Volume: In Press

Year: 2021

Publisher: Springer

Abstract: Image captioning is a widely known problem in the area of AI. Caption generation from floor plan images has applications in indoor path planning, real estate, and providing architectural solutions. Several methods have been explored in the literature for generating captions or semistructured descriptions from floor plan images. Since only the caption is insufficient to capture fine-grained details, researchers also proposed descriptive paragraphs from images. However, these descriptions have a rigid structure and lack exibility, making it difficult to use them in real-time scenarios. This paper offers two models, Description Synthesis from Image Cue (DSIC) and Transformer-Based Description Generation (TBDG), for text generation from floor plan images. These two models take advantage of modern deep neural networks for visual feature extraction and text generation. The difference between both models is in the way they take input from the floor plan image. The DSIC model takes only visual features automatically extracted by a deep neural network, while the TBDG model learns textual captions extracted from input floor plan images with paragraphs. The specific keywords generated in TBDG and understanding them with paragraphs make it more robust in a general floor plan image. Experiments were carried out on a large scale publicly available dataset and compared with state-of-the-art techniques to show the proposed model's superiority.