Multimodal interaction is a natural way for humans to perceive and act in a coordinated manner to perform a given task. This involves the 5 senses namely touch (haptics), vision, audio, smell and taste. Thus, Multimodal Interaction Lab (MIL) in the Department of Electrical Engineering at Indian Institute of Technology Jodhpur seeks to perform fundamental and applied research that helps to: 1) understand how humans interact and respond to stimuli, 2) develop computational models that can mimic important functionality in human perception, 3) design interactive systems for diverse applications like AR/VR, immersive multimedia and medical domain, 4) design efficient algorithm for modeling and representation of 3D objects and scenes, 5) design efficient algorithms for 3D reconstruction, object detection and recognition, and low-level vision applications, 6) design novel computation based imaging system to perceive the desired information from the scene, and 6) develop state-of-the-art multimodal-information processing methods for real-time applications. The MIL mainly focuses on the following two subthemes:

1. Computational Vision and Geometry Processing Lab
2. Multimodal Perception Lab

Equipment: Haptic interfaces: Phantom Premium 1.5 HF/6 DoF, Omega 3.0, Phantom Premium 1.5, Displays, Robotic Manipulator UR3, Force/Torque sensors, Lidar Scanner, RGB-D Cameras, Eyetracker, Cameras, Ambisonic-Microphone, Workstations (Server)

Research Projects

1. Data-driven Haptic Modeling and Rendering of Normal Interactions on Inhomogeneous Viscoelastic Deformable Objects [Dec. 2020- Dec.2022] (Funding Agency: DST SERB, 33 Lacs). PI: Dr. Amit Bhadwaj
2. Telepresence and Teleaction System for Robot Assisted Dentistry [Dec. 2021- Dec. 2024] (Funding agency: IHFC IIT Delhi, Amount: 129 Lacs) PI: Dr. Amit Bhardwaj, Co-PIs: Dr. Suril Shah, Dr. Vineet Gokhale (TU Delft), Dr. Ankita Chugh (AIIMS Jodhpur)
3. Haptic Camera: Inter-relationship between Different Sensing Modalities for Textured Surfaces [March. 2022- March. 2025] (Funding Agency: Seed Grant IIT Jodhpur, Amount: 21.43 Lacs), PI: Dr. Amit Bhardwaj
4. Haptics based Medical Simulator for Abdomen Palpation and Pulse Behavior [Oct. 2022-Sep. 2025] [Funding agency: iHub Drishti, Amount: around 62 Lacs), PI: Dr. Amit Bhardwaj, Co-PIs: Dr. Himanshu Kumar, Dr. Manish Narwaria, Dr. Rajendra Nagar and Dr. Bharat Choudhary (AIIMS Jodhpur)
5. Reflection Symmetry Based 3D Surface Reconstruction and Restoration from Partial Point Cloud, PI: Dr. Rajendra Nagar, Funding Agency: SERB, End Date: Dec 2023, Theme: 3D Computer Vision and Geometry Processing
6. Real-Time 3D Scene Reconstruction and Localization of Autonomous Ground Vehicles in Unknown Environments, PI: Dr. Rajendra Nagar, Funding Agency: iHub-Drishti, TIH IIT Jodhpur, End Date: April 2025, Theme: 3D Computer Vision and Robotics
7. Animatable Human 3D Surface Reconstruction from Partial 3D Scans, PI: Dr. Rajendra Nagar, Funding Agency: IIT Jodhpur, End Date: Oct 2024, Theme: 3D Computer Vision and Geometry Processing
8. Object-based 3D Spatial Audio Rendering for Augmented and Virtual Reality Applications, PI: Manish Narwarua, Funding agency: SERB, Edn date: Dec. 2022, Theme: Augmented and Virtual Reality
9. Wide Area Sensing for Visual Intelligence, PI: Manish Narwaria, Funding agency: iHub-Drishti, TIH IIT Jodhpur, End Date: April 2024, Theme: Computer Vision and Robotics

Associated Faculty Members

1. Amit Bhardwaj
2. Himanshu Kumar
3. Manish Narwaria
4. Rajendra Nagar

Students

1. Kriti Datta
2. Ravi Prakash Meghvanshi
3. Baruri Sai Avinash
4. Deepanshu Singh Solanki
5. Basant