IIIT Allahabad develops AI tool for blind, visually impaired to access chart images

Experts from the Indian Institute of Information Technology-Allahabad (IIIT-A), Prayagraj, in collaboration with their counterparts at the Karlsruhe Institute of Technology, Germany, have developed a tool for blind and visually impaired persons capable of creating and accessing chart images.

The paper “Alt4Blind: A User Interface to Simplify Charts Alt-Text Creation” related to this creation has been accepted and will be presented at the prestigious International Conference on Computers Helping People with Special Needs, being held in Linz, Austria, between July 8-12, 2024.

This work is being undertaken by PhD scholar and working professional Shahid Ali Farooqui, who is a senior technical officer at the Indian Institute of Science Education and Research (IISER)-Kolkata and hails from Prayagraj city. He is working under the guidance of Mohammed Javed, assistant professor in the department of IT at IIIT-Allahabad.

The PhD work is aimed at developing useful tools for blind people who can use this tool to search for images and understand them with Alt-Text generated by our AI-based software.

“Alternative Texts (Alt-Text) for chart images are essential for making graphics accessible to people with blindness and visual impairments. Traditionally, Alt-Text is manually written by authors but often encounters issues such as oversimplification or complication. Recent trends have seen the use of AI for Alt-Text generation. However, existing models are susceptible to producing inaccurate or misleading information. We have addressed this challenge by retrieving high-quality Alt-Texts from similar chart images, serving as a reference for the user when creating Alt-Texts,” said Shahid Ali Farooqui.

“We have introduced a new benchmark comprising 5,000 real images with semantically labeled high-quality Alt-Texts, collected from Human-Computer Interaction venues. We have also developed a deep learning-based model to rank and retrieve similar chart images that share the same visual and textual semantics, and we have designed a user interface (UI) to facilitate the Alt-Text creation process. Our preliminary interviews and investigations highlight the usability of our UI,” he explained.