IIT Jodhpur develops software to convert digital comics to video

The software, Comic-to-Video Network (C2VNet), evolves panel-by-panel in a comic strip and eventually produces a full-length video (with audio) of a digitised or born-digital storybook. 

Researchers at the Indian Institute of Technology (IIT) Jodhpur are developing a software framework, Comic-to-Video Network (C2VNet), that converts digitised comics to video. Digital humanities contributes to a composite of approaches methods rather than different approaches that lay emphasis on preserving, reconstructing, transmitting, and interpreting human record historically and contemporaneously. The software is being developed by research scholars Vaibhavi Gupta, Vinay Detani and Vivek Khokhar under Prof Chiranjoy Chattopadhyay.  

The C2VNet evolves panel-by-panel in a comic strip and eventually produces a full-length video (with audio) of a digitised or born-digital storybook. The goal was to design and develop software that takes a born-digital or digitised comic book as input and produces an audio-visual animated movie from it. 

IIT Jodhpur researchers have also proposed a dataset titled “IMCDB: Indian Mythological Comic Dataset of Digitised Indian Comic Storybook” in English. This has complete annotations for panels, binary masks of the text balloon and text files for each speech balloon and narration box within a panel and plans to make the dataset publicly available.  

Chiranjoy Chattopadhyay, assistant professor, Department of Computer Science and Engineering, IIT Jodhpur, said the panel extraction model C2VNet has two internal networks to support the video creation. He said, “CPENet developed by the team gives over 97 per cent accuracy and the speech balloon segmentation model SBSNet gives 98 per cent accuracy with fewer parameters. Both have outperformed state-of-art models. C2VNet is the first step towards the big future of automatic multimedia creation of comic books to bring new comic reading experiences.” 

Read more: 

• Interview: Leadership, soft skills key to dream job, says IIT Kanpur student who bagged Rs 2-cr offer 
• COVID surge: 60 test positive for virus at IIT Kharagpur 
• Engineering in Kannada fails to attract students in Karnataka