Skip navigation
Please use this identifier to cite or link to this item:
Title: Word searching in scene image and video frame in multi-script scenario using dynamic shape coding
Authors: Pratim Roy, Partha
Bhunia A.K.
Bhattacharyya A.
Pal U.
Published in: Multimedia Tools and Applications
Abstract: Retrieval of text information from natural scene images and video frames is a challenging task due to its inherent problems like complex character shapes, low resolution, background noise, etc. Available OCR systems often fail to retrieve such information in scene/video frames. Keyword spotting, an alternative way to retrieve information, performs efficient text searching in such scenarios. However, current word spotting techniques in scene/video images are script-specific and they are mainly developed for Latin script. This paper presents a novel word spotting framework using dynamic shape coding for text retrieval in natural scene image and video frames. The framework is designed to search query keyword from multiple scripts with the help of on-the-fly script-wise keyword generation for the corresponding script. We have used a two-stage word spotting approach using Hidden Markov Model (HMM) to detect the translated keyword in a given text line by identifying the script of the line. A novel unsupervised dynamic shape coding based scheme has been used to group similar shape characters to avoid confusion and to improve text alignment. Next, the hypotheses locations are verified to improve retrieval performance. To evaluate the proposed system for searching keyword from natural scene image and video frames, we have considered two popular Indic scripts such as Bangla (Bengali) and Devanagari along with English. Inspired by the zone-wise recognition approach in Indic scripts [37], zone-wise text information has been used to improve the traditional word spotting performance in Indic scripts. For our experiment, a dataset consisting of images of different scenes and video frames of English, Bangla and Devanagari scripts were considered. The results obtained showed the effectiveness of our proposed word spotting approach. © 2018, Springer Science+Business Media, LLC, part of Springer Nature.
Citation: Multimedia Tools and Applications (2019), 78(6): 7767-7801
Issue Date: 2019
Publisher: Springer New York LLC
Keywords: Dynamic shape code
Hidden Markov model
Indic word spotting
Scene and video text retrieval
Word spotting in multiple scripts
ISSN: 13807501
Author Scopus IDs: 56880478500
Author Affiliations: Roy, P.P., Department of CSE, Indian Institute of Technology Roorkee, Roorkee, India
Bhunia, A.K., Department of ECE, Institute of Engineering & Management, Kolkata, India
Bhattacharyya, A., Department of ECE, Institute of Engineering & Management, Kolkata, India
Pal, U., CVPR Unit, Indian Statistical Institute, Kolkata, India
Corresponding Author: Roy, P.P.; Department of CSE, Indian Institute of Technology RoorkeeIndia; email:
Appears in Collections:Journal Publications [CS]

Files in This Item:
There are no files associated with this item.
Show full item record

Items in Repository are protected by copyright, with all rights reserved, unless otherwise indicated.