Skip navigation
Please use this identifier to cite or link to this item: http://repository.iitr.ac.in/handle/123456789/21645
Title: Indic handwritten script identification using offline-online multi-modal deep network
Authors: Bhunia A.K.
Mukherjee S.
Sain A.
Bhunia A.K.
Pratim Roy, Partha
Pal U.
Published in: Information Fusion
Abstract: In this paper, we propose a novel approach of word-level Indic script identification using only character-level data in training stage. Our method uses a multi-modal deep network which takes both offline and online modality of the data as input in order to explore the information from both the modalities jointly for script identification task. We take handwritten data in either modality as input and the opposite modality is generated through intermodality conversion. Thereafter, we feed this offline-online modality pair to our network. Hence, along with the advantage of utilizing information from both the modalities, the proposed framework can work for both offline and online script identification which alleviates the need for designing two separate script identification modules for individual modality. We also propose a novel conditional multi-modal fusion scheme to combine the information from offline and online modality which takes into account the original modality of the data being fed to our network and thus it combines adaptively. An exhaustive experimental study has been done on a data set including English(Roman) and 6 other official Indic scripts. Our proposed scheme outperforms traditional classifiers along with handcrafted features and deep learning based methods. Experiment results show that using only character level training data can achieve competitive performance against traditional training using word level data. © 2019 Elsevier B.V.
Citation: Information Fusion, 57: 1-14
URI: https://doi.org/10.1016/j.inffus.2019.10.010
http://repository.iitr.ac.in/handle/123456789/21645
Issue Date: 2020
Publisher: Elsevier B.V.
Keywords: Character level training.
Deep neural network
Handwritten script identification
Multi-modal learning
Offline and online handwriting
ISSN: 15662535
Author Scopus IDs: 57188719920
57211037698
57195999376
57203526133
56880478500
57200742116
Author Affiliations: Bhunia, A.K., Institute for Media Innovation, Nanyang Technological University, Singapore, Centre for Vision, Speech and Signal Processing, University of Surrey, England, United Kingdom
Mukherjee, S., Department of ECE, Institute of Engineering & Management, Kolkata, India
Sain, A., Department of EE, Institute of Engineering & Management, Kolkata, India
Bhunia, A.K., Department of Electrical Engineering, Jadavpur University, India
Roy, P.P., Department of CSE, Indian Institute of Technology, Roorkee, India
Pal, U., Computer Vision and Pattern Recognition Unit, Indian Statistical Institute, Kolkata, India
Corresponding Author: Roy, P.P.; Department of CSE, India; email: proy.fcs@iitr.ac.in
Appears in Collections:Journal Publications [CS]

Files in This Item:
There are no files associated with this item.
Show full item record


Items in Repository are protected by copyright, with all rights reserved, unless otherwise indicated.