Skip navigation
Please use this identifier to cite or link to this item: http://repository.iitr.ac.in/handle/123456789/5591
Title: Fingertip detection and tracking for recognition of air-writing in videos
Authors: Mukherjee S.
Ahmed S.A.
Dogra D.P.
Kar S.
Pratim Roy, Partha
Published in: Expert Systems with Applications
Abstract: Air-writing is the process of writing characters or words in free space using finger or hand movements without the aid of any hand-held device. In this work, we address the problem of mid-air finger writing using web-cam video as input. In spite of recent advances in object detection and tracking, accurate and robust detection and tracking of the fingertip remains a challenging task, primarily due to small dimension of the fingertip. Moreover, the initialization and termination of mid-air finger writing is also challenging due to the absence of any standard delimiting criterion. To solve these problems, we propose a new writing hand pose detection algorithm for initialization of air-writing using the Faster R-CNN framework for accurate hand detection followed by hand segmentation and finally counting the number of raised fingers based on geometrical properties of the hand. Further, we propose a robust fingertip detection and tracking approach using a new signature function called distance-weighted curvature entropy. Finally, a fingertip velocity-based termination criterion is used as a delimiter to mark the completion of the air-writing gesture. Experiments show the superiority of the proposed fingertip detection and tracking algorithm over state-of-the-art approaches giving a mean precision of 73.1% while achieving real-time performance at 18.5 fps, a condition which is of vital importance to air-writing. Character recognition experiments give a mean accuracy of 96.11% using the proposed air-writing system, a result which is comparable to that of existing handwritten character recognition systems. © 2019 Elsevier Ltd
Citation: Expert Systems with Applications (2019), 136(): 217-229
URI: https://doi.org/10.1016/j.eswa.2019.06.034
http://repository.iitr.ac.in/handle/123456789/5591
Issue Date: 2019
Publisher: Elsevier Ltd
Keywords: Air-writing
Fingertip detection and tracking
Hand pose detection
Handwritten character recognition
Human-computer interaction (HCI)
ISSN: 9574174
Author Scopus IDs: 57209496772
57190343458
35408975400
55808071612
56880478500
Author Affiliations: Mukherjee, S., Department of Electrical Engineering, National Institute of Technology Durgapur, Durgapur, 713209, India
Ahmed, S.A., Department of Mathematics, National Institute of Technology Durgapur, Durgapur, 713209, India
Dogra, D.P., School of Electrical Science, Indian Institute of Technology Bhubaneswar, Bhubaneswar, 751013, India
Kar, S., Department of Mathematics, National Institute of Technology Durgapur, Durgapur, 713209, India
Roy, P.P., Department of Computer Science and Engineering, Indian Institute of Technology Roorkee, Roorkee, 247667, India
Corresponding Author: Mukherjee, S.; Department of Electrical Engineering, National Institute of Technology DurgapurIndia; email: sm.20150025@btech.nitdgp.ac.in
Appears in Collections:Journal Publications [CS]

Files in This Item:
There are no files associated with this item.
Show full item record


Items in Repository are protected by copyright, with all rights reserved, unless otherwise indicated.