Skip navigation
Please use this identifier to cite or link to this item: http://repository.iitr.ac.in/handle/123456789/21797
Title: A multimodal-Siamese Neural Network (mSNN) for person verification using signatures and EEG
Authors: Chakladar D.D.
Kumar P.
Pratim Roy, Partha
Dogra D.P.
Scheme E.
Chang V.
Published in: Information Fusion
Abstract: Signatures have long been considered to be one of the most accepted and practical means of user verification, despite being vulnerable to skilled forgers. In contrast, EEG signals have more recently been shown to be more difficult to replicate, and to provide better biometric information in response to known a stimulus. In this paper, we propose combining these two biometric traits using a multimodal Siamese Neural Network (mSNN) for improved user verification. The proposed mSNN network learns discriminative temporal and spatial features from the EEG signals using an EEG encoder and from the offline signatures using an image encoder. Features of the two encoders are fused into a common feature space for further processing. A Siamese network then employs a distance metric based on the similarity and dissimilarity of the input features to produce the verification results. The proposed model is evaluated on a dataset of 70 users, comprised of 1400 unique samples. The novel mSNN model achieves a 98.57% classification accuracy with a 99.29% True Positive Rate (TPR) and False Acceptance Rate (FAR) of 2.14%, outperforming the current state-of-the-art by 12.86% (in absolute terms). This proposed network architecture may also be applicable to the fusion of other neurological data sources to build robust biometric verification or diagnostic systems with limited data size. © 2021
Citation: Information Fusion, 71: 17-27
URI: https://doi.org/10.1016/j.inffus.2021.01.004
http://repository.iitr.ac.in/handle/123456789/21797
Issue Date: 2021
Publisher: Elsevier B.V.
Keywords: CNN
EEG
LSTM
Multimodal
Siamese Neural Network
User verification
ISSN: 15662535
Author Scopus IDs: 57202036550
36012527200
56880478500
35408975400
57202922022
56926234700
Author Affiliations: Chakladar, D.D., Department of Computer Science & Engineering, Indian Institute of Technology Roorkee, Pin code-247667, Roorkee, India
Kumar, P., Institute of Biomedical Engineering, University of New Brunswick, Canada
Roy, P.P., Department of Computer Science & Engineering, Indian Institute of Technology Roorkee, Pin code-247667, Roorkee, India
Dogra, D.P., School of Electrical Sciences, Indian Institute of Technology Bhubaneswar, Pin code- 752050, Odisha, India
Scheme, E., Institute of Biomedical Engineering, University of New Brunswick, Canada
Chang, V., Artificial Intelligence and Information Systems Research Group, School of Computing, Engineering and Digital Technologies, Teesside University, Middlesbrough, United Kingdom
Funding Details: Prof Chang’s research is partly supported by VC Research (number: VCR 0000050 ). Dr. Scheme’s research is partly supported by the New Brunswick Innovation Foundation, Canada . VCR 0000050; New Brunswick Innovation Foundation, NBIF
Corresponding Author: Chang, V.; Artificial Intelligence and Information Systems Research Group, United Kingdom; email: v.chang@tees.ac.uk
Appears in Collections:Journal Publications [CS]

Files in This Item:
There are no files associated with this item.
Show full item record


Items in Repository are protected by copyright, with all rights reserved, unless otherwise indicated.