A Generalized Review Of Human-Computer Interaction Using Electromyogram
Signals

Sourav      Maity; Karan      Veer
Abstract

The primary use of human-computer interaction is in the smart home as well as in industry 4.0. Communication between computers and humans can be benefitted from a spontaneous interchange of emotions. The objective of the work is to provide an idea regarding the process of identifying various emotions using facial electromyography signals through the electrode placement method. Here, one contemplated the facial electromyography on masticatory function assessment and emotional articulation monitoring. Furthermore, we have also presented the measurement of facial electromyography including the selection of electrode, location of the electrode, and reduction of noise. Facial emotions have a significant effect on the cognitive process of the human brain such as doubt perception, ability to solve problems, learning capabilities, emotional interactions, and memory which is beneficial while interacting with patients suffering from depression and stress. Through the use of rehabilitation applications, patients are directed through their recovery process while becoming accustomed to their emotional state or wellbeing, which has the dual effect of boosting motivation and accelerating recovery. This review paper will motivate and inspire researchers and engineers for finding more suitable systems for various applications.
Keywords: EMG signal, facial expressions, signal processing, biomedical signals, analysis, electrode.
Graphical Abstract

[1]
E. Scheme,  and K. Englehart, "Electromyogram pattern recognition for control of powered upper-limb prostheses: State of the art and challenges for clinical use", J. Rehabil. Res. Dev., vol. 48, no. 6, pp. 643-659, 2011.
 [http://dx.doi.org/10.1682/JRRD.2010.09.0177] [PMID:  21938652]
[2]
A. Phinyomark, P. Phukpattaranont,  and C. Limsakul, "A review of control methods for electric power wheelchairs based on electromyography signals with special emphasis on pattern recognition", IETE Tech. Rev., vol. 28, pp. 316-326, 2011.
 [http://dx.doi.org/10.4103/0256-4602.83552]
[3]
T.S. Saponas, D.S. Tan, D. Morris, R. Balakrishnan, J. Turner,  and J.A. Landay, "Enabling always-available input with muscle-computer interfaces", In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology, Victoria, BC, Canada, 2009, pp. 167-176 
 [http://dx.doi.org/10.1145/1622176.1622208]
[4]
J. Yousefi,  and A. Hamilton-Wright, "Characterizing EMG data using machine-learning tools", Comput. Biol. Med., vol. 51, pp. 1-13, 2014.
 [http://dx.doi.org/10.1016/j.compbiomed.2014.04.018] [PMID:  24857941]
[5]
P. Padmanabhan,  and S. Puthusserypady, "Nonlinear analysis of EMG signals-A chaotic approach", In Proceedings of the 26th Annual
International Conference of the IEEE Engineering in Medicine and
Biology Society, vol. 1, San Francisco, CA, USA, 2004, pp. 608-611 
 [http://dx.doi.org/10.1109/IEMBS.2004.1403231]
[6]
E. Scheme,  and K. Englehart, "Training strategies for mitigating the effect of proportional control on classification in pattern recognition based myoelectric control", J. Prosthet. Orthot., vol. 25, no. 2, pp. 76-83, 2013.
 [http://dx.doi.org/10.1097/JPO.0b013e318289950b] [PMID:  23894224]
[7]
S. Thongpanja, A. Phinyomark, C. Limsakul,  and P. Phukpattaranont, "Probability density of electromyography signal for different levels of contraction of biceps brachii", In Proceedings of the 10th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, Krabi, Thailand, 2013, pp. 1-5 
 [http://dx.doi.org/10.1109/ECTICon.2013.6559497]
[8]
E.A. Clancy,  and N. Hogan, "Probability density of the surface electromyogram and its relation to amplitude detectors", IEEE Trans. Biomed. Eng., vol. 46, no. 6, pp. 730-739, 1999.
 [http://dx.doi.org/10.1109/10.764949] [PMID:  10356879]
[9]
I.W. Hunter, R.E. Kearney,  and L.A. Jones, "Estimation of the conduction velocity of muscle action potentials using phase and impulse response function techniques", Med. Biol. Eng. Comput., vol. 25, no. 2, pp. 121-126, 1987.
 [http://dx.doi.org/10.1007/BF02442838] [PMID:  3695613]
[10]
M. Bilodeau, M. Cincera, A.B. Arsenault,  and D. Gravel, "Normality and stationarity of EMG signals of elbow flexor muscles during ramp and step isometric contractions", J. Electromyogr. Kinesiol., vol. 7, no. 2, pp. 87-96, 1997.
 [http://dx.doi.org/10.1016/S1050-6411(96)00024-7] [PMID:  20719694]
[11]
F.S. Ayachi, S. Boudaoud,  and C. Marque, "Evaluation of muscle force classification using shape analysis of the sEMG probability density function: A simulation study", Med. Biol. Eng. Comput., vol. 52, no. 8, pp. 673-684, 2014.
 [http://dx.doi.org/10.1007/s11517-014-1170-x] [PMID:  24961179]
[12]
C. Darwin, The Expression of Emotions in Man and Animals, Murray: London, UK, 1872, pp. 30-180.
[13]
P. Ekman, W.V. Friesen,  and J.C. Hager, Facial Action Coding System The Manual, In Facial Action Coding System., Consulting Psychologists Press: Palo Alto, CA, USA, 2002.
[14]
S. Turabzadeh, H. Meng, R. Swash, M. Pleva,  and J. Juhar, "Facial expression emotion detection for real-time embedded systems", Technologies (Basel), vol. 6, p. 1, 2018.
 [http://dx.doi.org/10.3390/technologies6010017]
[15]
C-N. Huang, C-H. Chen,  and H-Y. Chung, "Speech pattern recognition with facial electromyography", In Biomedical Engineering Society Annual Symposium, 2003. 
[16]
B.C. Watson, "Measures of speech production", IEEE Eng. Med. Biol. Mag., vol. 7, no. 1, pp. 30-33, 1988.
 [http://dx.doi.org/10.1109/51.651] [PMID:  18244048]
[17]
F. Grandori, P. Pinelli, P. Ravazzani, F. Ceriani, G. Miscio, F. Pisano, R. Colombo, S. Insalaco,  and G. Tognola, "Multiparametric analysis of speech production mechanisms", Engineering in Medicine and Biology Magazine, IEEE, vol. 13, no. 2, pp. 203-209, 1994.
 [http://dx.doi.org/10.1109/51.281679]
[18]
I.J.T. Veldhuizen, A.W.K. Gaillard,  and J. de Vries, "The influence of mental fatigue on facial EMG activity during a simulated workday", Biol. Psychol., vol. 63, no. 1, pp. 59-78, 2003.
 [http://dx.doi.org/10.1016/S0301-0511(03)00025-5] [PMID:  12706964]
[19]
V. Surakka,  and J.K. Hietanen, "Facial and emotional reactions to Duchenne and non-Duchenne smiles", Int. J. Psychophysiol., vol. 29, no. 1, pp. 23-33, 1998.
 [http://dx.doi.org/10.1016/S0167-8760(97)00088-3] [PMID:  9641245]
[20]
D.M. Sloan, M.M. Bradley, E. Dimoulas,  and P.J. Lang, "Looking at facial expressions: Dysphoria and facial EMG", Biol. Psychol., vol. 60, no. 2-3, pp. 79-90, 2002.
 [http://dx.doi.org/10.1016/S0301-0511(02)00044-3] [PMID:  12270585]
[21]
S. Hu, K.A. Player, K.A. Mcchesney, M.D. Dalistan, C.A. Tyner,  and J.E. Scozzafava, "Facial EMG as an indicator of palatability in humans", Physiol. Behav., vol. 68, no. 1-2, pp. 31-35, 1999.
 [http://dx.doi.org/10.1016/S0031-9384(99)00143-2] [PMID:  10627059]
[22]
J. de Jong Peters, P. Madelon,  and V. Inge, "Disgust and disgust sensitivity in spider phobia", J. Anxiety Disord., vol. 16, no. 5, pp. 477-493, 2002.
 [http://dx.doi.org/10.1016/S0887-6185(02)00167-6] [PMID:  12396207]
[23]
L. Jäncke, J. Vogt, F. Musial, K. Lutz,  and K.T. Kalveram, "Facial EMG responses to auditory stimuli", Int. J. Psychophysiol., vol. 22, no. 1-2, pp. 85-96, 1996.
 [http://dx.doi.org/10.1016/0167-8760(96)00013-X] [PMID:  8799771]
[24]
C. Angkoon Phinyomark, "Limsakul, feature reduction and selection for EMG signal classification", Expert Syst. Appl., vol. 7, pp. 7420-7431, 2012.
 [http://dx.doi.org/10.1016/j.eswa.2012.01.102]
[25]
C. Kendell, E.D. Lemaire, Y. Losier, A. Wilson, A. Chan,  and B. Hudgins, "A novel approach to surface electromyography: An exploratory study of electrode-pair selection based on signal characteristics", J. Neuroeng. Rehabil., vol. 9, p. 24, 2012.
 [http://dx.doi.org/10.1186/1743-0003-9-24] [PMID:  22537650]
[26]
H.M. Al-Angari, G. Kanitz, S. Tarantino,  and C. Cipriani, "Distance and mutual information methods for EMG feature and channel subset selection for classification of hand movements", Biomed. Signal Process. Control, vol. 27, pp. 24-31, 2016.
 [http://dx.doi.org/10.1016/j.bspc.2016.01.011]
[27]
M.A. Oskoei,  and H. Hu, "Support vector machine-based classification scheme for myoelectric control applied to upper limb", IEEE Trans. Biomed. Eng., vol. 55, no. 8, pp. 1956-1965, 2008.
 [http://dx.doi.org/10.1109/TBME.2008.919734] [PMID:  18632358]
[28]
A. Phinyomark, C. Limsakul,  and P. Phukpattaranont, "Novel feature extraction for robust emg pattern recognition", J. Comput., vol. 1, pp. 71-80, 2009.
[29]
Y. Huang, F. Chen, S. Lv,  and X. Wang, "Facial expression recognition: A survey", Symmetry (Basel), vol. 11, p. 1189, 2019.
 [http://dx.doi.org/10.3390/sym11101189]
[30]
A. Paiva, I. Leite,  and T. Ribeiro, Emotion modelling for social robots., Ict. Usc. Edu, 2012.
[31]
K. Veer,  and T. Sharma, "A novel feature extraction for robust EMG pattern recognition", J. Med. Eng. Technol., vol. 40, no. 4, pp. 149-154, 2016.
 [http://dx.doi.org/10.3109/03091902.2016.1153739] [PMID:  27004618]
[32]
A. Alkan,  and M. Günay, "Identification of EMG signals using discriminant analysis and SVM classifier", Expert Syst. Appl., vol. 39, no. 1, pp. 44-47, 2012.
 [http://dx.doi.org/10.1016/j.eswa.2011.06.043]
[33]
B. Cesqui, P. Tropea, S. Micera,  and H.I. Krebs, "EMG-based pattern recognition approach in post stroke robot-aided rehabilitation: A feasibility study", J. Neuroeng. Rehabil., vol. 10, no. 1, p. 75, 2013.
 [http://dx.doi.org/10.1186/1743-0003-10-75] [PMID:  23855907]
[34]
Z.G. Zhang, H.T. Liu, S.C. Chan, K.D.K. Luk,  and Y. Hu, "Time-dependent power spectral density estimation of surface electromyography during isometric muscle contraction: Methods and comparisons", J. Electromyogr. Kinesiol., vol. 20, no. 1, pp. 89-101, 2010.
 [http://dx.doi.org/10.1016/j.jelekin.2008.09.007] [PMID:  19027325]
[35]
M.A. Oskoei,  and H. Hu, "GA-based feature subset selection for myoelectric classification", In IEEE International Conference on Robotics and Biomimetics, Kunming, China, 2006. 
 [http://dx.doi.org/10.1109/ROBIO.2006.340145]
[36]
Y-J. Liu, J-K. Zhang, W-J. Yan, S-J. Wang, G. Zhao,  and X. Fu, "A main directional mean optical flow feature for spontaneous micro-expression recognition", In: IEEE Trans. Affec. Comp., 2015.
[37]
S. Hu, J. Nathaniel, B.S. Riggan, C. Gordon, K.P. Gurton,  M. Thielke, P. Gurram,  and A.L. Chan, "A polarimetric thermal database for face recognition research", IEEE Conf. on Comp. Visi. Patt. Recog. Worksh., 2016.
[38]
K. Basterretxea, J. Echanobe,  and I. Campo, "A wearable human activity recognition system on a chip", IEEE Proceed of. Conf on
Des. and Arc. for Sig. Ima. Process, 2014.
[39]
Md. R. Ahsan, M.I. Ibrahimy,  and O.O. Khalifa, "Electromyography (EMG) signal based hand gesture recognition
using artificial neural network (ANN)", In IEEE 4th International Conf.on Mecha. (ICOM), 2011. 
[40]
Q. Wang, X. Chen, R. Chen, Y. Chen,  and X. Zhang, "Elecromyography-based locomotion pattern recognition and personal
positioning toward improved context-awareness applications", IEEE Trans. on Sys. Ma.and Cyb: Sys, vol. 43, no. 5, 2013.
[41]
M.I. Ibrahimy, M.R. Ahsan,  and O.O. Khalifa, "Design and optimization of levenberg-marquardt based neuralnetwork classifier for EMG signals to identify hand motions", Meas. Sci. Rev., vol. 13, pp. 142-1, 2013.
[42]
T. Lorrain, N. Jiang,  and D. Farina, "Influence of the training set on the accuracy of surface EMG classification in dynamic contractions for the control of multifunction prostheses", J. Neuroeng. Rehabil., vol. 8, p. 25, 2011.
 [PMID:  21554700]
[43]
Yu-Dong Zhang, Zhang-Jing Yang, Hui-Min Lu, Xing-Xing Zhou, Philips Preetha, Qing-Ming Liu,  and Shui-hua Wang, "Facial emotion recognition based on biorthogonal wavelet entropy, fuzzy support vector machine and stratified cross validation", IEEE Spe. Sec.on Emo. Awa. Mob. Comp, 2016.
[44]
M. Su, F. Li, S. Chen, Z. Huang, M. Qin, W. Li, X. Zhang,  and Y. Song, "Nanoparticle based curve arrays for multirecognition flexible electronics", Adv. Mater., vol. 28, no. 7, pp. 1369-1374, 2016.
 [PMID:  26644086]
[45]
E. Roh, B-U. Hwang, D. Kim, B.Y. Kim,  and N-E. Lee, "Strectchable, transparent, ultrasensitive and patchable strain sensor for human machine interfaces comprising a nanohybrid of carbon nanotubes and conductive elastomers", ACS Nano, vol. 9, no. 6, pp. 6252-6261, 2015.
[46]
M. Hamedi, Sh-Hussain Salleh, T.S. Tan, K. Imail, J. Ali, C. Dee-Uam, C. Pavaganun,  and P.P. Yupapin, "Human facial neutral activities and gesture recognition for machine-interfacing applications", Int. J. Nanomedicine, 2011.
[47]
C. Zong,  and M. Chetouani, "Hilbert-huang transform based physiological signals analysis for emotion recognition", In Proceedings of the 2009 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), Ajman, UAE, 2009, pp. 334-339 
[48]
W.V.F. Paul Ekman, A technique for the measurement of facial action. Facial Action Coding System (FACS)., Paul Ekman GroupManchester: UK, 1978.
[49]
B. Matzke, S.C. Herpertz, C. Berger, M. Fleischer,  and G. Domes, "Facial reactions during emotion recognition in borderline personality disorder: A facial electromyography study", Psychopathology, vol. 47, no. 2, pp. 101-110, 2014.
 [PMID:  24021701]
[50]
A. Boxtel, "Van Facial EMG as a tool for inferring a_ective states", Proc. Meas. Behav., vol. 2010, pp. 104-108, 2010.
[51]
P. Weyers, A. Mühlberger, C. Hefele,  and P. Pauli, "Electromyographic responses to static and dynamic avatar emotional facial expressions", Psychophysiology, vol. 43, no. 5, pp. 450-453, 2006.
 [http://dx.doi.org/10.1111/j.1469-8986.2006.00451.x] [PMID:  16965606]
[52]
S. Wioleta, "Using physiological signals for emotion recognition", In Proceedings of the 2013 6th International Conference on Human System Interactions (HSI), Gdansk, Poland, 2013, pp. 556-561 
 [http://dx.doi.org/10.1109/HSI.2013.6577880]
[53]
D. Girardi, F. Lanubile,  and N. Novielli, "Emotion detection using noninvasive low cost sensors", Proceedings of the 2017 Seventh International Conference on A_ective Computing and Intelligent Interaction (ACII)..
 2017. pp. 125-130, San Antonio, TX, USA23–26 October [http://dx.doi.org/10.1109/ACII.2017.8273589]
[54]
M. S. AL-Quraishi, I. Elamvazuthi, T.B. Tang, A. Q. Muhammad, S. Parasuraman,  and A. Borboni, "Multimodal fusion approach based on EEG and EMG signals for lower limb movement recognition", IEEE Sens. J., 2021.
[55]
A. Balbinot,  and G. Favieiro, "A neuro-fuzzy system for characterization of arm movements", Sensors (Basel), vol. 13, no. 2, pp. 2613-2630, 2013.
 [http://dx.doi.org/10.3390/s130202613] [PMID:  23429579]
[56]
M. Khezri,  and M. Jahed, "A neuro-fuzzy inference system for semg-based identification of hand motion commands", IEEE Trans. Ind. Electron., vol. 58, pp. 1952-1960, 2007.
 [http://dx.doi.org/10.1109/TIE.2010.2053334]
[57]
A. Subasi, "Classification of EMG signals using PSO optimized SVM for diagnosis of neuromuscular disorders", Comput. Biol. Med., vol. 43, no. 5, pp. 576-586, 2013.
 [http://dx.doi.org/10.1016/j.compbiomed.2013.01.020] [PMID:  23453053]
[58]
H. Sharon, I. Elamvazuthi, C.K. Lu, S. Parasuraman,  and E. Natarajan, "Development of rheumatoid arthritis classification from electronic image sensor using ensemble method", Sensors (Basel), vol. 20, no. 1, p. 167, 2019.
 [http://dx.doi.org/10.3390/s20010167] [PMID:  31892135]
[59]
E. Natarajan, V. Kaviarasan, W.H. Lim, S.S. Tiang,  and T.H. Tan, "Enhanced multi-objective teaching-learning-based optimization for machining of delrin", 
 [http://dx.doi.org/10.1109/ACCESS.2018.2869040]
[60]
E. Natarajan, V. Kaviarasan, W.H. Lim, S.S. Tiang, S. Parasuraman,  and S. Elango, "Non-dominated sorting modified teaching–learning-based optimization for multi-objective machining of polytetrafluoroethylene (PTFE)", J. Intell. Manuf., pp. 1-25, 2019.
[61]
A. Martínez-Rodrigo, R. Zangróniz, J.M. Pastor, J.M. Latorre,  and A. Fernández-Caballero, "Emotion detection in ageing adults from physiological sensors", Proc. Adv. Intelli. Sys. Comput., vol. 376, pp. 253-261, 2015.
 [http://dx.doi.org/10.1007/978-3-319-19695-4_26]
[62]
A. Nakasone, H. Prendinger,  and M. Ishizuka, "Procomp infiniti bio-signal encoder", In Proceedings of the 5th International Workshop on Biosignal Interpretation, Tokyo, Janpan, 2005, pp. 219-222 
[63]
J. Wagner, J. Kim,  and E. Andre, "From physiological signals to emotions: Implementing and comparing selected methods for feature extraction and classification", In Proceedings of the 2005 IEEE International Conference on Multimedia and Expo, Amsterdam, the Netherlands, 2005, pp. 940-943 
 [http://dx.doi.org/10.1109/ICME.2005.1521579]
[64]
C.J. De Luca, "The use of surface electromyography in biomechanics", J. Appl. Biomech., 1997.
 [http://dx.doi.org/10.1123/jab.13.2.135]
[65]
C-N. Huang, C-H. Chen,  and H-Y. Chung, "The review of applications and measurements in facial electromyography", J. Med. Biol. Eng., vol. 25, no. 1, pp. 15-20, 2004.
[66]
D.V. Moretti, F. Babiloni, F. Carducci, F. Cincotti, E. Remondini, P.M. Rossini, S. Salinari,  and C. Babiloni, "Computerized processing of EEG-EOG-EMG artifacts for multi-centric studies in EEG oscillations and event-related potentials", Int. J. Psychophysiol., vol. 47, no. 3, pp. 199-216, 2003.
 [http://dx.doi.org/10.1016/S0167-8760(02)00153-8] [PMID:  12663065]
[67]
U. Qidwai, M.S. Ajimsha,  and M. Shakir, "The role of EEG and EMG combined virtual reality gaming system in facial palsy rehabilitation - A case report", J. Bodyw. Mov. Ther., vol. 23, no. 2, pp. 425-431, 2019.
 [http://dx.doi.org/10.1016/j.jbmt.2019.02.012] [PMID:  31103130]
[68]
P. Rong,  and G.L. Pattee, "A multidimensional facial surface EMG analysis for objective assessment of bulbar involvement in amyotrophic lateral sclerosis", Clin. Neurophysiol., 2022.
 [PMID:  35033773]
[69]
Y. Golland, A. Hakim, T. Aloni, S. Schaefer,  and N. Levit-Binnun, "Affect dynamics of facial EMG during continuous emotional experiences", Biol. Psychol., vol. 139, pp. 47-58, 2018.
 [http://dx.doi.org/10.1016/j.biopsycho.2018.10.003] [PMID:  30300673]
[70]
S.L. Kroll, L.M. Mayo, A. Asratian, A. Yngve, I. Perini,  and M. Heilig, "Negative self-evaluation induced by acute stress indexed using facial EMG", Psychoneuroendocrinology, vol. 133, p. 105402, 2021.
 [http://dx.doi.org/10.1016/j.psyneuen.2021.105402] [PMID:  34530295]
[71]
A.J. Fridlund,  and J.T. Cacioppo, "Guidelines for human electromyographic research", Psychophysiology, vol. 23, no. 5, pp. 567-589, 1986.
 [http://dx.doi.org/10.1111/j.1469-8986.1986.tb00676.x] [PMID:  3809364]
[72]
E.A. Clancy, E.L. Morin,  and R. Merletti, "Sampling, noise-reduction and amplitude estimation issues in surface electromyography", J. Electromyogr. Kinesiol., vol. 12, no. 1, pp. 1-16, 2002.
 [http://dx.doi.org/10.1016/S1050-6411(01)00033-5] [PMID:  11804807]
Cite As
Recent Patents on Engineering

A Generalized Review Of Human-Computer Interaction Using Electromyogram Signals

Abstract

Graphical Abstract
