フクシマ　エドワルド　フミヒコ   福島 E.文彦    所属   工学部 機械工学科    職種   教授
言語種別	英語
発行・発表の年月	2025/06
形態種別	学術論文
査読	査読あり
標題	Motion Semantic Enhancement and Autonomous Information Mining for Static-Dynamic Visual Emotion Recognition in Human-Robot Interaction
執筆形態	共著
掲載誌名	IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT
掲載区分	国外
出版社・発行元	IEEE
巻・号・頁	pp.1-15
総ページ数	15
国際共著	国際共著
著者・共著者	Cheng-Shan Jiang, Zhen-Tao Liu, Edwardo F. Fukushima and Jinhua She
概要	The advancement of static-dynamic visual emotion recognition plays a pivotal role in the evolution of intelligent and empathetic machines. Currently, the progress of static visual emotion recognition (SVER) faces challenges, primarily the need to balance model efficiency with robust recognition performance. Moreover, obstacles such as noisy data and limited labeled datasets restrict models from effectively learning the appearance features and dynamic dependencies intrinsic to dynamic visual emotion recognition (DVER). In the realm of SVER, facial pixel and semantic representation are derived using a lightweight surface and landmark features embedding network, followed by neuron-energy-based feature fusion-filtering to enhance the fusion of semantic representations positively. For DVER, a Global Anchor-dependent Noisy Emotional Data Filtering (GADNEF) method is employed for noisy label learning, facilitating clip-wise filtering of ambiguous data via iterative computations of frame-wise attention statistics across batches. Furthermore, a self-supervised learning paradigm based on a unified appearance and motion-guided masked autoencoder is implemented, enabling large-scale knowledge transfer tailored for video-based facial expression analysis. Our approach has achieved accuracies of 97.07%, 98.90%, 61.39%, and 92.28% on SVER datasets including KDEF, RaFD, SFEW, and RAF-DB, respectively, while the weighted average recall (WAR) and unweighted average recall (UAR) on DVER datasets such as DFEW and MAFW were 75.12%, 64.37%, 55.26%, and 42.53%, respectively. A preliminary application experiment has also been conducted to validate the practical applicability of our method within human-robot interaction (HRI) scenarios.
外部リンクURL	https://ieeexplore.ieee.org/document/11029040