フクシマ　エドワルド　フミヒコ   福島 E.文彦    所属   工学部 機械工学科    職種   教授
言語種別	英語
発行・発表の年月	2024/01
形態種別	学術論文
査読	査読あり
標題	Adaptive key-frame selection-based facial expression recognition via multi-cue dynamic features hybrid fusion
執筆形態	共著
掲載誌名	Information Sciences
掲載区分	国外
出版社・発行元	ELSEVIER
巻・号・頁	(660),pp.1-19
総ページ数	19
国際共著	国際共著
著者・共著者	Bei Pan, Kaoru Hirota, Yaping Dai, Zhiyang Jia, Edwardo F. Fukushima, Jinhua She
概要	A multi-cue dynamic features hybrid fusion (MDF-HF) method for video-based facial expression recognition is presented. It is composed of key-frame selection, multi-cue dynamic feature extraction, and information fusion components. An adaptive key-frame selection strategy is first designed in the training procedure to extract pivotal facial images from video sequences, addressing the challenge of imbalanced data distribution and improving data quality. The similarity threshold used for key-frame selection is automatically adjusted based on the number of image frames in each expression category, creating a flexible frame processing procedure. Multi-cue spatio-temporal feature descriptors are then designed to acquire diverse dynamic feature representations from the selected key-frame sequences. With parallel computation, different levels of semantic information are extracted simultaneously to explore facial expression deformation in video clips. To integrate features from multiple cues, a weighted stacking ensemble strategy is devised, preserving unique feature characteristics while exploring interrelationships among the multi-cue features. The proposed method is evaluated on three benchmark datasets: eNTERFACE’05, BAUM-1s, and AFEW, achieving average accuracies of 59.7%, 57.5%, and 54.7%, respectively. The MDF-HF method exhibits superior performance, compared to state-of-theart methods in facial expression recognition, offering a robust solution for recognizing facial expressions in dynamic and unconstrained video scenarios.
外部リンクURL	https://doi.org/10.1016/j.ins.2024.120138