Publication of the week: Dongxu Han, Naseer Al-Jawad & Hongbo Du

“Facial expression identification using 3D geometric features from Microsoft Kinect device”, Proc. SPIE 9869, Mobile Multimedia / Image Processing, Security, and Applications 2016 (ed. Sos S. Agaian, Sabah A. Jassim), 986903-1 (May 19, 2016); doi:10.1117/12.2223029

Facial expression identification is an important part of face recognition and closely related to emotion detection from face images. Various solutions have been proposed in the past using different types of cameras and features. The Microsoft Kinect device has been widely used for multimedia interactions. More recently, the device has been increasingly deployed for supporting scientific investigations. This paper explores the effectiveness of using the device in identifying emotional facial expressions such as surprise, smile, sadness, etc. and evaluates the usefulness of 3D data points on a face mesh structure obtained from the Kinect device. The paper presents a distance-based geometric feature component that is derived from the distances between points on the face mesh and selected reference points in a single frame. The feature components extracted across a sequence of frames starting and ending by neutral emotion represent a whole expression. The feature vector eliminates the need for complex face orientation correction, simplifying the feature extraction process and making it more efficient. The authors applied the kNN classifier that exploits a feature component based similarity measure following the principle of dynamic time warping to determine the closest neighbours. Preliminary tests on a small scale database of different facial expressions show promises of the newly developed features and the usefulness of the Kinect device in facial expression identification.

The paper is based on Dongxu Han’s final year undergraduate project in 2015. The paper was accepted by the SPIE International Conference on Mobile, Multimedia/Image Processing, Security and Applications, an annual international forum for researchers and industry practitioners to share their research and development ideas and experiences in multimedia systems and image processing. This year’s conference was held in Baltimore USA between 17 and 22 April 2016, and Dongxu presented the paper at the conference and received positive and useful comments.

Dongxu graduated from Applied Computing with a first class honours degree in 2015. Dongxu also received the British Computer Society Buckingham Prize for the Best Graduating Students in Computing earlier this year. Dongxu is now registered as an MPhil/DPhil student, conducting research in machine learning based decision support systems for supporting medical diagnostic decisions and most effective treatments. The research is a collaboration between Applied Computing and Queen Charlotte and Chelsea Hospital, Imperial College London and KU Hospital Leuven, Belgium.

Hongbo Du and Naseer Al-Jawad are Senior Lecturers in the Department of Applied Computing.