Facial affect "in the wild": a survey and a new database

Zafeiriou, Stefanos, Papaioannou, Athanasios, Kotsia, Irene ORCID logoORCID: https://orcid.org/0000-0002-3716-010X, Nicolaou, Mihalis A. and Zhao, Guoying (2016) Facial affect "in the wild": a survey and a new database. 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Affect "in-the-wild" Workshop, 26 June - 01 July 2016, Las Vegas, USA. ISBN 9781509014378. ISSN 2160-7516 [Conference or Workshop Item] (doi:10.1109/cvprw.2016.186)

PDF - Final accepted version (with author's formatting)
Download (6MB) | Preview


Well-established databases and benchmarks have been developed in the past 20 years for automatic facial behaviour analysis. Nevertheless, for some important problems regarding analysis of facial behaviour, such as (a) estimation of affect in a continuous dimensional space (e.g., valence and arousal) in videos displaying spontaneous facial behaviour and (b) detection of the activated facial muscles (i.e., facial action unit detection), to the best of our knowledge, well-established in-the-wild databases and benchmarks do not exist. That is, the majority of the publicly available corpora for the above tasks contain samples that have been captured in controlled recording conditions and/or captured under a very specific milieu. Arguably, in order to make further progress in automatic understanding of facial behaviour, datasets that have been captured in in the-wild and in various milieus have to be developed. In this paper, we survey the progress that has been recently made on understanding facial behaviour in-the-wild, the datasets that have been developed so far and the methodologies that have been developed, paying particular attention to deep learning techniques for the task. Finally, we make a significant step further and propose a new comprehensive benchmark for training methodologies, as well as assessing the performance of facial affect/behaviour analysis/ understanding in-the-wild. To the best of our knowledge, this is the first time that such a benchmark for valence and arousal "in-the-wild" is presented

Item Type: Conference or Workshop Item (Paper)
Keywords (uncontrolled): Benchmark testing; Databases;Estimation; Face; Face recognition; Lighting; Videos
Research Areas: A. > School of Science and Technology > Computer Science
Item ID: 20030
Notes on copyright: © 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Useful Links:
Depositing User: Irene Kotsia
Date Deposited: 15 Jun 2016 10:10
Last Modified: 29 Nov 2022 21:46
URI: https://eprints.mdx.ac.uk/id/eprint/20030

Actions (login required)

View Item View Item


Activity Overview
6 month trend
6 month trend

Additional statistics are available via IRStats2.