Nonnegative decompositions for dynamic visual data analysis

Zafeiriou, Lazaros, Panagakis, Yannis ORCID logoORCID:, Pantic, Maja and Zafeiriou, Stefanos (2017) Nonnegative decompositions for dynamic visual data analysis. IEEE Transactions on Image Processing, 26 (12) . pp. 5603-5617. ISSN 1057-7149 [Article] (doi:10.1109/TIP.2017.2735186)


The analysis of high-dimensional, possibly temporally misaligned, and time-varying visual data is a fundamental task in disciplines, such as image, vision, and behavior computing. In this paper, we focus on dynamic facial behavior analysis and in particular on the analysis of facial expressions. Distinct from the previous approaches, where sets of facial landmarks are used for face representation, raw pixel intensities are exploited for: 1) unsupervised analysis of the temporal phases of facial expressions and facial action units (AUs) and 2) temporal alignment of a certain facial behavior displayed by two different persons. To this end, the slow features nonnegative matrix factorization (SFNMF) is proposed in order to learn slow varying parts-based representations of time varying sequences capturing the underlying dynamics of temporal phenomena, such as facial expressions. Moreover, the SFNMF is extended in order to handle two temporally misaligned data sequences depicting the same visual phenomena. To do so, the dynamic time warping is incorporated into the SFNMF, allowing the temporal alignment of the data sets onto the subspace spanned by the estimated nonnegative shared latent features amongst the two visual sequences. Extensive experimental results in two video databases demonstrate the effectiveness of the proposed methods in: 1) unsupervised detection of the temporal phases of posed and spontaneous facial events and 2) temporal alignment of facial expressions, outperforming by a large margin the state-of-the-art methods that they are compared to.

Item Type: Article
Research Areas: A. > School of Science and Technology > Computer Science
Item ID: 23769
Depositing User: Yannis Panagakis
Date Deposited: 06 Mar 2018 15:53
Last Modified: 24 Jan 2023 10:10

Actions (login required)

View Item View Item


Activity Overview
6 month trend
6 month trend

Additional statistics are available via IRStats2.