Disentangling the modes of variation in unlabelled data

Wang, Mengjiao, Panagakis, Yannis ORCID: https://orcid.org/0000-0003-0153-5210, Snape, Patrick and Zafeiriou, Stefanos P. (2018) Disentangling the modes of variation in unlabelled data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40 (11) . pp. 2682-2695. ISSN 0162-8828 [Article] (doi:10.1109/TPAMI.2017.2783940)

Preview

PDF - Final accepted version (with author's formatting) Download (14MB) | Preview

Abstract

Statistical methods are of paramount importance in discovering the modes of variation in visual data. The Principal Component Analysis (PCA) is probably the most prominent method for extracting a single mode of variation in the data. However, in practice, visual data exhibit several modes of variations. For instance, the appearance of faces varies in identity, expression, pose etc. To extract these modes of variations from visual data, several supervised methods, such as the TensorFaces relying on multilinear (tensor) decomposition (e.g., Higher Order SVD) have been developed. The main drawbacks of such methods is that they require both labels regarding the modes of variations and the same number of samples under all modes of variations (e.g., the same face under different expressions, poses etc.). Therefore, their applicability is limited to well-organised data, usually captured in well-controlled conditions. In this paper, we propose a novel general multilinear matrix decomposition method that discovers the multilinear structure of possibly incomplete sets of visual data in unsupervised setting (i.e., without the presence of labels). We also propose extensions of the method with sparsity and low-rank constraints in order to handle noisy data, captured in unconstrained conditions. Besides that, a graph-regularised variant of the method is also developed in order to exploit available geometric or label information for some modes of variations. We demonstrate the applicability of the proposed method in several computer vision tasks, including Shape from Shading (SfS) (in the wild and with occlusion removal), expression transfer, and estimation of surface normals from images captured in the wild.

Actions (login required)

View Item

Statistics

DownloadsShow export options

Export as CSVXMLJSON

Activity Overview

6 month trend

394Downloads

6 month trend

269Hits

Additional statistics are available via IRStats2.