IPAC Christian WOLF

In this talk I will give a short overview over the automatic learning of deep hierarchical representations including recent advances in this area. I will briefly cover the basic functionality of frequently used models such as convolutional neural networks and traditional applications such as object recognition and video classification.

In the second part of the talk I will address the problem of gesture recognition and pose estimation from videos, presenting two different strategies: (i) estimation of articulated pose (full body or hand pose) alleviates subsequent recognition steps in many conditions and allows smooth interaction modes and tight coupling between object and manipulator; (ii) in situations of low image quality (e.g. large distances between hand and camera), obtaining an articulated pose is hard. Training a deep model directly on video data can give excellent results in these situations.

We tackle both cases by training deep architectures capable of learning discriminative intermediate representations. The main goal is to integrate structural information into the model in order to decrease the dependency on large amounts of training data.

- We propose an approach for hand pose estimation that requires very little labelled data. It leverages both unlabeled data and synthetic data produced by a rendering pipeline. The key to making it work is to integrate structural information into the training objective.

- In the context of multi-modal gesture detection and recognition, we propose a deep recurrent architecture that iteratively learns and integrates discriminative data representations from individual channels (pose, video, audio), modeling complex cross-modality correlations and temporal dependencies.

Site : http://liris.cnrs.fr/christian.wolf/research/gesturerec.html