Single-frame 3D human pose recovery from multiple views

We present a system for the estimation of unconstrained 3D human upper body pose from multi-camera single-frame views. Pose recovery starts with a shape detection stage where candidate poses are generated based on hierarchical exemplar matching in the individual camera views. The hierarchy used in this stage is created using a hybrid clustering approach in order to efficiently deal with the large number of represented poses. In the following multi-view verification stage, poses are re-projected to the other camera views and ranked according to a multi-view matching score. A subsequent gradient-based local pose optimization stage bridges the gap between the used discrete pose exemplars and the underlying continuous parameter space. We demonstrate that the proposed clustering approach greatly outperforms state-of-the-art bottom-up clustering in parameter space and present a detailed experimental evaluation of the complete system on a large data set.