Human activity understanding for robot-assisted living

This thesis investigated the problem of understanding human activities, at different levels of granularity and taking into account both the variability in activities and annotator disagreement.
To be able to capture the large variations within each of the action classes, we propose a model that uses a sequence of latent variables which can be implicitly learned without human annotation. We also propose a model that incorporates the uncertainty of labeling, and introduce learning methods that can use the noisy labels for finding the optimal model parameters. The above two models both focus on recognizing actions, not activities. Therefore, thirdly, we propose to use a hierarchical model that can encode the activity labels with different complexity. In this model, the activity and action nodes are interconnected. The results show that by using joint modeling of actions and activities, the F-score is increased by 3 percentage points compared with the state-of-the-art. Fourthly, we propose a novel algorithm that can predict upcoming actions. This is particularly important when the robot needs to give pro-active assistance before an actual action is complete. We incorporate a human kinematic model for estimating the cost of reaching objects, and these costs are used as features for predicting future actions. The results show an improvement of 3 percentage points in F-score compared with the state-of-the-art. Finally, we propose to use overhead cameras for detecting and tracking persons in the room, which can be used to efficiently approach the person for tasks such as person identification, activity recognition, and intent prediction.