Robust expert systems for more flexible real-world activity recognition

Abstract

(Abstract) During the last years a tremendous interest in the analysis of human behavior has emerged to better understand and meet people's needs and demands. One of the most relevant research areas that investigate people behavior is human activity recognition. Human activity recognition aims at identifying human conducts in an autonomous fashion from the observation of a person's actions and their interaction with the surroundings. The flourishing of activity recognition is having a great impact in society and is extremely valuable in a wide variety of fields such as wellness, healthcare, sports or gaming. The use of wearable or on-body sensors to monitor the human behavior is now on the forefront of human activity recognition. Nevertheless, the actual results for human activity recognition are fairly constrained and generally restricted to ideal or laboratory scenarios. Activity recognition systems are designed to comply with ideal conditions and are of limited utility in realistic domains. To become real-world applicable, activity recognition systems must satisfy operational and quality requirements that pose complex challenges, most of which have been sparsely and vaguely investigated to date. Classic activity recognition systems assume that the sensor setup remains identical during the lifelong use of the system. However, in users' daily life, sensors may fail, run out of battery, be misplaced or experience topological variations. These changes may lead to significant variations in the sensor measurements with respect to the default case. Consequently, activity recognition systems devised for ideal conditions may react in an undesired manner to imperfect, unknown or anomalous sensor data. This potentially translates into a partial or total malfunctioning of the activity recognition system. In this thesis, novel expert systems are proposed to address the challenges of making activity recognition systems functional in real-world scenarios. A novel methodology, the hierarchical weighted classifier, that leverages the potential of multi-sensor configurations, is defined to overcome the effects of sensor failures and faults. This approach proves to be as valid as other standard activity recognition models in ideal conditions while outperforming them in terms of robustness to sensor failure and fault-tolerance. This methodology also shows outstanding capabilities to assimilate sensor deployment anomalies motivated by the user self-placement of the sensors. Furthermore, an novel multimodal transfer learning method that operates at runtime, with low overhead and without user or system designer intervention is developed. This approach serves to automatically translate activity recognition capabilities from an existing system to an untrained system even for different sensor modalities. This is of key interest to support sensor replacements as part of equipment maintenance, sensor additions in system upgrades and to benefit from sensors that happen to be available in the user environment. The potential of these advanced expert models leads to new research directions such as autonomous systems self-configuration, auto-adaptation and evolvability in activity recognition. Thus, this thesis opens-up a new range of opportunities for activity recognition systems to operate in real-world scenarios.