Activity Recognition

Recently, commercial systems have become popular that utilize a broad range of sensors to facilitate gesture and motion-based interaction.Examples range from multi-touch surfaces, through tilt control common in mobile phone applications, and complex motion-based games controllers, e.g. Nintendo(R) WII(TM) and Microsoft Kinect™. While these systems are mainstream, the next basic research challenge is activity-driven, implicit interaction. Two key differences to existing systems are: (1) the interpretation of complex human activities, and (2) the extension of interaction from periods where a user consciously performs control gestures to permanent monitoring of user activity. Conceptually, activity-driven interaction builds on the vision of context awareness developed since the 1990. Applications range from sports, through mobile gaming, information retrieval, personal healthcare to industrial work support. For example, monitoring certain activities can support therapy in areas ranging from cardiovascular diseases to psychiatric disorders and cognitive disabilities. Activity based support (automatically showing correct manual pages, pointing out mistakes) can speed up industrial maintenance tasks by up to 30%.

Despite demonstrated potential, currently only very simple activity based applications such as physical activity monitoring have managed to go beyond early stage lab demonstrations. From the scientific point of view the question is how to map information available from unreliable, often simple sources onto complex human activities. The main challenges stem from the combination of three factors:

In every day situations sensor choice, placement and configuration is often dictated by practicability, usability, and user acceptance constraints rather by the requirements of the recognition system. In addition, the system configuration may dynamically change. The diversity of human behavior. Even the simplest activities can be performed in a multitude of ways differing not only between people, but also between individual execution instances of a single person. (e.g. using different arms, different hand positions, or even the hip to close a drawer) The complexity of human behavior. Relevant human actions are seldom atomic and independent. Instead, a complex hierarchy of actions that may be executed in parallel, overlap and interleave is to be considered by the recognition system. Beyond the technological challenges involved in the recognition system, there are additional unsolved problems including application design, usability, user acceptance, and business models for commercialization. The field lacks also definitions for many commonly used terms including “action,” “sensor,” “evidence,” and even “activity” itself, leading to ambiguity in scientific discourse. The conceptual grounding provided by Nardi and Kaptelinin’s definition of Activity Theory (2006) are perfectly understandable to a human. Yet, they are not easily codified into machine programmable constructs. The theory recognizes that elements of “activity” such as goal and motive are socially constructed, depending on the perspectives of the actors in the system. Despite the complexities of “activity” at the human cognitive level, researchers demonstrated that some notions of activity can be utilized in computer systems, but meanings of terms differ among the various research groups. Currently, many different communities are involved in research related to activity recognition, including the core ubicomp community, human computer interaction, computer vision, cognitive science and artificial intelligence. Privacy concerns are a critical barrier to adoption of activity-based technologies. These concerns range from risk of criminal activities( e.g., stalking and identity theft), to social issues of managing personal relationships. Technological approaches to addressing the concerns must also be based on deep understanding of the psychological, sociological and political constraints under which people will operate activity-based systems.