From motion to human activity recognition

Alhersh, Taha

PDF
Alhersh_2021_Approved_to_be_submitted_and_printed.pdf - Veröffentlichte Version
Download (8MB)

URL:	https://madoc.bib.uni-mannheim.de/59204
URN:	urn:nbn:de:bsz:180-madoc-592041
Dokumenttyp:	Dissertation
Erscheinungsjahr:	2021
Ort der Veröffentlichung:	Mannheim
Hochschule:	Universität Mannheim
Gutachter:	Stuckenschmidt, Heiner
Datum der mündl. Prüfung:	26 März 2021
Sprache der Veröffentlichung:	Englisch
Einrichtung:	Fakultät für Wirtschaftsinformatik und Wirtschaftsmathematik > Practical Computer Science II: Artificial Intelligence (Stuckenschmidt 2009-)
Fachgebiet:	004 Informatik
Freie Schlagwörter (Englisch):	computer vision , motion , optical flow , activity recognition , metrics
Abstract:	Advancements in wearable technology have the potential to transform the quality of life, business, and the global economy. Body sensors can be used in human activity recognition, which has direct impact on various application domains such as surveillance systems, healthcare systems, robotics, and other physical and metrological applications. Human activity recognition can be considered as a problem in both computer vision and pervasive computing. In this research, we started from a computer vision problem based on optical flow, and then introduced open issues in respect to existing techniques. We went on to study the relation between optical flow and human activity recognition, taking into consideration the effectiveness of using optical flow combined with other wearable sensors. As a result, feasible solutions have been presented to solve those problems, and then useful insights have been given for implementing corresponding techniques. A comprehensive set of experiments and discussions were performed during the research. Firstly, we suggested an unsupervised optical flow fine-tuning that overcomes the need for a ground truth for training on the one hand and enhanced motion boundaries on the other. Secondly, we provided theoretical justification for optical flow evaluation metrics. Moreover, we suggested novel optical flow performance metrics that have been evaluated alongside current metrics. Our empirical findings examined the performance of all metrics with regard to their sensitivity to change in motion between estimated optical flow and the ground truth. Finally, we investigated methods regarding feature extraction for Inertial Measurement Units (IMUs) and visual data captured from wearable sensors, for instance, statistical features, local visual descriptors, and features extracted from deep learning. The features generated were tested for human activity recognition using Support Vector Machines and Recurrent Neural Network as the main recognition methods.