MA: Machine Learning-Enabled Radar-Based Human Pose Estimation Using a Raytracing-Enhanced Data Set
Task description:
Integration of smart home applications is becoming increasingly prevalent in the daily routines of individuals. While audio and RGB camera systems have already paved the way for voice and gesture control in certain households, concerns regarding privacy intrusion persist, particularly with the use of visual sensor technology. In this context, radar sensors offer a distinct advantage by minimizing privacy concerns, as they gather data that does not contain any directly identifiable personal references. Radar sensors offer significant potential for both private residences and public buildings for task relating to human behavior while maintaining stringent data protection standards. They can be used, for instance, to detect people in the building, their pose, general movements including gestures and even to measure vital parameters.
This thesis utilizes raytracing software to synthesize raw radar data, enhancing the dataset for machine learning-based pose estimation. The process involves deriving a human base-mesh model from optical motion capture (OMC) data, adapting its trajectory in the room and parameters within the environment, and generating raw radar data via raytracing software. Machine learning (ML) techniques are then employed to estimate the pose of the human body.
To achieve this goal, a thorough review of the literature is imperative to grasp the state-of-the-art ML techniques tailored for extracting the position from radar data. Measurements are then conducted to acquire a foundational dataset for ML training. Subsequently, various ML approaches are developed and accurately evaluated. Concurrently, additional training data is generated by fitting a human mesh model onto the measured OMC data and radar data is extracted by raytracing software. Finally, the accuracy of the joint positions is compared with and without augmented radar data, providing valuable insights into the efficacy of this novel approach.
Requirements: Profound python knowledge, Pre-knowledge in radar signal processing, Pre-knowledge in Blender
Supervisors: Prof. Dr.-Ing. Martin Vossiek, Lukas Engel, M.Sc.
Date of issue: April 2024
Language: German or English