Research on acoustic technologies and processing concepts to activate and control devices by gesture

Abstract

Human interaction systems (HSI) are evolving to mimic human communication. This implies that these systems are moving from being based on single input commands (such as key presses, voice commands, gestures) to a multi-command interpretation. In other words, interaction will become a multimodal system, capable not only of recognizing what the user is giving as input, but also of modifying the meaning of this input depending on the context. This implies the need to introduce new technologies in the field of device control to increase the information collected from the user. This doctoral thesis is the result of the investigation of acoustic technologies, specifically ultrasound, for the activation and control of devices through gestures. These results are presented in the form of a compendium of publications, compiling the scientific articles published during the doctoral period. The research has been carried out in parallel with the development of an innovative ultrasonic transducer. The objective has been to study the feasibility of using this sensor as a unique input method for gesture recognition systems. The possibility of using systems with limited resources, such as Edge devices, as devices for the acquisition and processing of the signal obtained by the ultrasonic sensor has been evaluated and demonstrated. Subsequently, algorithms for object positioning (e.g. a hand) based on time-of-flight (ToF) have been investigated and generalized to more than one sensor. Finally, an airborne writing system has been developed, capable of obtaining and recognizing a series of alphanumeric characters made by the user in the air. This system can be generalized to other types of trajectories or gestures. The research has been carried out under a doctoral contract at the facilities of Infineon Technologies AG, at its headquarters in Munich, Germany.