RF-HOI: Recognize Human-Object Interaction with Radio Frequency Signals

Abstract

RF-HOI is the first framework that recognizes Human-Object Interactions (HOI) using only radio frequency signals. It fuses mmWave radar and RFID for simultaneous action recognition and target object identification, and uses a physics-based simulator to synthesize multimodal RF data at scale, approaching vision-model performance while preserving privacy.

Publication
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies