Towards Robust and Natural Underwater Human-Robot Interaction

The underwater domain takes up almost four-fifths of the planet and is inherently hostile towards human exploration. However, in numerous applications in the marine environment (e.g., in surveillance, environmental monitoring, security, and search-and-rescue), human assessment is necessary for efficient and effective task completion. Current technology for underwater exploration sees limited applications of autonomous underwater vehicles (AUVs) but relies heavily on remotely operated vehicles, which unfortunately does not take advantage of robot autonomy. This project will develop novel algorithms and protocols to enable humans to communicate safely with AUVs while preserving and leveraging their autonomy. Specifically, the intent is to create novel methods for gesture- and motion-based bidirectional human-robot communication methods and enable autonomous underwater robots to detect, identify and interact with specific individuals. The research objectives will be evaluated individually and as an integrated, coherent system onboard underwater vehicles. The proposed research has the potential to create a fundamentally new direction in human-in-the-loop field robotics, with underwater robot companions being able to assist divers in a range of tasks and even learning to carry out these tasks in an autonomous manner, greatly reducing risk to humans. This research will also impact a broad range of disciplines, including human-machine dialog, machine vision, activity recognition, and robot control.

This research will develop novel algorithms and protocols to enable humans to communicate safely with AUVs while preserving and leveraging their autonomy. Specific goals include: development of a gesture-based human-to-robot language with multiple communication granularities; creation of algorithms for visual identification of humans by learning from spatial and periodic cues; and development of a non-verbal, motion-based underwater robot-to-human communication scheme. The research objectives will be evaluated individually and as an integrated, coherent system onboard underwater vehicles. The investigation into gesture-based visual languages will quantify the detectability, usability, and efficacy of such methods in realistic settings. Statistical, convolutional and generative learning-based approaches will be applied to understand both identifiable human features and gestural communication. Additionally, robot body language and motion will be used as cues for robot-to-human non-verbal communication. These methods will be quantitatively and qualitatively validated via user studies and robot field trials to characterize their advantages and limitations.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.