PUBLICATIONS

“Telekolympographic” Swim Gesture Recognition for Teleoperation of Remote or Virtual Boat

IEEE - ICMA 2026

Paper ID: 633621

Paper Title: Swim Gesture Recognition for Teleoperation of Remote or Virtual Boat

Authors: Ibrahim Hassan, Varun Seshadri, Li-An Yang, Yuxuan Zhang, Alexander Vicol, Steve Mann

Accepted on 14 May, 2026

Notinated for Best Conference Paper Award

https://ieeexplore.ieee.org/document/11647508

I. Hassan, V. Seshadri, L. -A. Yang, Y. Zhang, A. Vicol and S. Mann, "Telekolympographic Swim Gesture Recognition for Teleoperation of Remote or Virtual Boat," 2026 IEEE International Conference on Mechatronics and Automation (ICMA), Changchun, Jilin, China, 2026, pp. 572-577, doi: 10.1109/ICMA69663.2026.11647508.Abstract: This paper presents a Telekolympographic User Interface framework for remote or virtual vessel control through real-time swim gesture recognition on dry land. A head-mounted camera captures upper-body arm motion, which is processed by a computer vision pipeline that segments the user’s arms, extracts geometric contours, and converts motion into steering and propulsion commands. The resulting interaction paradigm treats swimming gestures not merely as body movements but as the primary control input of the system. To evaluate the approach, we applied the same gesture-recognition pipeline to both virtual and remote-physical domains. In the virtual domain, a Unity-based simulator incorporates a 3D reconstruction of Peter Street Basin, a safety pool that is part of Lake Ontario in the city of Toronto, allowing users to navigate a virtual vessel using upper-body swim gestures. In the physical domain, the same control logic is transmitted to a remote-controlled racing boat over a low-latency wireless link. Together, these implementations demonstrate that vision-based aquatic interaction can provide an intuitive Natural User Interface (NUI) alternative to conventional handheld controllers in environments where traditional interfaces are awkward or impractical. keywords: {Printing;Equations;Sports;Arm;Conferences;Visual systems;Tagging;Boats;Cameras;Computers;Natural User Interface;Mersivity;WaterHCI;Computer Vision;Gesture Recognition;Extended Reality;Virtual Reality;Ship of Kolympi;Telekolympographic User Interface;Kolympography;Kolympographic User Interface;Ankolympography},URL: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=11647508&isnumber=11647333