Robotics perception

Shared autonomy for safe assistive manipulation

Robotic arms that combine brain, muscle, and eye-tracking signals to decide when to act on a user's intent.

Shared autonomy for safe assistive manipulation

Assistive robots must act on what a person intends — and only when that intent is clear. With collaborators in Aerospace and Mechanical Engineering and in Computer Science, AIRLab contributes perception and learning to shared-autonomy systems that fuse EEG, EMG, and eye-tracking signals to decide when a user is committed to an action, and to transformer-based models that recognize motor intent from muscle signals.

In the NeuroCommitSSM study, synchronized EEG, EMG, and eye-tracking signals produce a continuous commit-readiness score, and a ROS 2 and computer-vision supervisor checks that each action is feasible before a Kinova Gen3 arm executes it, across five assistive daily-living tasks such as reaching for a clock, retrieving a dropped bottle, and toggling a fan.

Five assistive tasks compared across shared-autonomy strategies

Publications

T. Sultan, P. Sangani, K. Cool, P. Sikorski, G. Liu, H. Ali Akbarpour, and M. Babaiasl. NeuroCommitSSM: Decision-Centric Shared Autonomy for Safe Assistive Manipulation via EEG–EMG–ET Commit Readiness. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2026 (accepted).

J. Aikkarakudiyil Joby, P. Sikorski, T. Sultan, H. Ali Akbarpour, F. Esposito, and M. Babaiasl. EMG-TransNN-MHA: A Transformer-Based Model for Enhanced Motor Intent Recognition in Assistive Robotics. IEEE International Conference on Big Data, 2024. Paper

Collaborators

  • Dr. Madi Babaiasl (Aerospace and Mechanical Engineering
  • SLU)
  • Dr. Flavio Esposito (Computer Science
  • SLU)