Augmenting Human Cognition with Collaborative Robots

AugMEnted Learning InnovAtion (AMELIA)

Collaborative robotics is a growing application space in robot technology used in manufacturing, mining, construction, and energy industrial settings. This convergence research project will contribute new knowledge and theory of Human-Computer Interaction and Human-Robot Interaction, by augmenting human cognition for safer and more efficient collaborative robot interaction.

To meet these goals, the team of researchers planned to:

  • develop a novel HRI task/scenario classification scheme in collaborative robotics environments vulnerable to observable systems failures
  • establish fundamental neurophysiological, cognitive, and socio-behavioral capability models (e.g., workload, cognitive load, fatigue/stress, affect, and trust) during these HRI (i.e., the mind motor machine nexus)
  • use these models to determine when and how a human’s cognitive, social, behavioral and environmental states require adjustment via technology to enhance HRI for efficient and safe work performance; and finally
  • create an innovative and transformative Work 4.0 architecture (AMELIA: AugMEnted Learning InnovAtion) that includes a layer of augmented reality for human and robots to mutually learn and communicate current states.

The team proposed a novel communication scheme using artificial emotional intelligence in which robots and humans collaborate in potentially dangerous situations. The robot will detect the worker’s “cognitive state” using different machine learning techniques, and then take the appropriate action. Ultimately AMELIA seeks to empower the worker to focus on complex, cognitive problem-solving tasks, performed safely and efficiently, while ensuring that it adapts to both the worker's attitudes and cognitive states.

Edge Services

We employed MicroLambda, an edge-based framework, to develop real-time cognitive workload prediction and speed reduction interventions. Real-time tasks need high-speed and responsive services, which are very challenging due to the higher task processing and communication time. Edge computing is a technology where the edge servers are deployed very near to the user.

Publications

Demo Video: