University of Maryland researchers fuse facial expressions, EEG signals, and language model outputs with transformer architectures for low-latency, multimodal emotion recognition in human–robot interaction, advancing empathetic robotics.

Key points

• Multimodal fusion of facial expression, EEG neurophysiological signals, and LLM-based language embeddings using transformer architectures.
• On-device, real-time emotion inference optimized through model compression techniques for low-power hardware like microcontrollers and mobile GPUs.
• Portable EEG-based detection of P300 neural signatures for concealed information measurement with personalized calibration protocols.

Why it matters: Equipping robots with real-time emotional intelligence transforms human–robot collaboration by enabling adaptive, empathetic interactions beyond conventional automation.