Shelby Hall Graduate Research Forum Posters

Files

Download Available for download on Sunday, April 28, 2030

Download Full Text (1.2 MB)

Description

Edge AI models for real-time cognitive engagement detection will have major effects on clinical neuroscience, human-computer interface, and education. This work provides a lightweight deep learning model for frame-by- frame video classification and develops an edge device. The proposed edge device delivers low latency, energy-efficient processing, and high temporal resolution, achieving 29 frames per second (FPS) with each frame taking 30 milliseconds, all without reliance on cloud services. Our investigation also introduces a Multimodal Fusion Framework designed to integrate physiological signals and behavioral data into the engagement classification process, in contrast to conventional systems that rely solely on video-based categorization. Using synchronized EEG, ECG, and face video data from 17 student volunteers engaged in controlled cognitive activities, the study examined. While EEG alpha and beta frequency bands gave important new perspectives on cognitive workload, ECG-derived heart rate variability (HRV) acted as an indirect predictor of cognitive stress. Reaction times and task accuracy among behavioral measures provided outside confirmation of cognitive states. This work investigates the strengths and limits of each modality by means of independent and integrated data streams, therefore stressing the possibilities of improving classification accuracy and contextual knowledge of engagement levels. In contrast to conventional systems that mostly rely on video-based categorization, this study provides a Multimodal Fusion Framework that integrates behavioral and physiological information into the engagement classification process. Without requiring cloud infrastructure, the proposed edge device demonstrates low latency, energy-efficient computation, and excellent temporal resolution.

Publication Date

3-2025

Department

Electrical & Computer Engineering

City

Mobile

Disciplines

Bioelectrical and Neuroengineering | Biomedical | Digital Communications and Networking | Electrical and Electronics | Systems and Communications | Systems and Integrative Engineering | VLSI and Circuits, Embedded and Hardware Systems

Edge AI-Based Cognitive Engagement Detection and Multimodal Exploration

Share

COinS