FACE READERS: The Frontier of Computer Vision and Math Learning

Authors

Beverly Woolf, University of Massachusetts Amherst
Margrit Betke, Boston University
Hao Yu, Boston University
Sarah Adel Bargal, Georgetown University
Ivan Arroyo, University of Massachusetts Amherst
John J. Magee IV, Clark UniversityFollow
Danielle Allessio, University of Massachusetts Amherst
William Rebelsky, University of Massachusetts Amherst

Document Type

Conference Paper

Abstract

The future of AI-assisted individualized learning includes computer vision to inform intelligent tutors and teachers about student affect, motivation and performance. Facial expression recognition is essential in recognizing subtle differences when students ask for hints or fail to solve problems. Facial features and classification labels enable intelligent tutors to predict students’ performance and recommend activities. Videos can capture students’ faces and model their effort and progress; machine learning classifiers can support intelligent tutors to provide interventions. One goal of this research is to support deep dives by teachers to identify students’ individual needs through facial expression and to provide immediate feedback. Another goal is to develop data-directed education to gauge students’ pre-existing knowledge and analyze real-time data that will engage both teachers and students in more individualized and precision teaching and learning. This paper identifies three phases in the process of recognizing and predicting student progress based on analyzing facial features: Phase I: Collecting datasets and identifying salient labels for facial features and student attention/engagement; Phase II: Building and training deep learning models of facial features; and Phase III: Predicting student problem-solving outcome. © 2023 Copyright for this paper by its authors.

Publication Title

24th International Conference on Artificial Intelligence in Education Workshop

Publication Date

7-3-2023

ISSN

1613-0073

Keywords

artificial intelligence, AI, facial expression recognition, intelligent tutors, machine learning

Repository Citation

Woolf, Beverly; Betke, Margrit; Yu, Hao; Bargal, Sarah Adel; Arroyo, Ivan; Magee, John J. IV; Allessio, Danielle; and Rebelsky, William, "FACE READERS: The Frontier of Computer Vision and Math Learning" (2023). Computer Science. 6.
https://commons.clarku.edu/faculty_computer_sciences/6

APA Citation

Woolf, B., Betke, M., Yu, H., Bargal, S. A., Arroyo, I., Magee IV, J. J., ... & Rebelsky, W. (2023). FACE READERS: The Frontier of Computer Vision and Math Learning.

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright

© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0)