Conversation aid for people with low vision using head mounted display and computer vision emotion detection
People with central vision loss may become unable to see facial expressions during face-to-face conversations. Nuances in interpersonal communication conveyed by those expressions can be captured by computer vision systems and conveyed via an alternative input to the user. We present a low-cost system using commodity hardware that serves as a communication aid for people with central vision loss. Our system uses a smartphone in a head-mounted-display to capture images of a conversation partner’s face, analyze the face expressions, and display a detected emotion in the corner of the display where the user retains vision senses. In contrast to purpose-built devices that are expensive, this system could be made available to a wider group of potential users, including those in developing countries.
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
accessibility, computer vision, head-mounted-display, low-vision
Zuniga, Rafael and Magee, John, "Conversation aid for people with low vision using head mounted display and computer vision emotion detection" (2018). Computer Science. 22.
Zuniga, R., & Magee, J. (2018). Conversation aid for people with low vision using head mounted display and computer vision emotion detection. In Computers Helping People with Special Needs: 16th International Conference, ICCHP 2018, Linz, Austria, July 11-13, 2018, Proceedings, Part II 16 (pp. 44-50). Springer International Publishing.