PEDAGOGICAL CONVERSATIONAL AGENTS (2017 - present)

Student and Robot playing game Verbal Curiosity Expression in a Social Robot. As curiosity has been found to enhance learning, memory, and exploration, understanding how to elicit curiosity can inform the design of educational technologies. In this work, we investigate how a social peer robot’s verbal expression of curiosity is perceived, whether it can affect emotional and/or behavioural curiosity contagion in students, and how it impacts learning. https://doi.org/10.1145/3290605.3300636

Diagram of Activity Theory Participatory Design. Working with teachers to conceptualize possible robot functions and the effects they may have on the current way learning activities are designed, planned, and executed.


Picture of youtube video Voice-based Agents. Designed, built, and deployed an Amazon Alexa Skill Alien Invasion, with two colleagues: Nalin Chhibber and Sangho Suh, as part of the Alexa Skills Challenge: Kids, a competition to publish an Alexa skill for children under 13.

Example of chat with chatbot Teachable Chatbot. Designing and measuring effects of a chatbot’s social behaviours on students.





NEURAL CORRELATES OF CURIOSITY (2017-present)

Diagram of task used in study Understanding Curiosity. Towards measuring states of epistemic curiosity through electroencephalographic (EEG) signals. In collaboration with colleagues at Inria Bordeaux (Lead: Aurelien Appriou). https://hal.inria.fr/hal-02971795




VOICE PERCEPTION - COCHLEAR IMPLANTS (2014-2016)

In discriminating speakers’ voices, normal-hearing individuals effectively use two vocal characteristics, vocal pitch (related to fundamental frequency, F0) and vocal-tract length (VTL, related to speaker size). Typical cochlear-implant users show poor perception of these cues.

spectrograms of 4, 8, and 16 bands in cochlear implant Bimodal Hearing. In implant users with low-frequency residual acoustic hearing, this bimodal electro-acoustic stimulation may provide additional voice-related cues, such as low-numbered harmonics and formants, which could improve F0/VTL perception. In acoustic noise-vocoder simulations, where added low-pass filtered speech simulated residual hearing, a strong bimodal benefit was observed for F0 perception. No bimodal benefit was observed for VTL, which seems to mainly rely on vocoder spectral resolution. https://doi.org/10.1121/1.5034171

example of neurofeedback setup Neurofeedback (Brain-Computer Interface). Preliminary work on developing an electroencephalography (EEG) based neurofeedback system for improving perception of F0 and VTL for cochlear implant users.




COGNITIVE AND USER MODELLING (2015)

eeg plot Theta Oscillation Phase-Locking. EEG study on theta oscillation phase-locking after attentional blink (AB) training. The AB is a phenomenon which demonstrates the attention limitations of human visual processing. The cause of the AB is thought to be due to a reduction in processing capacity once the first target has been perceived, preventing the processing and conscious perception of a second target.

Graphs of ACTR output ACT-R Cognitive Architecture. How people make time estimations has been heavily researched over the years. This work provides a cognitive model based on analyses of data from a time estimation task. The model focusses on the role feedback plays on subsequent estimations and suggests ways in which memory traces are formed and retrieved.

Image showing interruption manaagement system Interruption Management System. With multitasking comes interruptions and there is a growing literature on the detrimental effects interruptions have on performance. When best to interrupt a user has become a major area of research in the field of HCI. We developed and tested a system which interrupted users based on their cognitive workload (measured by pupil dilation), and measured their performance by resumption lag. (Similar work published by colleagues: https://doi.org/10.1080/10447318.2016.1198525)




HUMAN-ROBOT INTERACTION (2015-2016)

Picture showing robot waving experiment The Search For A Waving Person. With the use of a Kinect sensor and ROS modules a behaviour was built which searched for a waving person in a room by rotating on the spot by small intervals. Once the person was found the robot navigated to him or her and stopped directly in front at a face-to-face angle. Experiments were conducted to measure the accuracy of the behaviour.

3d printed robot hand EMG-Powered Robotic Hand. Project for Science LinX, an organization which aims to familiarize high school students with science through interactive exhibits and workshops throughout the Netherlands.