Conversational Agents (2017 - present)

Conversational agents (CAs) are systems that have the capability of natural language discourse. Chatbots can be considered the most basic form, and commonly communicate through a simple text interface. Virtually-embodied CAs have a simulated virtual character resembling a human or other form, ranging from cartoon-like to highly detailed. And then there’s physically-embodied CAs -- entities that have a physical presence in the world, so as to connect with their environment, including both voice-based agents that communicate through a speaker, as well as robots.

Example of chat with chatbot

Humorous Teachable Chatbot

Previous studies have highlighted the benefits of pedagogical CAs using socially-oriented conversation with students. In this work, we examined the effects of a chatbot's use of affiliative and self-defeating humour on learners' perception of the agent and attitudes towards the task. [DOI]

Student and Robot playing game

A Curious Robot

As curiosity has been found to enhance learning, memory, and exploration, understanding how to elicit curiosity can inform the design of educational technologies. In this work, we investigate how a social peer robot's verbal expression of curiosity is perceived, whether it can affect emotional and/or behavioural curiosity contagion in students, and how it impacts learning. [DOI]

Diagram of Activity Theory

Social Robots for In-Class Learning Activities

With advances in AI, research is increasingly exploring the potential functions that social robots can play in education. As teachers are a critical stakeholder in the use and application of educational technologies, we conducted a study to understand teachers' perspectives on how a social robot could support a variety of learning activities in the classroom. [Work is under review].

Image of Youtube video

Voice-based Agents

Designed, built, and deployed an Amazon Alexa Skill Alien Invasion, with two colleagues: Nalin Chhibber and Sangho Suh, as part of the Alexa Skills Challenge: Kids, a competition to publish an Alexa skill for children under 13.

Neural Correlates of Curiosity (2017 - present)

Diagram of task used in study

Understanding Curiosity

Understanding the neurophysiological mechanisms underlying curiosity and therefore being able to identify the curiosity level of a person, would provide useful information for researchers and designers in numerous fields such as neuroscience, psychology, and computer science. A first step to uncovering the neural correlates of curiosity is to collect neurophysiological signals during states of curiosity, in order to develop signal processing and machine learning (ML) tools to recognize the curious states from the non-curious ones. In collaboration with colleagues at Inria Bordeaux (Lead: Aurelien Appriou). [DOI]

Voice Perception - Cochlear Implants (2014-2016)

Cochlear implants (CI) restore hearing in individuals with profound to severe hearing loss via electric stimulation of the auditory nerve. Unfortunately, modern CIs suffer from degraded fine spectrotemporal resolution. As a result, CI users have difficulty perceiving important voice characteristics for higher level speech perception, specifically, the fundamental frequency (F0) and the vocal-tract length (VTL).

spectrograms of 4, 8, and 16 bands in cochlear implant

Bimodal Hearing

Research has shown that CI users with residual low-frequency hearing greatly benefit from a hearing aid in the non-implanted ear. This is known as bimodal hearing and provides electric stimulation via the CI, in addition to acoustic amplification for the low-frequency sounds. In implant users with low-frequency residual acoustic hearing, this bimodal electro-acoustic stimulation may provide additional voice-related cues, such as low-numbered harmonics and formants, which could improve F0/VTL perception. In acoustic noise-vocoder simulations, where added low-pass filtered speech simulated residual hearing, a strong bimodal benefit was observed for F0 perception. No bimodal benefit was observed for VTL, which seems to mainly rely on vocoder spectral resolution. [DOI]

Example of neurofeedback setup

Neurofeedback (Brain-Computer Interfaces)

An electroencephalography (EEG) based neurofeedback system was developed, aimed at improving VTL perception using CI simulations. The results indicated a necessity for further research into event-related potentials (ERPs), specifically the P300 waveform, elicited by non-word, vocoded speech stimuli. [Part of Master's thesis].

Cognitive and User Modelling (2015)

EEG Plot

Theta Oscillation Phase-Locking

EEG study on theta oscillation phase-locking after attentional blink (AB) training. The AB is a phenomenon which demonstrates the attention limitations of human visual processing. The cause of the AB is thought to be due to a reduction in processing capacity once the first target has been perceived, preventing the processing and conscious perception of a second target.

Graphs of ACTR output

ACT-R Cognitive Architecture

How people make time estimations has been heavily researched over the years. This work provides a cognitive model based on analyses of data from a time estimation task. The model focusses on the role feedback plays on subsequent estimations and suggests ways in which memory traces are formed and retrieved.

Image showing interruption manaagement system

Interruption Management System

With multitasking comes interruptions and there is a growing literature on the detrimental effects interruptions have on performance. When best to interrupt a user has become a major area of research in the field of HCI. We developed and tested a system which interrupted users based on their cognitive workload (measured by pupil dilation), and measured their performance by resumption lag.

Human-Robot Interaction (2015-2016)

Picture showing robot waving experiment

The Search For A Waving Person

With the use of a Kinect sensor and ROS modules a behaviour was built which searched for a waving person in a room by rotating on the spot by small intervals. Once the person was found the robot navigated to him or her and stopped directly in front at a face-to-face angle. Experiments were conducted to measure the accuracy of the behaviour.

3d printed robot hand

EMG-Powered Robotic Hand

Project for Science LinX, an organization which aims to familiarize high school students with science through interactive exhibits and workshops throughout the Netherlands.