Oliver Lemon is Professor of Computer Science at Heriot-Watt University, Edinburgh, where he is also Director of the Interaction Lab, and Academic Co-Lead of the National Robotarium. He also co-founded the Conversational AI spin-out company Alana AI.
His research involves Artificial Intelligence techniques related to novel human-computer interfaces and Human-Robot Interaction (HRI), in particular natural language processing and the use of machine learning methods for optimisation.
Prior to this he was a Senior Research Fellow at the School of Informatics, University of Edinburgh (2003–2009), and a Senior Project Engineer at the Center for the Study of Language and Information (CSLI) at Stanford University (2000–2002).
Oliver will be presenting at the Language Sciences Annual Symposium on 16 November 2023 as part of a research dialogue with Micol Spitale on speech, language and multi-modal technologies in healthcare, including human-robot interactions.
My research sets out to build AI systems and robots that can talk to humans like real people. Future AIs and robots should be able to collaborate effectively with us using natural language conversation. That means both being able to understand human speech and to generate useful and appropriate responses for people, for example when answering questions or assisting in a task.
I’m especially focused on conversational interactions, where people and machines collaborate using sequences of utterances, and where there is a context of interaction, for example a shared task, or an image or video. We often work on medical AI and robotics applications.
I work in both the UK National Robotarium and at our spin-out company Alana AI. Most of my time is working with other scientists to develop new systems and models for human-language processing, collecting data to train the systems, and then evaluating them in different experiments with people.
I’ve been doing this for more than 20 years, so I have several career highlights so far. The first would be at Stanford University, where we created one of the very first systems for humans to interact using conversational speech with robots, back in 2001. This led to a collaboration with researchers at NASA.
Another highlight was collaborating with researchers at Cambridge University (Prof. Steve Young’s group) where we pioneered Reinforcement Learning methods for conversational AI, in a series of EU-funded projects that I coordinated.
More recently a highlight has been competing in the Amazon Alexa Prize for several years, where we deployed our conversational AI systems across the US, reaching the final each time. That real-world experience led to the formation of our spin-out company Alana AI, which is now working on medical applications of conversational AI.
In terms of my latest research, we are working on developing Large Language Models (LLMs) like ChatGPT which are more accurate and trustworthy in collaborative tasks; can be used in visual situations and can talk about images and videos; can handle more than one person in a conversation; and can be used in medical and assistive living settings.
I’m most excited by the immense advances that are happening now due to the invention of LLMs, and extending these to deal with visual situations and shared task collaboration between humans, AIs, and robots.
I hope my research will lead to everyone in the world having access to their own private and personal conversational AI assistant that can support their effectiveness, creativity, critical thinking, and wellbeing.
There are huge opportunities but also many potential pitfalls in the recent development of LLMs and conversational robots, and their application in medical and wellbeing settings. It’s important to me to engage with a variety of perspectives and experiences that are relevant for how we develop such technology in the future.