You have used advanced brain imaging techniques to come up with a model of how the brain processes vision – and it says that instead of just sorting through what we see, our brains actually anticipate what we will see next. Could you tell us a bit more?
‘We are interested to understand how the brain supports vision. A classical view had been that the brain is responding to visual information in a cascade of hierarchical visual areas with increasing complexity, but a more modern way is to realise that, actually, the brain is not meeting every situation with a clean sheet, but with lots of predictions.’
How does that work?
‘The main purpose of the brain, as we understand it today, is it is basically a prediction machine that is optimising its own predictions of the environment it is navigating through. So, vision starts with an expectation of what is around the corner. Once you turn around the corner, you are then negotiating potential inputs to your predictions – and then responding differently to surprise and to fulfilment of expectations.
‘So that's what’s called the predictive processing framework, and it's a proposed unifying theory of the brain. It's basically creating an internal model of what's going to happen next.’
'Vision starts with an expectation of what is around the corner.'
Lars Muckli, Centre for Cognitive Neuroimaging, Glasgow, Scotland
Why does this happen?
‘First of all, the outside world is not in our brain so somehow we need to get something into our brain that is a useful description of what's happening – and that's a challenge.
‘We become painfully aware of this challenge if we try to simulate this in a computer model – how do we get information about the outside world into a computer model? The brain does that in an unsupervised way. It segments the visual input into object, background, foreground, context, people and so on, and no one ever gives the brain any kind of supervision to do so.
‘To have meaningful models of the world, you need to have something like a supervisor in your brain that says: “This is Object A. This is another object, and you need to find a name for this.” We don't have a supervisor, but we have something – and that's the currency of surprise. (The need) to minimise surprise is used as a supervisor.’
How does surprise help us form models?
‘You can pick up an object, drop it, and you have a very good model of how that should sound (and) where it should stop. If it would suddenly not do that, you need to update your internal models. Those would be very rare situations.’
How does this work in our everyday experience – when we’re cycling on the street, for example?
‘You have forward models, so while you're cycling, you predict the trajectory of the cars, of your own movement on the entire world, in real time. You update your predictions (of) the future model that you create in order to cycle through the city without being run over.
‘These models are very good because you have this experience, and only now and then you need to update these models, or you update them (in real time) because you're turning a corner. So you're updating with your memory, your predictions, and a slight slip of the internal model comes about because you're surprised: “Oh it's not this street.” So, while you're cycling, you're negotiating a future model with another future model because you're updating these creations of your predictions.’
You’ve been working with the Human Brain Project (HBP), which is building a massive data and computing network to study the brain. How does it link to your work?
‘The Human Brain Project brings together different disciplines from computer science, neuroscience, computational neuroscience, robotics and medicine trying to answer a very simple question: “How does the brain work?”
‘It’s interesting because you need to become explicit about the model that each scientist has and use this on different scales and different species from mice to humans, from computational models to neuroscientific experiments and converge to a more general understanding. The predictive coding framework is something that is (being) tested by several teams within the HBP.
‘(To use a metaphor), if you want to build a large airplane (like an) Airbus you have different pockets in which you design the wheels, the aerodynamics of the wings and sub-parts and sub-materials. It is the vision of combining which drives the different groups, and that’s important because you work on small models, but having the idea of combining it drives the innovation.’
What’s next for your pursuit of brain science?
‘There is one thing that interests me a lot. We seem to have this capability of mind-wandering and predicting alternative scenarios in (the) future. We think about: “What am I going to do tonight? Should I go to the shops?” And you do this while you’re cycling on your bike and so on, and it seems like you’re doing three things at a time – thinking about planning your birthday, riding your bike and also calculating your next bill. There seems to be two scenarios, the emerging now, and the alternative – the plan.
‘How did this evolve? What are the rules? And how is this created? Since it’s happening in the brain, then we need to find a description for that – how is the mind wandering?’
What about your work motivates you?
‘I find the predictive coding framework a theory as important to brain science as evolution is to biology. It is the key explanation of how it makes sense that brains have been created through evolution to do a job for us and it is by creating predictions. The question of can we use this understanding for artificial systems (is) something that is ongoing and will keep us busy for a long time.’
The research in this article was funded by the EU’s European Research Council. If you liked this article, please consider sharing it on social media.
