Researchers have constructed the first comprehensive model of how neurons in the brain behave when faced with a complex decision-making process, and how they adapt and learn from mistakes.
The mathematical model, developed by researchers from the University of Cambridge, is the first biologically realistic account of the process, and is able to predict not only behaviour, but also neural activity. The results, reported in The Journal of Neuroscience, could aid in the understanding of conditions from obsessive compulsive disorder and addiction to Parkinson’s disease.
The model was compared to experimental data for a wide-ranging set of tasks, from simple binary choices to multistep sequential decision making. It accurately captures behavioural choice probabilities and predicts choice reversal in an experiment, a hallmark of complex decision making.
Our decisions may provide immediate gratification, but they can also have far-reaching consequences, which in turn depend on several other actions we have already made or will make in the future. The trouble that most of us have is how to take the potential long-term effects of a particular decision into account, so that we make the best choice.
There are two main types of decisions: habit-based and goal-based. An example of a habit-based decision would be a daily commute, which is generally the same every day. Just as certain websites are cached on a computer so that they load faster the next time they are visited, habits are formed by ‘caching’ certain behaviours so that they become virtually automatic.
An example of a goal-based decision would be a traffic accident or road closure on that same commute, forcing the adoption of a different route.
“A goal-based decision is much more complicated from a neurological point of view, because there are so many more variables – it involves exploring a branching set of possible future situations,” said the paper’s first author Dr Johannes Friedrich of Columbia University, who conducted the work while a postdoctoral researcher in Cambridge’s Department of Engineering. “If you think about a detour on your daily commute, you need to make a separate decision each time you reach an intersection.”
Habit-based decisions have been thoroughly studied by neuroscientists and are fairly well-understood in terms of how they work at a neural level. The mechanisms behind goal-based decisions however, remain elusive.
Now, Friedrich and Dr Máté Lengyel, also from Cambridge’s Department of Engineering, have built a biologically realistic solution to this computational problem. The researchers have shown mathematically how a network of neurons, when connected appropriately, can identify the best decision in a given situation and its future cumulative reward.
“Constructing these sorts of models is difficult because the model has to plan for all possible decisions at any given point in the process, and computations have to be performed in a biologically plausible manner,” said Friedrich. “But it’s an important part of figuring out how the brain works, since the ability to make decisions is such a core competence for both humans and animals.”
The researchers also found that for making a goal-based decision, the synapses which connect the neurons together need to ‘embed’ the knowledge of how situations follow on from each other, depending on the actions that are chosen, and how they result in immediate reward.
Crucially, they were also able to show in the same model how synapses can adapt and re-shape themselves depending on what did or didn’t work previously, in the same way that it has been observed in human and animal subjects.
“By combining planning and learning into one coherent model, we’ve made what is probably the most comprehensive model of complex decision-making to date,” said Friedrich. “What I also find exciting is that figuring out how the brain may be doing it has already suggested us new algorithms that could be used in computers to solve similar tasks,” added Lengyel.
The model could be used to aid in the understanding of a range of conditions. For instance, there is evidence for selective impairment in goal-directed behavioural control in patients with obsessive compulsive disorder, which forces them to rely instead on habits. Deep understanding of the underlying neural processes is important as impaired decision making has also been linked to suicide attempts, addiction and Parkinson’s disease.