Inefficient choices often persist because they are familiar. When faced with a decision between what we know and what we don't, human behaviour is far more rigid than we like to believe – despite our innate curiosity.
I study these decisions with bandit problems. In simple terms, a bandit problem explores how people balance sticking with an outcome they know against trying something new that might be better. Most real-world decisions sit in this grey zone: choices we can only partly understand before we make them.
Rigidly sticking to what we know has real consequences. When people and organisations fail to try new ideas, innovation slows and competition weakens. Miss an early opportunity and the loss tends to be compounded, as technologies and advantages often build on what came before.
What do we do when everything is uncertain?
In recent research at Macquarie University, we've examined how people actually behave under uncertainty. We ask participants to repeatedly choose between two unknown options; they are paid real money based on their performance, so their decisions have real consequences.
In theory, people should strike a balance between learning and earning – exploring new options when information is valuable and exploiting known options when they are confident.
In practice, they don't.
We've found that while people gradually learn which option performs better, they are also highly inertial. What they chose most recently influences them far more than the evidence justifies. Recent experiences are overweighted, while older information – which remains just as relevant – is discounted.
This helps explain why commuters keep taking the same route to work, or why households stick with the same energy plan, even when alternatives may suit them better. People do not update their behaviour as rationally as standard economic models assume.
Another key finding is that people do not explicitly value learning itself. Rarely do they think, I should try this to get better information. Instead, the smaller the perceived difference between options, the more willing people are to experiment.
In our research, we call this behaviour intelligent noise . People behave somewhat randomly – 'noisily' in research terms – which allows them to gather information about different options. But this randomness is not blind: people are still more likely to choose what they currently believe is the better option.
For policymakers, this matters. If the goal is to help people discover better alternatives, interventions need to break inertia and support regular, low-cost experimentation.
Should we force people to try new things?
A tempting policy response is to force experimentation. Think of transport strikes that disrupt familiar commutes, pushing people onto bikes, buses or ferries. Or regulations that restrict access to high-emission products, nudging consumers towards cleaner alternatives.
The idea is simple: force people to try something new and they will realise if it is better. The reality is more complicated.
Follow-up research shows a clear backlash to forced change. When people are pushed into an option they didn't choose, they pay less attention to what they learn from the experience. The frustrated commuter, annoyed by disruption, may fail to notice the new route is actually quicker.
Even when two people end up with the same information, those who were forced tend to behave differently – actively avoiding the option they were pushed into.
This has uncomfortable implications for policymakers. Forcing change can delay genuine belief change, even when the outcome is objectively better. Interventions that look effective on paper may perform poorly once real human behaviour is taken into account.
What this means for climate change
The same decision logic applies to climate action. Many sustainable choices feel worse in the short term. Yet people cannot know whether these trade-offs are as costly as they imagine – or as beneficial to the environment – without actually trying them.
Our ongoing research at Macquarie explores this by introducing real-world carbon consequences into the bandit problem. In laboratory experiments, participants make choices that affect not only their own payoffs but also carbon outcomes. So far, these decisions have led to the purchase of carbon credits offsetting more greenhouse gas emissions than the average annual footprint of 10 Australians.
The uncomfortable conclusion
This research points to an uncomfortable truth: innovation, sustainability and productivity don't fail because better options do not exist; they fail because people get stuck. We overvalue recent experiences, resist unintentional change and rarely value learning for its own sake.
If societies want to adapt more quickly – whether to new technologies or a changing climate – policies need to work with human psychology rather than against it.
Sometimes nudging beats forcing. Sometimes reducing friction matters more than incentives. And sometimes the hardest part of progress is not discovering new ideas but getting people to try them.
