AI-based prediction has profoundly upended what was previously thought possible in the field of algorithmic forecasting. Today, its uses vary widely, from credit decisions and health diagnostics to hiring and insurance. But as AI’s influence in prediction has grown, so have concerns about potential societal implications, as even the most advanced machine learning models are not always able to understand the enormous amount of data they are processing.
The Future of Prediction conference, hosted by Columbia Business School’s Digital Future Initiative, brought together leading researchers and practitioners with multidisciplinary backgrounds to discuss these major consequences.
“Today, the amount of data that can be ingested to be able to produce forecasts has grown significantly,” says Omar Besbes, the Vikram S. Pandit Professor of Business at CBS and co-organizer of the event. “But we now have access to alternative ways to digest and process the data. We potentially could have better forecasts, but there is also an element where we have larger black boxes too.”
Those so-called black boxes are where problems with AI-based forecasts often begin, according to Besbes. For starters, the inner workings of AI models are far from transparent, with little incentive for firms to reveal the quality or source of their data. Second, large language models (LLM) are prone to incorrect results — commonly known as hallucinations — often by design. That does not mean models can never be trusted but that their results should be scrutinized.
The following are other key takeaways from the Future of Prediction conference and some of the new societal questions that have emerged.
On Its Own, AI-Based Forecast Doesn’t Always Stick to the Facts
Besbes explains that for applications where facts are required, such as health diagnostics, retrieval-augmented generation is needed, whereby LLM outputs are combined with traditional data querying systems to achieve more accurate results. For more creative tasks, however, hallucination may be more welcome.
“If you use an LLM to generate interesting texts or a poem, then maybe we want it to hallucinate,” Besbes says. “More and more, the power of these new AI tools is coming when we specialize them to particular tasks, as opposed to using a generic prediction across different tasks.”
Protecting society from problematic hallucinations and other predictions gone awry requires some level of regulation, but the extent of such intervention needs to be measured, Besbes argues. Otherwise, regulators risk stifling innovation and preventing smaller firms from being able to compete. He points to the European Union’s AI Act as a good model, noting it clearly differentiates between inappropriate LLM applications and risky LLM applications, which should be monitored but not outright banned. Calls for blanket moratoriums on AI models are unproductive, not to mention practically unenforceable, Besbes says.
“Progress can be steered in the right direction, but regulations where only the good actors respect it and bad actors don't respect it can be dangerous,” he says.
Models Don’t Understand Underlying Causes
From a sociological perspective, the use of AI-based algorithmic prediction is fundamentally different from traditional statistical prediction methods, according to Elena Esposito, a professor of sociology at Bielefeld University and the University of Bologna who co-organized the conference along with Besbes and David Stark, the Arthur Lehman Professor of Sociology at Columbia University.
The correlation-based approach of algorithms means they can make useful predictions without understanding the underlying causal mechanisms. This can be both an advantage and a limitation, depending on the specific use case, says Esposito. Like Besbes, she cautions that business leaders and regulators need to grapple with balancing the efficiency of a model with the need to ensure transparency and avoid bias.
“In medicine, you don't only want to know if a prediction is correct; you want to avoid false negatives. In policing, it is the opposite. You don't want to put innocent people in jail, so you have to avoid false positives,” she says. “You have to measure the correctness of the prediction, not necessarily in terms of knowing the future but in terms of how reliable it is for the specific needs of a singular case.”
Traditional Prediction Versus Algorithmic Prediction
Esposito notes that distinguishing between algorithmic prediction and traditional statistical prediction is critical in efforts to better understand the greater societal implications. Algorithmic prediction tends to be individualized rather than focused on population averages, since it is based on correlations in data rather than on causal explanations. The individualized nature of algorithmic prediction can be very useful, but it also means the predictions may not generalize well to other individuals.
These issues, according to the conference co-organizers, highlight the need for new forms of transparency in the forecasting process, as well as reliability criteria that allow society to trust the results of processes we don’t quite understand yet.
