top of page

Humans and Weather Forecasting: How ChatGPT’s 'Strawberry' Mistake Highlights the Dangers of AI-Only Forecasts

Can AI be trusted with our weather forecasts? In an age where computers and artificial intelligence (AI) are making significant strides across various industries, it's easy to assume that machines can replace human expertise in weather forecasting. However, the art of weather forecasting presents unique challenges that the experience, intuition, and nuanced understanding of meteorologists can far outshine. This is especially critical in complex weather scenarios when people rely on weather information the most.

Bundy Storm.jpg

The Complexity of Weather Patterns

Weather forecasting is a complex science involving understanding and interpreting a plethora of variables. While computers and AI can process vast amounts of data quickly, they often lack the ability to understand the intricacies and subtleties that seasoned meteorologists can identify. Most importantly, AI forecasting works by identifying previous patterns and extrapolating them to predict future patterns. The challenge is that no two weather setups are exactly alike—each one is different. When you throw in an element of chaos (small, random events that cannot be predicted), it adds further complications.

Meteorologists spend decades studying the atmosphere, learning to recognize patterns, and understanding how various weather phenomena interact (particularly with microclimates). Their training and experience allow them to make informed judgments that AI and computers simply cannot yet replicate. They can apply intuition and experience to their forecasts, even when weather patterns are erratic and unpredictable. Meteorologists can quickly adapt to changing scenarios, recognize errors, and make adjustments to forecasts that computers and AI may miss. This ability to adapt and respond to evolving conditions is crucial for ensuring public safety and minimizing the impact of extreme weather.

Smoothing Out of Extreme Weather Forecasts

Most automated forecast systems provide an average-weighted forecast across all models. On paper, this can provide higher levels of accuracy, as most events lie within a normal distribution pattern, and average-weighted forecast systems do well when the weather is simple and normal. However, during patterns of extreme weather (particularly severe weather or extreme rainfall), it's rare that every single outcome or forecast model provides a correct forecast. In this case, AI may remove or "smooth out" the extreme values. However, a meteorologist can connect the dots and determine that an extreme event may be likely even if one (or no) computer models support this.

The 2022 Brisbane and Lismore floods were excellent examples of this. Many computer systems smoothed the forecast rainfall to see 300-500mm occur over four days. While this was true for many locations, some computer models forecasted 1000-1500mm. Had "only" 300-500mm fallen, the impacts of the floods would have been far less significant. However, some locations recorded well in excess of 1000mm, which caused significant issues.

The Human Element: Communication, Trust & Accountability
 
Meteorologists provide a critical element to forecasting—the human level of accountability. Imagine you need to make a decision at work. If the decision is wrong, the company will lose $10. With this risk at stake, it's unlikely much thought will be placed into the forecast. If the stakes were raised and $1000 was at risk, a little more thought and care would be placed into the decision. However, if the stakes were raised again, and human life would be lost if the wrong decision is made, then a significant degree of care and thought would be placed into the forecast. Meteorologists can make decisions within the context of their impacts because they understand what's at stake and have a level of trust and accountability. This is something that computers and AI do not have.

A relatable example of AI's limitations is a recent instance where ChatGPT, an advanced language model, made an error in something as straightforward as counting the letters in the word "strawberry." While this mistake is relatively harmless, it highlights the potential for inaccuracies in more complex and critical tasks like weather forecasting. Human oversight remains essential to ensure accuracy and reliability. Even when faced with risking a human life, the AI model confidently said it would risk some one's life over a counting mistake that was obviously incorrect.

Profits Vs Accuracy

AI and computer-generated forecasts are favoured by many organizations because they're cheaper to run than meteorologists. An analogy is the consideration between fast food and a restaurant. Fast food is cheap and convenient, but if you're after the best meal, it may not be the best choice. Both a fast food outlet and a restaurant may use the same fundamental tools to provide their food; however, one will individually prepare and craft each meal while the other will provide a "one size fits all" approach to keep costs down. This means that forecasts made by meteorologists can cost more, but when you consider what's at stake (often millions of dollars of company assets—or even more precious—human life), it can sometimes cost more to use an automated service than one that's accountable and accurate.

Profits_edited.jpg

Embracing Technology, Enhancing Expertise

It's important to note that while AI has its limitations, it still plays a valuable role in modern meteorology. AI can process vast amounts of data quickly and highlight patterns so humans can make the final decision. The most effective approach combines the strengths of computers and AI with the expertise of human meteorologists.

At Weatherwatch, we pride ourselves on combining cutting-edge computer and AI weather forecasting technology with the invaluable expertise of our dedicated meteorologists to deliver the most accurate and reliable weather information available. We leverage computers and AI to handle the data processing and initial analysis so that our meteorologists can focus on interpreting the results and apply their knowledge and expertise to deliver accurate, enhanced weather forecasts. When weather decisions can cost (or save) you millions of dollars or protect human lives, there is simply no substitute for the trust, experience, and accountability of a human meteorologist.

If you have any questions about what forecasts are right for you - reach out to our friendly team via our contact page.

bottom of page