Brief History of Weather Forecasting

Since ancient time, human’s livelihood is tightly related to weather. Farmer’s harvest depends on weather. As early as in the Shang Dynasty (13th century BC), there were written records of weather observations and weather predictions on oracle bone inscriptions. Studies indicated that in 1217 BC, China not only had 10 consecutive days of actual weather records, but also made 10-day weather forecast, which exemplified the importance of weather in ancient times.

In the ancient article called "Lunheng·Mingyu" (論衡·明雩; about 88 AD), there was a story about Confucius (孔子) forecasting weather:

Confucius went out and asked Zilu (子路) to bring along rain gear. After a while, it really rained heavily. Zilu enquired the reason for the forecast. Confucius said: "The moon was close to the location of Net constellation (畢宿) last night." The Net constellation is one of the twenty-eight constellations (二十八星宿) in ancient China, and is located in Taurus. Ancient people observed the relative position of the moon and the constellations to infer weather changes. This was based on experience accumulated through long-term observations. Ancient people considered the Net constellation as the "rain" constellation. This is exactly the origin of Confucius’ prediction of rain. Apart from observing the constellations, people in ancient time also predicted weather based on the 24 solar terms (二十四節氣). For instance, near the end of the Eastern Han Dynasty (東漢), Zhuge Liang (諸葛亮) "borrowed easterly winds" to set Chibi (赤壁) on fire, defeating the Cao (曹) fleet. This was made possible by using the pattern of local weather changes around the time of winter solstice to predict the timing of easterly winds. Moreover, through years of observations, farmers have come up with many easy-to-remember agricultural proverbs to help predict weather.

(Source: 《Confucius Sacred Relics Picture》Thirty-Four: Observing Celestial Phenomena to Infer Rain)

Agricultural proverbs are wisdom of life handed down from generation to generation, and many of them are about weather or climate. Many agricultural proverbs concerning weather can be explained by science. For example, "if there are swells while winds are calm, typhoon will come by in three days" (無風起橫浪，三天台風降). The scientific explanation for that is: high winds near a typhoon’s centre induce big waves. These big waves travel faster than the typhoon and gradually move away from the typhoon’s center to form the so-called swells. Before a typhoon comes, swells will arrive first, thus creating the situation where swells are seen before arrival of a typhoon. Another example of a weather-related agricultural proverb is: "rainbow in the east will be followed by sunshine, while rainbow in the west will end up in rain" (東虹日頭，西虹雨). Rainbow is an optical phenomenon caused by refraction of sunlight through small water droplets in the air. The prerequisite for rainbow in the east is that there is rain in the east while sunny in the west. Sunshine from the west hits the small water droplets in the east and got refracted to form rainbow. As weather system generally moves from west to east, the rain area in the east will continue to move further away to the east, resulting in improvement in local weather. On the contrary, when rainbow appears in the west, the sun shines from the east to rain area in the west to form rainbow. The rain area in the west will move east closer, thus bringing rain to the local area. The situation described in this agricultural proverb is very similar to that in another agricultural proverb, "don’t go out in morning sunglow; travel thousands of miles in evening sunglow". Both proverbs suggest the ideal time to go outdoor. Of course, the above several weather-related agricultural proverbs only describe the general situations, and there can still be exceptions.

(Left) Rain area moving away when there is rainbow in the east
(Right) Rain area moving in when there is rainbow in the west

Development in Weather Observations

Experience alone has its own limitations. It is difficult to predict precisely the future changes in weather. In other words, most of the empirical methods in ancient times only achieved qualitative predictions, but have not yet achieved quantitative predictions. To achieve quantitative prediction, we must first make quantitative observations, viz making measurements with meteorological instruments.

According to records, China had used "Xianfeng Bird" (相風鳥) to measure wind direction as early as the Eastern Han dynasty. It was also believed that some form of rain gauges were in use to measure rainfall. When technologies in the West began to develop rapidly in around the 16th century, various types of meteorological instruments were invented to detect temperature, pressure, wind direction and wind speed, etc. These meteorological instruments were also introduced to China through Jesuit missions. These instruments were mainly used to measure the weather conditions on the ground. Later on, kites and balloons were used to launch meteorological instruments to the upper air to detect high-altitude weather conditions, widening the coverage of meteorological measurements.

Xianfeng Bird was used to measure wind direction in ancient China (source：http://weather.sz.gov.cn/szsqxjwzgkml/szsqxjwzgkml/qt/kpfw/content/post_5618480.html)

As we all know, the land area in the earth only accounts for about 30% of the global surface area. Making weather observations only at land stations is somewhat inadequate. After the Second World War, remote sensing technology continued to develop. First, there came weather radar which could detect the location and direction of movement of rain areas within one to two hundred kilometres. Then, there were meteorological satellites which capture the distribution of clouds across the whole earth from space and is nowadays indispensable for weather observation.

(Left) Weather radar station at Tate's Cairn
(Right) Radar image of Typhoon Hagos that affected Hong Kong on 19 August 2020

Geostationary satellite capturing the distribution of clouds across the whole earth (Source: Fengyun-4A
meteorological satellite of China Meteorological Administration)

The United Nations World Meteorological Organisation is responsible for coordinating cooperation in weather observations of countries and organisations around the world. It has established a global observing system with a view to integrating all observing systems and promoting standardisation of observational instruments and methods. For ground-based meteorological observations, it is stipulated that observations are made around the world at the same time every three hours, and data are exchanged via dedicated network, so that all meteorological centres can get hold of weather changes around the world every three hours. Nowadays, observational technologies are much diversified, with wider coverage and more frequent observations. These provide indispensable meteorological data for quantitative forecasting.

Global Observing System (Source: World Meteorological Organisation)

Recent Development in Meteorological Technologies

With weather observational data, weather maps can be produced to track the movement of weather systems and to predict weather. On a weather map, the locations of high and low pressure areas (including tropical cyclones) can be identified based on surface air pressure data, and cold or warm fronts can be located based on temperature and cloud cover data. From consecutive weather maps, one can track the direction and speed of movement of weather systems such as high-pressure areas, low-pressure areas, cold fronts, and warm fronts, so as to assess the local weather changes brought by these weather systems. For example, when a cold front arrives, the temperature drops significantly, and when an area of low pressure moves close, it often brings along rain or even thunderstorms. Before radio became popular, it was difficult to collect observational data from different places in real time. Therefore, when weather maps emerged in the early 19th century, they were drawn afterwards.

The earliest weather map of the Hong Kong Observatory (Source: Hong Kong Observatory)

Nowadays we rely on computer models to make weather forecasts. The technology concerned is called numerical weather prediction. First of all, weather observational data are collected. Then computers are used to analyse the current weather conditions (technically called "initial conditions"). Thereafter the evolution of future weather is calculated in accordance with the laws of physics. The results of calculation from the computer models are then used to generate the forecast maps that represent the future weather.

Computer model results are used to generate various forecast maps (Source: Hong Kong Observatory)

Uncertainty in Weather Prediction

Now that we have computer models, why are there still errors in weather forecasts? To understand why, we need to take a look at the characteristics of computer models. Changes in the atmosphere are continuous in time and space, but computer models cannot fully simulate this situation. The main reason is that in a computer model, the atmosphere is divided into many grids to simulate the changes in air pressure, temperature, humidity, etc. in each grid. The larger the size of the grid, the smaller the number of grids to cover the area, and the lower the capability to simulate continuous changes in the atmosphere. Conversely, the smaller the grid size, the greater the number of grids to cover the same area, and the greater the capability to simulate continuous changes in the atmosphere. However, this requires higher computing power of computers.

3-D grids adopted in a computer model (Source: Hong Kong Observatory)

Although the technologies in weather observation are rather advance nowadays, the data so collected nonetheless do not fully cover every corner of the entire three-dimensional atmosphere, especially above the ocean and in the upper-air. Therefore, the initial conditions generated from incomplete observational data cannot fully represent the actual initial conditions. As there is some degree of error in the initial conditions, the future weather conditions calculated based on the laws of physics are not immune to errors.

In addition, the atmosphere is a chaotic system. I believe everyone has heard of the butterfly effect. Edward Lorenz, the founder of Chaos Theory, once said: "Will a butterfly flap its wings in Brazil cause a tornado in Texas?" This actually means that a small change in the air flow initially (flapping of butterfly wings) may become larger and large in time (tornado). This may sound a bit exaggerated, but it actually means that in a chaotic system, slight error at the beginning will grow with time. Putting this to a computer model, error, however small initially, may increase with forecast time. When the forecast error increases, the accuracy of forecast will decrease. This is precisely the limitation of computer model forecasting, and it is also a major challenge in weather forecasting.

Schematic diagram of butterfly effect. A change in the airflow from flapping of butterfly wings may evolve into a tornado.

Two to three decades ago, the computer forecasting technology was less sophisticated than of late. Weather data collected then were also less than those now. The forecast period of computer model was not long, just about two to three days. The amount of weather data collected in recent years has increased geometrically. The error in initial conditions analysed based on larger amount of data is smaller than before. The forecasting period of computer model has also increased substantially. Even so, the forecast error of computer model yet exists and will still increase with time.

(Left) Computer forecast is more accurate, being close to the actual variation
(Right) Computer forecast is less accurate, being deviate gradually from the actual variation

Ensemble Forecast

If the error of computer forecast is not large, then the forecast can be considered accurate. Otherwise, if the error is rather large, the forecast becomes unreliable. So how can we foretell if a computer forecast is reliable or not? A common practice now is to resort to "ensemble forecast". In ensemble forecast, the same computer model is run multiple times with slightly different initial conditions to make predictions, so as to estimate the likely scenarios in future weather and to calculate the chance of each scenario. If the forecasts of many scenarios are consistent with each other, their forecasts are more trustworthy. If the forecast of an individual scenario deviates significantly from the other scenarios, the reliability of this scenario will be in doubt. For example, when using ensemble forecast to predict the movement of a tropical cyclone, many scenarios suggest that the tropical cyclone will move to Fujian but individual scenario indicate that the tropical cyclone will move to Hainan Island. Then the scenario of the tropical cyclone moving to Fujian is more reliable, while the scenario of the tropical cyclone moving to Hainan Island is less trustworthy.

Different forecast scenarios in ensemble forecast

To indicate the reliability of a forecast, we normally calculate the probability of forecasts. The higher the probability of a forecast, the higher its reliability. For instance, on 31 December 2020, the temperature at the Hong Kong Observatory dropped significantly to a minimum of 8.1°C. The probability forecast 5 days ago showed that the minimum temperature on 31 December 2020 would have a higher chance of being around 8°C, which is consistent with the actual situation. Of course, we must also point out that ensemble forecasting is not immune from error and there can still be uncertainty. For example, the probability forecast on 26 December 2020 shows that the forecast minimum temperature on 4 January 2021 would have a higher chance of being around 13°C. However, this 9th day forecast is not very accurate. The actual minimum temperature was 16.9°C, differing from the forecast by about 4°C.

Probability forecast of minimum temperatures issued on 26 December 2020 (Source: Hong Kong Observatory)

Conclusion

With the development in weather observations and computer modelling, the forecast accuracy nowadays is higher than before. However, the atmosphere is a chaotic system. It still comes with uncertainty in the evolution of weather, and poses a big challenge to forecasters. While ensemble forecast allows us to better understand the probability of forecast, enhancing the confidence of forecasters, uncertainty in forecast still exists. In recent years, many researches have been made in the application of artificial intelligence in weather forecasting, especially the use of deep learning methods. Whether or not artificial intelligence can unlock the curse of chaotic system? We will wait and see.

Reference

1. Wen Kegang, Editor in chief, 2003, "Chinese Meteorological History", China Meteorological Press (in Chinese)

2. Weather China website, "Meteorological Observations in Our Ancient Country" http://www.weather.com.cn/jiangsu/qxkp/rdzt/03/1826382.shtml (in Chinese)

3. Lee-Di, 1978, "On the invention of the meteorological instruments in the ancient China", Chinese Journal of Atmospheric Sciences, 2(1): 85-88 (http://www.iapjournals.ac.cn/dqkx/cn/article/doi/10.3878/j .issn.1006-9895.1978.01.10). doi: 10.3878/j .issn.1006-9895.1978.01.10 (in Chinese)

4. Hong Kong Observatory, "Formulation of Weather Forecasts", https://www.weather.gov.hk/en/education/weather/forecasting/00134-formulation-of-weather-forecasts.html

5. Hong Kong Observatory, "Butterfly Effect, Model Ensemble and Probabilistic Forecast", https://www.weather.gov.hk/en/education/weather/forecasting/00136-butterfly-effect-model-ensemble-and-probabilistic-forecast.html

Author:
Dr. Cheng Cho-Ming, Director of the Hong Kong observatory

May 2021