We constructed a random forest (RF) model that identifies relationships between crop development and meteorological variables during the growing season. For crop data, we adopted yearly soybean yields (t/ha of dry matter) generated by the global gridded crop model (GGCM) EPIC-IIASA , which is based on the Environmental Policy Integrated Climate EPIC; field-scale crop model. This GGCM simulates complex relations between weather conditions and crops at planetary scales by reproducing biophysical processes in the soil–plant–atmosphere system and providing crop-related outputs based on climate-related inputs. Simulation outputs used in this study were performed for phase 3a of the Intersectoral Impact Model Intercomparison Project (ISIMIP; see https://isimip.org, last access: 20 June 2021, for details and protocols) and the Global Gridded Crop Model Intercomparison (GGCMI) initiative. It used as climatic input in the GSWP3-W5E5 dataset, which is a merger of the GSWP3 (Global Soil Wetness Project phase 3) dataset and the W5E5 dataset . It captures the period from 1901 to 2016 at a 0.5∘×0.5∘ resolution. The reasons for using yields from crop models are the longer time series that allow for more years to be included in the training of the statistical model and uniformity of data quality for regions where there is low quality of observational data. Another advantage of using simulated crop models is that management and technology trends are static, whereas they are intrinsically embedded in the observed yield datasets. Other works have also adopted simulated yields for climate–crop analysis . Details of the EPIC-IIASA model's performance can be seen in , . We used 100 years of data (1916 to 2016) to train the model and we limited the analysis to grid cells from the top 10 US soybean producer states, which are (ordered by production volume): Illinois, Iowa, Minnesota, Indiana, Nebraska, Ohio, South Dakota, North Dakota, Missouri and Arkansas. Together, they represent over 70 % of the US national soybean production and approximately 20 % of the global soybean production . In addition, to ensure only rainfed soybeans were considered, we selected only grid cells that contained at least 90 % of the grid as rainfed, which corresponded to 84 % of the region studied MIRCA2000;. For validation of the crop model, we compared the EPIC-IIASA simulated yields with the observed yields from the US Department of Agriculture (USDA, https://www.nass.usda.gov/Quick_Stats/, last access: 15 August 2021) for the region considered. EPIC-IIASA has higher mean and standard deviations values than the observed because the simulated yields are potential . To evaluate the interannual variability, we obtained a coefficient of determination, R2, of 0.674. We also observed a good correlation between the two standardised datasets (Fig. ). We consider EPIC capable of replicating the interannual variability of the observed data.

For weather data, we used observed data obtained from the Climate Research Unit (CRU) TS4.04 dataset . They have global coverage at a resolution of 0.5∘×0.5∘, cover the period from 1901 to 2019 at a monthly scale and are based on weather station observations. The CRU dataset has a comprehensive range of climatic variables at monthly resolution (Table ), which makes the dataset suitable for agricultural studies, as shown in , , and .

Because the analysis involves also the exploration of future scenarios of global warming (GW), we included large ensembles of synthetic weather data produced by the global climate model (GCM) EC-Earth V2.3 . As a global coupled climate model, it combines atmospheric, ocean, land surface and sea ice models at a resolution of 1.125∘×1.125∘. We used large ensembles of short-time periods to represent the full range of possible realisations at different levels of global warming . Three scenarios are considered: a benchmark representing the present-day climate of 2011–2015 (referred to as PD), a 5-year period representing an average global mean temperature 2 ∘C above the pre-industrial levels (referred to as 2C) and another 5-year period corresponding to an average of 3 ∘C above pre-industrial levels (referred to as 3C). To create the large ensembles for each GW scenario, we combined the 5-year periods with 16 different initial conditions and 25 different realisations based on stochastic physics. Together, they culminate in 2000 years of different simulations for each warming level scenario (Fig. ; see , for more information on the ensemble setup). Finally, we resampled the large ensembles in 20 members of 100 years to be consistent with the length of the historical dataset (referred to as grouped ensembles hereinafter).

To facilitate the comparison of observed and modelled data, we first upscaled the CRU and crop model data to the same resolution of the EC-Earth data with the first-order conservative remapping method . We spatially averaged all data for the region studied (Fig. ) to focus on the regional scale of weather events and their crop yield impacts, as these have larger influence on global markets. Aggregating data spatially might lead to loss of information, especially on local extreme conditions, but the RF model performance is comparable when running on aggregated data and on all grid points (Table ).

The setup of this work relies on comparing large uniform samples at different levels of global warming. Therefore, long-term trends in the data needed to be removed. The yield data from the EPIC-IIASA model have a significant long-term trend, which is a result of the inclusion of atmospheric CO2 concentration levels in the biomass growth calculation, a process called CO2 fertilisation . We regressed the yield data against the global CO2 concentration levels to remove the long-term trend (Fig. ). Here, we explore the probability of soybean failure, which is defined by means of a threshold, similarly to , , . Every season with a yield of 1 standard deviation below the mean was considered a failure. The meteorological variables from the CRU dataset were also detrended linearly to remove global warming influence. This way, we isolated the interannual variability component from long-term trends in both meteorological variables and crop yield time series.

We chose a random forest model to detect failures in soybean yields because of its high performance, flexibility and interpretability. Random forest is a non-linear and non-parametric statistical model for classification and regression. The model consists of an ensemble of independent decision trees. The decision trees are each trained on random subsamples of the data to provide different predictions, and the final estimate of the model takes into consideration all predictions together, accounting for internal variability. Random forest has become widely popular and is applied in different fields of science. It presents high accuracy while providing low overfitting levels and is ranked among the best classifiers for real-world problems .

The first step in designing the random forest model was the feature selection. Among the multiple meteorological variables considered (Table ), some variables are more relevant than others in predicting crop failures for the region studied. There is also a temporal factor, where the importance of a meteorological variable shows a seasonal cycle. By removing non-relevant variables and months, the data fed to the random forest model are simplified and the model performance possibly increased. We considered multiple feature selection methods because there is no universal method, providing robustness to the selection. The methods are analysis of variance (ANOVA) , mutual information selection , the χ2 test and the internal feature selection of the random forest model . At the end of the feature selection step, we obtained the most important meteorological variables to soybean failures.

Random forest models require the tuning of internal parameters for an optimal performance. We tuned the random forest's parameters following a resampling technique called cross-validation. It consisted of dividing the data in 10 different splits, where nine splits are used to train the model and the remaining one is used for validation. The process is run 10 times so that every split is used once for testing. In addition, this process was repeated five times with different random divisions, leading to 50 runs in total. The final configuration of the RF parameters can be seen in Table . Because the crop yield dataset has less failure seasons than non-failure seasons, the dataset is imbalanced, affecting the model's capacity in identifying the minority class. To address this issue and improve the model performance, we assigned weights to the predictions of each class with values inversely proportional to their frequency. This increases the penalties for underrepresentation of the minority class, balancing the model.

With the RF setup complete, we trained and validated the RF model on the historical data following a 80/20 split, where 80 % of the data were used to train the model and the remaining 20 % were used to validate the model's performance on unseen data. Whilst non-parametric and non-linear models like random forests tend to achieve higher performance than simpler linear models, their complexity renders it more difficult to interpret the outcomes. We included partial dependence plots in the analysis. Partial dependence plots illustrate how model outputs vary according to alterations in one or more inputs, while preserving other inputs values . These plots make the random forest model more interpretable and demonstrate the interactions between meteorological variables and crop yields.

In order to evaluate the random forest performance, we used the Matthews correlation coefficient (MCC) metric on the validation data. It is considered more informative and truthful for the evaluation of binary classification models than other metrics . The MCC assesses the performance of the model by quantifying the number of true positives (TPs), the number of true negatives (TNs), the number of false positives (FPs) and the number of false negatives (FNs), as illustrated in Eq. (). It ranges from -1 to 1, and a score of 0 is equivalent to a random prediction. The model requires both positive data and negative data to be correctly predicted to have a high score, which makes it particularly useful for imbalanced datasets . For a comprehensive overview of the model performance, we included additional performance metrics, which are accuracy, precision, recall and F1 score. They can be seen in Appendix .

1MCC=TP⋅TN-FP⋅FN(TP+FP)(TP+FN)(TN+FP)(TN+FN)

We compare the random forest's performance to threshold-conditioned methods, which are frequently used for multivariate risk assessment . Two cases are adopted here: the “AND” and the “OR” cases. The “AND” case requires all variables to be equal to or above hazard limits simultaneously for the failure definition. The “OR” case considers at least one of the conditions surpassing the limit to classify as failure . The threshold values for this work were defined as the average conditions of failure seasons in the observed data for each variable minus the corresponding standard deviation of that variables across the sample of failure seasons.

To explore soybean failures at different levels of global warming, we used large ensembles of meteorological data from the EC-Earth model. The large ensembles have the advantage of explicitly simulating extreme events that would not be found in smaller datasets due to their rare nature . Because the RF model relies on thresholds that may be exceeded more or less frequently in biased climate model data, bias corrections were applied for the analysed regions. We estimated adjustment factors between the detrended CRU dataset between 1916 and 2016 and the PD scenario. Bias adjustment factors were calculated with Detrended Quantile Mapping , considering 25 degrees of freedom. The adjustment factors were then applied to the scenarios 2C and 3C, assuming a constant bias (see Fig.  for bias correction results for each scenario and Fig.  for spatial variability of corrected bias).

To assess the importance of the correlation of the conditions leading to failures, we created permuted versions of each large ensemble by randomly reshuffling the meteorological variables , so that the correlation structure between them was removed (referred to as shuffled versions). We also defined a metric called the relative compound contribution. Relative compound contribution measures the importance of the correlation structure between the meteorological variables leading to crop failures. It is a statistical interpretation comparing crop failures under different correlation structures. Relative compound contribution is calculated as the ratio of the failure ratio obtained with the original data to the failure ratio obtained with the shuffled data, Relativecompoundcontribution=FailureratiooriginalFailureratioshuffled. The closer to 1 the relative compound contribution gets, the less important the correlation structure between the variables is. For scenario exploration in this study, we use six scenarios: PD, PD shuffled, 2C, 2C shuffled, 3C and 3C shuffled, each of which was fed into the RF model to obtain the soybean failure probabilities. We assessed the impact of climate change by comparing the failure probabilities for different return periods (calculated as the inverse of the exceedance probability of the specific event to occur) for the PD, 2C and 3C scenarios. Then, we quantified the relative compound contribution by comparing each level of global warming with their shuffled variants. Finally, we compared the RF model with the threshold-conditioned methods to account for differences in the approaches to predict changes with global warming and to quantify the relative compound contribution.

Machine learning algorithms do not extrapolate well for data outside the training range . Given the random forest model in this work is trained on historical conditions but applied to GW scenarios, we tested the influence of data outside the training range in the analysis following a three-step test: (1) identify the number of cases outside the training range for each meteorological variable; (2) restrict the values outside the training range to be within the training range; (3) quantify possible differences and inconsistencies in the results between the data before and after the conversion of values outside the training data range (see Appendix  for more information).

Storylines were here used to identify the driving components of a historical extreme event and to generate future analogues based on the same event. The 2012 soybean season is our case study, which presented extremely low yields across the main producing regions of the United States. We first verified whether the trained RF model was able to correctly predict 2012 as a failure event and the failure probability assigned to it. To explore the possible analogues of the 2012 season in warmer scenarios, we took two approaches based on the event definition: (1) the first approach was based on the physical conditions that led to the 2012 extreme event, defined here as “event analogues”. We quantified the joint occurrences of meteorological conditions exceeding the 2012 season conditions in the global warming (GW) scenarios; (2) the second approach was based on the impact metric of the event, its yield failure probability estimated by the random forest model, and we defined this approach as “impact analogues” as, e.g. in. Using this approach, we quantified the number of seasons in the GW scenarios with equal or higher failure probability predicted by the random forest model. Last, we compared the 2012 season meteorological conditions with the mean meteorological conditions of impact analogues to account for possible changes in the physical aspects of the future analogues.

Following the methods section, the first step to build a random forest model is the feature selection. The four feature selection methods (Table ) show potential evapotranspiration, diurnal temperature range, monthly maximum temperature and precipitation as the most important features. The considered soybean season in the US ranges from May to October, and we see that the months with the highest sensitivity to meteorological conditions are July and August (Fig. ). The two months correspond to the reproductive phase of soybeans in the region studied , which is a vulnerable crop stage to weather stress . Based on this, we test an alternative version of the meteorological variables limited to July and August and aggregated along the two months. Results show the aggregated data outperform the monthly data (Table ). We therefore adopt for this study the aggregated version of the meteorological variables along July and August and compare them to their climatology to identify general features (Fig. ). Failure years are warmer, have lower levels of precipitation and fewer wet days, larger daily temperature range, higher levels of potential evapotranspiration and lower fractions of cloud cover when compared to the climatology. Vapour pressure and ground frost frequency do not show significant differences between normal and failure seasons, so they are removed from the model training.

The remaining meteorological variables have high correlation levels between themselves, as seen in Fig. b. To separate which correlations have statistical redundancy and which have not, we examine each meteorological variable individually. Among the monthly temperature variables, mean (tmp), minimum (tmn) and maximum (tmx) temperature variables are highly interconnected. Tmx shows the best performance among the three variables, so we select tmx as the representative of temperature variables. Monthly precipitation and wet days per month are also highly interconnected, but their performance is similar. As precipitation is a precursor of wet days per month in the CRU dataset , monthly precipitation is selected. Potential evapotranspiration exhibits high correlation values to all other variables because potential evapotranspiration is derived from temperature, vapour pressure and cloud cover . Therefore, we consider potential evapotranspiration redundant for this specific experiment. Finally, the three variables selected are maximum monthly temperature, precipitation and diurnal temperature range. They still have considerable levels of correlation, but these have a physical meaning, highlighting the compound nature of meteorological variables leading to crop failures in the region.

To evaluate the performance of the random forest (RF) model to link meteorological variables to crop yield failures, we use the Matthews correlation coefficient (MCC) metric (Eq. ) and compare the results with the “AND” and “OR” threshold-conditioned approaches for reference. The RF model has the highest MCC score at 0.61, while the “AND” approach has 0.54 and the “OR” method only 0.34. The random forest model also performs better than the other methods in the additional metrics as seen in Fig. . Therefore, the RF model is successful to link inputs with outputs and outperforms the threshold-conditioned methods.

Partial dependence plots (Fig. ) explore which variables contribute to successfully distinguish between failure and non-failure conditions. They show the relationships between the meteorological variables' perturbations and crop failure probability. For the selected time period, crop failures are proportional to diurnal temperature range and maximum temperature. Precipitation shows a general inverse proportion to crop failure probability, suggesting low values of precipitation to increase failure probability, as indicated by Fig. . Soybean failures in the region studied are thus associated with high levels of monthly maximum temperature, high levels of diurnal temperature range and low levels of precipitation. Furthermore, we observe that the links between crop failures and the meteorological variables are non-linear.

The random forest model manages to successfully link weather conditions with crop failures, to capture the complexity of the failure, including non-linear relationships, and to outperform the approaches “AND” and “OR”. The success in reproducing crop failure impacts from combinations of weather features for the historical datasets makes the model suitable to explore different climate scenarios.

The trained and validated random forest model is applied to the grouped ensembles to estimate crop failure probabilities for different global warming scenarios. First, we determine if the crop failure probabilities obtained for the present-day (PD) scenario is consistent with the historical data from EPIC-IIASA (Fig. a). The width of the grouped ensemble (shading) indicates the many possible manifestations due to natural variability. The observed data are just a single realisation over that period, and we see the observed data are within the range of the ensemble. For the estimation of global warming influence on crop failure probabilities, we quantify the return periods of failure probabilities for the PD, 2C and 3C scenarios (Fig. b). The 2C scenario shows increased failure probabilities for any given return period with respect to the PD scenario, while the 3C scenario shows slightly higher failure probabilities than the 2C scenario. Global warming is therefore likely to increase the occurrence of soybean failures, but the difference between 2C and 3C is not significant.

The relative compound contribution is quantified by considering two versions of data arrangement: original (ordered) and shuffled (unordered, no correlation between variables). Figure  presents the crop failure probabilities for different return periods. In all scenarios, the return periods for the 0.5 failure probability threshold are shorter for the original data than for the shuffled data. Thus, the combination of the meteorological variables is relevant for the failure likelihood of soybeans (e.g. low precipitation concurs with above average temperatures), which highlights the compound nature of the crop failure drivers. For the 2000 years of data in each scenario, the RF model predicts 276 failure seasons for the PD original data, whereas the shuffled version has 63 failure seasons predicted. The failure seasons for the 2C original and shuffled data are, respectively, 616 and 241 seasons, and for the 3C scenario, 621 and 353 seasons. Thus, while the number of failure seasons increases with global warming for the original data, the increase in the number of failure seasons for the shuffled data is larger. A reduction in the relative compound contribution for the 2C and 3C scenarios suggests the correlation structure between meteorological variables becomes less relevant for our definition of crop failure.

We demonstrate the changes in the physical conditions for failure and non-failure seasons at each scenario (Fig.  for 3C and Fig.  for 2C). Median temperature values along July and August of failure years increase by 1.1 ∘C for 2C and 2.6 ∘C for 3C. Median precipitation values of failure years show little change, slightly increasing by 2.4 mm/month for 2C and 0.18 mm/month for the 3C compared to PD. Median diurnal temperature range values of failure years show a slight decrease of 0.33 and 0.36 ∘C for 2C and 3C scenarios, respectively. Despite median values of precipitation not changing significantly, we see an increase in the failure rate at the low end of the rainfall distribution for 2C and 3C (Fig. a top). Together with temperature extremes, the results suggest more frequent and intense joint warm and dry conditions in the future (Fig. a). However, we see also an increase in the failure distribution at higher (and thus less critical) precipitation levels for both 2C and 3C than for PD. A similar pattern is observed for the diurnal temperature range values, where less critical levels of DTR still incur failures (Fig. b). Extreme temperature values dominate the failure probability at warmer levels, which reduces the need for other variables to be extreme as well to generate failures. Therefore, the univariate increase in the temperature values due to global warming is associated with the increase of soybean failure ratios and with the decrease of the relative compound contribution.

We compare the RF model with the “AND” and “OR” approaches to account for differences in the approaches in quantifying the relative compound contribution and the predicted changes with global warming. The “AND” approach shows the lowest ratio of failure for both original and shuffled data (Fig. a) but also the highest levels of relative compound contribution (the difference between ordered and permuted datasets, Fig. b). The “OR” approach only requires one critical variable for failure definition, which implies the highest failure ratios (Fig. a). Moreover, breaking the correlation structure and shuffling the meteorological variables for this approach means increasing the number of failure seasons, which implies relative compound contribution has a decreasing role. Finally, the random forest model predicts an intermediate number of failure seasons. The RF model suggests relative compound contribution as an enhancing factor for crop failures (in contrast to the “OR” method) but not as much as suggested by the “AND” model (Fig. a). Because the RF model presented the highest performance scores previously, we consider it to be the most reliable in quantifying relative compound contribution, while the others either underestimate or overestimate the importance of compound structure for crop failures. Nevertheless, all three methods agree that relative compound contribution loses importance under warmer scenarios, trending towards a level of 1.0 (no relative compound contribution, Fig. b).

The extrapolation test indicates all three meteorological variables in the climate projections have values outside the training range, with temperature extremes exceeding the historical range as the most frequent case of extrapolation. However, the conversion of values outside the training range into values within the training range does not change the results obtained. This is because the model is a binary classifier, and the failure is defined within the historical data. Unprecedented extreme values do not change the failure definition, as it follows the assumption that values outside the training range are similar to the extremes in the training range (see Appendix for further information). With the extrapolation test demonstrating that values outside the training range do not affect the results, the impacts of different levels of climate change on soybean failure probability in the region are validated. Next, we investigate if these general conclusions also hold for specific cases like the extreme 2012 season using a storyline approach.

The year 2012 had an extreme loss in US soybean production, with exceptionally low yields in the majority of the producing region (Fig. a). We test the random forest model for the 2012 season to identify the meteorological variables associated with the failure event. The season stood out as all three selected meteorological variables presented extreme values (Fig. b), with precipitation approximately at -2 SD (standard deviations), temperature above 2 SD and DTR scoring the highest recorded value exceeding 3 SD. When used to predict the probability of crop failure due to the meteorological conditions described above, the RF model indicates a 0.96 likelihood of the 2012 season to be a failure.

We create 2012 season analogues using both impact and event perspectives. Event analogues are defined as the joint occurrences of the 2012 climatic conditions (as in Fig. b), while impact analogues are defined as the number of seasons with failure probability assigned by the random forest model as equal or higher than the probability of the 2012 season (i.e. probability >=0.96). The event analogues have a occurrence ratio of 0.0015 (three events in 2000 years, 666-year return period) for both PD and 2C scenarios, but it is not seen in the 3C scenario (Fig. ), suggesting a highly rare event that does not increase with global warming. However, we find a higher ratio of 2012 impact analogues for the PD scenario, 0.022 (44-year return period), 14 times the PD event analogues. Impact analogues also increase in frequency at warmer scenarios, 0.038 (26-year return period) for 2C and 0.042 (24-year return period) for 3C. The difference in results between the two types of analogues is mainly due to the DTR values, illustrated by Fig. a–c, which show the ratio of exceedance (the frequency of seasons exceeding a given threshold) of the 2012 meteorological conditions for each variable individually. Event analogues require all variables to exceed the corresponding 2012 conditions. While we observe an increase in the number of seasons with temperature and precipitation exceeding the 2012 conditions in global warming scenarios, DTR does not increase with GW. Therefore, DTR becomes a bottleneck in the generation of event analogues. Impact analogues, on the other hand, are predicted to increase with global warming because they are defined based on the impact metric (failure probability) and bypass the DTR limitation. In addition, by relying on the impact metric, the meteorological conditions of the analogues can be analysed for changes due to global warming (Fig. d–f and ). The impact analogues of the year 2012 show warmer temperatures during summer with respect to the original event. For precipitation, the analogues are significantly drier than those in the year 2012 during July and August, the months in which the RF model takes into account. Finally, the analogues present lower DTR values during most of the year.