Traditional statistical/machine learning methods generate one model (e.g., neural
network) based on all the available training data and use that model to predict a quantity
of interest. Ensemble methods instead generate multiple models. They then calculate
predictions by combining the predictions of each of their component models using
averaging, voting, or other appropriate means. The resulting ensembles typically
outperform each of their component models in prediction accuracy. The idea is that, just
as a committee of diverse people tend to make better decisions than each individual
alone, an ensemble of diverse yet high-performing models tend to perform better than a
single model. There are multiple ways to generate a diverse set of models that work well
together. Within machine learning, there are "Pattern-level" methods such as bagging and
boosting that generate models using different subsets of the training set. "Feature-level"
methods such as input decimated ensembles generate models that use different subsets of
the input features. Another method involves generating multiple neural networks by
starting with different random weights.

Along these lines, in climate modeling, different climate models with multiple initial
conditions per model are used to generate multiple forecasts, which are then combined.
Active areas of research include determining the best way to generate the multiple initial
conditions and determining the best way to combine different forecasts. One issue in
combining is assessing the conditions (e.g., different times of year and different regions
of the world) under which different models perform well and combining appropriately.
Climate models are computationally expensive to run, so it is important to not run models
under conditions where they are known to perform poorly.

We have demonstrated the usefulness of ensemble methods in aircraft health monitoring
and are working on using them in satellite image understanding.