Data Fusion

A reminder of Bayes Theorem

\[ \underbrace{p(\theta|D)}_\text{posterior} \; \propto \; \underbrace{p(D|\theta)}_\text{likelihood} \;\; \underbrace{p(\theta)}_\text{prior} \; \]

The posterior

The posterior is the conditional probability of the parameters given the data \(p(\theta|D)\) and provides a probability distribution for the values any parameter can take,

This allows us to represent uncertainty in the model and forecasts as probabilities, which is powerful for indicating the probability of our forecast being correct.

A reminder of Bayes Theorem

\[ \underbrace{p(\theta|D)}_\text{posterior} \; \propto \; \underbrace{p(D|\theta)}_\text{likelihood} \;\; \underbrace{p(\theta)}_\text{prior} \; \]

The likelihood

The likelihood \(p(D|\theta)\) represents the probability of the data \(D\) given the model with parameter values \(\theta\), and is used in analyses to find the likelihood profiles of the parameters.

This term looks for the best estimate of the parameters using Maximum Likelihood Estimation, where the likelihood of the parameters are maximized for a given model by choosing the parameters that maximize the probability of the data.

A reminder of Bayes Theorem

\[ \underbrace{p(\theta|D)}_\text{posterior} \; \propto \; \underbrace{p(D|\theta)}_\text{likelihood} \;\; \underbrace{p(\theta)}_\text{prior} \; \]

The prior

The prior is the marginal probability of the parameters, \(p(\theta)\).

It represents the credibility of the parameter values, \(\theta\), without the data, and is specified using our prior belief of what the parameters should be, before interrogating the data. This provides a formal probabilistic framework for the scientific method, in that new evidence must be considered in the context of previous knowledge, providing the opportunity to update our beliefs.

Advantages of Bayesian approaches

Data fusion

Least Squares

Least Squares makes no distinction between the process model and the data model.

Least Squares

Here I’ll use the example used in the practical, exploring time-series of the Normalised Difference Vegetation Index (NDVI ~ vegetation “greenness” measured from the MODIS satellites) to look at vegetation recovery with time since fire in Fynbos.

Least Squares

Here I use Nonlinear Least Squares (NLS) to fit a negative exponential model:

\[\begin{gather} \text{NDVI}_{i,t}=\alpha_i+\gamma_i\Big(1-e^{-\frac{age_{i,t}}{\lambda_i}}\Big) \end{gather}\]

This process model is the only opportunity for inputting data, and we input time series for \(NDVI\) and date (or postfire \(age\), since our time series started at the time of a fire). Parameters \(\alpha\), \(\gamma\) and \(\lambda\) are estimated by the model.

Least Squares

The only way to add other data sources is to make the process model more complex.

• E.g to capture the seasonal fluctuations in NDVI we can add a sine term that requires us to specify the month (\(m\)) of the fire, like so:

\[\begin{gather} \text{NDVI}_{i,t}=\alpha_i+\gamma_i\Big(1-e^{-\frac{age_{i,t}}{\lambda_i}}\Big)+ A_i\text{sin}\Big(2\pi\times\text{age}_{i,t}+\Big[\phi+\frac{\pi}{6}(m_{i,t}-1)\Big]\Big) \end{gather}\]

A quick aside on the parameters

\[\begin{gather} \text{NDVI}_{i,t}=\alpha_i+\gamma_i\Big(1-e^{-\frac{age_{i,t}}{\lambda_i}}\Big)+ A_i\text{sin}\Big(2\pi\times\text{age}_{i,t}+\Big[\phi+\frac{\pi}{6}(m_{i,t}-1)\Big]\Big) \end{gather}\]

Maximum Likelihood

Maximum Likelihood makes a distinction between the data model and the process model:

Maximum Likelihood

I don’t make the distinction in the equations presented in the practical, but do in the functions, where we included the data model, where we specified a Gaussian (normal) distribution around the mean, \(\mu\) (which is described by the process model):

\[\begin{gather} NDVI_{i,t}\sim\mathcal{N}(\mu_{i,t},\frac{1}{\sqrt{\tau}}) \\ \end{gather}\]

Where \(\mathcal{N}(\mu_{i,t},\frac{1}{\sqrt{\tau}})\) is just a normal distribution with mean \(\mu\) and variance \(\frac{1}{\sqrt{\tau}}\).

Maximum Likelihood

We then have a separate function for the process model, which describes \(\mu\) as a function of the covariate (vegetation \(age\)), with parameters \(\alpha\), \(\gamma\), \(\lambda\), \(A\), \(\phi\) and \(m\):

\[\begin{gather} \mu_{i,t}=\alpha_i+\gamma_i\Big(1-e^{-\frac{age_{i,t}}{\lambda_i}}\Big)+ A_i\text{sin}\Big(2\pi\times\text{age}_{i,t}+\Big[\phi+\frac{\pi}{6}(m_{i,t}-1)\Big]\Big) \end{gather}\]

Maximum Likelihood

The beauty of a separate data model is that you have flexibility to specify probability distributions that suit the data observation process (i.e. not just the normal distribution), e.g. Binomial coin flips, Poisson counts of individuals, Exponential waiting times, etc.

You can even specify custom data models with their own covariates, which is useful if you have information on things like instrument drift and calibration, etc - i.e. another opportunity for data fusion.

Single-level Bayes

When we use Bayesian models we now have the priors, which are essentially models describing our prior expectation for each of the parameters in the process model, like so:

Bayesian models include parameter models that describe our prior expectation for the parameters in the process.

Single-level Bayes

When implementing Bayesian models, we specify the priors as parameter models, e.g.:

\[\begin{gather} \alpha_i\sim\mathcal{N}(\mu_{\alpha},\frac{1}{\sqrt{\tau_{\alpha}}})\\ \gamma_i\sim\mathcal{N}(\mu_{\gamma},\frac{1}{\sqrt{\tau_{\gamma}}})\\ \lambda_i\sim\mathcal{N}(\mu_{\lambda},\frac{1}{\sqrt{\tau_{\lambda}}})\\ \end{gather}\]

Where (in this case) we’re saying we believe the three parameters from our simpler postfire recovery model are all sampled from normal distributions with independent¹ means and variances.

Single-level Bayes

Note that you need priors (i.e. parameter models) for all parameters. They don’t all have to be independent though, which can be useful, for example if you have multiple separate sets drawn from the same population, etc.

Single-level Bayes

When combining Multiple Likelihoods, there need to be links between all terms - e.g. two datasets (\(D_1\) and \(D_2\)) that share the same parameters (\(\theta\)):

\[ \underbrace{p(D_1|\theta)}_\text{likelihood 1} \;\; \underbrace{p(D_2|\theta)}_\text{likelihood 2} \;\;\underbrace{p(\theta)}_\text{prior} \; \]

or one dataset (\(D_1\)) conditional on another (\(D_2\)), that is conditional on parameters (\(\theta\)):

\[ \underbrace{p(D_1|D_2)}_\text{likelihood 1} \;\; \underbrace{p(D_2|\theta)}_\text{likelihood 2} \;\;\underbrace{p(\theta)}_\text{prior} \; \] etc.

Hierarchical Bayes

Hierarchical Bayes is a form of multilevel modelling that provides incredible flexibility for model specification through specifying the priors as models with inputs and specifying relationships among the priors or from one prior to multiple layers in the model.

• i.e. Hierarchical Bayesian models allow you to fuse data through the parameter model, but when you start treating your priors as parameters, you have to specify priors on your priors (hyperpriors) to your parameter model.

Hierarchical Bayesian models allow considerable flexibility through the inclusion of hyperparameters that can drive the priors.

Hierarchical Bayes

I won’t explore all the options (because they’re almost endless). In the example below, we used environmental covariates (soil, climate, topography, etc.) to explain the variation in the priors and constrain the parameters of the postfire recovery curve.

Until now we have only been considering the postfire recovery curve for a single location, but one can fit every MODIS satellite pixel in the Fynbos Biome in one model, like Wilson, Latimer, and Silander (2015) did, (including the seasonality term in the process model).

Hierarchical Bayes

Hierarchical Bayes

Hierarchical Bayes

Advantages of including the regression of the parameters on environmental covariates in this model include:

Data fusion, model complexity and uncertainty

Increasing model complexity provides opportunities for fusing new data into your model, but be aware that this comes with trade-offs.

Utility of the postfire model

The model does not make explicit near-term forecasts, but gives estimates of the expected NDVI signal for any location with known age since fire and time of year.

We’re using this to develop a near-real time satellite change detection system for Fynbos.

The Ecosystem Monitoring for Management Application (EMMA). See www.emma.eco for more.

EMMA workflow

EMMA

Managers can get an overview of change in the landscape every time new MODIS satellite data are collected (daily, or 16-day averages).

Overview map highlighting some of the major changes detected by Slingsby, Moncrieff, and Wilson (2020).