Kalman filter is a Bayesian method to update and improve parameter estimates from observations with uncertainty. It can be applied to time series formulated as a Linear Gaussian State Space Model. The kalman filter method was used by NASA in the Apollo Project to estimate trajectories of the manned spacecraft landing on the Moon and back (wow!).
In this and the next post, I will apply Kalman filter to two time series datasets: (1) simulated GPS locations (in honour of the Apollo 11 Lunar Module), and (2) a power market electricity price dataset.
Problem Formulation
Assumption: all noises are Gaussian, in which case the Kalman filter minimises the mean square error of estimated parameters. If the noises are not Gaussian, given only the mean and standard deviation, the Kalman filter is the best linear estimator.
The State Space Model, with observations
where
Using conclusions from the marginal and conditional distributions of multivariate Gaussians, and let
- At time 0, initial guess is
- At time t,
- Prediction updates:
- Measurement updates:
Simulated Race Car Location Data
Suppose a race car is on a track with the following equations:
Suppose unfortunately the car jerks when accelerating (such that the acceleration is not constant), we consider the third order derivatives of the car’s position as Gaussian variables
By the Newton’s law of motion,
The dynamics are similar for dimension
Let the car carry a GPS device to observe its current physical position, with measurement error following
Applying Newton dynamics to Kalman Filter, the state and observation vectors contain up to second order derivatives of the position (note
The process error matrix covariance
where
The GPS observes hidden states directly, so
and the measurement error covariance matrix
At time 0, suppose the car starts at the rightmost race track location (2,0) with known velocity and acceleration, and the hidden state covariance is a relatively large value due to uncertainty.
Here’re the results. The Kalman Filter algorithm updates the car’s predicted position as each observation comes in, which effectiveliy reduces noises and smoothes the raw GPS observations.