| Case | Forecast | Observation | Error | Error2 |
| 1 | 7 | 6 | 1 | 1 |
| 2 | 10 | 10 | 0 | 0 |
| 3 | 12 | 14 | -2 | 4 |
| 4 | 10 | 16 | -6 | 36 |
| 5 | 10 | 7 | 3 | 9 |
| 6 | 8 | 5 | 3 | 9 |
| 7 | 7 | 5 | 2 | 4 |
| 8 | 8 | 13 | -5 | 25 |
| 9 | 11 | 12 | -1 | 1 |
| 10 | 13 | 13 | 0 | 0 |
| 11 | 10 | 8 | 2 | 4 |
| 12 | 8 | 5 | 3 | 9 |
| SUM | 114 | 114 | 0 | 102 |
To calculate the Bias one simply adds up all of the forecasts and all of the observations seperately. We can see from the above table that the sum of all forecasts is 114, as is the observations. Hence the average is 114/12 or 9.5. The 3rd column sums up the errors and because the two values average the same there is no overall bias.
However it is wrong to say that there is no bias in this data set. If one was to consider all the forecasts when the observations were below average, ie. cases 1,5,6,7,11 and 12 they would find that the sum of the forecasts is 1+3+3+2+2+3 = 14 higher than the observations. Similarly, when the observations were above the average the forecasts sum 14 lower than the observations. Hence there is a "conditional" bias that indicates these forecasts are tending to be too close to the average and there is a failure to pick the more extreme events. This would be more clearly evident in a scatter plot.
To calculate the RMSE (root mean square error) one first calculates the error for each event, and then squares the value as given in column 4. Each of these values is then summed. In this case we have the value 102. Note that the 5 and 6 degree errors contribute 61 towards this value. Hence the RMSE is 'heavy' on larger errors. To compute the RMSE one divides this number by the number of forecasts (here we have 12) to give 9.33... and then take the square root of the value to finally come up with 3.055.
Y = -3.707 + 1.390 * X RMSE = 3.055 BIAS = 0.000
(1:1)
O 16 + . . . . . x . . . . . +
|
b | . . . . . + .
|
s 14 + . . . . . . . x . + . .
|
e | . x . . x . .
|
r 12 + . . . . . . x + . . . .
|
v | . . . + . . .
|
a 10 + . . . . . x . . . . . .
|
t | . . + . . . .
|
i 8 + . . . + . x . . . . . .
|
o | . + . x . . .
|
n 6 + . + x . . . . . . . . .
|
| + . x x . . . .
|
4 +-------+-------+-------+-------+-------+-------+
4 6 8 10 12 15 16
F o r e c a s t
| Case | Forecast | Observation | Error | Error2 |
| 1 | 9 | 7 | 2 | 4 |
| 2 | 8 | 5 | 3 | 9 |
| 3 | 10 | 9 | 1 | 1 |
| 4 | 12 | 12 | 0 | 0 |
| 5 | 13 | 11 | 2 | 4 |
| 6 | 9 | 10 | -1 | 1 |
| 7 | 9 | 7 | 2 | 4 |
| 8 | 9 | 6 | 3 | 9 |
| 9 | 12 | 9 | 3 | 9 |
| 10 | 14 | 13 | 1 | 1 |
| 11 | 9 | 5 | 4 | 16 |
| 12 | 8 | 8 | 0 | 0 |
| SUM | 122 | 102 | 20 | 58 |
In this case the sum of the 12 forecasts comes to 122, which is 20 higher than the sum of the observations. Hence the forecasts are biased 20/12 = 1.67 degrees too high. Of the 12 forecasts only 1 (case 6) had a forecast lower than the observation, so one can see that there is some underlying reason causing the forecasts to be high which hasn't been properly addressed. A good verification procedure should highlight this and stop it from continuing. The bias is clearly evident if you look at the scatter plot below where there is only one point that lies above the diagonal.
There are no really large errors in this case, the highest being the 4 degree error in case 11. Consequently the tally of the squares of the errors only amounts to 58, leading to an RMSE of 2.20 which is not that much higher than the bias of 1.67. This implies that a significant part of the error in the forecasts are due solely to the persistent bias. If in hindsight, the forecasters had subtracted 2 from every forecast, then the sum of the squares of the errors would have reduced to 26 giving an RMSE of 1.47, a very respectable result. Hence to minimise the RMSE it is imperative that the biases be reduced to as little as possible.
Y = -2.409 + 1.073 * X RMSE = 2.220 BIAS = 1.667
(1:1)
O 16 + . . . . . . . . . . . +
|
b | . . . . . + .
|
s 14 + . . . . . . . . . + . .
|
e | . . . . + x .
|
r 12 + . . . . . . . x . . . .
|
v | . . . + . x . .
|
a 10 + . . . . x + . . . . . .
|
t | . . + x x . .
|
i 8 + . . . x . . . . . . . .
|
o | . + . x . . . .
|
n 6 + . + . . x . . . . . . .
|
| + . x x . . . .
|
4 +-------+-------+-------+-------+-------+-------+
4 6 8 10 12 15 16
F o r e c a s t