link to propertyforecasts.co.uk homepage spacer spacer

home | products | about us | news & press | help | feedback | member login

house photo
free reports for
search
Browser Information

Historic accuracy of forecasts  June 2005 - 2008

Obtaining a forecast is all very well – but how accurate is it? Can one rely on it?

We at www.PropertyForecasts.co.uk are always trying to ensure that, when you buy a forecast from us, you will be buying one that is as reliable as current technology can make it. Where we cannot justify a forecast mathematically, we will not provide one.  Please note:  this year, we are introducing three-year accuracy checks, to replace our earlier two-year values - a much more difficult proposition!

Annual forecasting accuracy checks

Annually, we carry out a check on our historical forecasting accuracy performance, across the vast majority of the 8000 postcode sectors in England & Wales, each on average covering around 2000 dwellings of up to four property types: detached, semi-detached and terraced houses, and flats & maisonettes. All series used in our forecasts (including the background series) are cut back two years, and then forecast to the current period. The results obtained from the two-year forecasts are then compared to the latest smoothed Land Registry values, and those historic forecasts graded according to their accuracy.

Of course, we can never guarantee the future accuracy of any specific forecast. However, where relationships between historic series are strong, the forecast tends to continue to be good while that relationship continues. This implies that, where a forecast has been accurate in the past, that accuracy tends to continue for a while.

The actual calculation for assessing accuracy of a forecast is highly complex, as it takes into account not only the accuracy over two years, but also the match over the two-year period between the forecast and actual series, and the fluctuations in the two series over the forecast period.

For example, the “accurate” forecast in the diagram on the left is not promising a useful forecast for the future. After all, it is not very helpful to be spot-on accurate at the end of two years, if over the course of the two years there has been no match of any kind between forecast and actual! Three months earlier, and probably also three months later, there is a wide divergence in values.
On the other hand, in the diagram on the right, there is a difference between the forecast value, and the actual value achieved at the end of the forecast period. But overall, the match in shape of the forecast curve with the actual values over the period is a reasonable confidence-booster that the forecast over the next period will be much closer to what is actually most likely to happen, than the exact match in the diagram above.

 

back to previous


Historic forecast accuracy definitions

In summary, the measure of forecast accuracy used by Technical Forecasts Ltd, and adopted on www.PropertyForecasts.co.uk combines:

  • a measure of how accurately the actual and forecast behaviours match throughout the forecast period (the root mean square (RMS) error)
  • the comparative shapes of the curves (volatility; since the volatility of a data series can render the actual accuracy of a forecast at one specific time period as a totally misleading indicator of the overall accuracy of a forecast data set), and
  • the ratio between the two (RMS/volatility), measuring accuracy with respect to the volatility over the period.
  • There is no consideration of final accuracy value in the accuracy classification, as the final point of comparison at the end of the two years is essentially arbitrary, therefore has no place in the assessment.

The result of this is a grading system of historic forecast accuracy of A / B / C / U, where A historically has given the ‘best’ measure of forecast accuracy at any specific point. A ‘U’ classification does not necessarily imply a poor forecast (although historically it has either been inconsistent, poor, or non-existent due to lack of data). An additional classification of ‘u’ has been introduced, to identify those series with insufficient data history to allow a comparative forecast to be made. We cannot assess the historic accuracy of these series.

Always exercise caution in the use of forecasts for financial planning – always consult a professional adviser. Remember that past performance can never guarantee future performance.
Out of interest, the forecast in the first diagram above would probably be classed as ‘U’, whereas the second would probably be classed as 'A' or ‘B’.

Historic accuracy figures summary, neighbourhoods ( tested Sept 2008 )

Comparisons are between actual forecasts we made in Aug 2005, from Land Registry data to June 2005, and filtered Land Registry data up to the end of second quarter 2008. 

  • For semi-detached and terraced houses, 90% or more of historic forecasts are classified A or B over the three-year period.  For detached houses, this has dropped a little to 85%, while for flats & maisonettes, nearly 78% are classified A or B.  For houses, around 45% are classified as A (nearly 65% for terraced), while for flats and maisonettes this figure stands at over 37%.
  • Around 90% of A forecasts are within 15% of the smoothed Land Registry values at June 08 (after three years’ forecast), with 42-45% within 5%.
  • Around three-quarters of A and B forecasts are within 15%, and 67-76% of all forecasts (including U) for houses are within 15% at the end of three years. For flats & maisonettes, this figure has dropped slightly, to nearly 60% of all forecasts within 15%.
  • Less than 1.5% of all house price forecasts are classified as U, while for flats (the most difficult to forecast, since most series are quite short) this has risen to nealy 6%.   Please remember that ‘u’ forecasts cannot be assessed for accuracy, since there is usually insufficient data remaining when three years’ worth of history has been removed.
  • 'u' forecasts - those which cannot be assessed for accuracy once three years' worth of data has been removed, due to short or erratic data series - account for less than 8% of all houses, but around 15% of flats.
  • Around 96% of all forecasts are in the same overall direction as the smoothed Land Registry data. For A category forecasts, this has increased to over 99%.  Average correlation - the measure of how closely the forecast series matches the actual data - has dropped markedly over the last year, now standing at 0.65-0.69 for all properties, reflecting the effect of the 'credit crunch' on house prices since a year ago.
  • Average error at the end of three years for A classified forecasts is under +4% for all property types (standard deviation SD around 8%).  Error for A and B classifications together for detached houses and flats is around +5%, while for semis and terraced houses error is around +3.5%. This year, errors are predominantly positive, implying that growths over the three years have been over-estimated - again, a likely reflection of the reductions seen during the credit crunch.  SDs have also increased slightly, to around 12% for houses and 13.5% for flats, for the same likely reason.
  • Overall average error was around +4% for semis and terraced houses, with a SD of around 15%, over +7% for detached (SD over 16%), and +9.6% for flats & maisonettes, with a SD of around 24%.

A ‘-’ sign in error values shows underestimates while positive values show over-estimates.

In comparison with last year (where we were showing only 2-year historic accuracy data), average correlations in A and B data series are significantly higher, but the proportion of A and B forecasts within the 15% error band is significantly lower this year than last (around 75%, as opposed to over 80% previously).  Perhaps this indicates that we were right in the shape and direction of many of our forecasts from three years ago, but not quite as close as we might have been in real terms!

There has been a significant drop in accuracy of forecasts for flats overall, indicating that the effects of the credit crunch are probably felt more profoundly in the market for flats and maisonettes than that for houses.


back to previous

Historic accuracy figures summary, Towns & Cities (tested Sept 2008)

As with the Neighbourhoods, comparisons are between forecasts we made in Aug 2005, from Land Registry data to June 2005, and filtered Land Registry data up to the end of second quarter 2008.  Averages for Towns & Cities are calculated from postcode sector (neighbourhood) forecasts, using (mainly) the Royal Mail definitions for towns and averaging relevant sector values .

  • Overall for houses, around 91-96% of three-year historic forecasts are classified A or B;  64-70% are classified as A, slightly down from last year’s 70% or so for two-year forecasts.
  • Around 90 % of A forecasts for houses are within 15% of the filtered Land Registry values at June 2008 (after three years’ forecast), with around half within 5%.
  • For houses, 78-84% of A and B forecasts are within 15%, and 73-81% of all forecasts (including U) are within 15% at the end of three years. For flats & maisonettes, this figure has fallen somewhat to 73% of A and B forecasts within 15%, and two thirds of all forecasts.
  • Less than 1% of all house price forecasts are classified as U;  for flats & maisonettes, around 2% are classified U.
  • Around 98% of all forecasts are in the same overall direction as the smoothed Land Registry data. For A category forecasts, this is over 99%.
  • Average error at the end of three years for A classified forecasts is +3% for semis and terraced houses, and+5% for detached houses (standard deviation SD of 7.5-8%), while that for A and B classifications together varies between +2% and +6% (SD around 11%). For flats & maisonettes, average error for ‘A’ and ‘B’ classifications was +6.2%, with an SD of 12.4%.
  • Overall average error was between +2.5 and +7.3% for houses (SD 12-13.7%), and +8.4% for flats & maisonettes, with a somewhat higher SD of around 17%.
Once again , a ‘-’ sign against an error value would show an underestimate, while positive values show over-estimates.  The surprising factor this year is that average correlations over the last three years - which for neighbourhoods are somewhat worse than previously - are significantly improved for towns, at around 0.75 overall.

back to previous

Examples of A, B, C and U category forecasts

sample of 'A' category forecast
The blue line of the forecast follows both the trend and the value of the pink line showing the actual (smoothed) Land Registry values. You could have confidence that this trend is likely to continue for a while.
sample of 'B' category forecast
The blue line of the forecast follows the overall trend of the actual value, but sllightly displaced. You could have reasonable confidence that this trend is likely to continue for a time.
sample of 'C' category forecast
The blue forecast line reflects some of the erratic nature of the pink line of actual (smoothed) Land Registry values, but under-estimated the rising trend. You can have some confidence that the forecast is likely to be in the same direction, that values may be similar at times, and that an erratic forecast may well be reflected in erratic local values.
sample of 'U' category forecast
Here we have two samples of a ‘U’ category forecast, illustrating that, in spite of the classification, the result is not necessarily always poor. With an erratic data series in particular, it may be that the zigs of one do not match the zags of the other, so this could well result in a ‘U’ classification.
 
second sample of 'U' category forecast

Historic forecast accuracy over three years – detailed results for neighbourhoods (postcode sectors)

Detached houses:
'A'
'B'
'C'
'U'
ALL
Proportion of dataset (%)
42.6%
43.1%
12.9%
1.4%
100%
Average error
4.0%
6.7%
16.6%
40.0%
7.3%
Standard Deviation of error
8.3%
15.5%
91.0%
54.6%
16.4%
Proportion positively correlated
99.5%
96.2%
73.9%
81.2%
96.7%
Average correlation
0.75
0.62
0.53
0.39
0.65
Within 5% after 3 years
42.4%
21.5%
9.8%
0%
28.3%
Within 10% after 3 years
71.8%
42.7%
19.0%
1.2%
51.2%
Within 15% after 3 years
89.7%
60.0%
28.7%
5.9%

67.5%

Semi-Detached houses:
'A'
'B'
'C'
'U'
ALL
Proportion of dataset (%)
46.7%
43.3%
9.2%
0.7%
100%
Average error
2.7%
3.9%
9.9%
47.3%
4.2%
Standard Deviation of error
8.0%
16.1%
27.9%
62.3%
15.6%
Proportion positively correlated
99.1%
96.9%
92.3%
80.8%
97.4%
Average correlation
0.75
0.63
0.53
0.39
0.67
Within 5% after 3 years
45.3%
20.8%
9.8%
5.8%
31.1%
Within 10% after 3 years
73.5%
41.3%
19.0%
7.7%
55.4%
Within 15% after 3 years
91.8%
61.2%
28.7%
9.6%
72.2%
Terraced houses:
'A'
'B'
'C'
'U'
ALL
Proportion of dataset (%)
64.2%
30.6%
4.7%
0.5%
100%
Average error
4.0%
3.5%
4.8%
48.4%
3.9%
Standard Deviation of error
8.0%
17.0%
32.1%
42.9%
14.1%
Proportion positively correlated
99.1%
92.5%
80.0%
63.6%
96.0%
Average correlation
0.75
0.53
0.29
0.23
0.66
Within 5% after 3 years
43.3%
19.1%
4.2%
0%
33.8%
Within 10% after 3 years
73.5%
33.2%
7.4%
0%
57.6%
Within 15% after 3 years
91.8%
53.9%

16.8%

9.1%
76.2%
Flats & Maisonettes:
'A'
'B'
'C'
'U'
ALL
Proportion of dataset (%)

37.6%

40.1%
16.5%
5.8%
100%
Average error
3.7%
6.1%
18.4%
47.9%
9.6%
Standard Deviation of error
8.4%
16.7%
29.0%
54.9%
23.8%
Proportion positively correlated
99.1%
96.9%
90.7%
85.7%
96.0%
Average correlation
0.80
0.67
0.56
0.48
0.69
Within 5% after 3 years
42.1%
18.3%
5.2%
0.7%
24.1%
Within 10% after 3 years
72.5%
37.7%
12.9%
1.1%
44.6%
Within 15% after 3 years
89.1%
56.8%
20.5%
1.4%
59.8%

back to previous

Historic forecast accuracy – detailed results for Towns & Cities

Detached houses:
'A'
'B'
'C'
'U'
ALL
Proportion of dataset (%)
64.1%
27.4%
7.9%
0.7%
100%
Average error
5.2%
8.6%
17.4%
36.5%
7.3%
Standard Deviation of error
7.6%
15.1%
26.4%
49.2%
13.7%
Proportion positively correlated
100%
96.8%
89.0%
66.7%
98.0%
Average correlation
0.81
0.62
0.46
0.45
0.73
Within 5% after 3 years
43.5%
18.3%
11.0%
16.7%
33.8%
Within 10% after 3 years
70.8%
36.9%
17.8%
16.7%
57.0%
Within 15% after 3 years
88.1%
54.4%
24.7%
16.7%
73.4%
Semi-Detached houses:
'A'
'B'
'C'
'U'
ALL
Proportion of dataset (%)
70.6%
25.6%
3.7%
0.1%
100%
Average error
3.1%
6.8%
3.3%
59.6%
4.1%
Standard Deviation of error
7.4%
17.2%
25.1%
0%
12.0%
Proportion positively correlated
99.7%
97.0%
94.1%
100.0%
98.8%
Average correlation
0.82
0.66
0.51
0.58
0.77
Within 5% after 3 years
51.1%
15.3%
8.8%
0%
40.3%
Within 10% after 3 years
79.0%
33.6%
29.4%
0%
65.5%
Within 15% after 3 years
92.8%
56.2%
47.1%
0%
81.6%
Terraced houses:
'A'
'B'
'C'
'U'
ALL
Proportion of dataset (%)
65.2%
29.0%
5.4%

0.3%

100%
Average error
2.8%
0.6%
8.4%
6.0%
2.5%
Standard Deviation of error
7.4%
15.2%
32.8%
53.0%
13.2%
Proportion positively correlated
99.7%
97.4%
88.0%
33.3%
98.2%
Average correlation
0.82
0.64
0.40
0.05
0.74
Within 5% after 3 years
49.0%
18.4%
12.0%
33.3%
38.0%
Within 10% after 3 years
81.5%
41.2%
16.0%
33.3%
66.1%
Within 15% after 3 years
93.8%
62.6%
28.0%
33.3%
81.0%
Flats & Maisonettes:
'A'
'B'
'C'
'U'
ALL
Proportion of dataset (%)
55.6%
32.8%
9.4%

2.2%

100%
Average error
4.1%
9.9%
21.5%
39.3%
8.4%
Standard Deviation of error
8.1%
16.7%
26.8%
44.2%
16.9%
Proportion positively correlated
99.5%
97.4%
95.4%
93.3%
98.3%
Average correlation
0.85
0.70
0.65
0.63
0.78
Within 5% after 3 years
44.4%
13.3%
1.5%
0%
29.2%
Within 10% after 3 years
71.5%
31.4%
6.2%
6.67%
50.8%
Within 15% after 3 years
89.3%
46.0%
18.5%
6.67%
66.6%

back to previous

house image