APRIQResearch

The forecast scorecard

We publish rent forecasts, and we grade them. Then we grade everyone else’s, by the same code and the same rule. Our own record comes first because a scorecard you are not on is not accountability.

Our record

Every forecast this method could have made, judged against what actually happened, using only what was known at the time. Skill is the improvement over assuming nothing changes — a market that only rises makes “rents will rise” correct and worthless, so being right is not the test.

Horizon Forecasts graded Landed in range Median width Mean score Skill
1 quarter ahead 10,476 83.8% ±7.9% 0.258 1.7%
2 quarters ahead 10,088 83.2% ±8.8% 0.277 6.0%
4 quarters ahead 9,312 83.4% ±11.3% 0.322 14.3%

Lower scores are better. We publish no forecast beyond four quarters: at eight the method failed its own gate, and a wider band would not have fixed that.

Why a wide range earns nothing

Two deliberate frauds are graded beside us as a check that the scoring is not flattering anyone. They found a real fault: under a simple right-or-wrong rule, the forecaster whose range cannot miss scored a perfect record. Scoring is now the width of a claim plus a penalty for falling outside it, against a baseline carrying the same width — so widening a range buys exactly nothing.

Forecaster Landed in range Median width Mean score Skill
always up 5%a year ahead 0.8% ±0.0% 0.647 21.7%
wide enough to never missa year ahead 99.9% ±45.5% 0.940 0.0%

The hedger lands inside its own range almost every time and demonstrates nothing. The weather vane never lands inside, because it names a single number — and still beats assuming nothing changes, because rents genuinely rose. Both facts are published.

Everyone else

Nothing is graded here yet. The register is open and empty, which is the honest state: entries are added by hand, reviewed as a change to a file, and each one has to clear a bar that most published commentary does not.

A claim can only be graded if it names four things:

  1. A metric we publish
  2. A geography we cover — rents in New South Wales, Queensland, Victoria and South Australia; sale prices in New South Wales and South Australia only
  3. A period that ends, so it can be settled
  4. A number, not a direction

“Sydney rents will rise strongly” cannot be scored, and recording it as though it could would be putting words in someone’s mouth. Every entry carries the forecaster’s own words, a link to the archived source and its hash, and the reading we took of them — so a forecaster who says we read them wrong is making a correction we publish, not an argument we have to win.

Method

Forecasts are the market’s growth over the preceding year applied to each area’s own observed level. An area’s own recent momentum was measured and rejected: it loses to assuming nothing changes at every horizon tested, while the market’s beats it at all of them. Ranges are the measured spread of this method’s own misses over 10,476 walk-forward cases, not a model’s confidence. No single number is published, because a single number gets quoted without whatever sat beside it. More research · Sources and licences.