How to evaluate the presidential model's performance
Probabilistic projections such as Plano Político's presidential model should not be read as a guaranteed forecast of the election result. On the contrary, the purpose of a statistical model like this one is to quantify the uncertainty that exists around the outcome, informed by the record of past elections. If the model is perfectly calibrated, a candidate given a 10% chance will win one election in ten.
Presidential elections, however, are rare events: they happen only once every four years, each with a different cast of candidates and a different political context. It would be easy to artificially inflate the model's uncertainty so that it is "never wrong", but that would undermine its analytical usefulness. On the other hand, an overconfident model creates the opposite problem: mistaking a temporary trend for inevitability.
We are twenty days from the first round of the presidential election. A great deal can happen in twenty days, and the electorate reacts increasingly quickly the closer it gets to polling day: in 2014, for instance, at this same stage of the campaign, Marina was favourite to face Dilma in the runoff; in the final week Aécio overtook her and finished the first round twelve percentage points ahead.
A probabilistic model cannot predict the exact result; instead, it maps out those possibilities so that they are on the radar of anyone following the election. The best way to evaluate the model, then, once we know the result on 4 October, is to be guided by the following points:
- Were each candidate's individual results within the model's confidence intervals?
- If so, how far were they from the median projection?
- Did the direction the model pointed in over its final days indicate a greater or a lesser likelihood of the eventual result?
- Finally, which terms in the model were decisive in landing closer to — or further from — the final result?
In the days following the election, Plano Político will publish an analysis of the presidential model's results in light of these questions, pointing out what it got right and wrong and, should there be a runoff, an explanation of how the model will work in the race's second phase. The reason is simple: although it is technically a "new election", the runoff takes place in the shadow of the first round's results, which allow the two remaining candidates' probabilities to be read with far greater clarity.