Engineering Thinking: "PolitiFact Earns 'Pants On Fire' Rating"

Blogger Ed Walker of "Engineering Thinking," produced a list of deficiencies at PolitiFact likely to lead to biased findings.  After presenting the list, Walker drops the hammer:

PolitiFact scores a big fat zero, ranking it among sites devoted to UFOs, ghosts, psychic phenomena, and other organizations that dabble in pseudoscience.

This does not mean that PolitFact is completely biased or always wrong. It does mean that they have no published scientific standards, so it is not possible to evaluate their work. Without such standards, evaluation criteria may shift from issue to issue, perhaps allowing them to indulge in subtle favoritism toward people or issues they like, while awarding “pants on fire” ratings to those they don’t.

Obviously Walker is evaluating PolitiFact on some level, so when he says it is not possible to evaluate their work, he seems to be saying that PolitiFact's system (the continuum between "Pants on Fire" and "True") does not feature criteria adequate for separating one grade from another, as with the "ridiculous" criterion noted here, as though there is an objective determination of "ridiculous."

Walker accurately indicts PolitiFact on the issue of shifting standards--PolitiFact certainly does vary in its approach to fact checking, as shown by results such as finding it simultaneously "True" that Joe Biden did not advocate partitioning Iraq while also finding it "Half True" that Joe Biden advocated partitioning Iraq.

Walker's short post makes some terrific points, so please read it all.