Last night I was invited to examine the methodology of surveys conducted by the John Locke Foundation. Having been in the market research business for more than 20 years, I consider myself at least partially qualified for the task.
To those not familiar with the industry, there is as much art as science involved. For example, knowing how to ask questions to get unbiased responses is one aspect of the art – assuming of course that you actually want unbiased responses. This is especially challenging for researchers who come to their projects with ideological biases and agendas. The tobacco industry, for example, is widely known for its abuse and misuse of all kinds of research.
The science half of the equation is equally fraught with challenges. For example, in today’s busy world, it’s getting harder and much more expensive to do research well. Telephone response rates have plummeted in recent years as fewer people are willing to even pick up the phone, let alone sit through a survey that can take 30 or 40 minutes. As a result, many research firms simply don’t do the number of call-backs necessary to obtain representative samples. Nor do they build internal reliability into their surveys by getting at the same question in several different ways.
Which brings us to the survey research company that does polling for the John Locke Foundation. The company is called Tel Opinion Research.
In 1997, a loose-knit group of professionals agreed to serve as members of the Board of Directors of an organization called "The National Association of Republican Campaign Professionals." This organization allows us to share ideas about fundraising, survey research, media and other vital parts of campaigns. And we can come together in a social setting with people who share one common goal of electing more Republicans at all levels.
And then there’s the president of the company, who has this to say about himself:
Bill Lee has been professionally involved in political efforts and campaigns for more than twenty-five years in more than half of the United States, Central America and Africa. He is an acknowledged expert in campaign planning and strategy and has taught these and other subjects in virtually every national conservative or Republican campaign school, for a variety of professional associations and as a guest lecturer at the Harvard Institute of Politics and American University. Bill Lee co-founded and serves as President of Tel Opinion Research, a political and commercial survey research firm. Lee has faced Bill Clinton in three gubernatorial contests, winning one, and is the only consultant to have ever defeated him.
The John Locke Foundation and Tel Opinion Research have every right to do whatever they want, of course. That’s the great thing about America. But any policymaker who comes across this kind of polling should take it for what it is. Survey research done on the cheap by proudly partisan hacks.
PS Just to be clear, I am not saying this research does not reflect the real opinions of those surveyed. I'm just saying we have no way of knowing whether that's the case - and good reason to suspect ideological bias.