Last Updated on March 14, 2018, 9:34 AM | Published: March 14, 2018
A professional pollster says multiple serious problems plague the online survey being used by the Oklahoma City Public Schools Board of Education in their superintendent search.
And, the problems with the survey are so pronounced, the pollster calls it a “manipulation of research.”
But, the director of the Oklahoma State School Boards Association that generated the survey, argues that it is “just one of many tools” they will be using in the process.
The OSSBA is being employed to assist the board in their superintendent search.
Manipulation
Bill Shapard, founder of the research and polling company Sooner Poll, responded to questions from Free Press about the survey currently on the district’s website where anyone is invited to participate.
He called the survey a “manipulation of research” because it seems to be a survey and yet has multiple problems that cause the results to be unreliable as a survey.
“The reason why I say that is because they want to say that it is a survey. But, they are abusing the use of that word so that people feel like it’s a survey of say Oklahoma City.”
And there are so few controls on who is taking the survey it can be easily manipulated by about any group that decides to tilt the results toward a particular type of candidate for the job.
“There may be people from India taking the survey,” Shapard said about the ability for anyone, anywhere to take the survey as long as they have an internet connection.
Non-scientific
Shapard pointed out four reasons why the survey fits “a textbook definition of a non-scientific survey.”
• “It does not give everyone within a given population the equal opportunity to participate.”
• “It is a non-probability sample, meaning that participants are self-selecting.”
• “It does not control for everybody that does participate, to participate only once.”
• “It is not controlling for only those that should have a say in the survey.”
Problems
The survey had several structural problems between questions that would allow for “results that may contradict with each other.”
Citing one question on the survey where the participant was asked to rank 26 different traits from one to 26, Shapard said the survey could be “overwhelming for the average survey taker.”
At our request, Shapard tried out different parts of the survey and gave his assessment.
He observed that the survey did not separate those who have children in the district and those who do not.
And, it did not contain demographic questions that would help users of the survey to understand whose results they were viewing.
There is no way to tell whether the respondent is male or female.
“Since the population living in OKCPS boundary is heavily Hispanic, I would think there would be a question about race but there is not,” Shapard said.
He concluded that “the person who wrote it did not have any experience in survey instrument design.”
The survey is hosted by Survey Monkey, which seems to allow controls for measuring Internet Protocol or IP address, but software settings that allow users to mask their IP address are quite common.
OSSBA response
Free Press contacted Sean Hime, executive director for OSSBA.
He said that they are aware that it is not a scientific survey and that it is “just a way for staff and community members to give their feedback to the Oklahoma City board.”
“It’s just one of many different ways to give feedback,” said Hime. “It’s not the only way.”
Hime said they will use face to face discussions with groups of staff and others within the district to determine what they would like to see in the next superintendent.
Board chair responds
We asked Paula Lewis, board chair, what she thought about the survey after we told her about the problems the pollster found.
She said she thought the survey was “still useful” but was “not the only tool we are using for the search.”
“We won’t hire a superintendent just off of one survey,” she said.
Founder, publisher, and editor of Oklahoma City Free Press. Brett continues to contribute reports and photography to this site as he runs the business.
