by David Hulchanski, Professor, Principal Investigator, Neighbourhood Change Research Partnership, University of Toronto, 18 April 2014
It now seems that the 2016 census will be similar to 2011 when the traditional mandatory census long form was replaced by the Harper government with a voluntary survey, the National Household Survey (NHS).
The conclusion of a recent op-ed by four of us who extensively use census data:
“The income data in the National Household Survey is not valid. It should not be used or cited. It should be withdrawn. The 2016 census should be restored to the non-politicized, non-partisan scientific methodology that existed prior to the flawed 2011 National Household Survey.”
The voluntary nature of the NHS was controversial from the start due to one overarching question: Can a voluntary survey ever substitute for a mandatory census? As the head of Statistics Canada, Munir Sheikh, who was appointed to that position in 2008 by Mr. Harper, answered with his resignation in June 2010: “It can not.”
This question was not asked or directly addressed in any fashion in a Statistics Canada study released in 2012, Final Report on 2016 Census Options: Proposed Content Determination Framework and Methodology Options. The Statistics Canada authors mention the word “voluntary” 47 times and “mandatory” 45 times in the 33,000 word 75 page study. The objective “was to study options and deliver a recommendation to the federal government on the methodology of the 2016 Census Program.”
The report skates around the central issue for the 2016 Census, whether the long form would be mandatory or voluntary.
An option is to have no census at all and to rely on existing administrative and other data was not recommended because other existing data sources in Canada are not extensive enough to do away with the census. “The only viable approach for 2016 is a traditional Census Program.”
The “traditional” census program for many decades has been the mandatory 20% sample for the long form.
The report only hints at a number of fatal flaws in the NHS. For example, the problem of response rates:
“The move from the mandatory census long form in 2006 to the voluntary NHS in 2011 was expected to significantly impact the response rates. With lower response rates comes the risk of increasing the non-response bias. p. 23
The report also confirms that no studies were carried out prior to the switch to a voluntary survey:
“No tests to predict these impacts had ever been done on a survey in Canada that has both this magnitude and self-enumeration as the main collection mode.” p.23
We are told that more people who responded to the NHS did not answer as many of the questions as was the case with the mandatory long form:
“For households that chose to respond to the NHS, preliminary analysis of response rates to NHS questions seems to show that rates for the first modules of the NHS questionnaire up to the education module are not very different from the rates for the same modules in the 2006 long form. However, the differences are more important starting with the labour market activity module.” p.28
Why not? The NHS, even though more expensive than the traditional census, had no follow-up mechanism:
“The main reasons of the differences compared to 2006 could have to do with the absence of a follow-up for partial non-response to NHS questions and to the voluntary nature of the questionnaire, which may have had an impact on the perseverance of respondents.” P.28
Serious, actually fatal, problems with non-response rates are occasionally mentioned in passing:
“Of concern as well is that cooperation with a voluntary survey may be unevenly spread within the population, leading to non-response bias that may be difficult to fully correct. Further information on this will only become available as the NHS data are processed and analyzed.” P.45
and
“Statistics Canada must consider, however, the implications of collecting information on a voluntary basis rather than a mandatory basis, particularly for estimates for low geographic areas and small population groups, since the former does not achieve the same type of cooperation from the public.” P.47
But to this day, as in so many areas of federal scientific research, professionals are either silenced or write reports that are so heavily edited that they read like Conservative Party of Canada position papers.
The end of reliable detailed census data fits nicely with the transition from a society with a market mechanism to a market society. One of the more common quotes from Michael Sandel’s recent book, What Money Can’t Buy: The Moral Limits of Markets,
“The most fateful change that unfolded in the last three decades was not an increase in greed. It was the expansion of markets, and of market values, into spheres of life where they don’t belong.”
In a market society, where only market transactions matter, large private corporations can collect the information they need internally and privately. There is little need for public provision of information.
For some who believe they already have all the answers, facts don’t matter. They get in the way. In addition to the desire to starve government of funds, there are many today who want to starve voters of information.