Copyright Sociological Research Online, 1998

Elliott, C and Ellingworth, D (1998) 'The Practical Limitations of Survey Analysis: A brief response to Lynn'
Sociological Research Online, vol. 3, no. 2, <http://www.socresonline.org.uk/3/2/9.html>

Practical Limitations of Survey Analysis

1.1

In Elliott and Ellingworth (1997) an attempt was made to assess the representativenss of the 1992 British Crime Survey (BCS). In addition, statistical methods were employed to examine whether factors influencing area level response rates and the incidence of property crime could be identified. The authors also tested whether a relationship between crime rates and survey response rates could be identified.

1.2

We have been encouraged to find that our discussion of these issues has provoked such a useful and informative response by Lynn (1998). In particular, Lynn's knowledge and experience of being involved in the methodological design and implementation of the British Crime Survey sample have clearly provided him with privileged information and data with which he could reply to the issues we had raised. At the core of this additional knowledge is the ability of Lynn to distinguish between non-response and non- eligibility rates for the primary sampling units in the survey.

1.3

While we accept many of the comments made by Lynn, we feel it necessary to point out that shortcomings of our original work reflected the survey information made available to the 'typical' survey analyst. As we point out, the response rate that we calculated is the proportion of expected interviews carried out within each area (postcode sector). We have always been aware that the response rate is composed of two components: namely, the rate of eligibility (the proportion of addresses that are within the scope of the survey) and the rate of response (the proportion of eligible households that do respond). However, the only information available to us (as 'typical' users) are the data concerning the number of interviews achieved in each postcode sector, and the number of interviews that the BCS attempted in each of these postcode sectors. The reasons as to why households do not respond are not available to us. Consequently, we were unable to distinguish between potential types of non-response that were identified in Elliott and Ellingworth (1997). Lynn discusses alternative assessments of non-response bias: these rely on the additional information available to him, but not the academic survey analyst or user.

1.4

The work that Lynn carries out does inform analysis, and highlights issues that all users of the data should bear in mind. We acknowledge the great depth with which Lynn discusses the issues of coverage and sampling error. The particular criminological implications of response biases in high crime rate areas might usefully also be addressed.

1.5

By way of a plea to the producers of the BCS data, we would like to suggest that more of the kind of analysis discussed by Lynn (1988) be included in technical reports, rather than in the form of unpublished papers. It is noted that a preliminary look at the 1996 British Crime Survey technical report includes the capacity to record limited descriptive information about non-responding households. It is hoped, firstly, that this information will be made available with the main data files. Secondly, analysis of this additional data could highlight response related implications for analysis, and perhaps suggest strategies for improving response.

References

ELLIOTT, C. and ELLINGWORTH, D. (1997) 'Assessing the Representativeness of the 1992 British Crime Survey: the Impact of Sampling Error and Response Biases', Sociological Research Online, vol.2, no.4, <http://www.socresonline.org.uk/2/4/3.html>.

LYNN, P. (1998) 'The British Crime Survey Sample: A response to Elliott and Ellingworth', Sociological Research Online, vol.3, no.1, <http://www.socresonline.org.uk/3/1/12.html>.

Copyright Sociological Research Online, 1998