Copyright Sociological Research Online, 1997

Elliott, C. and Ellingworth, D. (1997) 'Assessing the Representativeness of the 1992 British Crime Survey: The Impact of Sampling Error and Response Biases'
Sociological Research Online, vol. 2, no. 4, <http://www.socresonline.org.uk/2/4/3.html>

Abstract

The paper highlights the importance of the representativeness of survey samples, using the 1992 British Crime Survey as an example. The success with which different demographic characteristics are represented in the survey sample is addressed by comparison to the 1991 Census Small Area Statistics for England and Wales. In addition, biases associated with different response rates in different areas are addressed, and given the nature of the survey, the impact of an area's crime rate on its response rate is also analysed. Finally, regression modelling is used to identify whether the same variables have explanatory power in explaining differences in crime rate and response rate.

Keywords:

Bias; British Crime Survey; Census; Deciles; Sampling; Regression Modelling; Representativeness; Response Rates

Introduction

1.1

When analysing survey data sources, the issue of how representative the sample should be at the forefront of the researcher's mind when looking to generalise findings to the general population. Although the representativeness of a survey sample is affected by many different influences (for example, the adequacy of the sampling frame, the effectiveness of the sampling strategy etc.) a vital factor that is frequently ignored is the response rate.

1.2

It is survey methodologists' lore that a higher response rate is better, but other than a general aim to ensure as high a response as possible, scant attention is paid to patterned response rates. If, for example a response rate is 70% across the sample, this suggests that on average each household has a likelihood of responding to the survey of 0.7. If, however, the scenario is one whereby the response rate shows a high degree of variability across households, the randomness of the sampling strategy would be potentially seriously undermined. If, in addition, this variability is in a sense predictable, a serious source of systematic error, or bias would exist.

1.3

This scenario, whereby the likelihood of responding to a survey could be (at least hypothetically) affected by certain characteristics, appears plausible to the authors. When the survey in question relates to crime victimisation, it seems even more pertinent to ask whether response rates are patterned, and whether the propensity of a household to respond to the survey may be affected by the very experiences that the survey is purporting to measure: specifically, crime experience.

1.4

The idealised analysis would compare the demographic and criminal victimisation characteristics of respondents with that of non-respondents, and assess whether, firstly, the extent to which both groups are broadly representative of the general population, and secondly, whether the experience of crime victimisation affects the likelihood of a household to respond to the survey. Clearly, however, the demographic and criminal victimisation characteristics of non-respondents remain (largely) unknown. While we are, therefore, unable to assess the existence and extent of the problem at the level of the individual household, this does not mean that the problem must remain uninvestigated.

1.5

The first part of the analysis assesses the characteristics of the respondents, in comparison to independent measures of the population characteristics from the census. (The characteristics of the sample that do respond are, clearly, available.) The sample characteristics represent the "end-point" of the sampling process, and by making a comparison between the sample characteristics and the population characteristics, an assessment can be made of the overall representativeness of the sample.

1.6

The second part of the analysis assesses the relationship between crime victimisation and propensity to respond to the survey. While, ideally, we would like to know the crime victimisation patterns of both respondents and non-respondents, clearly this information is unavailable. What can be calculated, however, is the response rates at an area level. In addition, the demographic characteristics of the area are available from the census. This allowed a regression analysis to be carried out to determine which variables determine area property crime incidence rates, and rates of response to the survey. Results can then be utilised to test to what extent area property crime rates and area demographic characteristics influence survey response rates. Although it is important that interpretation of these results is done with an appreciation that the data are at an aggregate level, the findings shed some light on what affects a household's propensity to respond to the survey.

Survey Methodology and Error

2.1

Statistical theory states that the most reliable way to achieve a representative sample is by a random-sampling method: each unit within a population frame has an equal probability of being selected for the sample. The aim of the procedure is that the measurement of a particular variable can be generalised, with a calculable degree of confidence, to the population from which the sample was drawn.

2.2

However, there are a number of different sources of error in the sampling procedure. The diagram below represents the different sources of error.

Figure 1: Different Sources of Error

2.3

Error Source A refers to the error in the operationalisation of the population. An example of an attempt to minimalise this is the change in population frame from the Electoral Register (ER) to the Postcode Address File (PAF). As a population frame reflecting the British population, the ER was perceived to be a relatively poor proxy for the British population: the young, the transient and ethnic minority groups were less likely to appear on the ER: a situation that apparently worsened due to the introduction of the Poll Tax. The large government surveys all shifted to the PAF from the ER at different stages in the 1980s: the General Household Survey, for example, moved to the PAF in 1984, while the British Crime Survey (BCS) was one of the last to make the change between the 1988 and 1992 sweeps. The PAF, however, is also (by necessity) an approximation to the British population. As a sample of addresses, rather than a sample of electors, decisions that lead people to register on the ER (or not) are removed from the process of appearing in the population frame. However, as a database of addresses, the PAF clearly does not include those sections of the population without an address: namely the homeless. In addition, the sampling strategy employed with the BCS uses a constrained version of the PAF: addresses referring to institutional establishments (for example old peoples' homes) are excluded, as their crime experience was felt to be substantially different from other addresses. Consequently, then, even a population frame that purports to cover the population of England and Wales will by necessity miss some sections of that population.

2.4

The second source of error (Error Type B) is that occurring between the operationalised population and the achieved sample. This error is commonly referred to as sampling error, and the primary aim of any sampling strategy is to minimise this error: sampling theory identifies a simple random sample as the method that will do this most effectively. In practice, however, the cost of carrying out a truly random sample of the population is generally too high. The 'compromise solution' generally employed is a stratified, multi-stage, random sample, and it is such a strategy employed by the BCS.

2.5

While full details of the BCS sampling strategy can be found in J. Hales (1992), briefly the sampling strategy employed was thus:- constituencies were divided into inner-city and non-inner city areas, and the two lists were stratified according to region, density and social class, and constituencies were selected, with a probability of selection corresponding to the number of households in each constituency, with inner-city areas deliberately over-sampled (see point 2.6). Within each constituency two postcode sectors were selected, again with a probability of selection corresponding to the number of households in the sector. The postcode sectors remain the smallest area that can be identified, but in order to limit the degree of dispersion of addresses, the postcode sector was divided into four quadrants, and addresses were selected within one of these quadrants.

2.6

A weighting procedure was employed to correct for the deliberate over-sampling of inner-city areas, and for the fact that individuals in large households are less likely to be selected as a respondent than individuals in small households. The weighting procedure adjusts for the fact that certain people in the population frame (for example, those living in inner-city areas) are more likely to be selected than others. By applying the weight, households that are less likely to be selected "count more" to allow for this, and the weighted sample therefore accords with the aims of a random sample.

2.7

One area where the aim of "randomness" was abandoned was the inclusion of an ethnic booster sample. All respondents in the sample were asked whether any households in the three addresses on either side of the respondent's house contained someone from an ethnic minority group. If there were, this household was then contacted. The 1992 technical report advises against the need to apply this weight for general analysis, but an analysis of the representativeness of the sample with regard to ethnicity requires the weight to be employed. As a result, the ethnic booster weight is only used when looking at variables relating directly to ethnicity.

2.8

The third type of error (Error Type C) is that which occurs because of the failure to achieve a valid response from a household within the identified sample: response bias. The technical report (Hales, 1992) identifies that of the 13,117 addresses identified as residential, 3,052 (23.3%) were unproductive. The size of this substantial minority suggests that to assume that this is not a source of bias is an assumption that should not go untested. As noted above, ideally we would like to know the characteristics of those not responding, but obviously this is not possible. However, we do know the number of interviews attempted within each area: 25 in non-inner city areas, and 28 or 29 in inner city areas, and that the aim was to produce an expected number of interviews in each postcode sector of 17.5. In addition, we do know the number of achieved interviews. As such, we can calculate a response rate by expressing the number of achieved interviews as a proportion of the expected number of interviews.

2.9

Having calculated the response rate for each area, we can address whether response rates are effected by firstly certain demographic characteristics, and secondly, by the crime rate.

Analysis and Results

3.1

Comparisons will be made between the demographic characteristics of the achieved sample, and those of the operationalised population.

3.2

The operationalised population is treated as the data coming from the 1991 Census Small Area Statistics. Although this was not the same as the population frame from which the BCS sample was drawn (the Postcode Address File) the notional population is the same: everyone living in England and Wales. While the Census proportions are treated as an 'objective truth', it must be recognised that this is to some extent an assumption: while the census is intended to be completed by every British household, and therefore not a sample, non-response is also an increasingly large problem, leading to an imputation process having been carried out for the first time in 1991. Having said this, the problems associated with this do not fall into the realm of sampling error due to the relatively small numbers involved. Comparisons between sample proportions and the census proportions are therefore a valid method for testing the representativeness of the effectiveness of the BCS sampling strategy, and the representativeness of the resulting sample.

3.3

The 1992 British Crime Survey was used for a number of reasons: while the British Crime Survey requests crime information pertaining to the previous year, the demographic information is that information at time of interview (i.e. early 1992). As such, the 1992 BCS demographic data corresponds reasonably closely to the 1991 Census. While the two data sources do not coincide precisely with each other, the timing problem arising from the employment of other BCS sweeps (e.g. 1988 or 1994) has been minimised.

Tests of Representativeness carried out by the SCPR

3.4

The Social and Community Planning Research Group (SCPR) who carried out the fieldwork for the BCS also carried out some limited test of national representativeness. The survey was tested for distribution across the regions, and across a number of different demographic variables. Adults under thirty years of age were found to be generally under sampled, and thirty to sixty nine year olds over sampled, with seventy plus households considerably under sampled. Students were under sampled nationally, as were council tenants. Those without the use of a car were also under sampled. A pattern emerges from these initial findings from SCPR, suggesting that young and relatively deprived groups tend to be under represented in the sample.

Further Analysis Carried Out

3.5

The analysis therefore concentrates on two main areas, corresponding to Error Type B (sampling error) and Error Type C (response bias).

Sampling Error

3.6

Which variables are better represented than others? Is it the case that certain variables are consistently over- or under-sampled across regions? Are nationally representative variables, representative at other levels as well? For example, a variable could be nationally close to the census proportion, but this may be the result of large variations at a regional level cancelling each other out. Is it the case that the sampling procedure results in certain regions being particularly badly sampled? These issues are addressed in the remainder of Section 3.

Response Biases

3.7

How do response rates vary across areas? How do area crime rates and response rates co-vary? What are the determinants of the two rates, and what are the implications for the reliability of the survey as a research instrument? See section four for an analysis of these questions.

Test Statistic

3.8

The tests for sampling error carried out compared the proportion of the BCS sample with a specific demographic characteristic, and the corresponding proportion from the census data. Using the two proportions, we calculated a test statistic, Z, thus:-

where:
Ps = the proportion from the sample;
P = the proportion from the population;
n = the sample size.

3.9

The null hypothesis is that the differences in proportions can be put down to sample error. The final columns, then, are the p - values: the probability of obtaining a value of Z at least as extreme under the null hypothesis, using a Gaussian distribution.^[1]

National Comparisons

---

Table 1: National Comparisons

---

National	Census	BCS
No. of areas = 572	%	%	P-value
Two Pensioner Households	9.90	11.56	0.02
Owner Occupying Households	67.76	71.82	0.00
Under 5 Households	12.81	13.73	0.10
Adult Females	52.22	50.34	0.00
No Car Households	32.56	30.06	0.02
Single Parent Households	4.10	4.02	0.52
Age 16 to 24 year olds	12.73	16.16	0.00
Male Unemployment	12.31	9.34	0.01
Single Pensioner Households	15.08	14.03	0.09
Ethnic Minority Households	4.30	4.73	0.14
Single Non-pensioner Households	11.59	11.40	0.44

---

3.10

The results show that two pensioner households; owner-occupier households and 16-24 year olds are significantly over-represented in the BCS proportions. Male unemployment, adult females and single pensioner households are among the under sampled variables.

Regional Comparisons

---

Table 2:Counts of Regions' with Over- and Under-sampled Demographic Variables (at the 10% Significance level)

---

Regional	BCS
No. of regions = 10	N +	N -	Sig +	Sig -
Two Pensioner Households	9	1	2	0
Owner Occupying Households	8	2	5	1
Under 5 Households	9	1	1	0
Adult Females	0	10	0	5
No Car Households	1	9	0	4
Single Parent Households	6	4	0	0
Age 16 to 24 year olds	10	0	10	0
Male Unemployment	1	9	0	4
Single Pensioner Households	4	6	0	1
Ethnic Minority Households	7	3	2	0
Single Non-pensioner Households	6	4	1	2

---

3.11

This table then shows the number of regions for which each variable was under- and over- sampled, and the number significantly under- and over-sampled at the 10% level. Perhaps reassuringly, variables that were over- or under-sampled at a national level generally showed the same pattern across each region, rather than the national figure being the result of one or two regions having a substantial impact.

Squared Differences

3.12

The next two points (3.10 and 3.11) address the issue of whether differences in representativeness are associated with certain regions and variables. To a great extent, if the degree of representativeness is constant across regions and across variables, the potential bias problem is a much smaller one than an alternative scenario in which the representativeness across regions or variables shows a high variability.

Differences Across Variables

3.13

In order to calculate the squared differences for each variable, the census proportions were subtracted from the survey proportions for each variable in each postcode sector. These differences were squared and totalled across each variable: squaring the magnitudes of bias ensured that positive and negative biases did not cancel each other out. This then produces a figure corresponding to, if you like, how different is the sample proportion from the census proportion, and allows us to look at whether the differences between the census and sample proportions were patterned by the variable being investigated.

---

Table 3: Squared Postcode Sector Biases

---

Variable	Sum of squared postcode sector biases
Single Parent Households	1.90
Adult Females	2.21
Two Pensioner Households	3.71
16 to 24 year olds	4.19
Ethnic Minority	4.37
Under 5 Households	5.48
Single Pensioner Households	5.92
Single Non-Pensioner Households	5.92
Male Unemployed	17.16
No Car Households	24.07
Owner Occupiers	33.47

---

3.14

No apparent pattern is observed across the variables: owner-occupying households is the most inaccurately sampled variable, but this is followed by non-car owning households and male unemployment. Consequently, no general conclusion can be made suggesting, for example that relatively deprived groups tend to be the most inaccurately sampled.

Differences Across Region

3.15

In addition to testing whether different variables exhibited better or worse representativeness, we tested whether different regions exhibited better or worse representativeness. The squared differences were calculated in a similar way as for each variable (i.e. census proportion - sample proportion for each postcode sector, squared, and then totalled across each region), however, this figure was then divided by the number of postcode sectors in each region, in order to adjust for different sized regions.

---

Table 4: Regional Postcode Sector Biases

---

Region	No. of postcode sectors	Average of squared postcode sector biases
South West	52	0.01
West Midlands	46	0.01
Wales	28	0.01
East Anglia	20	0.02
South East	112	0.02
Yorkshire	53	0.02
East Midlands	62	0.02
North	34	0.02
North West	74	0.02
Greater London	91	0.02

---

3.16

The table suggests that across regions, predominantly rural regions are found at the top of the list (the regional samples with the smallest squared differences), while the urban areas of London, the North West and the North are found at the end of the list. The West Midlands, however, appears to be a clear exception to this pattern.

Factors Affecting Survey Response Rates

4.1

An underlying question that this paper addresses is whether the level of crime in an area affects the response rate to surveys. As noted above (point 2.8), 23.3% of the identified sample failed to produce a valid interview, and of these unproductive 'interviews' 47% were categorised as 'Refused'. The question being posed here is, therefore, how does an area's crime experience (either through direct victimisation experience, or through psychological effects such as fear of crime) effect the area's response rate.

Ecological Explanations of Crime

4.2

A large and growing body of work highlights variations in crime experience across different areas. The study of area and crime is an established, and growing, area of work. Theories and approaches of 'social disorganisation' (Shaw and McKay, 1942), 'routine activities' (Cohen and Felson, 1979), 'community crime careers' (Reiss, 1986) , those addressing the ways in which structural and economic changes effect the functioning of an area (for example, Bottoms and Wiles, 1986), and those addressing the dynamics of crime rates such as James Q. Wilson analysis of 'tipping processes' (Wilson and Kelling, 1982) can all be characterised as ecological approaches to the crime problem, in that the incidence of crime can be affected by changes in the social networks existing within an area. If these are disrupted, either by macro-level economic or social trends, or by local changes in social practices, there will be implications for area crime rates.

Crime Distribution in Britain

4.3

The crime experience across areas of Britain has been shown to be unequally distributed, with the highest 10% of areas experiencing a substantial and marked increase in crime incidents over the next 10% of crime areas (Trickett et al, 1992). In addition, the distribution over time has shown that this inequality has increased over the eighties (Trickett et al, 1995). The driving force in the inequality across areas, and the changes in inequality over time has been multiple victimisation: it is to a great extent the case that changes in crime rate occur, not because more people get victimised, but because those that do get victimised, experience more incidents.

Response Rates and Crime Levels

4.4

Research therefore suggests that high crime rates are associated with high degrees of multiple victimisation, individuals with weak ties to the community, and correspondingly weak community structures. It would therefore seem reasonable to suggest that these same factors could also affect participation in a social survey. It is a survey methodologists' truism to say that response rates will be lower in inner-city areas (hence, the number of attempted interviews in inner-city areas was 28 or 29, while only 25 in rural areas). Consequently, the following sections attempt to directly address the link between crime rates and representativeness of the British Crime Survey. The question being asked is that if the level of crime effects the way people live their lives, which is presumably beyond doubt, it seems a reasonable question to ask whether the survey response rate in high crime areas is different from that in low crime areas.

Crime Deciles

4.5

Similar methodology to that adopted by Trickett et al (1992) is employed here to group the BCS areas into decile groups. The survey response rate in each of the deciles is calculated, and the representativeness of the different demographic variables is assessed across the deciles. The analysis follows the deciles approach adopted by Trickett et al (1992) because of the finding that the top ten percent of areas appear to experience a considerably more acute crime problem than the next highest crime decile: a higher level of aggregation (for example quartiles) would not provide the necessary distinctness to reveal this pattern. To calculate the crime deciles, the number of property crime incidents experienced in each area are totalled, and divided by the number of households sampled. This gives a mean number of incidents per household, or a property crime incidence measure for the area. The areas are then divided into those 10% with the lowest incidence rate (the first decile), the next ten percent, and so on up to the tenth, highest, decile. The 1992 BCS defines property crime as incidents suffered by the household of burglary, attempted burglary, theft inside and immediately outside the dwelling (for example from a garage), and criminal damage inside and immediately outside the dwelling. Crimes involving vehicles are not included. Property crime data were employed for two principal reasons: as these crimes are suffered at a property, we can be certain that the crimes occurred within a certain area, whilst crimes against a person, or against a vehicle could have occurred away from where the BCS interview took place. In addition, the measurement of property crime using a survey methodology is considerably more robust than the measurement of personal crime.

Response Rates Across Crime Deciles

4.6

We calculated the response rate for each postcode sector, and looked at the pattern across crime deciles.

---

Table 5: Response Rate for Postcode Sectors

---

Property Crime Decile	No. of areas	Proportion of Expected Interviews Carried Out
1	57	1.05
2	57	1.13
3	57	1.07
4	57	1.08
5	57	1.06
6	58	1.01
7	57	1.03
8	57	1.01
9	57	1.01
10	58	0.94

---

4.7

There is a clear pattern of the response rate diminishing as the crime rate increases. The tenth decile shows a substantial drop in response rate: indeed this is the only decile that does not reach the expected number of responses.

4.8

Is it the case that a high crime rate results in a low response rate? Or alternatively, is it the case that the groups that are less likely to respond to a survey (for whatever reason) are clustered in the high crime areas, and it is this clustering that results in a low response rate irrespective of the effect of a high crime rate.

4.9

We know that the response rate in the high crime deciles is lower than the other areas. If the representativeness of a certain demographic characteristic is demonstrated to be constant across all deciles, it would suggest that the lower response rate in the higher crime area is caused by the demographic peculiarities of the high crime deciles. Alternatively, if variables are noticeably less representative in the high crime deciles, it would suggest the it is the high crime rate per se that is affecting the representativeness of the variables, and the response rate.

4.10

The survey demographic proportions are calculated for the BCS postcode sectors in each decile, as are the census proportions. Previously, with the regional analysis, all the census data for a region was used: the question being asked was how representative was the BCS sample at the regional level. With the decile analysis, the analysis is at the BCS area level, so the question being asked is how representative is the BCS sample, once the area has been selected, and how does this vary as crime rates vary.

4.11

The following table reports the z-statistic for each of the demographic variables being investigated, across each of the deciles. For brevity, the actual proportions for each decile are not reported here, but can be found in Appendix B.

---

Table 6: Z-Statistic for Demographic Variables

---

Decile	Two Pensioner Hholds	Owner Occupier Hholds	Hholds with children under 5	Adult Females	Hholds with no car	Single Parent Hholds	Aged 16-24	Male Unemployed	Single Pensioner Hholds	Ethnic Minority Hholds	Single Non-Pensioner Hholds
1st	2.56	3.43 *	0.14	-0.81	-1.94	0.45	0.97	-1.82	0.33	-1.22	-0.72
2nd	2.58	5.29 **	0.96	-2.92 *	-5.20 **	-2.69	2.90 *	-2.20	-0.47	-3.05 *	-1.41
3rd	4.79 **	2.96	-0.37	-4.53 **	-5.87 **	-2.89 *	5.55 **	-2.09	-1.41	-0.86	-1.94
4th	3.83 *	1.79	0.26	-3.36 *	-3.04 *	-1.96	5.98 **	-1.89	-1.49	-2.02	-2.33
5th	0.81	1.87	-1.11	-3.83 *	-2.50	-1.01	12.68 **	-1.85	0.42	0.24	0.44
6th	0.73	0.62	0.68	-2.44	-3.01 *	-0.80	3.77 *	-2.19	-1.16	-1.71	-0.73
7th	3.57 *	0.65	-1.38	-2.62	-0.64	-0.60	8.31 **	1.63	-0.26	-2.15	-0.70
8th	1.54	1.06	0.38	-3.02 *	-3.38 *	-2.08	11.00 **	-1.41	-2.88 *	0.72	0.46
9th	2.24	3.43 *	2.30	-0.83	-4.54 **	2.10	11.10 **	-1.30	-3.00 *	-0.18	-1.57
10th	-1.22	-0.35	0.12	-3.24 *	-3.02 *	2.26	11.07 **	-0.82	-0.70	-2.50	-1.44

** - Significant at the 5% level
* - Significant at the 10% level

---

4.12

This table tends to suggest that there is little pattern to be discerned across the deciles. In general, the BCS proportions follow the census proportions quite closely, and those that do not (for example, households with no car, or population aged between 16 and 24), tend to be significantly different across most of the deciles, rather than in the high crime areas specifically. This tends to suggest that it is the case that living in high crime areas leads to non-response, rather than any demographic correlation.

4.13

Given this pattern in response rates, it is pertinent to ask whether it is possible to identify any demographic factors that can aid explanation of survey response rates, and whether these same factors partly determine an area's incidence of property crime. Ordinary least squares (OLS) regressions were carried out on the area level data. The two dependent variables were the response rate of an area to the BCS, and the incidence of property crime in an area ^[2]. Demographic data from the census^[3] were employed as explanatory data: the data were in the form of the proportion of the population of an area with certain demographic characteristics: the data were standardised (a mean across areas of zero, a standard deviation of one).^[4] Initial regression results, in which all the possible demographic variables were included as explanatory variables, indicated a large number of insignificant variables, and some unexpected coefficient signs on explanatory variables. Both of these problems indicate the presence of multicollinearity. This was to be expected: a number of variables employed were measuring closely related characteristics of an area, for example, the proportion of households with no car, and the male unemployment rate are very highly correlated at an area level. An iterative process which eliminated insignificant variables resulted in a reduced set of variables, which were all significant, suggesting that the problem of multicollinearity had been reduced. The following table gives the preferred models for the influences on response rates and property crime incidence levels.

---

Table 7: Preferred Models for the Influences on Response Rates and Property Crime Incidence Levels

---

Dependent Variable	Response Rate Model	Incidence of Property Crime Model
constant	0.56**	0.21**
South East	0.10**	0.09
North	0.15**	0.15
Yorkshire	0.14**	0.13
East Midlands	0.12**	0.16
South West	0.15**	0.07
West Midlands	0.11**	0.23*
North West	0.11**	0.12
Wales	0.14**	0.13
GLC	0.10**	0.06
Prop. of Vacant Housing	-0.02**	0.11**
Prop. of housing that is Flats	-0.05**
Single Parent households	0.01 *
Affluence factor		-0.03**
Prop. of population aged 5-15		0.08**
Prop. of Council Housing		0.04*
R-bar squared	0.21	0.17

**, * indicates significance at the 1% and 5% levels respectively.

---

4.14

The table shows that the coefficients are all significant at least the 5% level, most being significant at the 1% level. Regional dummy variables were included to ensure that regional variations did not mask relationships between other variables. The most important result to emerge is that while the decile analysis indicated some relationship between the incidence of property crime and survey response rates (table four), different demographic variables are found as having a significant input on these variables: only one variable was found to be significant in both models (the proportion of vacant properties).

Conclusion and Discussion

5.1

Across crime deciles there is a discernible pattern of a reduction in response rates as the property crime rate increases. As individual demographic variables do not appear to be any better or worse represented, this reduction in response rates appears to be due simply to the fact of living in high crime areas. This finding in itself has major implications for anyone using the BCS, and indeed any other survey instrument.

5.2

An analysis of the national and regional representativeness of the BCS was conducted: the authors conclude that the national patterns of representativeness were reflected at a regional level. This reinforces, to some extent, the reliability of crime surveys as a research instrument.

5.3

Attempts were then made to identify a link between a survey's response rate and the incidence of property crime in an area. While to some extent such a correlation was identified, different explanatory variables were found to account for area differences in survey response rates and the incidence of property crime. This finding also reinforces the validity of the sample: were it the case that the same variables found to have explanatory power in both models, the suggestion would be that response rate and crime are influenced by the same antecedent variables. Given that they do not appear to be suggests that although their appear to be errors associated with response rate, they do not directly affect the crime rate as measured by the survey.

5.4

There are many and varied implications for the crime surveys. For the simple calculation of crime rates, the question must be asked: if the non-respondents were to respond, would that affect the crime rate? There is obviously no easy way to answer this, as their crime experience and responses are obviously unknown. Two alternatives occur to the authors, though with the present data, these remain untestable. The first would be that there would be no significant effect on the crime rate: the households who do respond seem to be no more or less representative demographically in the high crime areas than anywhere else, so they are likely to provide representative responses for their areas. The second alternative is that the crime rate arrived at by the sample would be significantly effected by the inclusion of the non-respondents. Given that at the area level, crime experience is associated with a lower response rate, it seems most plausible that the inclusion of the non-respondents would result in a higher sample crime rate resulting.

5.5

The findings suggest that an area with a high crime rate tends to have a lower response rate. The danger here is to infer individual behaviour from area level findings, and suggest that people who suffer more crime are less likely to respond to the BCS. This is clearly one possible explanation: the finding, however, that predominantly different variables determine response rates and crime rates suggests that this inference may be too simplistic.

5.6

How can we explain the correlation between non-response rates and crime victimisation. The authors suggest that a plausible explanation for the link between victimisation and non-response may be sought in the notion of social isolation.

5.7

The direction of causality is not, however, clear, and the authors suggest three possible scenarios: crime may result in social isolation as a result of their experiences (avoidance of certain social settings, for example) , or indirectly (and perhaps more commonly) due to fear of crime. An implication of social isolation may be a reluctance to respond to the survey interviewer: even survey research requires delicate and skillful negotiation if a successful interview is gained. Previous experience or fear of crime may well upset this negotiation process.

Figure 2: First Scenario

5.8

An alternative scenario could be that social isolation may be a precursor to both crime victimisation and non-response:-

Figure 3: Second Scenario

5.9

The link between social isolation and non-response is as identified above, but in addition, social isolation may be considered a crucial aspects of explanations of area crime rates. James Q. Wilson's analysis, for example, identifies a crucial stage in the dynamics of crime rates. The result of 'broken windows' (or other visible signs of community deterioration) is the withdrawal from the community of those capable of maintaining social control (the capable guardians of Marcus Felson's routine activities theory). This then results in upward pressure on crime rates. This withdrawal from the community, and the resulting problems, we would argue, is merely the public manifestation of the private problem identified as social isolation.

5.10

The third scenario is a recognition that the pattern of causation is likely to be a combination of the two described above, as represented in the diagram below:-

Figure 4: Third Scenario

5.11

Social isolation at the individual level is influenced by crime victimisation, either directly, by people withdrawing from community to avoid further victimisation. The community level manifestation of this social isolation from community can set up the tipping process that Wilson identifies, thereby further reinforcing the feelings of social isolation.

5.12

The implications are, therefore, that levels of social isolation may effect response rates to surveys, and this may would have wider implications generally for survey research. It is not clear whether the specific nature of the survey under investigation (that is, crime) affects the relationship. Studies of response rates using other surveys could be investigated, assuming a plausible proxy measure for social isolation / community withdrawal could be obtained.

5.13

The authors suggestions for amendments of practice involve a consideration of including a response rate weighting procedure. The BCS sampling strategy appears to implicitly support this: the reason for over-sampling the inner-city areas was based on two understandings: firstly, that "higher 'deadwood' would exist in the Inner City areas" (Hales, 1992) (that is, a higher non-response rate), and secondly, that the Inner City areas would have a higher concentration of crime victimisation than non-inner city areas.

5.14

A weight consisting of

would weight the responses so that each area has an equal representation in any analysis. Analysis of surveys that show a wide disparity in response rates across different areas may be made more reliable by such a weighting procedure.

5.15

This weighting does, however, make a number of assumptions: mainly, that those that respond are typical of those that do not. Clearly, this assumption is difficult to treat uncritically. Empirically, initial attempts at quantifying the effect of such a weighting procedure provide little evidence that, at least in the case of the BCS, such a weighting procedure drastically effects results. In general, the authors would suggest that a situation where results are substantially effected by such a weighting procedure suggests a potentially unreliable dataset.

Notes

¹ The denominator of the Z statistics would ideally be inflated to incorporate the 'deft' factor for each variable. Deft factors indicate the extent to which standard errors are increased by the use of a stratified, as opposed to a simple random sampling technique (Maung, 1995). However, while some deft factors for BCS variables are reported in the 1992 Technical Report, the defts reported are only those associated with crime variables.

² Ideally, we would employ a simultaneous equation estimation technique, but this could not be employed, as it required the a priori selection of at least a slightly different set of demographic factors to be associated with each dependent variable.

³ See data appendix for the list and definition of all explanatory variables employed.

⁴ An additional variable was included here; the proportion of vacant properties. This is not included in the earlier analysis as it is not available in the BCS data, as the datafile only includes completed interviews. While we know from the BCS Technical Report (Hales, 1992) that nationally 5.2% of the attempted sample was vacant, and a further 0.5% were identified as not yet built or occupied, we cannot break the data down to specific areas. Clearly the number of vacant properties has a direct impact on response rates, and less directly on crime rates, so the census-derived variable was used here.

References

BOTTOMS, A.E., and WILES, P. (1986) 'Housing Tenure and Residential Community Crime Careers in Britain' in A.J. Reiss and M.Tonry (editors) Communities and Crime. Chicago: University of Chicago Press.

COHEN, L.E. and FELSON, M. (1979) 'Social Change and Crime Trends: A Routine Activity Approach' American Sociological Review, Vol 44: pp 588-608

HALES, J. (1992) British Crime Survey (England and Wales) Technical Report. London: SCPR.

MAUNG, N.A. (1995) 'Survey Design and Interpretation of the British Crime Survey' in M. Walker (editor) Interpreting Crime Statistics. Clarendon Press: Oxford.

REISS, A.J. (1986) 'Why are Communities Important in Understanding Crime?' in D.J. Evans, N.R. Fyfe, and D.T. Herbert (editors) Crime, Policing and Place:Essays in Environmental Criminology. London, Routledge.

SHAW, C.R. and MCKAY H.D. (1942), Juvenile Deliquency and Urban Areas. Chicago: University of Chicage Press.

TRICKETT, A., OSBORN, D.R., SEYMOUR, J. and PEASE, K. (1992) 'What is Different about High Crime Areas?', British Journal of Criminology, vol. 32, no. 1, pp. 81 - 89.

TRICKETT, A., ELLINGWORTH, D., HOPE, T. and PEASE, K. (1995) 'Crime Victimisation in the Eighties: Changes in Area and Regional Inequality', British Journal of Criminology, vol. 35, no. 3, pp. 360 - 365.

WILSON, J.Q. and KELLING, G. (1982) 'Broken Windows', The Atlantic Monthly, March, pp. 29 - 38.

Appendix A: Variable Definitions

Two Pensioner Households

Census:-

The proportion of all permanent households containing two or more pensioners, and nobody else. Pensioners are defined as men aged 65 or over, and women aged 60 or over.

BCS

The proportion of all permanent households containing two or more pensioners. Pensioners are defined as above.

Owner Occupying Households

Both data sources:-

The proportion of all households which are owned or being bought by the occupiers. Homes which are tied to a job, and shared-ownerships are not included (this is not a category in the BCS at all).

Under 5 Households

Both data sources:-

The proportion of all households with at least one child under 5 years of age.

Non-Car Owning Households

Both data sources:-

The proportion of households not owning a car, van, or other motor vehicle (not including motorbikes).

Single Parent Households

Both data sources:-

The proportion of households containing only one adult, and at least one child.

16 To 24 Year Olds

Both data sources:-

The proportion of the population aged between 16 and 24.

Single Pensioner Households

Both data sources:-

The proportion of all households exclusively containing one pensioner. Pensioners are defined as above.

Single Non-Pensioner Households

Both data sources:-

The proportion of all households containing exclusively one non-pensioner, and no children.

Male Unemployment Rate

Census

The proportion of economically active males who are unemployed, or on a government training scheme.

BCS

The proportion of economically active male heads of household who are unemployed.

Ethnic Minority

Both data sources

The proportion of heads of household who describe themselves as Black (Caribbean, African or other Black); Indian, Pakistani, or Bangladeshi; Chinese; or other non-White.

Appendix B: Demographic Proportions For Property Crime Deciles

Proportion of Two Pensioner Households DECILE BCS Census 1 .13 .11 2 .12 .10 3 .15 .10 4 .14 .10 5 .11 .10 6 .10 .10 7 .12 .09 8 .10 .08 9 .10 .08 10 .07 .08 Proportion of Owner Occupier Households DECILE BCS Census 1 .78 .74 2 .81 .74 3 .75 .71 4 .74 .72 5 .72 .70 6 .66 .65 7 .63 .62 8 .64 .63 9 .63 .58 10 .53 .54 Proportion of Households with children under 5 DECILE BCS Census 1 .13 .13 2 .13 .12 3 .12 .12 4 .13 .13 5 .12 .13 6 .14 .13 7 .11 .13 8 .14 .14 9 .16 .13 10 .15 .15 Proportion of Females in Adult Population DECILE BCS Census 1 .51 .52 2 .50 .52 3 .49 .52 4 .50 .52 5 .50 .52 6 .51 .52 7 .50 .52 8 .50 .52 9 .52 .52 10 .50 .53

Proportion of Households with no Car DECILE BCS Census 1 .22 .25 2 .20 .26 3 .21 .29 4 .26 .30 5 .28 .31 6 .31 .36 7 .37 .38 8 .32 .38 9 .37 .44 10 .39 .44 Proportion of Single Parent Households DECILE BCS Census 1 .03 .03 2 .02 .04 3 .02 .03 4 .03 .04 5 .03 .04 6 .04 .04 7 .04 .05 8 .04 .05 9 .07 .05 10 .08 .06 Proportion of Population aged between 16 and 24 DECILE BCS Census 1 .13 .13 2 .14 .12 3 .15 .13 4 .15 .12 5 .18 .13 6 .15 .13 7 .17 .13 8 .18 .13 9 .19 .14 10 .19 .14 Male Unemployment Rate DECILE BCS Census 1 .06 .09 2 .06 .10 3 .07 .10 4 .08 .11 5 .09 .12 6 .09 .13 7 .17 .14 8 .12 .15 9 .14 .17 10 .16 .18

Proportion of Single Pensioner Households DECILE BCS Census 1 .14 .14 2 .13 .14 3 .13 .15 4 .13 .15 5 .16 .15 6 .14 .16 7 .15 .15 8 .11 .14 9 .12 .15 10 .14 .15 Proportion of Ethnic Minority Households DECILE BCS Census 1 .03 .03 2 .02 .04 3 .03 .04 4 .02 .03 5 .06 .06 6 .06 .07 7 .05 .07 8 .08 .08 9 .07 .07 10 .06 .08 Proportion of Single Non-Pensioner Households DECILE BCS Census 1 .09 .10 2 .09 .11 3 .10 .12 4 .08 .10 5 .12 .11 6 .11 .12 7 .13 .14 8 .15 .14 9 .13 .14 10 .12 .14

Copyright Sociological Research Online, 1997