Copyright Sociological Research Online, 1997

Temple, B. (1997) '"Collegial Accountability" and Bias: The Solution or the Problem?'
Sociological Research Online, vol. 2, no. 4, <http://www.socresonline.org.uk/2/4/8.html>

Abstract

In a recent debate about bias in social research, Hammersley and Gomm (1997) discuss error, and bias as a form of error, as 'a matter of collegial accountability'. They argue that radical epistemologies are a growing threat to the research community. Only by using such a community to decide what is reasonable can researchers avoid the threatened slip into the abyss. This threat is illustrated for the authors by, for example, the growing emphasis on the role of users of services by funding agencies.

For those researchers who have struggled to be heard within academic life, the desire to install a single community as judges of research is a step backwards. The evaluation criteria used for research have been narrowly defined by some researchers within that community. Feminists, amongst others, have been trying to widen the definitions of validity. The desire to return to an authoritative voice, a particular and restricted group of 'colleagues' in Hammersley and Gomm's case, constitutes the threat rather than the solution for those researchers. It assumes that these colleagues speak for everyone and are only accountable to themselves. In this article I examine the way in which Hammersley and Gomm (1997) have set up the debate with feminist researchers. I then go on to discuss the notion of 'the research community' and the assumptions the authors make about the criteria for evaluating research. I finish by introducing an alternative way of being accountable which involves opening up dialogue with a wider audience.

Keywords:

Epistemology; Feminism; Knowledge; Research

Introduction

1.1

Hammersley and Gomm's (1997) article is a valuable discussion of the various ways in which the term bias is referred to in social research. As they point out, it is used by researchers to refer to different things. They discuss a variety of forms of error and define bias as 'culpable systematic error'. Some of the definitional problems of bias result, they argue, from abandoning a foundationalist epistemology. If it is not possible to be objective then there is no yardstick for deciding who is biased and who is not.

1.2

Hammersley and Gomm (1997) define foundationalism as a set of assumptions about research in the social world based on ideas of objectivity, validity, error and bias. The social world is independent of the researcher who carries out research. Findings would be the same for anyone carrying out research, regardless of who they were: if they followed a set of methodological procedures they would find the truth. The challenges to this kind of view of research are well documented and feminists have been at the forefront of these (see for example Stanley and Wise, 1990; Fonow and Cook, 1991).

1.3

Hammersley and Gomm then discuss an alternative epistemology: relativism. Relativists define all research as accounts of the social world as produced by researchers rather than discovered existing independently of them. Research inevitably reflects the position, social and material, of the researcher and all accounts are equally true. As Hammersley and Gomm point out it is rare for researchers to be totally relativist. They suggest that feminists often choose standpoint epistemology as an alternative.

1.4

A standpoint epistemology is described by the authors as the view that a particular social location gives unique access to the truth. In this way, they argue, feminists can claim to have their cake and eat it: claims to universal validity can be retained whilst accepting the argument that the validity of all knowledge is relative to social location. How then, Hammersley and Gomm (¶3.8) ask is one to judge the validity of statements about the source of a knowledge claim? They suggest that feminists judge claims by making appeals to the workings of ideology or false consciousness: those in the preferred social location can be influenced by the workings of ideology. The ground appears to have been cut from under feminists' feet: 'bias ... once again becomes a threat to validity that is universal, not restricted to those occupying the wrong standpoint' (¶3.8).

1.5

According to Hammersley and Gomm (1997: ¶3.9), feminists use the two approaches interchangeably: relativist arguments to critique unpopular views and standpoint ones to protect views they favour. This they describe as 'ontological gerrymandering' .

1.6

However, this picture of feminist failings is achieved only by silencing feminist epistemologies other than relativism and standpoint (see here Romm, 1997; Humphries, 1997) and engaging with feminist researchers who hold these positions on a level which makes them seem naive. Humphries (1997) describes the picture of feminism painted as static 'an homogeneous feminism which falls over with the touch of a finger' (1997: ¶2.4). Admitting other nonfoundational feminist epistemologies would spoil Hammersley and Gomm's line of argument (see for example Stanley and Wise, 1990). There are many varieties of feminism and many ways of being feminist. For example, the collection of essays in Maynard and Purvis (1994) and Fonow and Cook (1991) illustrate some of these. There are points of commonality, what Griffiths (1995: p. 59) refers to as 'common threads', but the presence of different voices and epistemologies are one of its strengths. Epistemological issues are central to feminist debates and feminists are themselves well aware, and actively debating, them (see for example Fonow and Cook, 1991; Stanley and Wise, 1993; Maynard and Purvis, 1994; Griffiths, 1995 to name but a few).

1.7

Moreover, the feminists Hammersley and Gomm do refer to are also engaged in these debates and their arguments are not as simplistic as presented. Hammersley and Gomm cite the early work of Smith (1974), for example. However, Smith has developed her arguments since and presenting her work as undeveloped and unresponsive to recent debates is a grave injustice. As Stanley and Wise point out (1990), Smith's position is complex. She is not a standpoint feminist in the way Hammersley and Gomm indicate. She does not argue for women's standpoint to be 'some transcendent and superior understanding' as in the Hammersley and Gomm caricature, but tries to use it to 'move between different standpoints, different contexts' (Stanley and Wise, 1990: p. 36).

1.8

Only by ignoring the rich and diverse nature of feminist methodologies and epistemologies can feminists be seen as guilty of not being able to advance a sustainable position of their own. The strategies feminists use to assess the work of others are not limited in the way Hammersley and Gomm suggest. Feminists do not have to resort to discrediting views we disagree with using a relativist epistemology and supporting those we support using a standpoint one. The voices of those we disagree with are not dismissed as simply the workings of ideology or false consciousness (¶3.8). For example, Fuss (1989) uses the notion of ideology not to dismiss the work of dissenters but as a tool to look at the work of all researchers (I return to this below).

The Solution?

2.1

Hammersley and Gomm make a number of points in describing their own position. The first is that the distinction between accounts and the phenomena they purport to represent is a viable one. Referring specifically to the use of language they argue that authors cannot stand outside of language. Secondly, they argue that researchers do not have direct contact with the phenomena they seek to describe and explain. By this they mean that researchers do in a limited sense construct the phenomena they describe but not in any directly referential way. Their third point is that researchers have to make judgments about the status of accounts. Here the research community is seen as having a crucial role 'in subjecting knowledge claims to assessment on the basis of criteria of plausibility and credibility that are generally more skeptical than those operating in other areas of social life; in the sense that they are primarily concerned with avoiding the danger of accepting as true what is in fact false'. (¶4.2). This research community, it is argued, must have objectivity as its guiding principle and the production of knowledge as its immediate goal: 'truth is the only value that constitutes the goal of research'. (¶4.12).

2.2

Feminists' problems arise, the authors argue, because they are guilty of bias, that is, culpable systematic error. This results from 'an active commitment to some other goal than the production of knowledge' (¶5.1). The pursuit of scientific knowledge is the only way a researcher can maximise 'the chances of discovering the truth about the matter concerned' (¶4.14). The best way to do this is to set up a form of 'collegial accountability' with the pursuit of scientific knowledge as the goal.

2.3

In my reading of Hammersley and Gomm I found that I agreed with much of their position. My conclusions, however, are substantially different. This is because I see problems with, and alternatives to, judgment of research solely by 'the research community'. Indeed their solution of 'collegial accountability' when pinned down turns out to be no solution at all.

2.4

My epistemology differs from that of Hammersley and Gomm's in not attempting to set up restrictive and exclusionary validity criteria which privilege any particular version of truth, such as that produced by researchers in traditional scientific mode. Not being standpoint in the sense depicted by Hammersley and Gomm, I would urge caution in any attempt to set up any one group as the guiding light to relevance and validity. I would argue that all research accounts are partial and constructed by the researcher. However, this does not mean that I am a relativist and believe that all accounts are purely constructions. As Fuss (1989) asserts experiences cannot be free from the influences of ideology but can be analysed as 'a window onto the complicated workings of ideology (1989: p. 114). If accounts are not free of the workings of ideology, neither are they free of the influences of material and social factors. We make our social worlds in part by discussing our views, that is intersubjectively, and by locating our individuality within these worlds. Part of the point of discussion and debate is to establish the overlaps and contradictions between accounts and to assess the influences of material and social location on perspective.

2.5

Elsewhere (Temple, 1997) I have written about how useful I have found the concept of intellectual autobiography (Stanley, 1990) as a way of discussing what my views and those of the participants in my research are. Briefly, this concept entails acknowledging that all research is a product of the experiences and views of both the researcher (the autobiographical component) and those involved in the research (whose biographies influence their accounts). These influences on research are not added frills that can be removed: they are part and parcel of the research. As I have noted, this does not make me a relativist. It does mean that I believe that to judge my work you cannot call on some notion of 'collegial accountability' or 'a representative of the research community' (Hammersley, 1995: p. 97). My perspective would involve asking: which representative? which community? Hammersley argues that:

... researchers must appeal without further justification only to what their colleagues would accept as plausible and credible on the basis of well established procedures and accepted knowledge (Hammersley, 1995: p. 97).

2.6

In other words, only those views already condoned by a group with particular standards of assessment are included within this community. I discuss the implications of this notion of one homogeneous research community and the basis assumed for its decision making below. It is only necessary to note here that this is just the strategy that radical researchers had to challenge to get a foot in the door and to allow in perspectives and topics previously chosen with reference to a predominantly male centred agenda.

2.7

My epistemology is an example of the many voices and conversations within feminism (see also Fonow and Cook, 1991; Crowley and Himmelweit, 1992; Maynard and Purvis, 1994). Some feminists would challenge it. Whatever our brand of feminism, traditional views of science, how knowledge is produced and who is allowed to speak in the name of the research community, would be central concerns. These concepts have all been defined in the past to silence dissent. Specifying the views of the researcher and the participants in research is a way many feminists envisage would maximise the potential of dialogue with others. Researchers can test the boundaries of the phenomena they are researching in a way that does not deny that they are in part construction but nevertheless also valid. In other words, feminists would argue that there is another way to approach the problem of bias caused by the demise of foundationalism (although many of us would argue that it is alive and well in many academic departments and funding bodies). By specifying their perspectives, rather than assuming that by becoming part of some group such as 'the research community' researchers are less likely to suffer from afflictions such as bias, they can begin to address our intersubjectivity.

Science as Usual

3.1

If we look at the everyday world of researchers, there is no one homogeneous group of people to appeal to. Research is undertaken by a mix of lecturers, contract staff and research students. Only a few researchers are at the top of the tree, with women under-represented. Many researchers are untenured and insecure, depending on funding which often comes from organisations which have their own agendas.

3.2

Neither are researchers united in their support for any particular set of criteria on which to judge research. There is increasing debate on validity and reliability criteria. The work of Silverman (1993) and Boulton et al (1996) illustrates just two positions on these. There is, therefore, no one community with a standard set of criteria to appeal to. The position researchers take on the kinds of criteria to use is bound up with their epistemology, which is based on their experiences within and outside of research. These are many and varied. However, the criteria set up to evaluate research are often still based on some notion of validity derived from a desire to practice 'scientific' research. The work of Addelson (1991) shows that such appeal to an all embracing view of science, in either the natural or social sciences, as the basis of establishing the reasonableness of research is itself problematic.

3.3

Addelson (1991) challenges the idea of a unified science based on one world view. She points to the existence of 'prestige hierarchies' in the natural sciences. I have already indicated that these have parallels in the social sciences. These hierarchies influence what is deemed to be scientific rationality and which theories will be used to demonstrate this. If science is seen as a stock of knowledge embodied in theories, then it may appear possible to separate politics from science. However, some researchers, not merely feminists, see science in a different light. If the concept of science is unpacked it can be defined in different ways.

3.4

I have found Addelson's (1991) approach using 'cognitive authority' a useful starting point to her challenge to the idea of one view of what science is about. She sees science as an investigation undertaken by people whose practices define what is the 'norm' in the field and argues that this links politics and science. Some world views and theories will be more widely supported, not because they are the only possible explanations, but because researchers have had more chance to exercise their authority. Researchers in powerful positions find it easier to spread their theories. There is, however, no single approach to science and researchers hold different perspectives. Certain perspectives and theories may be privileged but there are challenges to these. For example, feminist critics of traditional science question a model of scientific research which is hierarchical and which uses binaries such as emotion/reason, subjective/objective, subjects and objects of research (see Rose, 1983; Harding, 1986; Keller, 1989).

3.5

When the differences between the natural and social sciences are introduced into this debate the problems in establishing one view of what is scientific are compounded (see Gordon, 1991). Addelson states that if her view of science:

... seems to open up the possibility of bias, the way ahead is not to blame individual researchers for showing favoritisms because they depart from some mythical set of abstract canons. The way to correct it is to broaden rational criticism in science by requiring that both philosophers of science and scientists understand how prestige and power are factors in the way cognitive authority is exercised (Addelson, 1991: p. 28).

3.6

She offers the idea of cognitive authority as a way of examining the role of epistemology, methodology and method on the practice of science. It enables the scientist to expand the criteria of scientific rationality and criticism to include social arrangements implicit in the practice of science itself. In a conclusion which is particularly relevant to the debate on bias and collegial accountability she asserts:

This would not result in a sudden illegitimate politicization of science or an opening of the floodgates of irrationality. Quite the contrary. Because they have cognitive authority our scientists already are politicized. It is the unexamined exercise of cognitive authority within our present social arrangements which is most to be feared' (Addelson, 1991: p. 31).

3.7

In a similar way, Ramazanoglu (1992: p. 208) argues that Hammersley's (1992) earlier work does not face up to the implications of identifying the 'supposedly rational validators' as people who are as political as the feminists they are trying to exclude. Feminists have long argued that notions of what is rational and scientific have been used to exclude alternative views. The criteria for defining what is relevant, what is scientific and how it should be written about, are often male defined (Ramazanoglu, 1992; Griffiths, 1995). Griffiths (1995) also makes the wider point that 'mainstream men' often assume that their audience is pretty much like them, that is they write as if there is no question about the 'we' they are talking about. In Hammersley and Gomm's case an appeal to science and the pursuit of knowledge operates as a way of trying to enforce this exclusion. Setting up science, the pursuit of knowledge and truth as the ultimate checks on research ignores the contested nature of all these concepts.

3.8

In summary, there is a long-standing debate about what science itself is and how knowledge is produced. There can be no presumption of agreement over these terms, as the debate between Hammersley and feminists has shown (Gelsthorpe, 1992; Ramazanoglu, 1992; Hammersley, 1992 and 1995; Hammersley and Gomm, 1997; Romm, 1997; Humphries, 1997). Rationality, science and community are contested concepts and subsuming them under 'collegial accountability' only serves to mask their workings as political constructs. With the possibility of different epistemologies must come the possibility of different views of what reason and science are and how far their pursuit can be the aim of research. Putting up 'science' as the binding that holds the research community together is itself, therefore, problematic (Gordon, 1991; Harding, 1986; Stanley and Wise, 1990). There is disagreement between feminists and non-feminists about the character of the scientific endeavour.

3.9

I now go on to give an example of research using a perspective which invites the reader to see how the author comes to a particular interpretation. The researcher (Mykhalovskiy, 1997) is a man who wanted to build on the insights in his field that feminists had provided. I have chosen it as my example because it illustrates the point that difference does not have to be a bar to debate and that writing for, and being accountable to, different kinds of audiences can enrich research.

Other Voices

4.1

Mykhalovskiy's (1997) work is a reflexive piece written after the rejection of an application for doctoral studies in Canada. One of the pieces he submitted, and around which much of the discussion of the reasons for rejection focused, was an account of masculinity using an autobiographical perspective. This was entitled Table Talk. In it the author discussed his talks with his family around their kitchen table. These talks covered many topics: immigration, Ukrainian ethnicity in Canada, patriarchal family relations, family violence and anti-Semitism. Mykhalovskiy describes how he was drawn to feminist postmodern critiques of social science but was taken aback by the response he received. One evaluator described Table Talk as 'self-indulgent, informal biography ... lacking in accountability to its subject matter' (1997: p. 232). This notion of self-indulgence is examined in detail by the author. The concept of accountability is less directly drawn out but is nevertheless also relevant (see here Romm, 1997 for a discussion of accountability). The reviewer sets up a notion of accountability to 'the subject matter'. This disembodied accountability parallels somewhat Hammersley and Gomm's (1997) use of the research community, science and truth. When examined in closer detail what it means is that the only kind of assessment that is valid is one based on the principles held by the reviewer as to the nature of the subject matter and how it should be tackled.

4.2

Mykhalovskiy deals with the implications of the accusation of self-indulgence and addresses the question of who should judge his work and on what criteria. He argues specifically that the charge of self-indulgence is a contradictory reading which 'invokes a universal reader who shuts out the possibility of the text speaking to others' (1997: p. 235). His research, he argues, was judged by the criteria of a sociological orthodoxy expressed in a:

... masculine academic discourse or voice. ... Authoritative, and at times arrogant, it is a voice that speaks unitarily and with confidence. At its worst, it floats depersonalized, above actual speech, booming loudly with knowledge of the other, inviting its listeners/readers to be persuaded through its reason and reasonableness. (1997: p. 237)

4.3

Many charges have been leveled at Mykhalovskiy's work, some similar to those used to assess feminist work, and I believe that this is no coincidence since both challenge the view that there is one way to read and write about research and one audience, an academic research audience, who know how to read 'correctly'. Mykhalovskiy attempts to engage multiple voices, including those of his grandparents and others involved in the piece he describes as group writing. He analyses how his views differ from those others at the table and in the group and does not attempt a final resolution to his research questions. For his reviewer, he argues, this is in itself problematic in that as the author he refuses to finish his work with the definitive conclusion traditionally expected from a researcher.

Conclusion

5.1

Feminist work has often been dismissed by the booming voice of reason used to assess Mykhalovskiy's work. Invoking the research community with a single agenda as this voice in effect shuts down much valuable dialogue with researchers, and others not involved in research, who question what science 'is' and try to extend what is encompassed within the concept of reason. Such voices include those of service users that Hammersley and Gomm dismiss so easily. Without service users the definitions of need that emerge from research are restricted, as are the range of possible solutions. Involving users in the whole process of research from setting up the issues to putting forward solutions can change the research and the outcome substantially (Temple, 1994; Temple et al, 1996).

5.2

Feminists, however, are well aware that placing the researcher within the text, comes with its own dangers. Humphries (1997), for example, warns that one of these is the placement of the researcher as the norm against which others' views can be judged. It is by listening and learning from other people's experiences that the researcher can learn that 'the truth' is not the same for everyone.

5.3

For many feminists, and other researchers, therefore, the notion of collegial accountability to a research community is problematic. So also is the use of the one general umbrella of science, truth and knowledge. All these concepts have in the past been used to attack particular kinds of research. Their unpacked re-instatement constitutes the threat, rather than the saviour, to the work of these researchers. They can serve to silence dissent and debate about what research is about, who it is for and how it is to be judged. Rather than suggest any particular group as in a most favoured position for the job of judge and jury, I argue that a more open and honest move is to declare your hand: these are my views on the research subject. You are welcome to compare them to your own but not to those of some mysterious disembodied entity that is assumed to speak for everyone.

References

ADDELSON, K.P. (1991) 'The Man of Professional Wisdom' in M. Fonow and J. Cook (editors) Beyond Methodology: Feminist Scholarship as Lived Research. Bloomington, USA: Indiana University Press.

BOULTON, M., FITZPATRICK, R. and SWINBURN, C. (1996) 'Qualitative Research in Health Care: A Structured Review and Evaluation of Studies', Journal of Evaluation in Clinical Practice, vol. 2, no. 3, pp. 171 - 179.

CROWLEY, H. and HIMMELWEIT, S. (1992) Knowing Women: Feminism and Knowledge. Cambridge: Polity Press in association with The Open University.

FONOW, M. and COOK, J. (editors) (1991) Beyond Methodology: Feminist Scholarship As Lived Research. Bloomington, USA: Indiana University Press.

FUSS, D. (1989) Essentially Speaking: Feminism, Nature and Difference. London: Routledge.

GELSTHORPE, L. (1992) 'Response to Martyn Hammersley's Paper 'On Feminist Methodology', Sociology, vol. 26, no. 2, pp. 207 - 212.

GORDON, S. (1991) The History and Philosophy of Social Science. London: Routledge.

GRIFFITHS, M. (1995) Feminisms and the Self: The Web of Identity. London: Routledge.

HAMMERSLEY, M. (1992) 'On Feminist Methodology', Sociology, vol. 26, no. 2, pp. 187 - 206.

HAMMERSLEY, M. (1995) The Politics of Social Research. London: Sage.

HAMMERSLEY, M. and GOMM, R. (1997) 'Bias in Social Research', Sociological Research Online, vol. 2, no. 1, <http://www.socresonline.org.uk/2/1/2.html>

HARDING, S. (1986) The Science Question in Feminism. Ithaca, NY: Cornell University Press.

HUMPHRIES, B. (1997) 'From Critical Thought to Emancipatory Action: Contradictory Research goals?', Sociological Research Online, vol. 2, no.1, <http://www.socresonline.org.uk/2/1/3.html>

KELLER, E. F. (1989) 'Feminism and Science' in A. Garry and M. Pearsall (editors) Women, Knowledge and Reality: Explorations in Feminist Philosophy. London: Unwin Hyman.

MAYNARD, M. and PURVIS. J (editors) (1994) Researching Women's Lives from a Feminist Perspective. London: Taylor and Francis.

MYKHALOVSKIY, E. (1997) 'Reconsidering "Table Talk": Critical Thoughts on the Relationship Between Sociology, Autobiography, and Self-Indulgence' in R. Hertz (editor) Reflexivity and Voice. London: Sage Publications.

RAMAZANOGLU, C. (1992) 'On Feminist Methodology: Male Reason Versus Female Empowerment', Sociology, vol. 26, no. 2, pp. 207 - 212.

ROMM, N. (1997) 'Becoming More Accountable: A Comment on Hammersley and Gomm', Sociological Research Online, vol. 2, no. 3, <http://www.socresonline.org.uk/2/3/2.html>

ROSE, H. (1983) 'Hand, Brain and Heart: A Feminist Epistemology for the Natural Sciences', Signs, vol. 9, no. 1, pp. 73 - 90.

SILVERMAN, D. (1993) Interpreting Qualitative Data: Methods for Analysing Talk, Text, and Interaction. London: Sage Publications.

SMITH, D. (1974) 'Women's Perspective as a Radical Critique of Sociology', Sociological Inquiry, vol. 44, no. 1, pp. 7 - 13.

STANLEY, L. (1990) 'Moments of Writing: Is There a Feminist Auto/biography?', Gender and History, vol. 2, pp. 58 - 67.

STANLEY, L. and WISE, S. (1990) 'Method, Methodology and Epistemology in Feminist Research Processes' in L. Stanley (editor) Feminist Praxis: Research, Theory and Epistemology in Feminist Sociology. London: Routledge.

STANLEY, L. and WISE, S. (1993) Breaking Out Again: Feminist Ontology and Epistemology. London: Routledge.

TEMPLE, B. (1994) Needs Assessment: Approaches and Dilemmas. Applied research and consultancy centre, Department of Social Policy and Social Work, University of Manchester.

TEMPLE, B., LLOYD, M., PRESTON-SHOOT, M., and WUU, R. (1996) 'Combining Methods, Constructing Needs and Consulting Service Users: Reflections on a Research Project', Health and Social Care in the Community, vol. 4, no. 5, pp. 300 - 303.

TEMPLE, B. (1997) 'Watch Your Tongue: Issues in Translation and Cross-Cultural Research', Sociology, vol. 31, no. 3, pp. 607 - 618.

Copyright Sociological Research Online, 1997