Copyright Sociological Research Online,

Qualitative Evaluation

Order this book

Qualitative Evaluation is an overview and discussion of the state of qualitative evaluation, by an enthusiast. The first chapter, "Encountering Qualitative Evaluation", is a fast run- through present debates and politics, a summary of "strands", and a synopsis. Qualitative work is in development against the background of renewed enthusiasm for evidence-based practice. The latter is (still) naïve about measurement in the social sciences, and its advocates "fail to take power seriously". Shaw's version of "qualitative" is catholic, with Clifford Geertz carrying perhaps most weight. His purpose is to avoid wrong turnings in qualitative evaluation, and his stance is summarised in "strands" which, he remarks, were the outcome of the work on the book: Evaluation is distinguished by its evaluative purposes; it is research; qualitative methodology should usually be the methodology of choice; it should be inter- professional; it should attend non-evaluation theory and thinkers; it should distinguish policy, programme and practice evaluation; it should contribute to causal analysis and outcome evaluation; it is able to "problematize" its own analysis and solutions. These strands recur throughout, so the argument is complex and fitful.

Ch.2. is again a quick, but a useful review of the arguments of major evaluation theorists ­ big names, plus one category, "Critical Evaluation". Ch.3, oddly titled "Evidence from qualitative evaluation", is, in fact, the paradigm chapter: Shaw keeps it under control by tying it closely to evaluation thinkers. Paradigms are standard: Realist Postpositivism, Critical evaluation, Constructivism, Pragmatism: issues are focussed on to Realism and Relativism, and Rigour or Relevance. Arguments vary, at speed, between the highest metaphysics ­ "is there reality out there?" and selections from the immediately practical. Constructivism is relativist: he settles for an open-minded realism of hidden causal machineries. Ch.4, "Values, validity and the uses of evaluation" focusses on issues specifically evaluative: what sort of validity can be aimed at, given evaluative purposes ­ and paradigmatic commitments. Ch.5 is on "Evaluating programmes and policies": qualitative programme evaluators include process, emphasise theorising, include social change, and worry about rationality assumptions and accountability. Enlightenment counts for as much as instrumental utilisation. Ch.6 is on "Practitioner Evaluation": Research and evaluation by practitioners, Participatory research, Evaluation as a dimension of practice, and Evaluation for practitioners. Topics include tacit versus theoretical knowledge, experience in education and social work, the place of ethnography, participation, involvement and radicalism. Schon gets a high rating on reflection on practice. Ch.7, "Decisions for Evaluation Design" , reviews evaluability, with quantitative and qualitative contrasted, includes a very truncated "critical" section on social justice, then reviews designs which give a realist weight to qualitative studies. There are sections on case studies, and action research. Ch.8 looks at fieldwork methods, with examples from qualitative evaluation rather than more generally, and concludes with a further section on politicised and committed evaluation, a brief and partial look at "ethics". Ch.9, "Analysis: establishing claims", concludes the book by looking at how to make qualitative evaluation persuasive to the sceptical: topics include (realist) causation, critical ethics (again), adaptations of survey logic, and generalisation of qualitative conclusions. Throughout, examples are given, with extended ones in separated boxes. They give the required sustenance, but are often rather brief. An appendix offers exercises for each chapter: these are useful and thought-provoking.

In a disarming remark, the author notes that writing the book was a disconcerting experience ­ a path from 'knowledge' to ignorance. It has a strong feeling of work-in- progress, and of self-education, with the strengths and weaknesses that go with that. Qualitative evaluation is itself learning, quickly, using multiple sources and arguments, and is anything but tidy. However, the arguments are complex, often with sources offering a final word where full argument is needed, and with repetition ­ of argument and quotation. Anyone will find much to argue with, with their opponent's position often curtailed. "Constructivists" of all shades are unlikely to be pleased or satisfied. It should not be a first book on evaluation, nor on qualitative analysis. Ch.2 is a better starting point than Ch.1, for relative beginners. For those with some acquaintance with the fields, it is a useful if breathless reminder and summary, and undeniably stimulating. Overall, the book is very worthwhile in a well-specified context of use: its final appeal is its dedication and enthusiasm.

Dianne Phillips
Manchester Metropolitan University

Copyright Sociological Research Online,