How do I know that they know that I know what they want to know?

Citation

Barker, A. (2007), "How do I know that they know that I know what they want to know?", Qualitative Market Research, Vol. 10 No. 1. https://doi.org/10.1108/qmr.2007.21610aaf.001

How do I know that they know that I know what they want to know?

How do I know that they know that I know what they want to know?

A Review of: “How do I know what I think until I hear what I say” by Geoff Bayley (from the AQR/QRCA Conference, Atlanta 2006)

Should we throw away the discussion guide and make the letter follow the spirit of qualitative research? This is the specific question that is posed by Geoff Bayley's excellent AQR/QRCA paper “How do I know what I think until I hear what I say”.

This paper will strike a resounding chord with many commercial qualitative research practitioners who feel specifically frustrated by “death by topic guide” in some qualitative projects and more generally deprofessionalised by research buyers' occasional micro-management of research projects.

For this article, I want to look in more detail at Bayley's paper and also to consider some of the more general questions it raises about the particular ironies and tensions in commercial practice, thereby raising the question of whether these occur in academic practice also.

A brief definition of terms: in the commercial qualitative world the discussion guide or topic guide is, ostensibly, the open ended script or questionnaire that shapes the group discussion or individual interview.

One might argue that the issues which Bayley discusses in his paper, largely focusing on the tensions between the discussion guide for the researcher and the discussion guide for the client, make the question irrelevant for academics who might have more control over the process – they are their own client? However, whilst this might reveal a naivety on this author's part about academic qualitative practitioners, it is also true that the question of the philosophically inconsistent application of a positivistic approach to discussion guide design for use in a phenomenological research encounter is a more generally applicable question.

So what is the meat of the argument in this paper? First of all Bayley makes an observation:

If you have either run or observed a `Focus Group' in which the style of interviewing leaves structure behind and emphasizes open-ended enquiry, you cannot fail to observe that the Discussion Guide has only a loose relationship to what actually happens.

This, Bayley argues, is strange and worthy of further exploration, if not downright problematic. The bulk of the paper discusses why this might be happening, why it is a problem and what to do about it.

First of all why is this happening? Bayley turns initially to a small sample of qualitative practitioners for assistance in answering this and other questions. Rather pointedly he adopts a “self-script” methodology with each researcher-respondent writing freely about the subject stated simply as “discussion guides”.

First of all you have to write a discussion guide – it is, not unfairly, expected by clients. However, whether the guide means the same for the researcher as it is perceived to do for the client is a moot point:

Without any pre-emption, all of our contributors made a distinction in the role of the Discussion Guide for themselves and their perception of its role for the client. Providing

the client with a Discussion Guide is a `given' in the contemporary marketplace; it is institutionalised as a component of professional conduct.

The tension to which Bayley refers is that between the use of the discussion guide for the researcher and the purpose it is perceived to serve for the client. For the former, the discussion guide is not necessarily a document which will be actively used in the group discussion or interview, whilst for the latter it is perceived to be just that – a script or questionnaire and herein lies the central tension. Clearly qualitative research and researchers are most effective when freedom and flexibility to answer the key questions of the research whilst (apparently) following the consumers' agenda and this is difficult to do if following a script.

Bayley discusses what, in my experience, is an often highly codified dance where the act of submitting and discussing the discussion guide serves a practical function of ensuring that all issues have been flushed and fleshed out; it also serves an emotional purpose of making all parties (especially client) comfortable that the researcher in whom they must trust, has “got” the brief. When this goes awry and causes professional tension and preciousness is when it spills over into a too detailed discussion of question/probe wording, as if the discussion guide is to be used as a questionnaire.

In spite of the tension which can sometimes arise, some practitioners take a pragmatic view (and they are represented in Bayley's paper) and welcome the role that the “dance” has in the sometimes delicate and complex relationship between client and researcher.

I have worked with clients who I know to have a highly detailed focus when it comes to discussion guide design, but who at the same time would not expect the guide to be followed slavishly in the sessions themselves (in fact they would be horrified if this were to happen). I used to ask myself what was going on here, but quickly realised that this was both an extended part of the briefing process, also an active projection for all parties into what might be said in the groups/interviews via the detail of the guide and also a sort of emotional contract between researcher and client that says “everything you might be interested in is here in black and white. Now let go and trust me to use my skills as a moderator to explore these issues for you”.

Trust is at the heart of the matter. Researchers often feel they can only perform best when they are trusted to get on with the job in their own way (Bayley's paper has some interesting discussion on “being in the zone”). Clients, on the other hand, live in a much more accountable world than they used to – the success of the project is how they are judged and so they need to trust that they have properly briefed the researcher and that some of that accountability has been transferred in the process. This often translates into a focus on detail in the discussion guide, as it is the only solid object in an otherwise somewhat fluid process.

The respondents in Bayley's research conceded that whilst the best qualitative discussions and interviews are (apparently) free flowing, natural, non-directive, spontaneous, unpredictable happenings where meaning is created, the topic guide does still have a value. It helps the researcher to memorise or internalise the detailed issues in the brief, it helps them to rehearse and develop a personal approach to the project and also to project into the session and start to visualise how it will run.

What Bayley's paper sometimes explicitly, but mainly implicitly, opens is a fascinating can of worms. The dance of qualitative research is apparently free flowing, but often highly and skilfully choreographed; it is full also of “double speak” and “benign duplicity” from the discussion-guide-waltz to the following-the-consumers'-agenda-tango. However, provided that all this is ethical then this is not a problem and in fact is central to the pleasure, interest and insight of qualitative research.

What this paper also does is to sound some alarm bells about the danger of undermining the essence of the qualitative process by overly micro-managing it at the discussion guide and other stages. Related to the forcing of too many detailed questions into the guide, which will/could never be asked, I would add to this the increasing amount of stimulus, ideas, issues, questions, etc. that are crow-barred into modern qualitative research. It is not just with nostalgia that I recall exploring a maximum of 3 or 4 product concepts in the early days of my career compared with sometimes now looking at six positionings, eight packs, two products and assorted claims and descriptors, with an adcept thrown in at the end for a “quick read”.

Bayley talks about the acknowledged need in marketing departments for a more creative and insightful (i.e. qualitative) approach and contrasts this with the dysfunctional micro-management of qualitative research. This is an important point. The phenomenological perspective is of immense contemporary value and those of us who work within that perspective should resist its inappropriate “positivisation”. When the quality of qualitative is under threat, then a line must be drawn.

In all this the need for the client perspective is critical and Bayley promises this as the next step in his investigation. In addition to enthusiastically recommending this paper to readers, I also look forward to hearing the clients' view.

Finally the academic practitioner view would also be fascinating – the dynamics of the academic world are different, but perhaps the client-researcher tension similarly exists and perhaps this tension finds its focus on the discussion guide. A debate of these issues in the various practitioner communities can only be beneficial.

For more details please contact: Andy Barker, andy.barker@spinach.co.uk

Andy BarkerSpinach Research