the-sage-handbook-of-qualitative-data-collection 2

Collecting Qualitative Data: A Realist
Approach
In: The SAGE Handbook of Qualitative Data Collection
By: Joseph A. Maxwell
Edited by: Uwe Flick
Pub. Date: 2018
Access Date: October 12, 2021
Publishing Company: SAGE Publications Ltd
City: London
Print ISBN: 9781473952133
Online ISBN: 9781526416070
DOI: https://dx.doi.org/10.4135/9781526416070
Print pages: 19-31
© 2018 SAGE Publications Ltd All Rights Reserved.
This PDF has been generated from SAGE Research Methods. Please note that the pagination of the
online version will vary from the pagination of the print book.
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
Collecting Qualitative Data: A Realist Approach
Joseph A. Maxwell
Introduction
This chapter considers qualitative data collection from the perspective of realism, a philosophical position
that has gained significant attention in discussions of research and evaluation methods (e.g. Clark, 2008;
Hammersley, 1992, 1998, 2008, 2009; Maxwell, 1990a, 1990b, 1992, 2012a, 2012b; Madill, 2008; Pawson,
2006; Pawson and Tilley, 1997; Sayer, 1992). I argue that this position has important implications for
qualitative research, including qualitative data collection. In this chapter, I provide a brief introduction to
realism, and then discuss how this approach can inform the theory and practice of qualitative data collection.
What is realism?
There are many varieties of realism across the philosophical landscape. In this chapter, I focus on what
is often called ‘critical realism,’ a term that is usually associated (at least in the UK) with the work of the
philosopher Roy Bhaskar (1975, 1978, 1989, 2011; Archer et al., 1998)1. However, this term was used even
earlier (apparently independently of Bhaskar's work) by the American social researcher and methodologist
Donald Campbell (1974, p. 432; Cook and Campbell, 1979, pp. 28–30). For both Bhaskar and Campbell,
a key feature of critical realism is the combination of ontological realism (the belief that there is a real
world that exists independently of our perceptions and constructions) with epistemological constructivism (our
understanding of the world is inevitably our own construction; there can be no perception or understanding of
reality that is not mediated by our conceptual ‘lens') (Bhaskar, 1989, pp. 12–25; Campbell, 1988, p. 447).
Many other philosophers and methodologists, including some well-known qualitative researchers
(Hammersley, 1992, 1998; Huberman and Miles, 1985), have taken similar positions, but used a variety of
different terms for these, obscuring the widespread acceptance of this stance (Madill, 2008). One prominent
advocate of realism in social research has explicitly aligned himself with Campbell's critical realism, rather
than Bhaskar's more recent advocacy of what he called ‘dialectical critical realism’ (Pawson, 2006, p. 20). For
this reason, I will use the term ‘critical realism’ broadly, to include a range of positions that explicitly combine
ontological realism with epistemological constructivism. (For a more detailed discussion of these issues, and
their implications for qualitative research, see Maxwell, 2012b.)
Following Kuhn's and others’ demonstrations that all observation is ‘theory-laden', and that our understanding
of the world is inherently shaped by our prior ideas and assumptions about the world, a constructivist
epistemology has been widely, although often implicitly, accepted by researchers. For example, Shadish
et al., in their influential work on experimental and quasi-experimental designs, stated that ‘all scientists
Page 2 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
are epistemological constructivists and relativists’ (2002, p. 29); there is no possibility of purely ‘objective’
or ‘theory-neutral’ description independent of some particular perspective and theoretical stance. This is
consistent with our common-sense belief that our everyday perceptions and interpretations may be mistaken.
Similarly, most of us accept a realist ontology in our everyday lives. When our cars break down, we believe
that there is a real problem that a mechanic can identify and repair. Most of us also believe that global
warming is a real phenomenon that is occurring independently of our beliefs about this. Thomas Schwandt,
in his Dictionary of Qualitative Research, argued that
On a daily basis, most of us probably behave as garden-variety empirical realists – that is, we act as
if the objects in the world (things, events, structures, people, meanings, etc.) exist as independent in
some way from our experience with them. We also regard society, institutions, feelings, intelligence,
poverty, disability, and so on as being just as real as the toes on our feet and the sun in the sky.
(2015, p. 264)
Although realism is often (incorrectly) associated with quantitative rather than qualitative approaches, a realist
ontology ‘is compatible with many qualitative methods and is the position of choice of many qualitative
researchers’ (Madill, 2008, p. 731).
However, a substantial number of qualitative researchers have adopted, at least in their espoused
philosophical views, a thoroughgoing constructivism that denies the existence of any reality external to our
constructions, a position most systematically developed by Guba and Lincoln (1989; Lincoln and Guba,
2013). A key argument for this view has been that a constructivist stance on epistemology makes a
realist ontology irrelevant, since we have no way to access this ‘reality’ that avoids the constraints of
a constructivist epistemology (Smith, 2008; Smith and Deemer, 2000). Lincoln similarly argued that ‘the
naturalistic/constructivist paradigm effectively brought about the irrelevance of the distinction between
ontology and epistemology’ (1995, p. 286), leading to what Lincoln and Guba called the ‘ontological/
epistemological collapse’ (2000, pp. 175–6).
Certainly, a constructivist epistemology rules out the possibility of any direct or ‘objective’ perception of reality.
However, Lincoln's and Smith's arguments ignore the possibility of testing our theories about reality, seeking
evidence that allows us to assess the plausibility of these theories (Campbell, 1984; Maxwell, 2012b, chapter
8). While this strategy can never lead to a complete or certain understanding, it does enable us to improve
the ability of our theories and interpretations to capture something about the phenomena we study, and thus
to increase our ability to deal with these phenomena.
A classic statement of this position was by Herbert Blumer, the main architect of the approach termed
‘symbolic interactionism', an important influence on the development of qualitative research (Williams, 2008).
Blumer argued that
I shall begin with the redundant assertion that an empirical science presupposes the existence of an
empirical world. Such an empirical world exists as something available for observation, study, and
Page 3 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
analysis. It stands over against the scientific observer, with a character that has to be dug out and
established through observation, study, and analysis … ‘Reality’ for empirical science exists only in
the empirical world. (1969, pp. 21–2)
However, Blumer joined this ontological realism with epistemological constructivism (although, since this term
was not available to him, he referred to this position as ‘idealism'). He asserted that
the empirical necessarily exists always in the form of human pictures and conceptions of it. However,
this does not shift ‘reality', as so many conclude, from the empirical world to the realm of imagery and
conception … [This] position is untenable because the empirical world can ‘talk back’ to our pictures
of it or assertions about it – talk back in the sense of challenging and resisting, or not bending to, our
images or conceptions of it. (1969, p. 22)
The combination of a realist ontology and a constructivist epistemology is not only an important (if often
implicit) stance in qualitative research, but is a prominent feature of our everyday strategies for dealing with
the world. The philosopher Hilary Putnam stated that
The notion that our words and life are constrained by a reality not of our own invention plays a
deep role in our lives and is to be respected. The source of the puzzlement lies in the common
philosophical error of supposing that the term ‘reality’ must refer to a single superthing instead of
looking at the ways in which we endlessly renegotiate – and are forced to renegotiate – our notion
of reality as our language and our life develop. (1999, p. 9)
Ontological realism is also shared by many versions of pragmatism (Maxcy, 2003, p. 56; Biesta, 2010, p. 111);
Putnam said he should have called his version of realism ‘pragmatic realism’ (Kuenne, 2002, pp. 149–50).
Buchler (1940) said of Charles Peirce, the founder of American pragmatism, that
Underlying every phase of Peirce's thought is his realism. The supposition that there are real things
– the real is ‘that whose characters are independent of what anybody may think them to be’ – he
regards as the ‘fundamental hypothesis’ of science, for it alone explains the manner in which minds
are compelled to agreement. (p. xiv)
There are a number of positions typically incorporated in critical realism that are particularly compatible with
qualitative research. First, most critical realists hold that meaning (including concepts, beliefs, intentions,
values, and other ‘mental’ phenomena) is just as real as physical phenomena (as implied in the Schwandt
quote above); we simply don't have access to the kinds of evidence about these that we have for many
physical entities. This acceptance of the reality of mental phenomena is a major difference between critical
realism and logical positivism. The latter was fundamentally anti-realist in rejecting any reference to
‘unobservable’ entities unless they could be ‘operationally defined’ in terms of the methods and data we use to
gain an understanding of these; positivists typically denied that theoretical terms were more than ‘convenient
fictions’ that were useful in prediction, but had no claim to any ‘reality'.
Page 4 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
However, the actual nature of mental phenomena, and their relationship to physical phenomena, is currently
a contested issue in philosophy. The most credible theory, in my view, is that of the philosopher Hilary Putnam
(1999), who argued for the legitimacy of both ‘mental’ and ‘physical’ ways of making sense of the world,
and for a distinction between mental and physical perspectives or languages, both referring to reality, but
from different conceptual standpoints. This is a rejection of both reductionist physicalism (mental phenomena
will ultimately be reducible to physical, e.g. neurological, phenomena) and dualism (mind and matter are
separate, independent, and irreducible entities with nothing in common). The ‘mental’ framework pertains to
phenomena that might, in principle, also be conceptualized in physical terms, but it provides an essential
perspective on these phenomena that physical theories currently lack, and for which the latter may never be
able to substitute (Maxwell, 2012b, chapter 2).
Thus, for critical realists in the social sciences, the meanings, beliefs, and values held by individuals are
part of the reality that we seek to understand, although an epistemologically distinct part from observable
physical objects and behavior. From this perspective, qualitative data and interpretations are data about real
phenomena, including behavior, meanings, and the processes that connect these.
Second, critical realists assert the reality of causation, a concept that has been rejected by many qualitative
researchers for some of the same reasons that they rejected ontological realism (Maxwell, 2012a, 2012b,
chapter 3). Accepting the ‘reality’ of causation seemed to acknowledge the existence of objective causal
‘laws’ that were independent of particular situations and the beliefs of participants; Guba and Lincoln (1989)
asserted that ‘there exist multiple, socially constructed realities ungoverned by natural laws, causal or
otherwise’ (p. 86) and that ‘“causes” and “effects” do not exist except by imputation’ (p. 44). This assumption
was based on a largely accurate perception of the positivist ‘covering law’ view of causation, which accepted
Hume's argument that all we could know of causation was the observable regularities in associations of
events, and which rejected any reference to unobservable entities and mechanisms. This view inherently
privileges quantitative research, with randomized experiments as the ‘gold standard’ for causal investigation
(Maxwell, 2004).
Unfortunately, qualitative researchers’ rejection of causation failed to recognize that an alternative, realist view
of causation was being developed at about the same time, following the demise of logical positivism. In this
view, causation inherently refers to the actual (even if unobservable) mechanisms and processes that result in
particular outcomes (e.g. Salmon, 1989, 1998). In this view, ‘explanatory knowledge opens up the black boxes
of nature to reveal their inner workings. It exhibits the ways in which the things we want to explain come about’
(Salmon, 1989, p. 182). This alternative understanding of causation is particularly compatible with qualitative
research, with its emphasis on the processes taking place in specific settings (Maxwell, 2012a). In a report on
a realist qualitative study, Fletcher (2016) stated, ‘I identify two key causal mechanisms shaping the lives of
farm women and suggest a future direction for feminist political economy theory to more effectively analyze
women's work in agricultural contexts’ (p. 1).
This stance also permits combining mental and physical perspectives in a single theory, and accepts that
mental phenomena are causally connected to physical ones. As the critical realist Bhaskar stated,
Page 5 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
Two crude philosophical distinctions, between mind and body and reasons and causes, have done
untold damage here … reasons, and social forms generally, must be causes (as well as effects)
… we have to see the natural and social dimensions of existence as in continuous dynamic causal
interaction. (1989, p. 6)
Finally, critical realism emphasizes the importance of context. The positivist approach to causation focused
on general causal laws, and ignored (or treated simply as ‘error') the influence of contexts on outcomes. In
contrast, the mechanisms postulated in a realist causal theory are seen not as general laws, or as having
invariant outcomes, but as situationally contingent; their actual context is inextricably part of the causal
process (Cartwright, 1999: 73; Little, 1998, 197 ff.; Pawson and Tilley, 1997). The focus of qualitative research
on particular contexts and processes, mental and social as well as physical, provides a powerful tool for
understanding the actual causal interactions that led to a specific outcome in that situation.
A realist ontology has many broader implications for qualitative research – for example, questioning the
ontological status of the ‘preconceptualized experience’ postulated by some phenomenological approaches.
However, the remainder of this chapter focuses on the collection of qualitative data, through interviewing,
observation, and other means, and how a realist perspective can suggest ways to make these more
productive. The primary emphasis is on seeing qualitative data, not simply as texts to be interpreted, or as
the constructions of participants (although they obviously may be these), but as evidence about the real
phenomena (physical, behavioral, and mental) that the researcher wants to understand (Maxwell, 2012b,
chapter 8). This inherently involves issues of validity, but the focus will be on theories of, and procedures for,
data collection.
It is also useful to view research designs as real entities – not simply a plan for the research, but also the
actual conceptualizations and practices employed in the study, which may be different from the intended
or stated design (Maxwell, 2013). Kaplan (1964, p. 8) described this as the difference between the
‘reconstructed logic’ and the ‘logic-in-use’ of a study. The ‘reconstructed design’ includes what may be
presented in a proposal or research publication, or even what the researcher believes about the study;
the ‘design-in-use’ is the design as realized ‘on the ground', in the actual conduct of the research. A
planned ‘unobtrusive’ observation, or an ‘open-ended’ interview, may be reactive or leading in ways that the
researcher isn't aware of, and researchers may have goals, assumptions, and questions that they haven't
consciously recognized. Thus, a realist stance implies a reflective and critical attitude toward one's own
beliefs, plans, and actions.
There are several implications of critical realism that apply generally to all strategies for qualitative data
collection. The first is that, because context matters, the results of any qualitative study are local results,
with no inherent generalizability or transferability. Systematic attention to sampling issues can support the
internal generalizability of the results to the particular setting, case, or population studied, but generalization
or transferability beyond these depends on different sorts of arguments and evidence (Maxwell and Chmiel,
2014). In addition, the transferability of conclusions about the processes occurring in a particular case or
population doesn't imply that the outcomes will necessarily be the same in other contexts (Becker, 1991;
Page 6 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
Cartwright and Hardie, 2012).
Similarly, data collection (and analysis) strategies need to be appropriate to the context in which they are
used; there are no generic strategies that can automatically be applied in any situation to provide valid results.
There is general agreement, across measurement, quantitative, and qualitative approaches, that no method
or design guarantees the validity of the inferences or conclusions (Brinberg and McGrath, 1985; Maxwell,
2017; Messick, 1995, 1998; Phillips, 1987, p. 121; Shadish et al., 2002). Certainly methods matter, but they
can never be assumed to generate valid data or conclusion without consideration of contextual issues, as
well as of the purposes and questions the data are intended to address.
Finally, a realist focus on local and context-specific phenomena needs to recognize that individuals and
situations are diverse, and thus that qualitative researchers need to use methods that respect and clarify
that diversity. Both quantitative and qualitative methods contain biases that, intentionally or inadvertently,
tend to conceal the existence of diversity and make it more difficult to understand its nature and influence
(Maxwell, 2012b, chapter 4). In quantitative research, data on a diverse set of individuals or settings are
typically reduced to a mean value (Rose, 2016), and diversity itself is summarized as the ‘standard deviation’
of that data from the mean, ignoring the complex reality of that diversity.
However, qualitative research, which accepts the particularity of its subject and is aimed at developing indepth understandings of single cases or small samples, also has methodological biases toward uniformity.
The sample size and sampling strategies used in qualitative studies are often inadequate to identify and
characterize the actual diversity that exists in the setting or population studied. In addition, qualitative
researchers often use data collection and analysis methods that emphasize uniformity, such as relying on key
informants and focusing on shared themes and concepts. An important step toward avoiding these errors is
to recognize both the reality of diversity and the potential for uniformist biases in interpreting this.
Researcher subjectivity and research relationships
In order to obtain data about the settings or participants that qualitative researchers select, they need to
establish relationships, both with potential participants, and with gatekeepers or other influential persons
who may control or facilitate access to these settings or participants. With few exceptions, researchers need
to actually interact (including electronic interaction) with participants and other people in the settings they
study. Hammersley and Atkinson (2007, pp. 16–22) referred to this interaction as ‘reflexivity': ‘the fact that
we are part of the social world we study’ (p. 21) and must therefore understand how we influence, and are
influenced by, this world. This mutual influence is both a necessary aspect and facilitator of data collection,
and a potential validity threat to your conclusions.
The personal characteristics that you bring to the research play a major role in this interaction. There
is a saying that in quantitative research, the researcher has instruments, but in qualitative research, the
researcher is the instrument (Brodsky, 2008). These two components of methods, the researcher's identity
Page 7 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
and perspective, and the research relationships that these influence, are part of the actual methods used in
the research. Here, I use ‘methods’ in a broad sense that includes all of the things that the researcher actually
does to acquire and make sense of the data collected. By calling this a ‘realist’ model, I mean that I see these
components (as well as others not discussed here) not simply as theoretical abstractions or methodological
principles, but as real phenomena, things that have an influence on the research, the data collected, and the
conclusions.
Researcher Subjectivity
The traditional view of ‘subjectivity', derived from logical positivism, treats this as ‘bias', something to be
eliminated, or at least controlled, in the interest of ‘objectivity'. The collapse of positivism implies that no
such ‘immaculate perception’ exists, and that the standpoint of the researcher is inevitably involved in, and
interacts with, the data that are collected; this is a major reason for the ‘epistemological constructivism’ of
critical realism. This perspective requires the researcher to take account of the actual beliefs, values, and
dispositions that she brings to the study, which can serve as valuable resources, as well as possible sources
of distortion or lack of comprehension.
The grain of truth in the traditional view of subjectivity as bias (with which critical realists would agree) is that
researchers’ personal (and often unexamined) motives, beliefs, and theories have important consequences
for the validity of their conclusions. If your research decisions and data analyses are based on personal
desires or theoretical commitments without a careful assessment of the implications of these for your methods
and conclusions, you are in danger of creating a flawed study or reaching incorrect conclusions.
However, rather than treating subjectivity as a variable to be controlled and ideally reduced to zero, critical
realists see it as a component of the actual process of understanding, one that can have a variety of
consequences, both good and bad. Even researchers who take an interpretive or constructivist stance toward
qualitative research often approach this process in ways that are quite consistent with a critical realist
understanding. For example, Tappan states,
the interpreter's perspective and understanding initially shapes his interpretation of a given
phenomenon, but that interpretation is open to revision and elaboration as it interacts with the
phenomenon in question, and as the perspective and understanding of the interpreter, including his
biases and blind spots, are revealed and evaluated. (2001, p. 50)
The main issue for data collection is how, specifically, one becomes aware of this subjectivity and its
consequences, and how one uses this subjectivity productively in the research. One strategy for
understanding the influence of these beliefs and prior experiences on your research is reflective analysis and
writing, or what qualitative researchers often call ‘memos'.2 Peshkin, in order to better understand his own
subjectivity during his study of a multiethnic school and community (1991), ‘looked for the warm and the cool
spots, the emergence of positive and negative feelings, the experiences I wanted to have more of or to avoid’
Page 8 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
(p. 287), and recorded these on 5 × 8-inch cards. (A diary or field journal, or a computer file of memos, would
also work, but index cards allow more immediacy of recording.) In analyzing these cards, he identified six ‘I's’
– aspects of his identity that influenced his research – and was able to better understand both the benefits
and the risks of these identities.
Another strategy is an exercise that I call a ‘researcher identity memo'. The purpose of this memo is to
help students examine their background, purposes, assumptions, feelings, and values as they relate to their
research, and to discover what resources and potential concerns their identity and experience may create.
Preissle (2008) refers to such memos, which may be published as well as used for personal reflection, as
‘subjectivity statements'. For more discussion, and examples of such memos, see Maxwell (2013, chapter 3).
Research Relationships
The second component of the researcher's role is the researcher's relationships with those studied. This
relationship is usually addressed in qualitative methods books and research reports, but it has often been
reduced to narrow and oversimplified concepts such as ‘entry', ‘access', or ‘rapport', something to be ‘attained’
at some point. These concepts obscure the complexity of research relationships, which are real, evolving
phenomena that shape the context within which the research is conducted, and have a profound influence on
the research and its results. Thus, their actual nature and operation need to be understood in order to use
them productively. As stated earlier, these relationships are an essential part of the ‘design-in-use’ of a study;
they form one component of the methods that the researcher uses to collect data, as well as influencing the
analysis of these data. Hammersley and Atkinson argued:
Once we abandon the idea that the social character of research can be standardized out or avoided
by becoming a ‘fly on the wall’ or a ‘full participant', the role of the researcher as active participant in
the research process becomes clear. (2007, p. 17)
There are several potential pitfalls in developing relationships with participants. Seidman (1998, pp. 80–2)
argued that it is possible to have too much rapport, as well as too little, but the nature of the relationship,
and its consequences, and not simply the amount of rapport, is critical. ‘Rapport’ may be an exploitative
or oppressive imposition on a participant; Burman (2001, p. 263) criticized the concept of rapport as a
commodification of relationship into manipulative strategies to promote disclosure. Lawrence-Lightfoot and
Hoffman Davis (1997, p. 135) also criticized the tendency to treat relationship as a tool or strategy for
gaining access to data, rather than as a connection. They argued that ‘relationships that are complex, fluid,
symmetric, and reciprocal – that are shaped by both researchers and actors – reflect a more responsible
ethical stance and are likely to yield deeper data and better social science’ (1997, pp. 137–8), and
emphasized the continual creation and renegotiation of trust, intimacy, and reciprocity. From a realist
perspective, this involves seeing research relationships neither as variables to be controlled or manipulated,
nor simply as ‘constructions’ created by the researcher and/or participants, but instead as real, complex
processes that have profound, and often unanticipated, consequences for the research. Glesne (2011, pp.
Page 9 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
141–50), Tolman and Brydon-Miller (2001), and McGinn (2008) provide detailed discussions of the different
aspects of field relations that need to be considered in planning and conducting a qualitative study.
However, there is often an unstated assumption that difference per se is an inherent problem for relationship
and dialog, one that must be overcome by recognizing or creating commonalities. This is a belief with
deep roots in Western social thought (Maxwell, 2012a, chapter 4), one that has been challenged by
postmodernism. Critical realism is consistent with the postmodern view that diversity is real and fundamental;
researchers need to guard against romantic and illusory assumptions of equality and intimacy that distort the
actual relationships they engage in, and they need to develop strategies that enable them to understand the
actual nature, amount, and consequences of diversity in their relationships, as discussed above.
Data Collection Methods
The main implication of realism for qualitative data collection is that data are usefully seen, not simply as
‘texts’ to be interpreted, or as the ‘constructions’ of participants (although some of them are these), but as
evidence for real phenomena and processes (including mental phenomena and processes). These data can
be used to make inferences about these phenomena, which can then be tested against additional data.
This follows directly from the basic premises of critical realism: that there is a real world that we seek to
understand, but that our understandings of this world are inevitably incomplete and fallible, and unavoidably
shaped by the particular assumptions and perspective that we bring to the research – the ‘lens’ through which
we view the world.
A key implication of this view for data collection is the importance, in planning and conducting data collection,
to consider how the data that you collect can enable you to develop and test your emerging understandings
of the phenomena you are studying. Both the use of data to develop theory and the ‘verification of statements
against data’ (Strauss and Corbin, 1990, p. 108) are key aspects of the ‘grounded theory’ approach to
qualitative research, but the latter use of data, to explicitly test your ideas, is less commonly addressed in
the qualitative literature. This is sometimes equated with the ‘hypothesis testing’ of quantitative research, and
seen as inappropriate for qualitative research.
However, thinking about the data that you could collect in terms of how these could expand, support, or test
your current understandings is not the same as the statistical testing of prior, theoretically derived hypotheses,
which in any case is now highly problematic in quantitative research (Cohen, 1994; Gigerenzer, 2004; Nuzzo,
2014; Trafimow and Marks, 2015). It simply requires constantly being aware of how your views might be
wrong or inadequately developed, and planning your data collection explicitly to address these issues, rather
than simply proceeding along a predesigned path and accumulating data with no clear sense of how to use
your data to support and test your preliminary conclusions.
Two key issues in selecting and using data collection methods are the relationship between your research
questions and data collection methods, and the joint use of different methods, often termed ‘triangulation'.
Page 10 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
The Relationship Between Research Questions and Data Collection
Methods
The methods used to collect data (including interview questions) don't necessarily resemble, or follow by
logical deduction from, the research questions; the two are both real parts of the design, but distinct and
separate parts. This can be a source of confusion, because researchers often talk about ‘operationalizing’
their research questions, or of ‘translating’ the research questions into interview questions (see Roulston
and Choi, Chapter 15, this volume). Such language is a vestige of logical positivist views of the relationship
between theory and methods. From a realist perspective, although research questions necessarily inform
data collection methods, there is no way to mechanically ‘convert’ research questions into methods. Your
methods are the means to answering your research questions, not a logical transformation of the latter,
and depend fundamentally on the actual context of the research and the researcher's relationships with
participants. Research questions formulate what you want to understand; your interview questions are what
you ask people in order to gain that understanding. The development of good interview questions (and
observational strategies) requires creativity and insight, rather than a simple translation of the research
questions into an interview guide or observation schedule, and depends fundamentally on how the interview
questions and observational strategies will actually work in practice in that context.
There are two important implications that the lack of a direct logical connection between research questions
and interview questions has for your research. First, you need to anticipate, as best you can, how particular
questions will actually work in practice – how people will understand them, and how they are likely to respond
in the actual context of the interview. Try to put yourself in your interviewee's place and imagine how you
would react to these questions (this is another use of ‘thought experiments'), and get feedback from others
on how they think the questions (and the interview guide as a whole) will work. Second, if at all possible,
you should pilot-test your interview guide with people who are as much like your planned interviewees as
possible, to determine if the questions work as intended, and what revisions you may need to make.
This lack of a deductive relationship between questions and methods also holds, more obviously, for
observation and other data collection methods. As with interviews, you need to anticipate what information
you will actually be able to collect, in the setting studied, using particular observational or other methods,
and how this information will contribute to your understanding of the issues you are studying, as discussed
above. If possible, you should pretest these methods to determine if they will actually provide this information.
Your data collection strategies will probably go through a period of focusing and revision, even in a carefully
designed study, to enable them to better provide the data that you need to answer your research questions
and to address any plausible validity threats to these answers.
The Uses of Multiple Qualitative Methods
Collecting information using a variety of sources and methods reduces the risk that your conclusions will
Page 11 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
reflect only the systematic biases or limitations of a specific source or method, and allows you to gain a
broader and more secure understanding of the issues you are investigating. This strategy is usually called
‘triangulation’ (see Flick, Chapter 34, this volume), but the term has been used in both a broad and a narrow
sense. The former refers to any use of multiple methods; the latter refers specifically to using a second
method as a check on the results of another method, to confirm or challenge these results (Rothbauer, 2008).
The narrow definition has been widely used, but Greene (2007, pp. 98–104) has argued that this purpose
(corroboration or convergence on a single conclusion) is often less valuable than using multiple methods
for complementarity (revealing different aspects of a single complex phenomenon), expansion (investigating
different phenomena that interact and need to be understood jointly), or initiation (to generate divergent or
contradictory interpretations and fresh insights, or to force the researcher to seek a deeper and more complex
understanding).
One belief that inhibits triangulation is the widespread (though often implicit) assumption that observation
is mainly useful for describing behavior and events, while interviewing is mainly useful for obtaining the
perspectives of participants. It is true that the immediate result of observation is description, but this is equally
true of interviewing: the latter gives you a description of what the informant said, not a direct understanding
of their perspective. Generating an interpretation of someone's perspective is inherently a matter of inference
from descriptions of their behavior (including verbal behavior), regardless of whether the data are derived
from observations, interviews, or some other source such as written documents (Maxwell, 1992).
While interviewing is often an efficient and valid way of understanding someone's perspective (see Roulston
and Choi, Chapter 15, this volume), observation (see Wästerfors, Chapter 20, this volume) can enable you
to draw inferences about this perspective that you couldn't obtain by relying exclusively on interview data.
This is particularly important for getting at tacit understandings and ‘theory-in-use', as well as aspects of the
participants’ perspective that they are reluctant to directly state in interviews. For example, watching how a
teacher responds to boys’ and girls’ questions in a science class (e.g. Sadker and Sadker, 1994) may provide
a much better understanding of the teacher's actual views about gender and science than what the teacher
says in an interview.
Conversely, although observation often provides a direct and powerful way of learning about peoples’
behavior and the contexts in which this occurs, interviewing can also be a valuable way of gaining a
description of actions and events–often the only way, for events that took place in the past or to which
the researcher can't gain observational access. Interviews can also provide additional information that was
missed in observation, and can be used to check the accuracy of the observations.
However, in order for interviewing to be most useful for this purpose, interview questions need to ask about
specific times and events, tapping into what has been termed ‘episodic memory', an important and distinct
neurocognitive memory system (Dere et al., 2008; Tulving, 1983; Tulving and Craik, 2000). In this system,
information is organized by temporal sequencing and spatial connection, rather than abstractly in terms of
semantic relationships.
Page 12 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
Tulving argued that this memory system makes possible mental ‘time travel', uniquely enabling interviewees
to retrieve their own previous experiences, and Flick (2000) applied this concept to qualitative interviewing,
developing a specific procedure for accessing episodic memory that he called episodic interviewing. Similarly,
Weiss (1994, pp. 72–6) stated that asking a question in present tense (e.g. ‘What happens while you're
waiting to be called [in a court case]?') elicits a generalized account, and that when respondents provide such
an account, ‘their description expresses a kind of theory about what is most typical or most nearly essential’
(1994, pp. 72–3) in such situations, rather than a concrete description of what actually happened. The latter
is better obtained by using past tense (‘What happened while you were waiting to be called?') to refer to a
particular occasion, or by questions such as ‘Can you walk me through that incident?'
Weiss (1994) also argued, however, that generalized accounts permit respondents to minimize information
about which they feel diffident, and to avoid potentially embarrassing details; these are more difficult to do in
recounting an actual experience. For this reason, a researcher should be reasonably sure that her relationship
with the participant will support asking for a description of a particular event, and should have thought about
how to respond if the participant seems uncomfortable. In this situation, the joint use of both generalized,
present tense and specific, past-tense questions, as with the joint use of observations and interviews, can
address the same issues and research questions, but from different perspectives. In both of these situations,
use of multiple methods can provide a more complete and accurate account than either of them could alone.
Conclusion
In summary, I have tried to indicate some ways in which a realist perspective can usefully inform qualitative
data collection. The common-sense, though usually implicit, ontological realism of many qualitative
researchers is already well incorporated into much qualitative research practice – for example, in the attention
given to the actual perceptions of participants, the context within which the research takes place, and the
ways in which these influence, and are influenced by, the methods used and the results of these. However,
explicitly seeing the goal of qualitative research as gaining a better understanding of real phenomena, mental
and physical, as well as seeing the processes and practice of qualitative research as conducted in a real
context, which the researcher must interact with and adequately understand, can help researchers to better
address all of these aspects of collecting qualitative data.
Notes
1. Bhaskar originally used the terms ‘transcendental realism’ (1975) and ‘critical naturalism’ (1978) for his
views, and only later combined these as ‘critical realism', http://en.wikipedia.org/wiki/Roy_Bhaskar.
2. Although ‘memo’ is used in the grounded theory tradition specifically for theoretical reflections written
during data analysis, I am using the term more broadly for any reflective writing done during, or in preparation
for, research (Bazeley, 2013, p. 103; Schwandt, 2015, pp. 196–7).
Page 13 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
Further Reading
Hammersley, Martyn (1992) ‘Ethnography and realism', in Martyn Hammersley, What's Wrong with
Ethnography? Methodological Explorations. London: Routledge, pp. 43–56.
Maxwell, Joseph A. (2012b) A Realist Approach for Qualitative Research. Thousand Oaks, CA: Sage.
Pawson, Ray, and Tilley, Nick (1997) Realistic Evaluation. London: Sage.
References
Archer, Margaret, Bhaskar, Roy, Collier, Andrew, Lawson, Tony, and Norrie, Alan (Eds.) (1998) Critical
Realism: Essential Readings. London: Routledge.
Bazeley, Pat (2013) Qualitative Data Analysis: Practical Strategies. Thousand Oaks, CA: Sage.
Becker, Howard S. (1991) ‘Generalizing from case studies', in E. W. Eisner and Alan Peshkin (Eds.),
Qualitative Inquiry in Education: The Continuing Debate. New York: Teachers College Press, pp. 233–42.
Bhaskar, Roy (1975) A Realist Theory of Science. London: Verso.
Bhaskar, Roy (1978) The Possibility of Naturalism. Brighton: Harvester Press.
Bhaskar, Roy (1989) Reclaiming Reality: A Critical Introduction to Contemporary Philosophy. London,
England: Verso.
Bhaskar, Roy (2011) Critical Realism: A Brief Introduction. London: Routledge.
Biesta, Gert (2010) ‘Pragmatism and the philosophical foundations of mixed methods research', in Abbas
Tashakkori and Charles Teddlie (Eds.), The SAGE Handbook of Mixed Methods in Social and Behavioral
Research (2nd edn). Thousand Oaks, CA: Sage, pp. 95–118.
Blumer, Herbert (1969) ‘The methodological position of symbolic interactionism', in Herbert Blumer (ed.),
Symbolic Interactionism: Perspective and Method. Berkeley, CA: University of California Press, pp. 1–60.
Brinberg, David, and McGrath, Joseph Edward (1985) Validity and the Research Process. Newbury Park,
CA: Sage.
Brodsky, Anne (2008) ‘Researcher as instrument', in Lisa Given (Ed.), The SAGE Encyclopedia of
Qualitative Research Methods. Thousand Oaks, CA: Sage, pp. 766–7.
Burman, Erica (2001) ‘Minding the gap: Positivism, psychology, and the politics of qualitative methods',
Page 14 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
in Deborah L. Tolman and Mary Brydon-Miller (Eds.), From Subjects to Subjectivities: A Handbook of
Interpretive and Participatory Methods. New York: New York University Press, pp. 259–75.
Campbell, Donald T. (1974) ‘Evolutionary epistemology', in Paul Arthur Schilpp (Ed.), The Philosophy
of Karl Popper, pp. 413–63. Reprinted in Donald T. Campbell, (1988). Methodology and Epistemology for
Social Science: Selected Papers (S. Overman, Ed.) Chicago, IL: University of Chicago Press, pp. 393–434.
Campbell, Donald T. (1984) ‘Can we be scientific in applied social science?' in Ross F. Conner, David
G. Altman, and Christine Jackson (Eds.), Evaluation Studies Review Annual, vol. 9, pp. 26–48. Reprinted
in Donald T. Campbell (1988) Methodology and Epistemology for Social Science: Selected Papers (S.
Overman, Ed.) Chicago, IL: University of Chicago Press, pp. 393–434.
Campbell, Donald T. (1988). Methodology and Epistemology for Social Science: Selected Papers (S.
Overman, Ed.) Chicago, IL: University of Chicago Press.
Cartwright, Nancy (1999) The Dappled World: A Study of the Boundaries of Science. Cambridge:
Cambridge University Press.
Cartwright, Nancy, and Hardie, Jeremy (2012) Evidence-based Policy: A Practical Guide to Doing it Better.
Oxford: Oxford University Press.
Clark, Alexander M. (2008) ‘Critical realism', in L. Given (Ed.), The SAGE Encyclopedia of Qualitative
Research Methods. Thousand Oaks, CA: Sage, pp. 167–70.
Cohen, J. (1994) The Earth is round (p<.05). American Psychologist, 49(12), 997–1003.
Cook, Thomas D., and Campbell, Donald T. (1979) Quasi-experimentation: Design and Analysis Issues for
Field Settings. Boston, MA: Houghton Mifflin.
Dere, Ekrem, Easton, Alexander, Nadel, Lynn, and Huston, Joe P. (Eds.) (2008) Handbook of Episodic
Memory. Amsterdam: Elsevier.
Fletcher, A. J. (2016) ‘Applying critical realism in qualitative research: Methodology meets method',
International Journal of Social Research Methodology, 20(2), 181–94.
Flick, Uwe (2000) ‘Episodic interviewing', in Martin W. Bauer and George Gaskell (Eds.), Qualitative
Researching with Text, Image and Sound. London: Sage, pp. 75–92.
Gigerenzer, G. (2004) ‘Mindless statistics', Journal of Socio-Economics, 33(5), 587–606.
Glesne, Corinne (2011) Becoming Qualitative Researchers: An Introduction (4th edn). Boston, MA: Pearson.
Guba, Egon G., and Lincoln, Yvonna S. (1989) Fourth Generation Evaluation. Thousand Oaks, CA: Sage.
Page 15 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
Hammersley, Martyn (1992) ‘Ethnography and realism', in Martyn Hammersley (Ed.), What's Wrong with
Ethnography? Methodological Explorations. London: Routledge, pp. 43–56.
Hammersley, Martyn (1998) ‘Get real! A defence of realism', in P. Hodkinson (Ed.), The Nature of
Educational Research: Realism, Relativism, or Postmodernism. Crewe, England: Crewe School of Education,
Manchester Metropolitan University. Reprinted in Heather Piper and Ian Stronach (Eds.) (2004) Educational
Research: Difference and Diversity. Aldershot: Ashgate, pp. 59–78.
Hammersley, Martyn (2008) Questioning Qualitative Inquiry: Critical Essays. Thousand Oaks, CA: Sage.
Hammersley, Martyn (2009) ‘Why critical realism fails to justify critical social research', Methodological
Innovations Online, 4(2), 1–11.
Hammersley, Martyn, and Atkinson, Paul (2007). Ethnography: Principles in Practice (3rd edn). London:
Routledge.
Huberman, A. Michael, and Miles, Matthew B. (1985) ‘Assessing local causality in qualitative research', in
David N. Berg and Kenwyn K. Smith (Eds.), Exploring Clinical Methods for Social Research. Beverly Hills,
CA: Sage, pp. 351–82.
Kaplan, Abraham (1964) The Conduct of Inquiry: Methodology for Behavioral Science. San Francisco, CA:
Chandler.
Lawrence-Lightfoot, Sara, and Hoffman Davis, Jessica (1997) The Art and Science of Portraiture. San
Francisco, CA: Jossey-Bass.
Lincoln, Y. S. (1995). ‘Emerging criteria for quality in qualitative and interpretive research', Qualitative Inquiry,
1(3), 275–89.
Lincoln, Yvonna S., and Guba, Egon G. (1985) Naturalistic Inquiry. Thousand Oaks, CA: Sage.
Lincoln, Yvonna S., and Guba, Egon (2000) ‘Paradigmatic controversies, contradictions, and emerging
confluences', in N. K. Denzin and Y. S. Lincoln (Eds.), Handbook of Qualitative Research (2nd edn).
Thousand Oaks, CA: Sage, pp. 163–88.
Lincoln, Yvonna S., and Guba, Egon E. G. (2013) The Constructivist Credo. New York: Routledge.
Little, Daniel (1998) Microfoundations, Method, and Causation. New Brunswick, NJ: Transaction.
Madill, Anna (2008) ‘Realism', in Lisa Given (Ed.), The SAGE Encyclopedia of Qualitative Research
Methods. Thousand Oaks, CA: Sage, pp. 731–5.
Maxcy, Spencer J. (2003) ‘Pragmatic threads in mixed methods research in the social sciences: The search
for multiple modes of inquiry and the end of the philosophy of formalism', in Abbas Tashakkori and Charles
Page 16 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
Teddlie (Eds.), Handbook of Mixed Methods in Social & Behavioral Research. Thousand Oaks, CA: Sage,
pp. 51–90.
Maxwell, J. A. (1990a) ‘Up from positivism (essay review of DT Campbell, Methodology and epistemology
for social science), Harvard Educational Review, 60(4), 497–501.
Maxwell, J. A. (1990b) ‘Response to “Campbell's retrospective and a constructivist's perspective”', Harvard
Educational Review, 60(4), 504–8.
Maxwell, J. A. (1992) ‘Understanding and validity in qualitative research', Harvard Educational Review, 62(3),
279–300.
Maxwell, J. A. (2012a) ‘The importance of qualitative research for causal explanation in education',
Qualitative Inquiry, 18(8), 655–61.
Maxwell, Joseph A. (2012b) A Realist Approach for Qualitative Research. Thousand Oaks, CA: Sage.
Maxwell, Joseph A. (2013) Qualitative Research Design, An Interactive Approach (3rd edn). Thousand
Oaks, CA: Sage.
Maxwell, Joseph A. (2017) ‘The validity and reliability of research: A realist perspective', in D. Wyse, L.
E. Suter, E. Smith, and N. Selwyn (Eds.), The BERA/SAGE Handbook of Educational Research. London:
Sage. pp. 116–40.
Maxwell, Joseph A., and Chmiel, Margaret (2014) ‘Generalization in and from qualitative analysis', in Uwe
Flick (Ed.), SAGE Handbook of Qualitative Data Analysis. London: Sage, pp. 540–53.
McGinn, Michelle K. (2008) ‘Researcher–participant relationships', in Lisa Given (Ed.), The SAGE
Encyclopedia of Qualitative Research Methods. Thousand Oaks, CA: Sage, pp. 767–71.
Messick, S. (1995) ‘Validity of psychological assessment', American Psychologist, 5(9), 741–9.
Messick, S. (1998) ‘Test validity: A matter of consequence', Social Indicators Research, 45, 35–44.
Nuzzo, R. (2014). Statistical errors. Nature, 506 (13 February 2014), 150–52.
Pawson, Ray (2006) Evidence-based Policy: A Realist Perspective. London: Sage.
Pawson, Ray, and Tilley, Nick (1997) Realistic Evaluation. London: Sage.
Peshkin, Alan (1991) The Color of Strangers, the Color of Friends: The Play of Ethnicity in School and
Community. Chicago, IL: University of Chicago Press.
Phillips, Denis Charles (1987) Philosophy, Science, and Social Inquiry: Contemporary Methodological
Controversies in Social Science and Related Applied Fields of Research. Oxford: Pergamon Press.
Page 17 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
Preissle, Judith (2008) ‘Subjectivity statements', in Lisa Given (Ed.), The SAGE Encyclopedia of Qualitative
Research Methods. Thousand Oaks, CA: Sage, p. 844.
Putnam, Hilary (1999) The Threefold Cord: Mind, Body, and World. New York: Columbia University Press.
Rothbauer, Paulette (2008) ‘Triangulation', in Lisa Given (Ed.), The SAGE Encyclopedia of Qualitative
Research Methods. Thousand Oaks, CA: Sage, pp. 892–4.
Sadker, Myra, and Sadker, David (1994) Failing at Fairness: How Our Schools Cheat Girls. New York:
Simon and Schuster.
Salmon, Wesley C. (1989) ‘Four decades of scientific explanation', in P. Kitcher and Wesley C. Salmon
(Eds.), Scientific Explanation. Minneapolis: University of Minnesota Press, pp. 3–219.
Salmon, Wesley C. (1998) Causality and Explanation. New York: Oxford University Press.
Sayer, Andrew (1992) Method in Social Science: A Realist Approach (2nd edn). London: Routledge.
Schwandt, Thomas A. (2015) The SAGE Dictionary of Qualitative Inquiry (4th edn). Thousand Oaks, CA:
Sage.
Seidman, Irving E. (1998) Interviewing as Qualitative Research: A Guide for Researchers in Education and
the Social Sciences (2nd edn). New York: Teachers College Press.
Shadish, William R., Cook, Thomas D., and Campbell, Donald T. (2002) Experimental and Quasiexperimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin.
Smith, John K. (2008) ‘Relativism', in Lisa Given (Ed.), The SAGE Encyclopedia of Qualitative Research
Methods. Thousand Oaks, CA: Sage, pp. 749–53.
Smith, John K., and Deemer, Deborah K. (2000) ‘The problem of criteria in the age of relativism', in Norman
K. Denzin and Yvonna S. Lincoln (Eds.), Handbook of Qualitative Research (2nd edn). Thousand Oaks,
CA: Sage, pp. 877–96.
Strauss, Anselm, and Corbin, Juliet (1990) Basics of Qualitative Research. Thousand Oaks, CA: Sage.
Tappan, Mark (2001) ‘Interpretive psychology: Stories, circles, and lived experience', in Deborah L. Tolman
and Mary Brydon-Miller (Eds.), From Subjects to Subjectivities: A Handbook of Interpretive and Participatory
Methods. New York: New York University Press, pp. 45–56.
Tolman, Deborah, and Brydon-Miller, Mary (2001) From Subjects to Subjectivities: A Handbook of
Interpretive and Participatory Methods. New York: New York University Press.
Trafimow, D., and Marks, M. (2015) ‘Editorial', Basic and Applied Social Psychology, 37(1), 1–2.
Page 18 of 19
Collecting Qualitative Data: A Realist Approach
SAGE
SAGE Research Methods
2018 SAGE Publications, Ltd. All Rights Reserved.
Tulving, Endel (1983) Elements of Episodic Memory. Oxford: Oxford University Press.
Tulving, Endel, and Craik, Fergus I. M. (Eds.) (2000) The Oxford Handbook of Memory. Oxford: Oxford
University Press.
Weiss, Robert S. (1994) Learning from Strangers: The Art and Method of Qualitative Interviewing. New York:
The Free Press.
Williams, J. Patrick (2008) ‘Symbolic interactionism', in Lisa Given (Ed.), The SAGE Encyclopedia of
Qualitative Research Methods. Thousand Oaks, CA: Sage, pp. 848–53.
http://dx.doi.org/10.4135/9781526416070.n2
Page 19 of 19
Collecting Qualitative Data: A Realist Approach