| Discuss naturalistic and participant observation.
• Discuss the three main phases of coding observational data.
|
The main ways to record observational data.
|
First steps in note taking.
|
Early descriptive notes and preliminary coding.
|
Organizing and culling your early notes.
|
Technologies of recording observational data.
|
When to make the transition from recording to coding.
|
When to use an observation protocol.
—Appendix 4.1: Example of a Site Visit Protocol.
|
| Phase 3: Coding
|
When should you use computer software for coding?
|
Example of a data organizing and coding grid.
|
• Conclude with tips for completing an observational study.
|
intRoDuction to oBSeRVAtionAl ReSeARch
If you have decided that your research question is one that can be best answered by onsite observation of events and people in natural settings, then you have collected or will soon be collecting data using the methods of observational research. More than in any other research design, in observational research, most of the work of coding and analysis is “entangled” with data gathering, and it is also “back loaded.” What we mean by entangled and back loaded is best highlighted by a comparison with survey research (see Chapter 1). In surveys the questions have to be written before any data can be col-lected; the questions strongly imply codes, and the codes suggest analysis plans. By con-trast, in observational studies, data collection and recording precede coding; collection and recording begin to suggest codes as you prepare fieldnotes after each observation session. This is an inductive approach to coding, which we discuss further in this chap-ter and in Chapchap-ter 11. In brief, in surveys, much of the work of coding comes before data gathering; in observational research, most of it comes during and after.
The term observational research is used in more than one way by methodology writers.1 We use it to mean studying phenomena as they occur without the researcher intervening. The oldest of the observational sciences is astronomy, which has generated data usable by modern researchers dating back thousands of years to ancient China and Mesopotamia. Astronomy pioneered in another sense: It was the first observational method to systematically employ photography to record data. Of course, all research involves observation of one sort or another, but astronomical observations, like the observations studied in this chapter, are not based on manipulations of, or interventions in, the phenomena observed. Experimenters observe participants’ reactions to their treatments, and survey researchers and interviewers observe respondents’ answers to their questions. The researchers discussed in this chapter are distinguished from experi-menters, interviewers, and surveyors because they observe without shaping the phenom-ena studied. In other words, observational researchers seek to study human phenomphenom-ena as they occur naturally, as if the researchers were not present. The meteor that crashed into Jupiter on September 10, 2012, would have done so had amateur astronomers not observed and documented it.2
Other terms besides observational research are also used to identify investigations aimed at studying social phenomena as they occur more or less naturally, without the kinds of interventions typical of surveys, interviews, and experiments. Field methods and ethnographic research also denote the same class of approaches to research as those discussed in this chapter. We prefer observational only because other terms can be somewhat less descriptive. For example, field methods may be too narrow and connote a particular anthropological approach, and the term ethnographic may have become, through widespread use, too broad.3 In its original sense of long-term onsite observation
1 Often “observational” is used as a crude residual category to label any research that is not experimen-tal— a designation that is less than helpful.
2 Only one actually observed it; another had made a video recording that captured the event and was able to confirm the observation.
3 See Culyba, Heimer, and Petty (2004) for a study of the use of the term and Gans (1999) for a critique of newer trends in ethnography, such as autoethnography. A case for autoethnography is made in Hughes, Pennington, and Makris (2012).
of human cultures, ethnographic research was most highly developed by anthropolo-gists, and we have relied heavily on their work.4
Some naturalistic observers of social life try to be as unobtrusive as possible, if not by interfering as little as an astronomer bent over a telescope, then at least by being as inconspicuous as the proverbial fly on the wall. The investigator’s goal of neutral, observer- free research is seldom if ever totally met, because, to one degree or another in social research, the observers’ presence usually affects the phenomena being observed (even a fly on the wall can be a distraction). But effect- free observation remains an ideal for many researchers. In participant observation, in which the researchers join the social activities that they are studying, their purpose is not to alter the phenomena,5 although they inevitably do so. Rather, the goal is to gain a closer, insider perspective on the phenomena. In practice, participant and naturalistic observation are often less dis-tinct than their names suggest. Differences are a matter of degree. The extent to which the researcher participates in the phenomena being studied ranges on a continuum from minimal (naturalistic) to extensive (participant). Finally, one type of participant obser-vation is important in most types of research: The obserobser-vations made by researchers as they are conducting their surveys, interviews, and experiments frequently provide important insights that complement the main method of data gathering.
By saying that pure observation is a goal, we do not mean to suggest that there can be an immaculate perception, an observation free of any prior researcher knowledge or filtering concepts or motivations. While admitting that all observation is, to use the popular term, “theory- laden,” we think it is possible and desirable to keep observations from becoming theory‑ obliterated, to be nothing other than poorly disguised exten-sions of ideological commitments. We would not want to work in social research if we believed that there was no reality that could be observed credibly, that no observation was more accurate than any other, or that one could not improve the quality of one’s observations through study and practice. If there was really nothing in the world apart from various epistemological posturings, what would be the point? Practically, we think it is possible to learn to use the phenomenological approach called bracketing, which means suspending judgment by recognizing and temporarily setting aside one’s assump-tions in order to better conduct a study.6
One key characteristic of observational researchers is that they tend to use several kinds of observation in the same study. Because observational researchers often have to improvise, as they conduct their investigations with minimal researcher control, they almost always resort to more than one way of gathering data— listening at a meeting, watching people interact in a social setting, or making a comment in a discussion and noting how it is received. Therefore, multiple sources and types of observation are typi-cally used in data collection, and researchers frequently employ triangulation7 to con-firm and analyze their data. Improvisation and multiple types of observation are impor-tant advantages of observational research, but they make it somewhat more difficult for
4 See Bernard (2000), Van Maanen (2011), and Chapter 11 in this book.
5 Action research is an exception.
6 Fischer (2009) is a recent discussion.
7 The term comes from trigonometry, in which distances and angles between two points are used to calcu-late a third. In research methodology, it is used loosely to refer to the combined use of two or more methods or types of data.
text and reference book writers to systematize their advice, especially since the multiple types of observation are used to investigate many types of phenomena.8
Probably the most frequently studied kinds of phenomena investigated in observa-tional research involve verbal interactions: conversations, discussions, debates, meet-ings, shouting matches, and so on. These verbal interactions, like interviews, are often recorded and transcribed. The difference is that, unlike in interviews, the researcher rarely directs the course of the verbal interactions by asking questions. Rather, ers watch and listen as the verbal exchanges develop. Of course, observational research-ers also often conduct interviews, both as formal interviews and as informal convresearch-ersa- conversa-tions. But interviews (see Chapter 2) and observations are distinct enough that we have found it helpful to discuss them in separate chapters. Perhaps the greatest of the distinc-tions between the two is that in interviews the emphasis is more on what interviewees say; the researcher interprets the interpretations of the interviewees and analyzes what the interviewees’ words reveal about what they think or feel. In observational research, the emphasis is more on the actions of “observees,” on what they do. Of course, one of the things they do is talk, sometimes to the researcher. So this distinction, like many others, is a matter of degree rather than kind.
Because observational researchers intervene least and have the least control over the phenomena they study, it is difficult to systematize descriptions and recommenda-tions about effective research methods. Also, observational researchers are least likely to have already gathered their data before they begin the work of coding and analyzing them. The codes tend to emerge in the data collection and the review of records. Of course, this can happen in surveys, interviews, experiments, and archival research, too.
All researchers make adjustments as they go along, but in observational research more emphasis is usually placed on learning by doing. We have assumed in writing this book that many readers will have already collected their data and are now thinking about how to code and analyze them. This assumption is probably less applicable to observa-tional researchers. Because the processes of gathering and recording data are more fun-damentally involved in coding them, we go back to stages of a research project earlier than those we looked at for surveys, interviews, and experiments.
In this chapter we examine three phases of gathering and preparing data for analysis in observational research: (1) methods of observing, (2) methods of recording the obser-vations, and (3) approaches to coding the recorded observations. The phases overlap and can be cyclical. For instance, early observations may suggest ideas to use in coding; or the work of coding recorded observations often leads researchers to want to engage in further observation and to pay attention to different or additional things when they do.9 Such overlaps, returns, and repeats occur in all types of research, quantitative or quali-tative, but they tend to be more extensive in observational research. After discussing the three phases of preparing data for analysis in observational research— observing, recording, and coding— the chapter concludes with recommendations about where in the book to seek advice about effective analysis techniques for data from observational research, recommendations for research on observational practice, and suggestions for how to avoid common barriers to success.
8 Zussman (2004) provides many examples of this variety.
9 Yin’s (2011) felicitous term for this process is the “analytic cycle”; Creswell (2007) uses and illustrates the
“data analysis spiral.”
PhASe 1: oBSeRVing
There are many ways to observe and many occasions, settings, and persons that can be observed. Where and when will you observe whom doing what? Everything cannot be observed, even with the aid of excellent recording equipment. Selection is always involved. In addition to practical considerations, there are two important ways in which the selection of what, where, when, and who to observe is shaped in a research project:
(1) by your research question and (2) by your role as a researcher.
Your Research Question
What you observe is inevitably and appropriately influenced by your research question and its relation to the theory or sensitizing ideas undergirding it. Different varieties of theories and types of research questions can lead to different foci in your observations.
If your research question is exploratory, you may be concerned with building a theory;
in your observations you might seek perceptions of phenomena that will give you ideas.
Looking for ideas is a mysterious process, to be sure, but it is one many researchers have pursued. As your initial research question you might ask, Why does this occur? That question is based on a simple initial theory: It occurs; therefore it can be explained. The question also implies your initial codes. For example, the question might be, Why are some clinics more successful than others? This presumes that there are differences in success and that the reason(s) for the difference can be discerned through observation.
You have to have some kind of working definition of, or code for, success so that you can recognize it when you see it.
If the question involves assessing a theory, you may be looking for confirming or disconfirming instances and for insights into what features of the phenomena being observed lead them to be confirming or disconfirming instances. For example, if the theory is that crime rates are lower in some neighborhoods because of the presence of certain kinds of social capital, you would observe neighborhoods for social patterns that confirmed or disconfirmed the salience of social capital in those neighborhoods. What kinds of neighborhood institutions and interactions would constitute the type of social capital specified in your research question? How will you observe them? If you observe them in different ways (e.g., attending a block meeting, having lunch in a popular res-taurant), these have to be noted, and they become some of your first codes.
Whatever the type of research question and its underlying theory (e.g., exploratory or confirmatory), the sites of the research and the observations made at those sites must both be relevant to the research question and the theory upon which it is built.10 A site might not be relevant because it is not representative of the category of phenomena being studied. For example, the success rate at some of the clinics might be high only because of unusual methods of recording outcomes. Or the crime rates in some of the neighborhoods might not take into account an important category of crime.
And the observations at the sites also need to be representative and relevant. If you visit a site at an unusual time or observe atypical people and places, you will not
10 These issues are also important at the design and sampling stages of the research; see Vogt, Gardner, and Haeffele (2011), especially Chapters 4 and 10, for further discussion.
be observing “business as usual.” For example, if you wish to investigate the organiza-tional climate of a particular company, you need to observe it over an extended period and at different times of the day and week. Otherwise, you may be observing only the short-term weather, not the long-term climate. Your efforts to make your observations representative and relevant also need to be recorded and included in your final research report. They are forms of sampling that have to be included as part of your early coding.
Your Role as a Researcher
The kinds of observations, recording, and coding that are possible depend on the status of the observer in the social situation being studied. Observers can be active participants in the situation or passive observers. Also, observers can conduct their observations overtly or covertly. Two continua can be used to describe the roles and positions of researchers conducting observational studies: active– passive and overt– covert. One con-tinuum reaches from full participation in the social situation being studied at one end to the role of a completely passive observer at the other. The other continuum extends from the researcher being clearly identified as such to the researcher who is “under cover” (see Figure 4.1). If you imagine 5 points on each continuum, there are 25 possible combina-tions. Of course, many more subtle shadings and nuances are possible. What you will be able to observe, record, and therefore code will differ depending on your positions on these two continua and how they are combined. Reading the observational research literature, it is possible to find examples of all possible combinations of active– passive and overt– covert researcher roles.
Opinions differ about which approaches are best in what circumstances. Beliefs are strong, debates are spirited, but the evidence is weak.11 Your choice is, of course, constrained by what is possible in a particular setting. Among the possible options, you choose those that are most effective for answering your research question. The range of roles that are feasible may not always include the options you believe would be most effective. Observational research begins with the art of the possible. In brief, there are three questions: (1) Should you participate or be passive, (2) should you do this overtly or covertly, and (3) how do you best combine the two choices in specific settings?
We provide some examples of the different ways researchers have made these choices that fall at various points of the active– passive and overt– covert continua illustrated in Figure 4.1. Briefly, the extremes of an overt but passive researcher (1–5 in the figure) would be studying law enforcement through police “ride alongs.”12 An active but covert
11 One study that presents some systematic evidence is Tope, Chamberlain, Crowley, and Hodson (2005).
12 Spano (2006); Van Maanen (1982).
1. Active/Passive Active Participant <----1----2----3----4----5---> Passive Observer 2. Overt/Covert Openly Researching <----1----2----3----4----5---> Covertly Researching
figuRe 4.1. Continua describing observational researchers’ roles.
researcher (5–1 in the figure) would be taking a job in order to study what it was like to do that job but not informing your coworkers that you are a researcher as well as an employee.13
The researcher’s roles may not be stable. For example, researchers have described, sometimes rather poignantly, how challenging it is to maintain an outsider role when observing, especially when it seems that it is hard to gain people’s trust if you don’t participate.14 At other times you might want to participate, but everyone knows you are an outsider, “just” there to do your research, so your involvement is viewed with suspicion.15 In general, it can be very difficult to maintain a purely passive external observational role; it can be equally difficult to be a full participant; and the fluidity of roles in ongoing social interaction means that you might not always be sure exactly what your role is or how it is evolving. It is not entirely up to you. An excellent example is Duneier’s study of the culture of street vendors in New York City. Over the several years of his research, he played several more or less active roles—even, at the close of the project, partnering with one of the members/participants to teach his course on research methods.16
One of the ways that your role is not entirely up to you is the requirement to clear your research with institutional review boards (IRBs), which are the official guard-ians of research ethics, as well as propagators of bureaucratic rules. One of the biggest
One of the ways that your role is not entirely up to you is the requirement to clear your research with institutional review boards (IRBs), which are the official guard-ians of research ethics, as well as propagators of bureaucratic rules. One of the biggest