Can you elaborate on that? Why the qualitative / quantitative distinction is fundamentally important for social science

In a recent blog post, Howard Aldrich argued that social scientists should drop the distinction between quantitative and qualitative research. I want to push back here and argue that there are important differences between the two methods which must be recognized to ensure high quality research. To be sure, the starting point of the discussion should be recognition of the underlying unity of research methodology, about which Charles Ragin has written eloquently. Quantitative and qualitative methods are both tools for advancing theory and knowledge. But these methods advance theory in distinct, complementary ways. To realize the full potential of research methodology requires recognizing these differences.

It is important to remember the long and distinguished tradition of the interpretivist philosophy of science, which undergirds qualitative methods, against the positivist basis of quantitative methods. Interpretivism – beginning with Max Weber’s 1904 treatise on objectivity in social science – holds that to understand human society means going beyond – but not rejecting – positivist methods of statistical hypothesis testing in order to generate data capable of providing meaning. How do humans understand their social world and particular social contexts? This tradition has been elaborated and refined by methodological luminaries from Clifford Geertz to Howard Becker and Michael Burawoy.

To be clear: I have zero interest in paradigm battles between positivism and interpretivism. Both are necessary. Positivism refers to a broad set of positions holding that all knowledge must come from empirical or “positive” data based on sense experience (against metaphysical speculation). The various versions of positivism also maintain that scientists can understand the world as an external object of analysis; for social scientists this means maintaining a certain distance from the people and relations under study, primarily through the use of surveys.

By contrast, interpretivists hold that the social world is distinct from the natural world. The goal of interpretive social science, according to Geertz, is to produce “thick description,” which goes beyond surface appearances to penetrate the “webs of significance” within which humans construct and understand the social world. Continuing with an example from Geertz, this means not mistaking a wink (a coded behavior aimed at another person) for a twitch of the eye. To ensure such mistakes are not made requires prolonged engagement with respondents. In contrast to the positivist attempt to maintain as much distance as possible, Burawoy has argued that prolonged engagement with research subjects helps unpack situational experiences to reveal social process. Disturbing the setting helps reveal social order.

Now, it is not the case that there is not a one-to-one correspondence between positivism-statistics and interpretivism-qualitative methods. Many (most?) contemporary researchers – both quantitative and qualitative – have appreciated the positivist-interpretivist debate and moved forward by appropriating insights from both, rather than slavishly holding to one or the other. It should also be noted that surveys which gather objective data (e.g. labor market data from the US Bureau of Labor Statistics or industry data from the Bureau of Economic Analysis) and statistics gathered from sources such as archives do not have the same interpretive problems as surveys which gather subjective data.

With that said, it is very difficult to extract meaning in the qualitative sense from statistical survey data. As Becker argued, it is difficult for statistics to produce thick description precisely because they operate at a distance from their objects of analysis, using remote indicators. This is by positivist design: in order to produce objective data, researchers should try not to disturb the social world they study. Surveys are thus based on standardized, primarily closed-ended questions, designed so that every respondent should be able to understand them in the absence of any contact with the researcher. But what if there are differences in how diverse respondents understand the question? What if the question does not make sense to some respondents? What if the set of predefined answers is does not include the answers most appropriate for a given respondent? What if these questions mean little to the respondent???

These are serious problems! By no means do they imply that surveys are useless – they are of fundamental importance for social science – but such questions do show that surveys are necessarily incomplete, and that survey data are indeed distinct from other types of data.

Aldrich suggests that “ethnographic fieldwork, archival data collection, long unstructured interviews, simple observational studies” etc. are a “heterogeneous set” that have in common only the fact that they are “not using the latest high-powered statistical techniques to analyze data that’s been arranged in the form of counts of something or other. … Beyond that, however, commonalities are few.” With the exception of archival techniques, I think Howard is fundamentally incorrect on this.

Qualitative scholars have long made a compelling argument that ethnography, in-depth interviews, unstructured interviews and non-participant observation have in common prolonged engagement in the field. In Becker’s analysis, this means that such methods are able to produce data having a number of a critical characteristics that survey data do not have: accuracy (ability to produce close, detailed observations), precision (ability to produce new information on issues not anticipated in the original formation of the research question), and breadth (ability to produce knowledge on a wide range of matters bearing on the research question).

If Weber, Geertz, Becker and Burawoy are correct, then it is eminently sensible – indeed, necessary – to label these methods qualitative, because they produce a different type of data from surveys. Aldrich effectively admits this when he argues that qualitative data can generate counts, but then adds the proviso that “the meaning of what has been observed derives not from ‘counting’ something but rather from understanding how to interpret what was observed,” which “depends upon a researcher’s understanding of the social context for what was observed.”

Data based on survey research are quantitative, and they are fundamentally important but incomplete. Data based on methods of prolonged engagement with respondents are qualitative, also important but incomplete. They are united in their goal of advancing knowledge and theory, but in order to use them correctly, it is important to understand their differences.