Grounded Theory's Best Kept Secret: The Ability to Build Theory

Introduction

As a lifelong passionate adherent of grounded theory method (GTM), and having watched GTM become more and more popular, and relevant, in a range of academic disciplines, I have one outstanding question. Why is it that the recommendations of Glaser and Strauss (1967) for moving from substantive to formal theory are not more widely discussed? If we take as our starting point Glaser and Strauss's definition of substantive theory, which is one developed for a substantive, or empirical area of inquiry, it is clear that the majority of theories generated using GTM fall into this definition. Why is it that we do not then attempt to build formal theory, which Glaser and Strauss define as one developed for a formal, or conceptual area of inquiry? Especially as Glaser and Strauss point out that substantive and formal theories exist on distinguishable levels of generality, and that each type can shade at points into the other?

Given an ongoing concern about theory building in many newer academic disciplines (Weber 2003; Lester 2005; Oswick, Fleming and Hanlon 2011; Johnson, Eksted and Jacobson, 2012), which centre on those disciplines’ ability to build formal theory, it seems timely to examine this question in more detail, and to explore GTM's undoubted contribution to a larger theory building effort in academic disciplines. While some social sciences have a long history of abstract theorizing (e.g., Stinchcombe 1991), many newer disciplines do not. Even in those disciplines, the advice of Glaser and Strauss (1967) with regard to theoretical sampling, looking at dissimilar and similar groups with unsaturated and saturated categories, seems rarely acted upon, presumably because of some difficulties operationalizing the advice. I remain convinced that this advice gives us valuable [Page 90]guidance when building theory, because it gives us advice on extending the scope and generality of a theory, which in turn can lead us to building bigger, more formal theories.

The rest of this chapter looks at theory building debates in the newer academic disciplines and considers GTMs contribution to the idea of theory. I then look at Glaser and Strauss's (1967) recommendations about theory building in detail, with special reference to minimizing and maximizing group differences. What might those recommendations tell us in 2017? What might we learn when we apply those recommendations?

Theory and Grounded Theory

Glaser and Strauss (1967) set out the role of theory in sociology as both enabling prediction and explanation of behaviour, and also as ‘being readily understandable to sociologists of any viewpoint, to students and to significant laymen'. They also state that the theory needs to be clear enough to be operationalized in quantitative studies where appropriate. They were also concerned with issues of fit of the theory, by which they meant that the theory must be able to explain the phenomena, and not be forced on to the phenomena as an explanation. Above all, they pointed out that enduring theories are based on data, are grounded in that data, and cannot easily be refuted (Glaser and Strauss 1967). They also made the important point that how theories are assessed is also dependent on how the theory was generated.

If we fast forward to recent debates about theory in management, and in information systems, we see some similar themes. For instance, Corley and Gioia (2011) are as concerned with the utility of a management theory as its originality. Oswick, Fleming and Hanlon (2011) wonder if theory borrowing from other disciplines constrains the development of new and indigenous theories. This also raises issues of fit of those borrowed theories, and whether those theories are adequately grounded in data from their home discipline. In software engineering, the question has been raised about the absence of theories in the discipline (Johnson et al. 2012), and the utility of theories to the discipline.

In information systems, a lively debate about the importance of theory to the discipline occurred in 2014. Avison and Malaurent (2014) railed against what they saw as a fetishing of theory in the discipline, the fact that various theories fell in and out of favour, and again, raised questions about the fit of borrowed theories. Their piece was accompanied by several commentaries from Lee (2014), Markus (2014), and Gregor (2014). One striking agreement among the commentaries was that there was indeed a lack of consensus about what a theory is.

Overall then, we see in these debates a lack of consensus about what theory is, and also a concern about the fit of theories imported from older reference disciplines. While the first is probably attributable to the newness of some of these disciplines, it is interesting to note that there are also concerns about fit and groundedness of theories, whether imported from outside the discipline (fit) or within the discipline (groundedness).

Gregor (2006) is quoted in a number of these debates, and her definition of theory bears repeating here because it is both broad and helpful. She defines theories as ‘abstract entities that aim to describe, explain, and enhance understanding of the world and in some cases, to provide predictions of what will happen in the future and to give a basis for intervention and action’ (p. 616). She further categorizes theories into those that analyze and describe (but do not contain causal relationships), theories for explanation (but with no testable propositions), theories for prediction (testable propositions but no causal explanations), and theories for both explanation and prediction. She also adds theories [Page 91]for design and action, which can be seen as more specific to information systems and software engineering.

Gregor also defines the components of all theories as containing a means of representation, constructs, statements of relationship, and scope. It is useful to consider how these conventional components compare to the grounded theory process of theory building, simply because it seems rarely that grounded theory is discussed in the same way as theory in general – yet the components are the same, as Table 4.1 comparing Gregor's components to GTM indicates.

A few points in the above table bear further discussion. First, and most importantly, the constructs in a grounded theory are grounded in data. Second, the founding fathers of grounded theory knew that a parsimonious theory was important – hence the recommendation that one or two ‘core’ categories (or constructs) should make up a grounded theory. This recommendation helps to raise the level of the theory and widen its potential scope. Third, grounded theories rarely have relationships described in mathematical terms, because they are grounded (usually)1 in qualitative data. That said, it is entirely possible to generate propositions about causal relationships in a grounded theory. If we look at the range of academic theories available to us, there are many different relationships represented in theories. For instance, in structuration theory, Giddens (1984) posits that social systems are created and reproduced by an inseparable duality of structure and agency. Theories like structuration theory are of course precisely the sort of ‘grand’ armchair theories that Glaser and Strauss (1967) felt did not provide sufficient theory for new areas, or simply did not work. Interestingly, Giddens did not think that his theory should be used for guiding the practice of research, but rather as a sensitizing device, which is entirely reasonable, as it can be seen as a ‘grand’ or formal theory.2

This brings us to the final point in Table 4.1, which is the issue of scope. The inevitable consequence of a grounded theory is that it tends to be a substantive theory, pertaining to the area being investigated, because if the theory needs to be grounded in data, it will naturally limit the scope of the theory to what can be observed in a single study. It is this issue of scope – and the associated boundary conditions and context – that seem to be less discussed in the literature. While in the physical sciences, boundary conditions are typically solutions expressed as equations, in the social sciences, these conditions are more likely to be boundaries of race, gender and class – and the truly universal theories might transcend these boundaries (Spatz and Kardas 2008). Davison and Martinsons (2016) have discussed the issue of theory scope in information systems, asking to what extent are the findings of a study strictly limited to the original context, and the importance of boundary conditions. Within grounded theory, there is a degree of clarity around the issue of scope, because it is clear that the scope of the theory can be extended by following the advice on theoretical sampling. The next sections look at Glaser and Strauss's definitions of substantive and formal theory, then examine their recommendations on moving substantive theory to formal theory, using theoretical sampling.

Defining Substantive and Formal Theory

Glaser and Strauss (1967) define substantive theory as ‘that developed for a substantive, or empirical area of sociological inquiry’ (p. 32). They define formal theory ‘that developed for a formal, or conceptual area of sociological inquiry’ (p. 32). They are careful to [Page 92][Page 93]say that both types of theory can be considered middle-range and are not all-inclusive ‘grand theories'. Glaser and Strauss point out that substantive and formal theories only differ in terms of the degree of generality – that is, the scope of the theory. Formal theory is created by comparative analysis among different kinds of substantive cases. So, gradually the scope is increased, by different contexts and boundary conditions, represented by different substantive cases.

They suggest that, for substantive theory, the theorist can select groups from the same substantive class, regardless of where they are found (p. 53). They give the example of comparing an ‘emergency ward’ to all kinds of medical wards, in all kinds of hospitals, in all countries. If, however, one was to progress towards a formal theory, they suggest that it would be useful to see the emergency ward as a subclass of emergency organizations.

So let us look more deeply at the definitions of substantive and formal theory. One key difference is that formal theory categories are not underpinned by high empirical content,3 despite the fact that they have their origin in empirical data. For the purpose of Table 4.2, categories can also be seen as constructs, as defined by Gregor (2006). In grounded theory, the aim is to get to one or two core categories for a parsimonious theory (Glaser 1978; Strauss 1987). This helps the researcher to get to a reasonable level of abstraction, and mitigates against having too many low-level categories, sometimes an inevitable consequence of detailed line-by-line coding. Importantly, it helps the researcher to pull together and integrate the theory around one central concern. In [Page 94]turn, those core categories are underpinned by lower level categories that are firmly grounded in the data.

As the scope of a substantive theory is extended towards a formal one, category or construct names by necessity become simpler, and more abstract, because that construct has to apply to a large number of substantive cases.

Relationships between constructs or categories in grounded theory are named ‘theoretical codes’ by Glaser (1978) in his famed book Theoretical Sensitivity. The idea of theoretical sensitivity, articulated in the 1967 book, is that the grounded theorist has to be sufficiently aware of theories, and how they are constructed, in order to be able to construct their own. Glaser proposed 18 ‘coding families’ in Theoretical Sensitivity that give inspiration for relationships between categories. These families (now totalling 41, with the families proposed in Glaser (2005)) are themselves often inspired by how other theories conceptualize phenomena. A few examples of theoretical families are given in Table 4.3.

Of course, there is also guidance on relating categories in the Straussian version of grounded theory. Strauss in 1987 proposed a coding paradigm of conditions, consequences, interactions, strategies and consequences, using a procedure he named ‘axial coding'. He says the purpose of this element of axial coding is to ‘hypothesize about conditions, consequences, interactions, strategies and consequences’, that is, to think about how the categories relate. Later, Strauss and Corbin (1990) published a further coding paradigm, which prompted the now famous dispute between Glaser and Strauss. It is interesting to note, however, that the strictures about this paradigm moved from mandatory to optional, over time. Table 4.4 explains the evolution of the coding paradigm.

Moving to the question of scope, Glaser and Strauss (1967) distinguish between scope of the population covered by the theory and the conceptual level of the theory (p. 52). So for a substantive theory, different groups would be selected of exactly the same substantive type – they give the example of federal bookkeeping departments. However, if the aim is to progress to formal theory, increasingly dissimilar – and larger – groups need to be selected. The key sentences in the 1967 book are as follows:

Clearly, Glaser and Strauss considered in great detail, from the beginning, the differences between substantive and formal theory, and how to move from one to the other, from a practical point of view. The next section discusses their recommendations for theoretical sampling, which are rarely discussed in the grounded theory literature, despite the undoubted insights they give us for the theory-building process.

Glaser and Strauss's Recommendations for Building Theory Using Theoretical Sampling

It is very clear that Glaser and Strauss were engaged from the outset with practical considerations for building theory – for them, the key issue of what data to collect next [Page 95][Page 96][Page 97]was always controlled by the emerging theory. The great contribution of their chapter on theoretical sampling in the 1967 book was their assertion that the selection of comparison groups gave control over two aspects of the developing theory – first, conceptual level, and second, population scope. This is expressed in a table on p. 58 of the book, reproduced in this section as Table 4.5, that shows the consequences of minimizing and maximizing differences in groups, and considering the variations within the data of those groups. Obviously, variations in the data are expressed by variations in categories.

For Glaser and Strauss, the purpose of selecting groups is not about comparisons between and inside substantive groups, useful though that may be. Their criteria are ‘theoretical purpose and relevance – not of structural circumstance’ (p. 48).

Glaser and Strauss also lay great stress on either maximizing or minimizing differences between groups to control ‘theoretical relevance’ (p. 55) of data collection. They also point out that this then has a bearing on categories. For instance, if a similar group is chosen, many more instances of a category will be collected, while important differences might be spotted that were not noticed in earlier data collection. This is important for establishing the properties of a particular category, and Glaser and Strauss suggest that these properties are established before differences in groups are maximized.

When maximizing differences between groups, data in a particular category will vary. This allows the researcher to think about the ‘strategic similarities’ between groups, [Page 98]which give the uniformities of scope within the theory. They state that maximizing ‘brings out the widest possible coverage on ranges, continua, degrees, types, uniformities, variations, causes, conditions, consequences, probabilities of relationships, strategies, process, structural mechanisms, and so forth, all necessary for elaboration of the theory’ (p. 57). We can see here too an early reference to theoretical coding (Glaser 1978) and a recognition that a widening of scope means that relationships between constructs have to be reconsidered. They summarize their recommendations in the table that is reproduced here (Table 4.5).

Another way of thinking about this table is that its key contribution falls into two areas. First, it helps us see that maximizing differences between groups helps us extend the scope of the theory we are working on. Second, we can raise the conceptual level of our substantive theory by considering the categories themselves. In particular, we can consider the unsaturated categories in our analysis as a promising avenue to raise the conceptual level of the theory, by guiding us to more interesting groups. The fascinating thing about this table is that it draws attention not only to issues of scope, but issues of the data, as expressed by the categories themselves, and how considering categories can improve the quality and conceptual level of the theory.

While Eisenhardt (1989) popularized theoretical sampling for group differences, she did not consider theoretical sampling of concepts in the same way as suggested above. However, she talked about within-case comparison based on categories, which can be seen as a variation of sampling for concepts in the data. The differences in strategy between grounded theorists and mainstream qualitative researchers can perhaps be seen as a consequence of different methodologies for theory building – Eisenhardt (1989) used cases, while grounded theory methodology relies upon slices of data. So, how might these recommendations for theoretical sampling, as suggested by Glaser and Strauss, be carried out? What happens if we try to follow what Table 4.5 suggests?[Page 99][Page 100]

Enacting the Recommendations for Theoretical Sampling

In Vaast and Urquhart (2017), I briefly discuss what Glaser and Strauss's advice on theoretical sampling might mean in practice for the generation of social media theory, and use the example of Facebook statuses. In this section, I expand in detail on that example, and the steps of theory building that I believe are implied by Table 4.5, in order for us to explore what can be learned from those recommendations.

Glaser and Strauss state that, at the beginning of generating a substantive theory, differences should be minimized in comparative groups. We can choose to minimize or maximize differences in groups along several dimensions, such as age, country, language, political affiliation, and so on. Maximizing those differences helps use theorize on relationships, conditions, patterns and mechanisms (Glaser and Strauss 1967, p. 57). In our Facebook example explained below, we may wish to look at slices of data from other online sources, such as personal blogs, Twitter feeds, or how a person responds to other comments from people below the line (BTL) when commenting on a news article. The theoretical sampling and choice of data slices depends on how the theory develops. It is important to realize that the motivations for theoretical sampling are theoretically motivated; it is not a question of simply verifying the category in various populations. How might we operationalize the advice from Glaser and Strauss on theoretical sampling, and how does it help the development of a theory? Each step in the example is cross referenced to Table 4.6 below.

Table 4.6 Example of theory building using Glaser and Strauss's recommendations for comparison groups

Levels and Scope of Theory

The previous section discussed, in detail, the guidance available from Glaser and Strauss for theory building using similar and dissimilar groups. It is helpful guidance, so, why is it not more known in the mainstream of qualitative research, and considered in numerous qualitative research textbooks consulted by graduate students? To me, the answer to this question resides in two places. First, in comparatively new disciplines, such as information systems and management, we are still unfamiliar with the process of theory building, and lack theoretical sensitivity, hence the debates on and lack of consensus on theory building in those two disciplines. Grounded theory is unique in how it encourages us to build theory systematically by use of constant comparison and theoretical sampling of groups. It also helps us understand, in detail, how theories shade into another from the substantive to the formal. So many of the newer disciplines lack a deep appreciation for the structures of theories, conveyed in grounded theory by theoretical families and the coding paradigm, and by the idea of theory mechanisms in the social sciences.

Second, by contrast, in the social sciences, there has been indeed a great deal of debate about the nature of theory, largely focused on mechanisms. However, a lot of current discussion on mechanisms tend to focus on opening up the black box, and work from higher levels of theory to lower ones4 to provide explanations. Davison and Martinsons (2016) complain about a rush to universalism in the information systems discipline, where theories borrowed from other reference disciplines are applied, regardless of the context of that application. They make a plea for the understanding and clear explication of boundary conditions when using theories, lest the context of the study be ignored. My commentary on that article (Urquhart 2016) suggested that particularism versus universalism is a false dichotomy. Grounded theorists know that Glaser (1978) states that levels of theory shade into one another as a theory becomes more and more abstracted, from substantive to formal. Stinchcombe (1991) was keenly aware of the issue of levels of theory when writing about theory mechanisms in social science. His proposition was that paying attention to lower-level mechanisms would result in a more ‘supple’ theory at the higher level. His definition of mechanisms is:

This description will be immediately be recognizable to grounded theorists as a description of theoretical coding, and the need to ground those theoretical codes at the lower level, while at the same time making sure that the level of abstraction holds.

Figure 4.1 is offered as a summary of Glaser and Strauss's vision of theory building, without compromising the original vision. For me, the key insight of their approach is not that we extend the scope of the theory by sampling different groups, but that we are theoretically guided by category development as we do so.[Page 103]

Figure 4.1 Theory Scope and Conceptual Level

The x axis represents Conceptual Level, and the y axis Theory Scope. These are the two dimensions that are acted upon by following Glaser and Strauss's guidance on theoretical sampling. The process of sampling similar or dissimilar groups extends the scope, but it is by paying attention to saturated and unsaturated concepts that the conceptual level of the theory, and its direction, is considered and worked on.

Discussion and Conclusion

It is clear from the discussion that Glaser and Strauss did indeed have clear and systematic recommendations for theory building from 1967 onwards. One problem seems to be that, because grounded theory is seen as producing rich, qualitative theories that are about micro phenomena, these recommendations seem not have permeated into the wider academic disciplines in which we work. Layder's critique (Layder 1998) about the hopelessly detailed nature of a grounded theory, it seems, has not been fully put to rest. It is easy to see why – the very nature of the grounded theory process means we start with a very detailed and inimate relationship with the data. This is both the strength and weakness of grounded theory. The scale of the task of building a grounded theory means that we often leave it to others to then take our theories and either apply them to other contexts or test them. One possible route is to take our substantive theories and widen their scope, but the emphasis on novelty, at least in my own discipline, seems to mean that once a substantive theory is published, it cannot be elaborated upon.

That said, my observation of the newer disciplines I work with – management, information systems (IS) and software engineering – is that these disciplines often struggle with theory building and the nature of theory. Unlike my sociologist colleagues, we are largely unfamiliar with the notion of mechanisms or how formal theories are used. Grounded theory offers clear and practical guidelines on how a theory that applies to one area can be extended to another area, and also how the conceptual level of a theory can be pushed to levels of further abstraction. I would contend that newer disciplines can learn from the clarity and systematic nature of Glaser and Strauss's recommendations, because, for those disciplines, advancing new and larger theories are important. There seems to be very little discussion of levels of theory in the newer disciplines – theory is either treated as universal, generally coming from other reference disciplines, and grounded theories are not considered as contributions to the overall theory building effort. Worse, we seem to lack the language to discuss theory and theoretical contribution. In information systems, Burton-Jones, McLean andMonod (2014) make an excellent contribution to the IS theory debate when they suggests moving away from ideas of process/variance theories that are prominent in the discipline towards conceptual latitude and fit, and focusing on constructs and relationships. As Davison and Martinsons (2016) suggest, it would also be helpful if people using and advancing theories considered the context in which they were created, and the context in which they are subsequently applied. Categorizing the type of theoretical contribution could also be fruitful; for instance, Colquitt and Zapata-Phelan (2007) suggest that contributions can be categorized from those which introduce significant new concepts, and examine new relationships and processes, to those that attempt to replicate theories. They then consider to what extent that contribution is grounded, empirically or theoretically (Colquitt and Zapata-Phelan 2007).

The division between grounded theorists and their colleagues will hopefully fade as the division between qualitative and quantitative research recedes. Often, the phenomenon itself will demand a combination of quantitative and qualitative research. We need [Page 105]to change policies via quantitative evidence, but before we do that, we need qualitative research into what people do. The advent of Big Data and the ubiquity of social media calls for both qualitative and quantitative research – understanding patterns in data requires a qualitative understanding of why people do what they do. For too long, in my opinion, Glaser and Strauss's stellar contribution to understanding how we build theory, and how we scale up that theory, have been ignored by the mainstream of many of our academic disciplines. It is my sincere hope that this chapter contributes to interpreting the intentions of Glaser and Strauss's advice on theoretical sampling. Glaser and Strauss represent a huge intellectual contribution to theory building in the twentieth and twenty-first centuries, and these ideas can and should help theory builders everywhere.

Acknowledgements

I am very grateful to Udo Kelle, Stefan Seidel, Olga Volkoff and Hamid Pousti, all of whom gave extremely helpful comments on an earlier draft of this chapter.