The Perception of Deviant Behaviours by Young People: Triangulating Survey and Focus Group Data

Learning Outcomes

By the end of this case students should be able to

• Have a better understanding of what triangulation in the social sciences is, with a critical view of its strengths and weaknesses
• Be able to assess the usefulness of adopting multiple data-gathering techniques
• Be able to project a research design based on triangulation

Introduction

Beginning in the late 1960s, the strategy of integrating several data-gathering techniques began to gain ground. To improve the quality of information available on social research, many scholars, experts in field research and methodology, offered proposals of integration between different sociological methods of data collection. In particular, Margaret Stacey (1969) proposes ‘combined operations’, Norman K. Denzin (1970) speaks of ‘methodological triangulation between different techniques’, Jack D. Douglas (1976) leans towards the use of ‘mixed strategies’ and Robert G. Burgess (1982) towards the ‘multiple strategies’ technique. It was clearly not a real innovation: On other occasions, other researchers had used similar integration systems with a view to matching the research design more closely to the specific requirements of the investigation.

However, in that period, the methodological debate focused on the possibility of checking the faithfulness of the results obtained as a concrete alternative to the classic validity and reliability data tests, which proliferated in the various disciplinary fields. In fact, classic assessment instruments, such as inter-rater reliability, inter-scores reliability, test–retest and split-half, although they had been positively responsible for a deeper methodological reflection, can be considered more appropriate for the assessment of tests and attitude scales (particularly those most used by psychometrics, such as projective tests, personality tests and Rorschach test). Moreover, classic assessment instruments hardly appear to be applicable in field research and in merging qualitative and quantitative techniques. For this reason, the need to identify new forms of combination of search tools better suited to social research emerged. The debate that opened up on these strategies of combining different research instruments has continued until the present day.

Starting with a reconstruction of this debate, in the case that follows I seek to clarify the triangulation technique by conducting a critical analysis of the goals of integrating different data-collection instruments. In the second part, I consider a practical example of integration between two research techniques: surveys and focus groups. By applying this triangulation to an investigation on the perception of juvenile deviance, I attempt to identify the advantages and limitations of this strategy.

The research I present in this case study belongs to an open concept of triangulation as a data enrichment strategy. It aims to combine different collection techniques – such as focus groups and surveys – in in-depth research, with a view to testing new directions for analysis and providing starting points for further consideration.

What Is the Purpose of Triangulation between Different Data-Collection Techniques?

The term triangulation derives from topographical studies and refers to a trigonometric survey method through which the exact distance between a fixed point and an observation point can be identified. It has been applied in various spheres: from nautical, to military, to geometry. In each context, it is understood as referring to the need to draw on several points of reference to obtain a more accurate result (Smith, 1975).

In the social sciences, triangulation refers to a strategy of integrating different techniques in studying the same phenomenon, with a view to increasing the reliability of the research results. Denzin (1970), in particular, has been credited with the attempt to classify the uses of triangulation in the social sciences. Speaking specifically of methodological triangulation, he distinguishes two main uses of this strategy: the within-method approach and the between-methods approach.

The first, the within-method approach, is defined by Herman W. Smith (1975) as the most primitive form of triangulation. This is because it envisages both a comparison between different indicators relating to the same property and a comparison between the results obtained by those same indicators in different surveys, the aim being to check both internal validity and data reliability. So, this technique simply requires the use of the same instrument in different occasions. For example, in a survey to detect the same trait, the researcher may use scales with different items and then compare the results of each of them. Likewise, in a research within a qualitative approach, the researcher could use many observational groups and then compare the results of the work of each group.

The other type of methodological triangulation, the between-methods approach, can be defined as the most popular, albeit one of the most complex, form of triangulation (Smith, 1975). It enables researchers to compare the results obtained by different techniques and has an illustrious precedent in the multi-trait/multi-method procedure of Donald T. Campbell and Donald W. Fiske (1959), which is distinguished by two innovative concepts: convergent validity and divergent validity. The first concept relates to the validity obtained by the congruence assessment of the same trait detected by different techniques so that the expectation is that valid indicators lead to very similar results, even if detected using different techniques. Instead, the second concept refers to the divergence or to a measure of the distance between the results of the detection of different traits obtained through the same technique or similar techniques so that the expectation is that the results of valid indicators relative to the detection of different characteristics differ significantly. In order to do that, Campbell and Fiske (1959) propose to carry out repeated detections of both the same trait and different traits using the same technique and different techniques and then to compare the results in the multi-trait/multi-method matrix.

On this last form of triangulation, a debate opened up and still continues. The person who first fully grasped and espoused this proposal, adapting it to the needs of anthropological research, is Douglas (1976), who enunciated the principle of ‘mixed strategies’. In his view, each study must start from a low degree of methodological structuring and then slowly and gradually arrive at an increasingly precise definition of the strategy to be adopted, going from natural to less and less natural, and increasingly structured and standardized forms of interaction. In other words, the chief principle of mixed strategies is that the researcher should begin with as little control in methods and with as much natural interaction as possible. An example is the study of nude beaches (Douglas et al., 1977) started with natural conversations with the respondents and continued by conducting interviews and then engaging in natural experiments in the field setting. Here, too, the results obtained using different techniques are compared, but with the aim not so much of validating the assertions as of bringing the objective of the research into ever-closer focus and delineating the problem under investigation in an increasingly targeted manner.

In most cases, however, the debate has focused on the techniques on which integration strategies should be used, in an attempt to neutralize the weaknesses of each instrument while at the same time enhancing its strengths. This strategy attracted a great deal of criticism, especially in relation to the integration imperative, which in a few years became for many authors a ‘must’ that was often not backed by sufficient study. As Todd D. Jick (1979) underscores, combining techniques can be effective and useful only if applied within a research design that has been studied in its finest details and supported by a sufficiently sound conceptual and theoretical apparatus.

This is the case with research projects whose design itself envisages certain important aspects of application such as the breakdown of survey groups, different data-collection stages and specific data analysis and comparison methods. Again, according to Jick (1979), if the project design lacks this precision, the risk is that triangulation will create more confusion and the researcher will be condemned to an inundation of information that he or she will be able to process only superficially.

The decision to adopt a multiple-technique strategy should in fact be followed by a specific focus on the nature of the data available (Fielding & Fielding, 1986). It is clear that data deriving from different techniques cannot always be easily related and compared, especially if the context from which said data were obtained and the survey instrument used are not given sufficient consideration. The risk, in short, is that the results obtained will be extremely approximate and relatively superficial. This is because each type of combination inevitably entails a choice fraught with compromises, cuts and simplifications.

In some cases, the criticisms were much more trenchant and struck to the heart of the combined technique approach. David Silverman (2000), for example, speaks of the illusion of building a unitary framework. Taking up the concept of Martyn Hammersley and Paul Atkinson (1983), Silverman asserts that researchers should not adopt an ingenuously optimistic viewpoint, according to which data from different sources will always be aggregated in a complementary and non-problematic manner. In this sense, Silverman (2000) argues that triangulation ignores the fact that different sets of data will always be generated in different contexts, so he claims that counterposing different data ignores the context-bound and skilful character of social interaction.

Moreover, Silverman (2000) reiterates that a naive use of triangulation only leads to ‘an illusory search for “the full picture”’ (p. 100). Supporting the use of multiple methods – for instance, combining interviews, participant observation and document reading in an ethnographic study – does not ensure having the most valid and reliable data. In other words, two parts do not necessarily give us a whole, just two pieces of the picture, and hence the combination of different methods is not the way to obtain the complete picture, or the ‘true’ state of affairs (Silverman, 2001). On the contrary, Silverman (2000) points out that ‘it is usually far better to celebrate the partiality of your data and delight in the particular phenomena that it allows you to inspect (hopefully in detail)’ (p. 112).

This indicates that triangulation has in many cases become a ‘methodological fashion’ rather than a real requirement. In other words, in all the various triangulation designs one basic assumption is buried. The triangulation rests on the promise that the weaknesses in each single method will be compensated by the counter-balancing strengths of another (Jick, 1979). But this assumption has to be deconstructed. With that in mind, we need to reflect deeply before adopting a strategy based on integrating multiple techniques. It is necessary to guard against adopting an ingenuous vision that attributes to the mediation of different sets of data the illusion of obtaining certain, more accurate or reliable results. These criticisms of combined technique strategies help us adopt a more informed standpoint with respect to an automatic and generalized use of such strategies.

In social research, we rarely deal with only a solid theoretical–conceptual framework of reference and an in-depth knowledge of the field of investigation that would enable us to design data ex ante that are suitable for triangulation. Rather, social research normally deals with exploratory and descriptive designs that start from a low degree of knowledge of the subject under study and a fragile theoretical–conceptual framework. For this reason, triangulation in the social sciences should not be used solely to evaluate and check the validity and reliability of the data, but should be extended to other purposes. But for which purposes, then, should triangulation strategy (i.e. a strategy combining different techniques) be used? Researchers have identified three specific uses:

• Help with systematizing a theory. The strategy of integrating different techniques can be a valuable help in systematizing a specific theory with respect to the problem under analysis (Jick, 1979): The emergence of contradictory viewpoints can prompt a redefinition of the theory in a circular integration between theory and research. An example of this use is offered by Jick (1979) in his study on the effects of a merger on employees. Although different measures of the same construct were shown to yield similar results, in the multi-method results there were also some surprises and discrepancies, which led to unexpected findings. For example, in one instance, interview data helped to suggest a relation between job insecurity/anxiety and certain attitudinal symptoms (Jick, 1979). In this case, the divergence turns out to be an opportunity for enriching the explanation.
• Support in interpreting the results of a study. Using different techniques together (e.g. techniques envisaging a quantitative approach with those envisaging a qualitative approach) is a particularly useful strategy to bear out the interpretation of controversial empirical results. This is the case when researchers want to conduct an in-depth analysis of certain specific cases of the subject under study, such as deviant cases or outliers. Researchers know that survey is more appropriate to people coming from the middle class and that it is difficult to study in a standardized way the attitudes of deviant people or people belonging to specific groups, such as outliers, subcultural groups or suburban groups. It also applies in cases where researchers want to examine specific aspects of the results of an initial investigation that cannot be satisfactorily explained or that do not provide satisfactory empirical support for the hypotheses formulated. In this respect, Silverman (2000) and Robert Dingwall (1997) view triangulation as an opportunity that can be exploited not so much during the ‘how’ stage (i.e. in the early stages of the research), as in the ‘why’ stage (i.e. those times when some empirical results do not ‘add up’ and require additional data to obtain a more satisfactory interpretation of the phenomenon being analysed).
• Information enrichment. Different techniques can be combined to increase the amount of data available or to find new stimuli and new opportunities to expand knowledge of the subject under study by envisaging a flexible and open research design. The case presented in the second part of this case study focuses on this purpose of triangulation, giving a specific example.

The Research on Juvenile Deviance: An Example of Triangulation in Action between Surveys and Focus Groups

The research work I wish to present here is part of a wider reflection that aims to re-evaluate and revisit the research studies. The studies are seen not just as an opportunity to analyse contingent reality but also as a basis for a purely cumulative empirical and theoretical knowledge in which survey results provide the starting point for a more precisely focused investigation.

The principal goal of the research on juvenile deviance can be read from this perspective. It starts from the need to clarify certain results obtained from a survey on the value orientation and attitudes of about 800 young people living in Rome (Cataldi, 2006). The in-depth analysis concentrated on a specific section of the questionnaire used for the broader survey, concerning the perception of deviant behaviour and the concept of legality.

The investigation used eight discussion groups segmented on the basis of the sample characteristics of the young people taking part in the previous survey. It followed, with some amendments, the factorial sampling grid adopted by the interviewers. So, in the in-depth investigation, participants were selected on the basis of three criteria: age, divided into two groups (18–25 and 26–32 years); educational qualifications, distinguishing between school-leaving certificates and university students and graduates, grouped in a single category; and political orientation, distinguishing between subjects defining themselves as centre-right and those defining themselves as centre-left. The specific aim of the research was to bring out non-generic, but additional and significantly relevant, information that could cast light on a number of problem issues that had emerged from the analyses of the broader data collected.

For this reason, the focus group sessions were structured in two stages: a private compilation stage and a collective debate stage. The first stage, whose function was to define the field of discussion and bring participants to focus on the issues under study, envisaged the self-compilation of the four main questions in the survey questionnaire. These consisted of two scored questions, concerning the concepts of acceptability and gravity of deviant behaviours,1 and two questions with pre-codified answers on the concept of rules2 and on the role of other people in guiding or influencing individual behaviour.3 The specific task of the second stage was to explore, through collective discussion, the principles, values and scenarios on which individuals base their answers to the questions put to them.

The expectation in conducting this in-depth and data-integration analysis was to obtain some stimuli for reflection and feedback that could be grouped along three main lines:

• 1.
  The technical–operational dimension. One of the objectives of the investigation was to conduct an in-depth analysis of the instruments used in the survey, especially in relation to the way they are perceived and used by respondents. From the analyses of the acceptability and gravity scales, many contradictions emerged, including numerous answers that go against common sense and contrast with the legislative approach of penalizing certain behaviours. These suggested response, score-attribution and modality anchoring mechanisms that were very different from those tacitly assumed by the investigation based on a rational and behaviourist questionnaire. In relation to these aspects, the in-depth analysis aims to achieve clarification and suggests possible changes or recommendations to be followed when drawing up questionnaires.
• 2.
  The information–evaluation dimension. The investigation aimed to analyse the motives underlying some of the answers obtained, by expressing the respondents' collective imagination, experiential assumptions and mental linkages. One of the goals of the research was to publicize the tacit dimension, that knowledge which is taken for granted and which informs respondents' manner of response. For example, the survey showed that, although Italian young people are very favourable towards the deinstitutionalization of the family (e.g. they are in favour of civil partnership, living together and gay rights), infidelity in a marriage or in couple is still considered serious. To give an idea, the results of the scored question on acceptability stated that behaviours against the (marital) fidelity are considered as unacceptable as drug consumption. The in-depth research tries to understand what experience and what tacit motivations are hidden behind these striking and difficult-to-explain results.
• 3.
  The theoretical–interpretative dimension. Starting from a consideration of the relationship between the respondents' interpretations and those of the researcher, the in-depth investigation was intended to stimulate, redefine and, if appropriate, redirect the considerations made previously on the basis of the data obtained from the survey. From the analyses conducted, an individualistic and hedonistic moral viewpoint emerged, which, while continuing to be fed by traditional familist principles, leads to a hyper-evaluation of the self and of personal well-being. In fact, the survey showed that behaviours deemed most eligible are those that cause excitement or satisfaction, and conversely, the behaviours that take positions closer to the pole of ineligibility are those bringing harm, suffering, sorrow or a sense of blame to the individual. However, from the survey it was not clear how respondents reconciled this hedonistic orientation with another apparently opposite attitude: the familism and the tendency to indicate family as the first most important value.

The overview that emerges from the survey appears to be fragmentary, contradictory and difficult to interpret using the classic social cleavages, such as the structural variables related to the socio-biographical part of the questionnaire. Gender, age, parent's job, parent's educational level as well as young people's political attitudes seem insufficient to understand some results obtained from the survey. That is why the in-depth research involved the same respondents in suggesting new keys to read and understand the information that emerged.

The analysis was therefore conducted primarily at the hermeneutic level, focusing on the meaning of the respondents' answers and investigating the phenomenon intensively rather than extensively. In particular, the study was conducted through a manual content analysis by comparing and classifying the response categories and providing a detailed description of the opinions that emerged, correlated with passages quoted from respondents' interventions (Stewart & Shamdasani, 1990). All of the reasons given in favour of or against the acceptability and gravity of individual behaviours and the selection criteria expressed by the respondents were then grouped together at different levels on the basis of clearly defined sets. The content analysis concerned the investigation of ethical–moral criteria, the values underlying the choices made, the imaginations and tacit/experiential assumptions of the respondents and the motivations underlying their positions with respect to each deviant behaviour indicated. Finally, a technical analysis was conducted that attempted to reveal the respondents' manner of understanding words and phrases, anchoring and score-attribution mechanisms and the analysis of the meanings attributed to the pre-codified answers.

Conclusions

On the basis of the case illustrated, the focus group technique lends itself very effectively to feedback on the investigation and proved to be an appropriate method of supporting the survey. This strategy is particularly useful in cases where the aim is to conduct what Aaron Cicourel (1974) would call a non-definitive triangulation, that is, a technique-combination strategy characterized by a specific focus on elaborating the assertions and by the contextual nature of the reports, according to the teachings of ethno-methodological practice. From this perspective, the originality of the focus group's role in bearing out the analytical–interpretative stage of the investigation cannot merely refer to the possibility of obtaining confirmation or non-confirmation of one's own hypotheses.

A new concept of triangulation has thus proved successful, in which the integration of different data-collection techniques is used more to enrich the information than to disclose its authenticity. The combined focus group-survey offers a number of different starting points for reflection, which expand upon the knowledge already acquired from a broader investigation. The objective of the comparison, therefore, was focused not on seeking informational concordance or discordance but specifically on the possibility of expanding the interpretative horizons and examining specific aspects of the problem under investigation, thus embarking upon a pathway of reflection and feedback that involves all aspects of the research. The in-depth investigation sought to analyse the motivations for the answers obtained, by expressing the respondents' collective imagination, experiential assumptions and mental linkages.

From the point of view of the intensity of the discussions and the profundity of the arguments, the focus group technique was more suitable for penetrating the respondents' response mechanisms, to the point of stimulating the expression of their tacit assumptions, or assumptions normally taken for granted, underlying subjective opinions. The focus groups stimulated interpersonal discussion and sharing by helping respondents retrace the processes of structuring opinions and publicly problematize their ideas, in a continuous effort to discuss and clarify.

The group discussions also proved to be a suitable forum to bring out respondents' individual contexts, enriched by very private aspects of their subjective experience. In a number of sessions, the group spirit and the complicity between the participants succeeded in overcoming inhibitions, making it possible to gradually examine the arguments in greater depth in a process of collective introspection. In one case in particular, one of the respondents told of a very dramatic and sensitive personal experience, thanks to the atmosphere of mutual acceptance and fruitful exchanges of views that had developed in the group. Episodes of this type are significant because they enable the researcher to enter into respondents' experience and share in their experiential background and to succeed in uncovering the tacit baggage that guides their response mechanisms.

However, on some occasions focus groups played a part in bringing out discordance and contradictions within individual thought and indeed revealed, in their various facets, the fragility and weakness of respondents' subjective convictions. This shows the bilateral role that the group plays on the individual. While it sometimes acted as a brake inhibiting the free expression of ideas, it more often acted as a stimulus to the frank admission of personal vulnerability. It encouraged respondents, in a climate of mutual understanding and complicity, to recognize their failings and to publicly declare the incoherence of their thoughts and opinions and to reveal evident cognitive dissonances.

Finally, on several occasions the group discussions made it possible to broaden the field of investigation to collateral arguments that the respondents themselves signalled as being relevant and in keeping with the subject of the survey. This happened, for example, in the discussion of respondents' perception of the gravity of fraud. Participants suggested, in parallel, information technology fraud and cyberfraud – including downloading files illegally from the Internet – topics that could be included as items for evaluation in a subsequent investigation. In this respect, the focus groups were open to accepting new ideas and broadening the field of research to innovative proposals on the basis of the conceptual proximity and proximity of thought discernible in the frames of reference of the respondents themselves.

Exercises and Discussion Questions

• 1.
  In your opinion, what are the weaknesses and strengths of triangulation in the social sciences?
• 2.
  Identify the main purposes of triangulation strategy to avoid the ‘methodological fashion’ of combining different data-gathering techniques without a specific reflection and a robust research design.
• 3.
  In the research on young Romans' deviance, why were some results difficult to interpret?
• 4.
  Imagine you decide to use a combination of survey and focus group for data collection in your research. Explain the reasons behind your choice, underlining the goals and the expectations of such a combination.
• 5.
  Explain the meaning of technical–operational dimension, information–evaluation dimension and the theoretical–interpretative dimension. Give some examples.
• 6.
  In your opinion, at the end what are the main advantages and disadvantages of combining focus group and survey data in the research on deviance in Roman young people? Would you repeat this choice of combination? Why?