Advantages and Disadvantages of Triangulation Research

Advantages and Disadvantages of Triangulation Research Triangulation Qualitative Analysis Triangulation is often used in qualitative methodology to increase the validity of findings, but it is not a panacea. Using British examples in the police and public sector, discuss the meaning of the term triangulation, how it can be used in qualitative research and its limitations and potential. Abstract This paper focuses on the use of triangulation within qualitative research using examples from British public sector and more specifically, police research. Qualitative research approaches have been increasingly used within public sector research. Different types of triangulation are explained and the relevance of each type is identified. The purpose of using triangulation within qualitative research; which is to increase validity, is critically discussed. Different types of validity are explained, as well as the significance of the concept of validity to research. Quantitative and qualitative research approaches are critically compared. The use of qualitative approaches within public sector research is defended, and the positivist perspective is challenged by arguing that the use of triangulation can strengthen the validity of qualitative research methodology. Using three examples of public sector research, some advantages and potential limitations of using triangulation within this area are identified. Several examples of police research studies that have adopted the use of triangulation as part of a qualitative approach are critically discussed with reference to different types of triangulation and their relationship to different types of validity. A synthesis of the issues that emerge from the selected police research studies in relation to triangulation is included. The paper concludes that while there are potential limitations associated with the use of triangulation within qualitative research, the cited examples of public sector and police research demonstrate that there are also several potential advantages and that triangulation can contribute to the validity of qualitative research in these domains. It is important to acknowledge that the application or the omission of different types of triangulation within a research study can correspond to its strengths and weaknesses, and there is an implication for the researcher to be knowledgeable and skilled in the use of triangulation in order to optimise its application within qualitative research. In other words, the limitations of triangulation might relate to the way that it is implemented rather than to the strategy itself. Introduction The purpose of this paper is to critically examine the use of triangulation within qualitative methodology as a means of increasing the validity of findings, with reference to research examples from the police and other public sector areas. Triangulation, Validity and Qualitative Research In simple terms, triangulation refers to the examination of a situation from different perspectives, using different methods (Laws et al 2003). Triangulation is often referred to as the use of mixed methods within research, however Denzin and Lincoln (2000) developed this concept further to include four different types: data triangulation, which refers to the use of a variety of sources of data within a study; investigator triangulation which involves several researchers; theory triangulation, which uses multiple perspectives to interpret a single set of data and methodological triangulation which involves the use of multiple methods to study a single problem. Data triangulation is commonly used within qualitative research as it facilitates understanding of the complexity of a poorly understood phenomenon, an example being the combination of holding in- depth discussions with participants and observing their behaviour within a natural setting (Polit and Beck 2004). Investigator trian gulation can help to overcome the potential problem of personal bias (Sim and Wright 2000). The advantage of using methodological triangulation is that it helps to resolve the limitations that single methods have in studying the complexity of social reality (Kendall 2005). Frequently triangulation involves the use of both quantitative and qualitative methods, the rationale being that the strengths of each are optimised (Flick 2006). This paper however, will focus on the use of triangulation exclusively within qualitative methodology. The purpose of using triangulation within qualitative research is to add rigour, depth, breadth, complexity and richness to the research process (Denzin and Lincoln 2003a). The use of triangulation within qualitative research is therefore, said to increase validity (Giles 2002). The term validity is often referred to as the ability of a research instrument to measure what it is intended to do (De Vaus 2002). There are however, different types of validity; internal validity, which asks whether researchers are actually measuring what they think they are; external validity which relates to the generalisability of the findings of a study to other settings; interpretive validity which is concerned with the accurate understanding and portrayal of data; theoretical validity which relates to the appropriateness of theory that emerges from the data and evaluative validity which refers to the quality of the research process, in relation to the actions of the researcher. Overall, within qualitative research; validity relates to the trustworthiness of the data, its analysis and its interpretation (Waltz et al 2005). Holloway and Wheeler (2002) however, suggest that triangulation does not always confer validity; that this is dependent on the nature of the individual research project and that only researchers who are experienced should use triangulation within qualitative research. Qualitative research approaches seek to understand the meaning of phenomena (Patton 2002a). They aim to represent the true complexities of human behaviour, gaining access to thoughts and feelings that cannot be accessed by using other methods Flanagan 2005). Quantitative research on the other hand, has been criticised for its reductionist approach towards the human experience; it has the ability to produce results which are statistically significant yet humanly insignificant. Qualitative research methods include interviews, observations and document analysis. The qualitative researcher tends to engage more with the research participants and aims to address any associated criticism of the potential for subjectivity by the application of rigorous and systematic methods of data collection and analysis. Qualitative research approaches have increasingly been used in public sector research within the UK, for example within the health care domain, where they have helped to enhance understanding of health, health behaviours and health services (Green and Thorogood 2004). There are also several examples cited within the literature, of qualitative methodology being used in police service research. (for example: Cassell and Symon 2004; Dick 2000; Dick and Jankowicz 2001; Dorn and Brown 2003; Noaks and Wincup 2004; Office for Public Management 2006). Triangulation in Public Sector Research Qualitative research methodology is often employed within the public sector because of the ‘people factor’; public sector organisations such as the police, health and social services and local councils employ large numbers of personnel (Grant and Vidler 2000), added to which these are helping organisations that provide a human service. Qualitative research is described as a naturalistic, interpretative approach that is concerned with understanding the meanings which people attach to phenomena within their social worlds (Ritchie and Lewis 2003). As stated previously, it is very much concerned with human behaviour, experiences, values and beliefs. It does however attract criticism from positivists in terms of not being valued as a scientific approach (Haslam and McGarty 2003). Positivism adopts the position that the purpose of science is to limit research activity to what can be directly observed and measured; that aspects such as feelings, attitudes and beliefs cannot be directly observed and measured and are not therefore, legitimate areas for scientific investigation (Trochim 2006). As previously stated, the implications of this are that there are huge areas within the human experience that would not warrant investigation within the quantitative paradigm. The purpose of triangulation within qualitative research is to challenge the criticism that this is not a scientific approach, by combining multiple observers, theories, methods and data sources in order to overcome the intrinsic bias that arises from single- methods, single observer and single- theory studies (Patton 2002b). There now follows an overview of three public sector research studies that have employed triangulation in order to demonstrate the effectiveness of this approach. McAdam et al (2002) used data triangulation of qualitative methodology to investigate how the principles of total quality management (TQM) are being sustained in the UK public sector by contributing to improved performance levels. The research methodologies included focus groups and semi-structured interviews. Using triangulation served to strengthen their finding which was that quality frameworks play a key role in improving organisational performance over time. A study by Atwal and Caldwell (2006) aimed to explore nurses perceptions of multidisciplinary teamwork in acute health-care. In order to do so, they adopted methodological triangulation by conducting interviews and direct observations of interactions between nurses and health-care professionals in multidisciplinary teams. The findings of this study identified three barriers that hindered teamwork: differing perceptions of teamwork; different levels of skills acquisitions to function as a team member; and the dominance of medical power that influenced interaction in teams. The combination of interview with observation in qualitative research is becoming increasingly popular (Ekstrom 2004). A study by Bryans (2004) aimed to articulate the health visiting expertise involved in recognizing and responding to client need during home visits. The researcher adopted what she describes as an innovative, multi-method approach used to articulate this expertise which involved a 20-minute simulated visit to an actress-client, a post- simulation focused interview and subsequent observation of actual home visits with 15 study participants. The researcher identified the strength of this approach as providing a blend of control and naturalism and considers that this approach could usefully be applied within multidisciplinary contexts. The researcher also acknowledged the limitations of the approach as arising from the variability of naturally-occurring, observational data which meant that simulation and observational data were not directly comparable in terms of content in a study of this type. Comparability of observational and simulation data, and the reliability of the observational data, were also diminished by the use of different methods of data-gathering. These studies collectively demonstrate that the use of triangulation within qualitative research can strengthen its findings; a potential limitation is that it may not be easy to make dependable comparisons between two sets of data due to differences in the ways that the data was collected. Triangulation in Police Research A review of the literature reveals that qualitative research approaches have been employed within police research involving the use of triangulation for some time. An interesting example of the use of triangulation within police research are studies that highlight discrepancies between information given about experiences of crime to the police by comparing this data obtained by other means, for example by using interviews conducted by external researchers (Bechhofer and Paterson 2000a). In other words, a more accurate picture can be gained by the use of triangulation. The same authors refer also to the use of qualitative research and multiple methodologies in order to obtain information about peoples’ perceptions about crime; for example relating to their fear of crime. This information might otherwise be inaccessible. It is possible that the use of triangulation within police research could be effective in dealing with information of a sensitive nature. Lee (2000a) makes reference to a study conducted by Norris et al in 1992 that demonstrated how the use of different methodologies can highlight the difference between perceptions held about, and the reality of, a situation. In this instance observations were made of the demeanour of police officers and citizens in situations where citizens were stopped by the police. During an encounter, blacks and whites were equally likely to present a calm and civil demeanour towards the police. There were also few differences in police demeanour and action towards the two groups. These findings did not support competing claims (i.e. derived from other sources of information) that the police are overtly hostile to blacks, or that blacks display disrespectful attitudes towards the police. Devine and Heath (1999) provide a critique of an ethnographic study conducted by Hobbs in 1988, of petty criminals and local CID detectives working in the East End of London. The ethnographic research consisted of both overt and covert participant observation of both the criminals and the police in a variety of different settings. The researcher was able therefore to gain a variety of sources of data which contributed to what is described by Devine and Heath as the ‘richness of his ethnography’ (p.8). However they also consider that the study would have benefited from investigator triangulation in order to overcome the problem of bias. They felt that this would have led to a more accurate picture of malpractice within the Metropolitan CID. An identified limitation of participant observation is that the researcher can become biased in favour of the group or indeed against them (Moore 2001). A study of this kind is also unlikely to have external validity; as being an ethno graphic study; it is very much informed by the uniqueness of its social and cultural context. The last two studies demonstrate the value of observation as a research method particularly as a component of triangulation. The advantages of observation are as follows: the researcher can observe interaction within a group as well as gain information from people individually via interviews, as the dynamics of a group might influence the nature of communication between its members; the researcher might gain a more objective view of a situation than a research participant who is likely to be immersed in their social world and may not as a result, identify aspects which are of interest to the researcher. Observation also enables the researcher to observe actions, which can be more meaningful in some ways than verbal communication (Weinberg 2001). The value of investigator triangulation is highlighted in a second study cited by Lee (2000b). This also looked at police- citizen interaction using multiple observers from different backgrounds. Data triangulation was also employed as the observers were asked to collect data in two forms, one on a checklist; the other as a narrative account. The two sources of data were then compared. The use of multiple observers should help to overcome the potential problem of personal bias that was said to arise from the Hobbs study. A study by Knock (2002) drew on the findings of a telephone survey of all police forces in England and Wales and interviews with key personnel in 16 police forces to gain information about the use of Sex Offender Orders (SOOs) which came into force in 1998; the aim of these being to provide an additional measure of protection to the public from sex offenders by prohibiting an offender from certain behaviours that had previously been precursors to offending. It was found that the pattern of SOO applications is not uniformly distributed across the country. The researcher commented that the low uptake of SOOs by some forces needs to be carefully interpreted. The use of triangulation allowed for the underlying reasons for low uptake to be identified. These included a general lack of understanding about the SOO process and concerns about inadequate resources for monitoring the Order; this information being gained via the interviews. Furthermore, some forces identified that other strategie s for managing sex offenders without resorting to an Order were considered to be adequate; others had negotiated voluntary agreements with offenders in order to modify their behaviour. The relevance of this study in relation to triangulation is that the ‘first level’ of presenting information could be misleading; the use of more than one methodology facilitated a deeper level of understanding about the variation in applying the SOOs. As with all qualitative research methods, there are advantages and disadvantages associated with the use of telephone interviews. It may be possible for the researcher to enter responses directly onto a computer which could increase time efficiency and accuracy of recording. Respondents might find it easier to respond to questions of a sensitive nature when they are not face- to- face with the interviewer. The disadvantages include a lower response rate than for face- to- face interviews, which reduces further when longer telephone intervie ws are required (Pocock 2000). It is also not possible to observe non- verbal aspects of communication. The use of triangulation therefore, helps to overcome the disadvantages of a single research method. The identified weaknesses of the telephone interview are all aspects which could be compensated for by the additional use of face-to- face interviews. Conversely, the strengths of the telephone interview should help to overcome the limitations of the face- to- face interview. Pain et al (2002) carried out research commissioned by the Home Office to produce a Police Research Series paper which examined homeless and school- excluded young people’s experience of crime and disorder; the rationale being that they had previously been poorly represented in consultation. The need to adopt a qualitative approach for this study was emphasised by the researchers, in order that the young people’s attitudes, experiences and understanding of crime could be explored in depth. The participants were invited to develop a range of visual materials, and observations and informal discussion were also used by the researchers. Findings included that there are negative outcomes of the vicious circle created by being labelled as either homeless or as troublemakers at school; that most of the crimes committed by the homeless young people are petty and victimless, such as begging; that the homeless feel most at risk in spaces where control can be exerted over them, su ch as within the home or in dealings with the police and as a result they actually feel safer on the streets; that crime and homelessness are two factors which are correlated; that school- excluded young people are more likely to be involved in crime, most of which is petty but some of which is serious; that school- excluded young people often feel victimised and have been bullied, some have been the victims of abuse within the home and others have been exposed to crime previously as non- offenders. The researchers highlight the benefits of participatory research approaches as enabling the young people to be involved in the process of research and strategy development, at the same time allowing their interests to be promoted. In other words, they are being empowered. The researchers also acknowledge the limitations of participatory research methods, which are not specified within this research report but which have been identified elsewhere as being time intensive; as having been criticised as involving reduced scientific rigour and that the researchers’ control over the project could be compromised (Arble and Moberg 2006). The researchers comment that the limitations of participatory methods could impact upon the effectiveness of their use as a basis for action. Triangulation involving informal discussion and observation in addition to the participatory methods therefore is a strategy which was used in this instance to overcome the potential limitations of the latter, th us increasing the validity of the research process and the interpretation and application of the data. This is a very interesting study which has revealed some powerful insights into the issues affecting an otherwise disenfranchised group, drawing upon their unique perspectives. Some of the findings are surprising and challenge widely held assumptions and negative perceptions held about homeless young people; that are reflected in the lack of resources, support and appropriate accommodation available for this group (Dean 2006). The impact of such a study could be therefore, to contribute towards improved service delivery for homeless and school- excluded young people. It could be argued that triangulation is a powerful strategy that adds credibility to a study of this kind, which in turn could have an impact on improving the lives of otherwise disempowered and misunderstood groups of young people. In summary, there are several examples within the literature of qualitative approaches adopted within police research which employ triangulation. A synthesis of the issues relating to the use of triangulation in this specific area of work is as follows: triangulation can help to address sensitive issues within police research that might not be effectively dealt with using a single method; triangulation can highlight discrepancies between data obtained as a result of using different sources and methods, which could also be interpreted as a limitation of triangulation, in that it could lead to data from different sources not being directly comparable; it enables the subjective perceptions of participants (and therefore those who may be affected by the outcomes of research studies) to be taken into consideration, in addition to the more objective observations of researchers, this allows for participants to be empowered and to potentially have some influence over the development of strat egy which results from the research process; triangulation potentially allows for the use of multiple researchers, multiple settings, multiple sources and multiple methods to be used within police research; the use of multiple methods enables the strengths to overcome the limitations of each method; triangulation facilitates different levels of research investigation to be carried out, for example underlying reasons for the findings emerging from the use of one method can be identified by the use of an alternative approach, thereby contributing to the depth, breadth, complexity and richness of the research process (Denzin and Lincoln 2003b) and triangulation could add to the credibility of a research study which in turn could impact upon changes being made to improve service provision. Conclusion Robson (2002) agrees with the statement within the question; i.e. that triangulation is not a panacea. It can be problematic, for example when two data sources are inconsistent or conflicting. Further investigation might lead to an even more complex set of understandings. Kirby (2000) agrees that while triangulation is used to overcome the problems associated with using one research method, it does not eliminate these problems, but disguises them by compensating for the weaknesses of one method with the strengths of another. Triangulation is not a unified overall methodology; it is a way of using complementary methods. Coleman and Briggs (2002) argue that while triangulation contributes to validity, it is not a panacea and that its value can be overestimated. Bechhofer and Paterson (2000b) suggest that there are weak and strong forms of triangulation. They describe a weak form as making multiple observations of a situation using the same method and a strong form as making several obs ervations using different methods. Despite these identified limitations of triangulation, examples cited within this paper from public sector and specifically, police research indicate that there are several potential ways in which it can contribute to the validity of the qualitative research process. The advantage of using qualitative methods within public sector and police research has been identified here as enabling the human experience to be valued and investigated. Potential limitations of qualitative research methods can be overcome by the use of triangulation as the strengths of each individual method can compensate for the weaknesses of others. Finally it is important to acknowledge that the appropriate skills of the researcher can contribute towards the quality of a mixed- methods study and that it might be necessary to involve two or more researchers who have complementary research skills in order to optimise the benefits of using triangulation within qualitative research (Tashakkori and Teddlie 2003). References Arble B and Moberg D. Participatory Research in Development of Public health Interventions. Population Health Institute. 2006. p. 3. Atwal A and Caldwell K. Nurses perceptions of multidisciplinary team work in acute health-care. International Journal of Nursing Practice. 12(6) 2006 pp.359-365 Bechhofer F and Paterson L. Principles of Research Design in the Social Sciences. Routledge. 2000. p. 58. Bryans A. Examining health visiting expertise: combining simulation, interview and observation. Journal of Advanced Nursing. 47(6) 2004 pp.623-630, Cassell C and Symon G. Essential Guide to Qualitative Methods in Organizational Research. Sage Publications. 2004. p. 316. Coleman and Briggs. Research Methods in Educational Leadership and Management. Sage publications. 2002. p. 70. Dean H. Social Policy. Polity Press. 2006. p. 99 Denzin N and Lincoln Y. Handbook of Qualitative Research. Sage Publications. 2000. p. 391. Denzin N and Lincoln Y Collecting and Interpreting Qualitative Materials. Sage Publications. 2003. p. 8. De Vaus D. Surveys in Social Research. Routledge. 2002. p. 53. Devine F and Heath S. Sociological Research methods in Context. Macmillan. 1999. p. 8. Dick, P. The social construction of meaning of acute stressors: a qualitative study of the personal accounts of police officers using a stress counselling service, Work Stress, 14 (3) 2000. pp.226-244. Dick P. and Jankowicz D social constructionist account of police culture and its influence on the representation and progression of female officers: A repertory grid analysis in a UK police force. Policing: An International Journal of Police Strategies and Management, 24 (2) 2001. pp. 181-199. Dorn L and Brown B, Making sense of invulnerability at work: a qualitative study of police drivers. Safety Science, 41(10) 2003 pp.837-859. Ekstrom K. Elusive Consumption. Berg Publishers. 2004. p. 57. Flanagan C. Research Methods for AQA A Psychology. Nelson Thornes. 2005. p. 46. Flick U An Introduction to Qualitative Research. Sage Publications. 2006. p. 33 Giles D Advanced Research Methods in Psychology. Routledge. 2002. pp. 220-1 Grant S and Vidler C. Economics in Context. Harcourt Heinemann. 2000. p. 64. Green and Thorogood. Qualitative Methods for Health Research. Sage Publications. 2004. p. 4. Haslam S and McGarty C. Research Methods and Statistics in Psychology. Sage Publications. 2003. p. 389. Holloway I and Wheeler S. Qualitative Research in Nursing. Blackwell publishing. 2002. p. 2. Kendall D. Sociology In Our Times. Thomson Wadsworth. 2007. p. 67. Kirby M. Sociology in Perspective. Harcourt Heinemann. 2000. p. 9. Knock K. The Police Perspective on Sex offender Orders: A preliminary view of policy and practice. Policing and Reducing Crime Unit: Police Research Unit. 2002. Laws S , Harper C and Marcus R. Research for Development: A Practical Guide. Sage Publications. 2003. p. 280. Lee R. Unobtrusive Methods in Social research. Open University Press. 2000. p. 37, 48 McAdam R, Reid R and Saulters R Sustaining quality in the UK public sector: Quality measurement frameworks. International Journal of Quality Reliability Management. 19(5). 2002. pp. 581 – 595 Moore S. Sociology Alive!Nelson Thornes. 2001. p. 34 Noaks L and Wincup E. Criminological Research: Understanding Qualitative Methods. Sage Publications. 2004. p. 101. Office for Public Management. Metropolitan Police Authority and the Metropolitan Police Service. Safer London Panel. Communication Matters. OPM. 2006. Pain R, Francis P, Fuller I, O’Brien K and Williams S. ‘Hard- to- Reach’ Young People and Community Safety: a model for participatory research and consultation. Police research Series paper 152. Home Office 2002. Patton M. Qualitative Research Evaluation Methods. Sage Publications. 2002. pp. 1, 555. Pocock R. Market Research in Health and Social Care. Routledge. 2000. p. 100. Polit D and Beck C. Nursing Research: Principles and Methods. Lippincott Williams and Wilkins. 2004. p. 36. Ritchie J and Lewis J. Qualitative Research Practice: a guide for social science students and researchers. Sage Publications. 2003. p. 3. Robson C. Real World Research: A Resource for Social Scientists and Practitioner-Researchers. Blackwell publishing. 2002. p.483. Sim J and Wright C Research in Health Care: Concepts, Designs and Methods. Nelson Thornes 2000. p. 137. Tashakkori A and Teddlie C. Handbook of Mixed Methods in Social Behavioural Research. Sage Publications. 2003. p. 553. Trochim W. Positivism Post-Positivism. Research methods Knowledge Base. Web Centre for Social research Methods. 2006. http://www.socialresearchmethods.net/kb/positvsm.php. Accessed 10th March 2007. Waltz C, Strickland O and Lenz E. Measurement In Nursing And Health Research. Springer Publishing. 2005. p. 217. Weinberg D. Qualitative Research Methods. Blackwell Publishing. 2001. p. 135.