Trustworthiness of research findings

I hereby proceed to discuss the key issues of research quality typical to this type of design. It will suffice to note here that in this discussion of research quality and its criteria, a parallel with the phenomenon under study (quality in HE) may easily be drawn. After all, research is one of the core HE activities. The criteria of research quality are embedded in competing discourses about the nature of reality, what valid knowledge is and how to discover it. The tensions and contestation regarding quality of research revolve around the issues of objectivity and subjectivity, a researcher’s role in the process of investigation, and one’s relationship to data and research subjects. More or less institutionalized research practices may be specific to a field of science, a particular discipline and a so called ‘epistemic community’.

According to research methodology literature (e.g. Healy and Perry, 2000), criteria for judging quality of research, e.g., generalizability, validity, and reliability depend on the paradigm a researcher is working from, as well as the purpose and type of research project. In conventional research terms, trustworthy research accounts are considered to be those which demonstrate internal validity and reliability. Despite varied opinions in the research methodology literature about the use and applicability of validity and reliability terms in qualitative studies, I find the underlying principle of trustworthiness central in judging quality of research findings. In a more naturalistic type of inquiry, like this study, where the flow of events and relationships unfold in the process of research, validity and reliability criteria are replaced by credibility and dependability. A criterion of reliability is meant to provide the readers with evidence that if a study was replicated with the same or similar respondents in the same context, and using the same techniques, its findings and conclusions would be repeated. Such an approach to the research

process and findings is arguable, especially from the epistemological view about the constructed and ongoing nature of reality. I agree with scholars who note that in qualitative studies, especially the findings derived from using non-standardized research methods (semi-structured interviews in this study) “are not necessarily intended to be repeatable since they reflect reality at the time they were collected, in a situation which may be subject to change” (Marshall and Rossman, 1999 in Saunders, Lewis and Thornhill, 2007). So how can then a reader know that the study findings are trustworthy?

For a case study, Yin (2003) identifies three types of validity: construct, external and internal validity. However, it is the latter type that is the most pertinent to qualitative case studies. Miles and Huberman’s (1994) observation about validity relates to Yin’s definition of internal validity, which can be addressed primarily at the data analysis stage. Miles and Huberman (1994) suggest, that in order to achieve internal validity, “meanings have to be tested for their plausibility, their sturdiness, their ‘confirmability’” (p. 11). Likewise, Gabriel (2008) suggests that plausibility, not accuracy in the process of establishing validity is important. For interpretivist research, like this study, plausibility concerns the truth value of research findings, whether the account, the inferences made, the experiences described appear to be true and real, whether the reader can relate to the subject’s world (Fulton, 2010). Constructivist research accounts, therefore, consider, among essential quality indicators, ‘truth’ or credibility, neutrality or confirmability, consistency or dependability and applicability or transferability (Lincoln and Guba, 1985).

In order to build a credible research account, I am cautious that the constructed realities which exist in the minds of respondents and those that I attribute to research objects and subjects are compatible (Lincoln and Guba, 1985). One of the major threats both to credibility and dependability are my, as principal investigator, bias and errors. In order to minimize these, scholars (e.g. Yin, 2003, p. 37) suggest first, to explicate my own bias, second, to support study findings and interpretations, either converging or competing explanations and conclusions with data, and third, to assure that they are internally coherent. Confirmability, as referred above, is thus a degree to which the findings are the product of the focus of inquiry and not my biases. My self-awareness, explicit assumptions, values and biases are, therefore, important elements in the course of data analysis and interpretation. Researcher’s reflexivity, analytical skills, a detailed account, and transparency of process for arriving at certain interpretations enhance the trustworthiness of research findings. In order to increase the dependability of the study, I apply theoretical insights drawn from extensive literature review (see

section 3.4. and Appendix D) as a guide, somewhat similar to what Yin calls “a standardized agenda for the investigator’s line of inquiry” (2003, p. 68) or a study protocol. Following recommendations in the research methodology literature (e.g.

Spiggle, 1994), it is supplemented with an account of field notes, research process log with notes and memos with recorded details of data collection and steps in the analysis. Another important aspect in data collection and analysis process that, I believe, increases both credibility and dependability of research findings is complementarity of methods and data sources. For a more detailed account of what data was collected, what analysis techniques were employed in the study in order to produce a trustworthy account, please see section 4.3.

The last criterion of research quality that I will discuss here is transferability. Many researchers consider generalizability as the most common drawback of case studies, especially when a study is based on a single case. I take a stance of researchers who argue that generalizability might be reached even in a single case at the level of theory “as long as one applies a clear set of analytical principles or theoretical assumptions to different settings, one can generalize about the basic processual factors which are likely to come into play across these settings” (Watson, 2001 in Gabriel, 2008, p.32, also Yin, 2003). Here I also find Eisenhardt’s (1989) point of view that findings can be related to the existing literature applicable (p. 545). In a theory-building rather than a theory-testing study, analytic generalization and “the conditions under which the construct or theory operates” (Miles and Huberman, 1994, p. 29), not a generalization of findings to other settings, is a prime concern.

This case is a qualitative study and, unlike quantitative studies, aims to provide rich context and meaningful (‘thick’) research account for its readers and let them determine how the situation described applies to their situation. In Geertz’s (1973, p. 26) viewpoint, providing a thick description rather than codifying “abstract regularities” and generalizing within a case are the essential tasks of theory-building research. While this study may yield transferable findings that apply to other organizational settings, studying the specificity of quality discourse and practices in higher education means that it may be “difficult to extrapolate results to other contexts beyond the specific academic institution and academic context”

(Kyriakidou, 2011, p. 587). Also, various other factors, e.g., the types of HEIs involved, a disciplinary field of a JP, nature of the national quality scheme will influence study findings and, as a result, may not necessarily yield broadly inclusive results. Nonetheless, I agree with scholars who argue that even if the knowledge that is gained from case studies “cannot be formally generalized does not mean that it cannot enter into the collective process of knowledge accumulation in a given field” (Flyvbjerg, 2006, p. 227). With regards to the issue of

transferability, I take Lincoln and Guba’s (1985) view that a decision to the extent to which the findings can be applied to other contexts and other respondents is left with the reader.