As you may have noticed during the course of yo

24 Aug As you may have noticed during the course of yo

As you may have noticed during the course of your research one key component of a research paper is the abstract. Often the abstract is something that scholars put together before they start work on their research to help them focus their ideas. Then as the paper develops they come to refine the abstract to ensure that it reflects what was actually done within the paper. A good abstract sets the stage for the research topic, explains the method in use, notes the findings, and mentions important implications of the study. They are typically 150-200 words in length and allow researchers to quickly see what a paper is about prior to reading the work in its entirety.

Please post an abstract that accurately reflects your study up to this point. If you need some examples of abstracts to help you get started take a look at some of the abstracts found within the peer-reviewed journals you referenced within your own proposal.

Questions to Answer

1. In thinking about your research, how are you planning to assess your data?

2. How might you code your data?

3. Finally, what additional insight did this week’s material help provide that we haven’t covered yet?

Your posts this week should demonstrate critical reflection upon the assigned readings.

Instructions: Must be at least 300 words. 

Reading: Article

Reliability, Generalisation and Reflexivity: Identifying Validity and Trustworthiness

In: An Introduction to the Philosophy of Methodology

By: Kerry E. Howell

Pub. Date: 2015

Access Date: June 27, 2019

Publishing Company: SAGE Publications Ltd

City: London

Print ISBN: 9781446202999

Online ISBN: 9781473957633

DOI: https://dx.doi.org/10.4135/9781473957633

Print pages: 182-192

© 2013 SAGE Publications Ltd All Rights Reserved.

This PDF has been generated from SAGE Research Methods. Please note that the pagination of the online version will vary from the pagination of the print book.

Reliability, Generalisation and Reflexivity: Identifying Validity and Trustworthiness

This chapter examines, reliability, generalisation and reflexivity as well as validity and trustworthiness. Chapter 12 also outlines a number of mechanisms relating to phenomenological or qualitative methodological approaches, which include worthiness, credibility, transferability and fairness. Reliability involves replication and asks how far another researcher could repeat the research undertaken at a given time. Replication is important for positivist and post-positivist approaches to research. A positivist and post-positivist study will usually use numerical data obtained through surveys from a particular sample population. Obviously, there are difficulties regarding replication but in such a study it is easier to repeat the experiment than it is to undertake observations on two or more occasions and come to the same understandings or interpretations reached during the previous studies. Indeed, reliability is a less important criterion for phenomenologists who follow criterion and procedures relating to trustworthiness and authenticity.

It is difficult to ensure high levels of both reliability and validity because if one is to accurately identify what is actually occurring in specific situations, it is necessary to go beyond the survey and involve oneself in the context of the research. Furthermore, it is more straightforward to generalise from a sample population to the population as a whole than it is to generalise from situation x to all situations. Consequently, it may be posited that generalisation is easier or more straightforward within a positivist or post-positivist research project. Within a positivist research project it is normally assumed that generalisation from sample to population should involve an intrinsic part of the research project. However, for a couple of reasons this does not always follow; on occasion one may be interested in specific cases alone because this has social significance in itself and/or it is important to understand that some research does not concentrate on generalisation but is more interested in theoretical inference. Reflexivity involves examining different conceptualisations of self when collecting and analysing data. This chapter assesses each of these areas in more detail then the next chapter relates these to methods of data collection in terms of surveys, interviews, focus groups and observations.

Reliability

For positivism, reliability is concerned with the extent that an experiment can be repeated or how far a given measurement will provide the same results on different occasions. Experimentation should reflect stability and ensure that any investigation of an individual or group at a given point in time can be repeated in exactly the same manner at another point in time. However, this is difficult to ascertain because one is never certain whether intervening factors during the two periods of time have changed the phenomenon and affected reliability. Given this problem it is useful to employ ‘equivalence validity’ and compare situations on separate occasions so as to determine whether different measurements of the same phenomenon correlate with each other. Phenomenological positions regarding reliability are concerned with whether observations made in an earlier research project can be observed in different or later projects; that is, projects in the future. Reliability

SAGE

2013 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods

Page 2 of 11 An Introduction to the Philosophy of Methodology

may be sought through categorisation and a synthesis of a positivistic and phenomenological position. Grounded theory attempts this through coding procedures, but this approach does have its shortfalls. For instance, even though an objectivist position is pursued through procedures and coding these can become extremely complex. Also the categories and procedures may alienate the reader from the research. Diagrams and conceptual maps can again complicate matters and obscure experience. Grounded theory may also make analysis opaque. Conversely, coding and categorising help to preserve images of experience as well as sharpen and direct questions.

Definition Box: Reliability

•

Stability: determines whether the measure is stable over time which provides confidence that the measure for a sample is consistent.

•

Internal reliability: consistency of indicators that involves the scale or index.

•

Inter-observer consistency: insurance that subjective judgements or the recordings and categorisations of data are consistent.

Reliability is extremely difficult for phenomenological studies as the ability to repeat research projects’ programmes is difficult to realise when individual situations in relation to multiple interpretation underpin the research process. Reliability is more easily realised when a structured, positivistic approach to the research programme is prioritised. In such a context, theory involves prediction so necessitates reliability through specified criteria and requires some form of hypothesis testing. Indeed, from a positivist perspective, validation requires a similar grounding to this as does the generalisation that follows in terms of laws that are immutable or remain until falsified. However, if we re-assess these positions from a phenomenological perspective different criteria emerge. For example:

credibility, validity and reliability in action research are measured by the willingness of local stakeholders to act on the results of the action research, thereby risking their welfare on the ‘validity’ of their ideas and the degree to which these outcomes meet their expectations. (Greenwood and Levin, 2000: 96)

The positivist position provides an image of a scientist in a lab with the work outlined in organised reports regarding concepts, evidence and procedures. Conversely, the phenomenological position identifies the image of the writer or storyteller balancing theoretical interpretation with aesthetics. With a phenomenological, critical theory or/and postmodern approach the reader is provided with an interpretation of the stories uncovered during the research.

Generalisation

SAGE

2013 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods

Page 3 of 11 An Introduction to the Philosophy of Methodology

As noted above, it is more straightforward to generalise using a positivist approach than it is when undertaking a phenomenological-based research project. The main point of a positivist investigation involves identifying relationships between samples and the general population, this is not the case for phenomenological, constructivist or participatory studies.

A generalisation of qualitative … studies is often called into question or regarded as infeasible, something which has weaknesses compared to quantitative set-ups … Only a statistical study that can establish the probability of the findings have not emerged by chance is … justified to make generalisations. (Alvesson and Skoldberg, 1999: 21)

However, this means that we must accept that a study of ‘surface regularities’ will provide identical patterns time after time. If it is accepted that non-observable phenomenon impact on the forms patterns take, such a position becomes untenable. Fundamentally, for a positivist research project generalisation involves the probability that patterns observed in a sample population can be extrapolated to a wider population from which the sample population is taken. Whereas a phenomenological project will be more concerned with generalisation from one setting to another; the extent that theoretical frameworks developed in one setting can be applied to other situations.

Difficulties for the social sciences have involved criteria for natural sciences being imposed upon it; generalisation is one such criteria. Positivism and post-positivism are based on empiricism, which as discussed in Chapter 3, was perceived as the correct and only way of undertaking scientific studies; a position that not only distorted social science but painted a false picture of how the natural sciences themselves actually worked. Given the necessities determined by natural science and quantitative perspectives, generalisation of phenomenological studies is often thought unrealistic. If simplified surface regularities are the benchmark then it is probable that such can be generalised from sample population to total population. It is probable that the findings are reliable and do not come about by some fluke or chance. However, this assumption depends on one’s interpretation of theory in relation to epistemological and ontological positions as well as what is considered to incorporate generalisations. There is a debate regarding the level natural science preconditions should be imposed on the social sciences, especially when underpinned by different paradigms of inquiry in terms of critical theory, constructivism and participatory.

Strauss and Corbin (1998) argued that generalisation from sample to population encompassed only one type of generalisation and a study may generalise from situation to situation. However, how may one generalise from stories? An answer is mainly through empathy and understandings that provide the basis for an acknowledgement of socialisation and the fact that we are human beings investigating human beings. Generalisation may be achieved through assessing how individuals feel in different situations and how they may act in certain circumstances. Consequently, given the dislocated nature of reliability and validity (see below) within the phenomenological approach a number of interpretations and subsequent generalisations may be forwarded in relation to a specific study. There must be connections between the researcher and researched as well as the intended, and in some contexts, unintended audience.

SAGE

2013 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods

Page 4 of 11 An Introduction to the Philosophy of Methodology

Reflexivity

The core of reflection incorporates reflexivity and involves how we are constructed in a social construct while at the same time acting as constructing agents. Without a constructed constructing self within a construction all is meaningless; through interpretation meaning is constructed. Construction requires something to be constructed, the researcher as a constructing subject and an object or a community that constructs the researcher. Reflexivity involves acknowledging the constructive elements without giving precedence to any part of the process; construction requires a continuum of interaction, a form of symbolic interaction or Hegelian recognition. Consequently, the relationship between selves and others provides the foundations for social constructivism. However, a concentration on reflexivity can conjure criticisms regarding ‘narcissistic self-centredness’, ‘self-absorption’ and ‘self-reflective isolationism’. One way of dealing with these criticisms involves researchers recognising themselves as elements of the wider social and political context and that we ourselves are caught up with and intrinsically linked to these contexts; others consider that the link between context and self should be negated. ‘The very idea of reflexivity … is the … ability to break away from a frame of reference and to look at what it is not capable of saying’ (Alvesson and Skoldberg, 2009: 270).

Reflexivity involves thinking critically about different conceptualisations of self; in a research project, self as researcher and respondent and wider self as learner, manager, parent, sibling and so on. Reflexivity involves a critical subjectivity in terms of coming to know self through the research process. For example, a doctorate is not simply about the completed thesis, it is also about the changes involved in students as they become active researchers; this is achieved through reflexivity undertaken during studies and the thinking required for the research process and successful completion of the thesis. In other words, the research project is as much about changes to the individual as the impact the research may have on the wider environment. ‘Self’ is brought to the research situation while at the same time self is developed through the research process. Indeed, types of self can be broken down into three categories: selves based around the role of researcher; selves formed through socio-historic existence; and selves determined by the situation or research environment (Reinharz, 1997). Each self requires interrogation regarding relationships with and formation of the research process. ‘We must question our selves … regarding how … binaries and paradoxes shape not only … identities called forth in the field … but also our interactions with respondents, in who we become to them in the process of becoming to ourselves’ (Guba and Lincoln, 2000: 183–4). Furthermore, through reflexivity researchers are able to locate themselves in the research; that is identify their socio-historical location and become aware that they carry a historical perspective of the situation or problem under analysis. Findings are co-created with researcher and researched involved in a saturation of the study with these juxtapositions of selves, situation and subject used to enhance understanding and enrich data interpretation. As well as observing participants the researcher is observing different selves and building interpretations of selves and these selves’ interpretations of data into the final outcome of the research. How the researcher fits and is involved with the research is not simply recognised but becomes an element of the research; personal investment in the research becomes part of the analysis. However, reflexivity has limited validity as it asks the reader to take interpretations at face value as an authentic attempt to explore selves

SAGE

2013 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods

Page 5 of 11 An Introduction to the Philosophy of Methodology

and be truthful and conscientious about the narrative accounts provided.

A reflexive attitude involves intensive scrutiny regarding how something is known and/or understood. Reflexivity incorporates an iterative and interactive dialogue about past experience in relation to present perspectives and future possibilities. A reflexive position is not simply reporting facts or truths but providing interpretations of data and issues raised in the field. ‘The outcome of reflexive social science is reflexive knowledge: statements that provide insight on the workings of the social world and insight on how that knowledge came into existence’ (Hertz, 1997: viii). Object and subject are brought back into the same space. Reflexivity may be perceived as the route to ‘radical consciousness of self in facing the political dimensions of fieldwork and constructing knowledge … Reflexivity becomes a continuing mode of self-analysis and political awareness’ (Hertz, 1997: viii). Reflexivity permeates the whole research process through continually challenging the researcher to be aware of ideology, culture and politics of the situation and self. Researchers should be aware of their own self, interests and social standing as these will direct and influence the research process. The impact of the researcher on the situation and individual being investigated needs to become part of the analysis. Previous experience should be assessed in relation to the research process. Reflexivity requires awareness of self that can only be partial, however it is important that this be made clear in the research text. The influence of previous work on individual perspectives should be openly acknowledged (Charmaz, 2000). Such awareness may be realised through memo-writing as this provides a discourse or conversation between the researcher and the data.

Reflexivity involves the double hermeneutical idea concerning the interpretation of the interpreting subject; this can be intensified through employing further levels of interpretation but the core of reflexivity incorporates reflection of interpretation and the self analysis of the person undertaking interpretation. Reflexivity provides ways of seeing or different lenses that act introspectively on data interpretations. Theory and abstraction are part of reflexivity and interaction between theoretical analysis and practical situations a necessary element of the process. Reflexivity is a two-way process of reflection or a primary concept reflecting on other secondary levels; fundamentally levels of interpretation are reflected in each other. Levels include involvement with data in terms of observations, focus groups and interviews, interpretation of materials through producing meanings, critique of interpretation through theoretical perspectives such as power, politics and ideological positions, reflection regarding the production of texts regarding selection of voices, claims of authority and self within the levels of interpretation. There is an incremental shift through levels of interpretation:

•

direct involvement with the data and low level interpretation of this data;

•

identification of relationships between theory and practice (praxis) or theoretical interpretation;

•

relationships between self data and theory or reflective interpretation.

The ideal situation incorporates theory being re-moulded by the data; norms, accepted values and personal perspectives should be challenged and reconstructed. Obviously, the extent of the interpretations will be curtailed by access to theory, consequently some comprehension of theoretical frameworks is required. A reciprocal relationship is needed between theoretical frameworks, the individual researcher and the data. The theories and individual researchers order the data and deal with anomalies and uncertainties in the early

SAGE

2013 SAGE Publications, Ltd. All Rights Reserved.

SAGE Research Methods

Page 6 of 11 An Introduction to the Philosophy of Methodology

stages of the study.

Important aspects of interpretation include:

•

creativity;

•

depth of theoretical knowledge;

•

breadth of theoretical knowledge;

•

variety of theoretical knowledge.

If the researcher has dedicated their career to a particular area or the student concentrated on a specific theory, then the research will be restricted to this area and reflexivity reduced. One may become emotionally attached to a theory and attempt to prove or confirm this. Consequently, knowledge of at least two theories will assist multiple interpretations and reflexivity. Controlled theories enable possible interpretations, the juxtaposition and synthesis of theories in relation to data delivers and enhances creativity. Fundamentally, wide reading is required as the sound theoretical knowledge engenders diverse interpretation and reflective capability.

Validity and Trustworthiness

For positivism, validity involves the extent to which measurement is accurate and what is supposed to be measured is actually being measured; how far can we be sure that a test measures the phenomenon we expect it too. Whereas, for phenomenological approaches the main emphasis regarding validity relates to whether access to knowledge and meaning has been realised. Questions relating to validity that require attention when evaluating the findings of research include: