STATISTICS

15 May STATISTICS

Your Paper was well written, however; I need you to follow the following Analysis Guidance for Intervention Data. I will give you a passing grade when you submit with these by the 26th of April at 1pm EST

This document is designed to provide a summary of the key steps for analysing intervention data. The main analysis is conducted using the general linear model function in SPSS. This document does not cover how to clean data for analysis. (Data for the PARS module has already been cleaned so students do not have to undertake this part of the analysis.) This document is written with the PARS assignment in mind, so please refer to statistical texts for details on how to check assumptions, and a broader overview of how to interpret the output of intervention analyses in SPSS.

Preparing Scales

When using scales, ensure you compute scale reliabilities (Cronbachs Alpha using the function Analyse>Scale>Reliability analysis). Make sure scales are recoded as required by the specific scale you’re using. If you find poor reliability, that might indicate scale items have not been coded as required (e.g. a scale item may need reverse coding). If scale reliability is poor, then you may want to exclude it from the analysis, remove a low-loading item, or report why you think the reliability is poor and justify why you decided to include it. Scale items should be aggregated or averaged using the compute variable function in SPSS (Transform>Compute variable) for the main analysis, as directed by the scale authors. (For the PARS assignment, scale reliability statistics can be reported in the appendix.)

Calculating Means and Standard Deviations

It is useful at this stage to calculate the means and standard deviations for the data using the function Analyse>Descriptive Statistics. For intervention data comparing more than one condition, you need to isolate a condition in the dataset before generating the means and standard deviations for that condition. The analyses testing the effect of an intervention with individuals in different conditions (i.e. between-subject) are essentially testing whether there is a significant difference in the means of groups in different conditions. The means for the different conditions show whether levels are increasing or decreasing, and this is useful for interpreting the results of the analysis.

Isolate study conditions using the function Data>Select cases, and use the function ‘If condition satisfied’. In the PARS data, use cohort as the variable in the rule (i.e. ‘Cohort = 1’ for the intervention group, or ‘Cohort = 2’ for the control group). When you have either of these rules applied, SPSS will only run the analysis on the cases selected by that rule. For example, if the rule applied is ‘Cohort = 1’ only cases with the value 1 in the cohort variable will be included in the analysis.

Bivariate Correlations

As part the analysis, you need to run bivariate correlations. Use the function Analyse>Correlate>Bivariate. (For the PARS assignment, this can be reported in the appendix). Significant bivariate correlations highlight relationships between the variables. This in itself may be of interest, but it also identifies variables to include as control variables in the main analysis. Demographic variables that correlate significantly with outcome variables can be included as covariates to partition out variance they account for.

Testing the Overall Effects of an Intervention

The first step in the main analysis is to conduct an overall MANOVA for all study items (and control variables) using the function Analyse>General Linear Model>Repeated Measures. (In the PARS assignment, the data have two conditions, with two main outcome variables with measures taken at four time points. Note that other study designs will require a different analysis, i.e. within-subjects designs vs. between-subjects designs, or repeated measures vs. cross-sectional.) What this analysis is telling you is whether there is an overall effect from the intervention across all time points and all outcomes.

The within-subject factor is the condition that all participants experienced. In the PARS assignment data this is time, with four levels to represent the four different time points in the data. Each outcome measure should be added. The between-subject factor is the condition that differed in participant groups. In the PARS assignment data this is cohort. To control for any demographic variables that correlated significantly with the outcomes, add them as covariates.

As well as reporting the F statistic and the p-value, it is becoming increasingly important to report and discuss the effect size when interpreting data. This is especially true when looking at the impact of interventions, as a significant result does not necessarily indicate a meaningful effect size. SPSS does not automatically calculate this for you, so ensure you select estimates of effect size from the options tab. You can also generate a plot in the output. Click on the plots tab and have your between-subject factor as separate lines (to represent the changes in different conditions) and your within-subject factor (in the PARS case this represents the different time points) as the horizontal axis. Don’t forget to click add when doing this, otherwise your plot will not be generated. When looking at the output, the main points of note can be found in the between-subject effects shown in the multivariate tests table. If there is no overall significant effect, you would have little justification to continue with the analysis past this point, as you have no evidence that the intervention has had an overall effect.

Testing the Effects of the Intervention on Specific Outcomes

The second step is to conduct separate MANOVAs for each of the study outcome variables. This essentially replicates the procedure above, but specific to each outcome. What this analysis is telling you is whether there is an overall effect from the intervention in this outcome across all time points and conditions. Use the function Analyse>General Linear Model>Repeated Measures, but include only one of the outcome variables as your measure. Include as covariates any demographic variables that significantly correlated with that outcome. When looking at the output, the main points of note are shown in the multivariate tests table in the factor*cohort section.

Testing for Simple Effects

For outcomes that have a significant overall effect, you can then run simple effects tests to isolate where significant changes occur. You can run simple effects tests for between-subjects effects (comparing different groups at the same point in time), and within-subjects effects (for each group separately comparing changes over time). Don’t forget to use a Bonferroni corrected p-value when interpreting these statistics. This is because you are running multiple analyses, and to avoid inflated error you need to use a more stringent test of significance for the results.

To explore between-subjects effects, use the function Analyse>General Linear Model>Univariate to run a one-way repeated-measures ANOVA. (We discussed in class whether to use t-tests or ANOVA, as they do a similar analysis. Both are acceptable, but ANOVA is more robust.) Enter your time point as the DV and your between-subject factor (cohort) as the fixed factor. If you are using a time point after time 1, then enter the previous time point as a covariate to isolate the effects that occur after that point in time. For example, if you are looking at effects at T3, then add T2 as a covariate to isolate in the analysis changes between T2 and T3 only. The line of interest in the output is that of your fixed factor (in this case cohort).

To explore within-subject effects, you need to isolate a condition in the dataset before running the analysis. Do this using the Data>Select cases function, and use the ‘If condition satisfied’ function. In the PARS data, use cohort as the variable in the rule (i.e. ‘Cohort = 1’ for the intervention group, or ‘Cohort = 2’ for the control group).

To run the within-subject simple effects tests, use the function Analyse>General Linear Model>Repeated measures. The within-subject factor is time. The PARS data has four levels, to represent the four time points. As with the prior MANOVAs, the between-subject factor in the PARS assignment data is cohort. In this part of the analysis we will use the Contrasts function. Once you have clicked on this, you will notice a section titled ‘Change Contrast’. Click on ‘contrast’ and you will have a list of options. Each compares time points in a different way. The two for this analysis are ‘simple’ and ‘repeated’. For the simple contrasts, you can then swap between ‘first’ and ‘last’. Don’t forget to click ‘change’ when swapping between any of the contrast options. A summary of these contrasts is below:

· Simple(First) compares the first time point (T1) to all the following time points

· Simple(Last) compares the last time point (T4) to all the preceding time points

· Repeated compares the time points sequentially (i.e. T1 to T2, T2 to T3, and T3 to T4).

The line of interest in the output is factor in the tests of within-subjects contrasts table.

***************************************************************************************************************

End of feedback

Institute of Management Studies (IMS)

Postgraduate Assignment Information Sheet

 

Module title:

Professional and   Applied Research Skills – Assignment 2

 

Assignment information (e.g., background info, assignment   question, further advice):

This assignment   involves an evaluation of an intervention run in an organisation.  In this assignment you are asked to critically   evaluate an intervention using a strategic and evidence-based approach to   evaluation.

One   of the specialist skills of occupational psychologists is that of evaluating   the effectiveness of interventions or processes that they, or others, have delivered.   Psychologists rely on evaluation work to inform many of the choices they make   in their practice. You are expected to demonstrate that you can conduct a robust   evaluation of practice. Evaluation can be carried out during a project, at   the end of a project and some considerable time after it has finished (i.e.   as a long-term follow-up). In this   assignment, data are provided for you to analyse, based on a real-world   intervention.

In   this assignment, you will analyse data measuring the impact of a coaching   intervention designed to improve employee performance and wellbeing. The   audience is the organisations senior team, who are highly competent and   intelligent, but have little psychology-specific knowledge. Therefore, think   about how to communicate effectively with this audience.

You will need to include some psychological   theory in your report, and reference appropriately in APA format, but think   carefully about balancing this with the other information required in the   report.

About the Intervention

Data were collected   from an ACT-based coaching intervention for senior managers in the   organisation. The study design was a randomised controlled trial (RCT)   design, which compared ACT-informed coaching to a waitlist control group.   Surveys were distributed by email. Survey 1 was sent by email one week before   the first coaching session (Time 1) and provided a baseline measure. Survey 2   was sent one week before the second coaching session (five weeks after the   baseline measure; Time 2). Survey 3 was sent one week before the third   coaching session (nine weeks after the baseline measure; Time 3). Survey 4   was sent four weeks after the third coaching session (14 weeks after the   baseline measure; Time 4). An online research randomiser tool   (www.randomizer.org) was used to randomly allocate coachees to either the   experimental group or the waitlist control group.

Of the 127 participants   in the sample, 90 (71%) were female, and 94 (74%) participants described   their ethnicity as white. There was an age range of 26 to 60 years (mean age   of 41.47 years). On average, coachees had worked in their job for between 3-4   years (mean of 3.27 years). Of the 127 coachees, 116 (91%) were educated to   degree level or above.

Coachees’ had three   face-to-face 90-minute coaching sessions delivered over a period of nine   weeks. The aim of the first session was to (a) introduce the coachee to ACT-informed   coaching, and the strategies ACT approaches employ, (b) identify core work   values for the coachee, (c) identify goals for the coachee to work on during   the coaching programme, and (d) introduce the coachee to mindfulness   practice. The three main exercises used in the session were a values   clarification exercise, a goal-setting process, and a short mindfulness   practice. Participants were asked to practice mindfulness between coaching   sessions: Two mindfulness practices were discussed in the coaching session,   and then emailed to participants following the session.

The aim of the second   coaching session was to (a) review progress towards the coachee’s goals, (b)   review the use of mindfulness since the previous session, and (c) introduce   defusion and acceptance as ways of moving past psychological blocks to   progress. There were three main exercises used in the session: A mindfulness   exercise focused on defusing the coachee from their thoughts, feelings and   physical sensations; a defusion and acceptance exercise focused on moving   beyond psychological barriers to coachees goal progress; and a metaphor   designed to increase the coachees willingness to experience difficult   thoughts and emotions in relation to their goals. Participants were asked to   use mindfulness practices between sessions, and practice using the defusion,   acceptance, and willingness exercises if they noticed psychological blocks to   progress. Copies of these exercises were emailed to participants after the   coaching session.

The aim of the final   session was to (a) review progress towards the coachee’s goals, (b) introduce   the observing perspective (i.e. self-as-context perspective), and (c)   encourage coachees to keep working towards their goals and increase their   values consistent actions. There were two main exercises used in the session:   A mindfulness exercise focusing on the observing perspective; and a values   consistency exercise, which asked coachees to reflect on what they are doing   day-to-day to live their values, where the inconsistencies with their values   are, and what else they might be able to do to bring their values to life.   Copies of these exercises were emailed to participants after the coaching   session. Following completion of the final survey, participants were emailed   a handout with information to help participants move forward with their gaols   and values following the coaching programme. This included (a) a short   mindfulness practice; (b) a life values clarification exercise; (c) tips and   suggestions for facilitating values-based living; (d) a resilience enhancing   exercise; and (e) resources for learning more about ACT.

Performance was   measured using the individual performance items from the Model of Positive   Work Role Behaviours (Griffin et al., 2007). This scale is based on a theoretically   derived model of performance, focusing on an individual’s proficiency,   adaptivity, and proactivity at work. All items are rated based on how often   participants have carried out the behaviour over the past month on a scale   ranging from 1 (very little) to 5 (a great deal). Responses were collected as   a self-report from participants. The scale consists of nine items.   Participants were asked to rate how often they had carried out each behaviour   over the past month on a scale ranging from 1 (“very little”) to 5 (a “great   deal”). An example self-report item for this scale is “Completed your core   tasks well using the standard procedures”. Higher scores indicate higher   performance.

General mental health   was measured using the General Health Questionnaire (GHQ-12; Goldberg, 1992).   This scale is a measure of current mental health; specifically the inability   to carry out normal functions, and the appearance of new and distressing   experiences. It consists of 12 items. An example item from this scale is “have   you recently felt capable of making decisions about things?” Items are scored   0 (more so than usual) to 3 (much less than usual). Scores have been coded so   higher scores indicate increased general mental health.

The Report

You   need to submit a report evaluating the coaching intervention. Specifically   within this report you need to:

1. Specify the purpose and aim of the        intervention being evaluated.
2. Identify and critically evaluate the        quality of the data available for the evaluation.
3. Identify the practical constraints        impacting on the evaluation process.
4. Select appropriate evaluation processes        and techniques.
5. Critically evaluate the quality of the        evaluation methodology (e.g. reliability, validity and sensitivity of        the data, sample size, timing of evaluations, rigour and appropriateness        of analysis etc.).
6. Evaluate outcomes systematically against        objectives using appropriate tools and analytical techniques.
7. Reach appropriate evidence-based        conclusions about the outcome of the intervention (e.g. what worked, in        what ways, for whom).
8. Reflect on the outcomes of the        evaluation and the implications of these for future evidence-based and        evidence-informed practice.
9. Provide advice and guidance to        stakeholders based on the results of the evaluation.

The report should   follow this structure below:

· Title

· Executive   summary

· Contents   page

· Introduction

· Main   body

· Conclusions   and recommendations

· References

· Appendices

 

Key/suggested references:

Barends, E., & Rousseau, D. M., (2018). Evidence-Based Management: How to Use Evidence to Make Better   Organisational Decisions. Kogan Page Limited; New York, NY.

Books on statistical analysis, such as Field (2017)   will be a helpful guide to running statistical analysis for this assignment.

Field, A. (2009). Discovering statistics using   SPSS. London: SAGE Publications.

 

Support for this assignment:

Support   for this assignment will be provided throughout the course, with the   following specific support sessions:

· 3x workshops on   evidence-based practice and intervention evaluation, including two case   studies.

· 1x workshop on progressing   your report, with the opportunity for feedback from tutors on your progress   so far. This formative workshop is aimed at helping students to progress   their ideas, and get helpful feedback before submitting their final reports

 

Word limit:

1,500   words.

This includes the main body of text, in text   citations [e.g. (Eisen et al., 2008)], quotations and footnotes. However, the   word limit excludes your title page, tables, figures, illustrations,   reference list and appendices.

 

Referencing style 

APA

 

Submission date and time:

11th   March at noon

 

 

Marking scheme:

Marking will be in accordance with the general IMS   postgraduate marking rubric:

1. Answer. (Does the   work answer the question or address the issue?)

2. Structure. (Is   the general structure of the work coherent?)

3. Flow. (Does each   statement follow sensibly from its predecessor?)

4. Argument. (Is   there a convincing quality of argument in the work?)

5. Evidence. (Are   claims supported by relevant evidence from the literature?)

P.S. The other file can be opened by IBM SPSS Statistics Software