PSYCHOLOGY RESEARCH PROJECT B

PSY3RPB: PSYCHOLOGY RESEARCH PROJECT B •Tutorial 4:

Between groups analysis Correlation and Regression analysis

Coordinator: Dr Melanie Murphy m.murphy@latrobe.edu.au

Assistant Coordinator: Aly Alipan a.alipan@latrobe.edu.au

1

Go to menti.com (you can use your phone)

Menti Code:

2

THE PROCESS • We will follow the steps below in tutorials over the first 5 weeks of semester in order

to make decisions about our approach, analyse our data and construct our report.

• Steps: ü Do our research to decide what we need to define and measure – this will also

include deciding on demographic characteristics of students as different students could have different experiences depending on their background.

ü State our aims in relation to these factors ü Select the most valid and reliable tools to measure these constructs. ü Construct and distribute a survey (* the SEG has directed that there will be no

contact with participants due to COVID-19 restrictions). ü Extract survey results from program and clean the data ready for analysis ü Finalise the variables we are going to use in our analysis ü Select and conduct appropriate analyses that address our aims 8. Create tables and graphs for analyses to help the SEG understand our statistical

outcomes 9. Summarise our findings 10. Make recommendations based on our findings for the SEG.

To assess the utility of the report in terms of rationale, methods, analysis and recommendations, the SEG as developed a set of Key Performance Indicators (KPI) against which they will benchmark your report.

3

PROJECT TIMELINE Task Class Time for Task Status

• Outline the rationale and approach for the study

Tutorial 1 Complete

• Decide on, and justify which constructs best represent the areas of interest outlined in the brief o (Nominal definitions for the study)

• Decide on a series of measures that help characterize the constructs of interest o (Operationalise the constructs)

• Define important demographic information needed to characterize the sample.

• Collect data. Before Tutorial 2 Complete • Clean dataset.

Tutorial 2 (Supplementary Activities

SGLA Module A) Complete

• Present characteristics of the sample. • Reduce variables into subscales

o Confirm the presence of categories within measures.

• Illustrate where there might be differences in the pattern of responses between groups in the sample.

Tutorial 3 (Supplementary Activities

SGLA Module B)

In progress

• Illustrate whether relationships between measures exist in the dataset, and potentially whether these could be used to predict certain outcomes.

Tutorial 4 (Supplementary Activities

SGLA Module B)

In progress

• Provide a summary statement and recommendations for the SEG Tutorial 5

• Submit Report to SEG LMS Turnitin 11.59pm Friday Aug 29th

4

WHAT RESOURCES DO WE HAVE? •The report brief and marking rubric •A dataset •DAI Assignment Resources Guide •A template for the report •Tutorial notes (and recordings) and SLGA activities

• Associated readings •The skills you’ve developed in three years of studying

psychology •The internet •LMS Discussion forum •Q&A session recordings (Sessions 1 and 2)

• Session 3 next week Tuesday 12pm • look for announcement on the LMS prior to this

5

DEVELOPING YOUR REPORT

Title page • Use the template

• Have a clear title • Change header • Remove highlighting

Abstract required? No

6

 

 

16/8/20

2

REPORT STRUCTURE: OVERVIEW

• Four sections 1. Report Context and Background

• Introduction • Report Aims

2. Method • Participants • Materials • Procedure • Data Analysis

3. Results I. General descriptive information II. Present analyses of between groups III. Present analyses of relationships and

prediction • can swap order – whatever makes

sense given the questions asked and the outcomes

4. Discussion 1. General Summary of Findings 2. Methodological Considerations 3. Report Recommendations

7

REPORT CONTEXT AND BACKGROUND

•Provides context of the report for the reader. • It can be brief because the SEG generally

knows what the report is going to deal with, but you’ll need to provide specific details regarding your focus and the constructs you are examining.

KPIs (AKA DAI Rubric)

0 1 2 3 4 5 6 7 8 9 10Each criterion is out of 10 and contributes 10% towards the total mark for this assignment.

Provides evidence- based introduction and rationale for study. (CT, IR)

No introduction or rationale is provided

Introduction included, but definitions and/or references m issing or problematic. Rationale unclear.

Introduction with some references and definitions that explains rationale and aims of the study provided.

Referenced introduction, w ith appropriate definitions that explains rationale and aims of the study provided with some issues.

Clear, succinct and referenced introduction, w ith appropriate definitions that explains rationale and aims of the study provided.

8

REPORT CONTEXT AND BACKGROUND

• Purpose of the study/report. • Specific questions can be mentioned as a

means of setting up a framework for the results and discussion sections.

• Provide a very brief summary of the literature that relates to the questions you will be asking.

• Relative to a “real lab report” this section will be much shorter than what you are used to.

• Writing concisely is very difficult but very necessary – this introduction will set the scene for the rest of the report.

9

REPORT AIMS

• The report aims and predictions should logically follow on from the background information. • Aims may be dot pointed to aid clarity, and a statement

of expected findings (hypothesis) for each aim should be included if possible. This does not have to be directional.

• Consider having a dot point aim the is clearly associated with an analysis in the results section (this will aid the logic and flow of your report).

10

• Include appropriate subsections • Participants • Materials, • Procedure, • Data Analysis

• These do not have to be as detailed as normally expected but should provide a brief description of the participants, the nature of the scales and how they were administered.

• Make sure to include a Data analysis section so that the audience knows what was done with the data prior to the analyses being run.

METHOD

Details methodology including description of participants, materials and procedure. (CT, CPS)

No details of methodology provided

Some methods included, but disorganized and missing major sections or components of information.

Some detail for all sections of methods included (participants, materials, procedure), but m issing key information in sections.

Methods including sample characteristics, scale description (with relevant statistics) and procedure explained with some issues.

All aspects of method including sample characteristics, scale description (with relevant statistics) and procedure explained.

KPIs (AKA DAI Rubric)

0 1 2 3 4 5 6 7 8 9 10Each criterion is out of 10 and contributes 10% towards the total mark for this assignment.

Data treatment and data reduction explained, allowing for an understanding of how variables used for analysis were derived. (QL, CPS)

Process of data treatment is not explained

Unclear how data was treated to produce variables used in the subsequent analyses.

Some explanation of data treatment and data reduction included, but key information is m issing or contains major errors.

Data treatment (including m issing cases, reverse scoring, normality testing, outliers) and data reduction process (factor analysis and subscale calculation) explained with some issues.

All steps of data treatment (including m issing cases, reverse scoring, normality testing, outliers) and data reduction process (factor analysis and subscale calculation) clearly explained.

11

METHOD Participants • Describe participants in the sample

• This will include information regarding the demographic characteristics you are interested in

Materials • Likely a short sentence or two generally introducing the

nature of measures (i.e. scales with likert response ranges) and survey software platform.

• Introduce materials table (if you wish to include it) and fill in table with your scales

• Only the scales you have included in your analysis need to be included.

• Don’t forget the reliability stats you calculated for the scales.

• This will be one or two short sentences

12

 

 

16/8/20

3

METHOD Procedure • Describe how survey was administered and who to.

Explain that the survey was extracted out of Qualtrics and what software it was imported into for analysis. • This will be one or two short sentences

Data Analysis • Data screening for problems and missing cases • Screening of outliers and how they were dealt with

(eg., trimming or winsorizing). • How was the PCA conducted and what did it show?

• One PCA needs to be presented (or both if you are including the DASS and BFI)

• How were the scales derived? • Checking of assumptions relevant to analyses. • Level of significance for analyses

13

FAQ •How much information do we need to include

about data cleaning? •As much as you need to show that you did this

as well as you could AND as little as you can get away with given the word count

•Think about what your readers really need to know to understand your report.

• i.e. No need to mention specific case #’s.

14

FAQ •How many tables of data do we need to

include from the PCA? • If you want, you can include the Pattern

Matrix in an Appendix. • This is not necessary though

• (not part of the rubric) • If you do include it in the Appendix, you must

refer to the Appendix in the body of the report so the reader knows it is there.

15

•Do we need to include all the output generated in an appendix?

•NO, As with a lab report (and the Project Report), only include in the report information that is necessary to answer your research questions.

• Clear and accurate reporting of the findings that is supplemented with relevant visual summaries is important

• Graphs and tables as appropriate

FAQ

16

• Report the results of between-groups analyses (t- tests and ANOVAs) and correlations, regressions, and in APA format

• Remember that you can’t possibly include EVERYTHING, so you have to choose what your report will focus on. • For each test, include what question the analysis

intended to address. • Explain relevant assumption testing and if changes

were made following assumption testing. •Supplement the findings with relevant visual

summaries • Graphs and tables as appropriate

RESULTS

17

RESULTS •When deciding which analyses to include,

remember the overarching brief is to understand student wellbeing and how it can be supported.

•Structure your results around the aims detailed in the introduction by following the same sequence of information

• In marking we are looking for quality of logic, not quantity of analyses.

•Marks are associated with correct selection of analysis technique, interpretation, and where possible a synthesis of the results.

18

 

 

16/8/20

4

RESULTS KPIs (AKA DAI Rubric)

0 1 2 3 4 5 6 7 8 9 10E a c h c r ite r io n is o u t o f 1 0 a n d c o n tr ib u te s 1 0 % to w a rd s th e to ta l

m a rk fo r th is a ss ig n m e n t.

Descriptive statistics to help characterise sample and outcome measures provided. (QL, CPS)

Descriptive statistics not included Inaccurate and/or inconsistent presentation of descriptive statistics, though some relevance to overall study present.

Descriptive statistics provided but missing some key information and/or reason for inclusion of some aspects unclear from introduction.

Descriptive statistics that characterise the nature of the sample and give an indication of mean responses to measures highlighted in the introduction provided with some issues.

All important descriptive statistics that characterise the nature of the sample and give an indication of mean responses to key measures highlighted in the introduction provided.

Between groups analysis examining important comparisons in data based on themes outlined in the introduction provided. (QL, CPS)

Between groups analysis not included

Between group analyses included, but major issues with statistical information and interpretation of outcomes. No link to study questions.

Between group analyses included, but relevance in terms of study questions unclear and/or issues with statistical information, follow analyses and interpretation of outcomes present.

Between group analyses addressing comparisons highlighted in the Introduction presented with relevant statistical information, post hoc analyses when appropriate, and interpretation of outcomes in terms of construct being examined with some issues.

Between group analyses addressing key comparisons highlighted in the Introduction presented with relevant statistical information, post hoc analyses when appropriate, and clear interpretation of outcomes in terms of construct being examined.

Analyses examining important relationships in data based on themes outlined in the introduction provided.

Analysis of relationships not included

Correlation and/or regression analyses exploring relationships included, but major issues with statistical information and interpretation of outcomes. No link to study questions.

Correlation and/or regression analyses exploring relationships included, but relevance in terms of study questions unclear and/or issues with statistical information, follow analyses and interpretation of outcomes present.

Correlation and/or regression analyses exploring relationships highlighted in the Introduction presented with most statistical information and interpretation of outcomes in terms of constructs being examined with some issues.

Correlation and/or regression analyses exploring key relationships highlighted in the Introduction presented with relevant statistical information and clear interpretation of outcomes in terms of constructs being examined.

(QL, CPS)

Data presented in the form of tables and graphs to summarise data and aid interpretation used. (QL, CPS)

No data presented as tables or graphs

Graphs and/or tables included, but titles not descriptive, format problematic, relevance unclear and not described in text.

Graphs and/or tables with titles used, but relevance not always clear and/or in-text description not provided.

Graphs and/or tables with titles used to facilitate communication of important patterns in the data and statistical outcomes with some issues.

Graphs and/or tables with appropriate titles used appropriately and effectively to facilitate communication of important patterns in the data and statistical outcomes.

• Include at least • One independent groups t test 👈 Comparing groups • One ANOVA of some form 👈 Comparing groups • One correlation 👈 Looking for relationships • One regression of some form 👈 Predicting relationships

19

RESULTS • Comparing groups

• Use the demographic information as IVs • Nominal/categorical variables

• Rows 3-13 in Variable view of spreadsheet

• Where are the differences in wellbeing (for example) between groups in our sample? • Are some groups more vulnerable to reporting poor

wellbeing? • Does this suggest targeted assistance?

• You do not need to look at all demographics, select and justify which you think are most important and present analyses associated with these

20

RESULTS • Looking for relationships

• Variables with continuous measures • The scale totals or subscale totals in the dataset – or COVID questions

• (Not demographics) • Are there any patterns in the relationships between our measures?

• What is associated with higher or lower wellbeing? • More general question than regression, more exploratory and could help justify

subsequent regression • Did the group comparisons suggest looking at correlations within subgroups?

• Predicting relationships • Again, variables with continuous measures

• The scale totals or subscale totals in the dataset – or COVID questions • (Not demographics)

• Can we use any of the measures to predict wellbeing? • Could this be used to focus intervention or support services?

• You do not need to look at all of the scales, select and justify which you think are most important and present analyses associated with these

21

RESULTS • Don’t mix up presenting and describing

the results with discussing and interpreting them. • Do use graphs and tables creatively to

present information most effectively. • Remember, this section is where many

of your marks will come from, and set the scene for the discussion section where many of your other marks will come from.

22

REPORTING OUTCOMES IN APA FORMAT

• The examples on the following slides are from difference studies that use similar analyses. • For tests where no example is provided

please seek out the information you need • eg., Pallant, Field, APA style manual,

various online sources, etc.

23

REPORTING ANOVA

24

 

 

16/8/20

5

REPORTING ANOVA Example One-way ANOVA: • An Analysis of Variance (ANOVA) was performed to

examine differences in Risk of heart disease between Age categories (under 40, 40-60, over 60). The differences in Risk of heart disease between Age categories were statistically significant, partial η2 = .87, F(2, 21) = 72.82, p < .001.

• Reporting of outcomes must be accompanied by means either in sentence format or in a table or graph.

• A description of the results in plain language with reference to means and effect sizes is a good idea.

• You would then describe the patterns in the post hoc tests to explain how the groups differ • No statistics are required to explain Post Hoc test results.

• **Please note: Age is NOT a categorical variable in our study and should not be used as an IV unless you have

25

REPORTING ANOVA Example Two-way ANOVA:

26

REPORTING ANOVA Example Two-way ANOVA: • A 2 x 3 between-subjects ANOVA was performed

to compare Risk of heart disease between sex (Females, Males) across Age categories (under 40, 40-60, over 60).

• A significant main effect was not found for Sex, F(1, 18) = 0.01, p = .92, partial η2 = .001.

• A significant effect was found for Age category, F(2, 18) = 64.69, p < .01, partial η2 = .88.

• A significant interaction was not found between Sex and Age category, F(2, 18) = 0.32, p = .73, partial η2 = .04. • Reporting of outcomes must be accompanied by means

either in sentence format or in a graph. • A description of the results in plain language with reference to

means and effect sizes is a good idea

27

REPORTING CORRELATION

Example:

• Pearson’s r correlation analysis was conducted to examine the relationship between hours spent studying and performance in school. A significant, moderate positive correlation was found, r = .48, p =.0, meaning that performance in school increased with increased hours spent studying.

28

REPORTING REGRESSION

29

REPORTING REGRESSION

Example:

• Multiple regression analysis was conducted using Intelligence, Hours spent studying, Barriers to performance, and School facilities to predict Performance in school. Results showed that the four variables significantly predicted Performance in school, adjusted R2 = .59, F (4, 20) = 9.66, p < .001, with a large effect (Cohen, 1988). Of the four variables, Intelligence made the largest unique contribution to predicting Performance in school, Sr2 = .43, p < .001.

30

 

 

16/8/20

6

• There are variations on how to phrase the reporting of outcomes. • The examples provided are one variation, others are

provided in Pallant and Field in the relevant chapters.

• Always be conscious of phrasing. • For example,

• ‘relationship’= correlation • ‘prediction’ = regression • ‘differences’ ’comparison’ = t test or ANOVA

• Each term has a highly specific meaning that corresponds to a different kind of analyses.

REPORTING OUTCOMES

31

REPORTING OUTCOMES

• Try not to get too caught up in stats speak • Make sure you include plain language

summaries for your reader • If you can’t decipher what your stats

mean when you write them out, your reader won’t be able to either

32

FAQ •Should I only report significant findings?

• You should report the numbers that address your research questions, and NOT just not the ones that are statistically significant.

• In some cases you may not expect to find differences, so if you test for these and don’t find any, that’s a perfectly legitimate outcome.

33

FAQ •Do the tables and graphs need to be APA

formatted?

•Yes • They need to be clear and readable, with titles

etc as per normal • You can include colour

• But, don’t be too adventurous as the report needs to look professional. Too much colour looks ridiculous and can make a report impossible to read once it’s been printed on a black and white printer.

34

• The report should finish with content that clearly summarises the findings of the analyses, drawing attention to important points that the SEG can act upon. • Don’t forget to think about limitations and offer clear

recommendations for action.

DISCUSSION AND RECOMMENDATIONS

KPIs (AKA DAI Rubric)

0 1 2 3 4 5 6 7 8 9 10

Each criterion is out of 10 and contributes 10% towards the total mark for this assignment.

Summary ties up main points from analysis w ith links to themes outline in introduction and consideration of methodology used. (CT, CPS)

Summary absent

Summary included, but relevance to analysis, study questions and literature not readily apparent.

Summary of findings included, but little reference to study questions provided. Examination of methodological issues has a number or issues.

A summary of key statistical findings relevant to aims detailed in introduction, as well as brief acknowledgment of methodological considerations is provided with some issues.

A clear, accurate summary of key statistical findings relevant to aims detailed in introduction, as well as brief acknowledgment of methodological considerations is provided.

Recommendations based on outcomes of data analysis that link back to rationale and research evidence provided to SEG. (CT, IR, CPS)

Recommendations absent

Recommendations provided, but with no reference to data analysis, introduction or research evidence.

Some recommendations made, but link between these and data analysis and/or introduction unclear. Use of supporting research not included.

Recommendations with links to findings of data analysis, themes outlined in the introduction and evidence-based research are provided with some issues.

Recommendations with clear links to findings of data analysis, themes outlined in the introduction and associated with evidence-based research are provided.

35

• Similar to a formal discussion but keep the applied context in mind

• Start (ALWAYS) by restating the overarching purpose and your specific research questions

• Summarise the findings and draw attention to/emphasise the most important findings.

• Use references to interpret / support your recommendations

• Do not be shy about interpreting findings in the context of ed psych, org psych, or any other relevant literature.

• Make sure you include clear and sensible recommendations for action (i.e., what should LTU do as a direct results of the findings outlined in your report?)

DISCUSSION AND RECOMMENDATIONS

36

 

 

16/8/20

7

• Map out the questions you are asking in plain terms and match the statistical analysis you need to use with each question • Address the most important/informative

questions first, and then if appropriate, move on to more complex questions • This will help structure the Results section

GENERAL POINTS

37

GENERAL POINTS • This report does NOT have to be strictly in APA

format, but APA 7th edition formatting must be used for reporting statistics, tables, figures, and references. • Any sources of information used should be cited

appropriately. • The word count is limited to 1000 words

• excluding figures, tables, reference list, appendices.

KPIs (AKA DAI Rubric)

0 1 2 3 4 5 6 7 8 9 10 Each criterion is out of 10 and contributes 10% towards the total mark for this assignment.

Writing and organization is clear and professional, aiding effective communication of investigation and outcomes.

Major issues with writing and no clear organisation

Many problems with expression, spelling and grammar. Organization and integration between sections of the assignment problematic.

Expression, spelling and grammatical has a number of issues. Organization is present but issues with integration between sections of the assignment.

Good expression, with few spelling or grammatical issues. Organization is good, with minor issues in integration between sections of the assignment.

Clear expression, with no spelling or grammatical issues. Logical organization with clear integration and flow between sections of the assignment.

38

APA FORMATTING • APA rules you need to retain:

• Spacing and format of statistical notation eg., F(2, 192) = 41.23, p = .06, Partial η2 = .08. • Use two decimal places – only use three when the first two

are 00 • Tables and Figures have Titles. • In-text citation and reference list format. • Double spacing and size 12 font.

• APA rules to bend (if desired): • Heading style and format. • Dot points can be used (eg., aims in intro and/or

recommendations in discussion.

39

FINAL POINTS FOR DAI • Q and A Session 3

• 12pm Tuesday Aug 25th

• Tutor consultations available • Time and links on LMS

• LMS Discussion Board • Final questions must be posted by 12pm Thursday Aug

27th

• Submit to LMS • 11.59pm Friday 28th August

40

PREPARING FOR NEXT WEEK: DDP GROUPS

• Careers Week in Psychology • Week 9

• DDP Day • Thursday 24th September

• Poster Presentations • 11.15 – 1.15

• Live Careers Q and A Seminar • Industry insight for Psychology students • 1.30 – 2.45

• Extended Project Meetings • All groups • 3pm – 5pm

• DDP Presentations • Groups of 5-6 students formed within tutorial class • More info on requirements and topic selection next week