Reliability/Precision of Psychological Tests

Chapter 5: Reliability/Precision of Psychological Tests

Activity 5A – 6 points

 

Complete this task as a Word document and submit via Turnitin

 

RUBRIC: 1.5 point for each (a, b, c, d) correctly answered with proper explanations. (.5 point deduction for each incorrect answer or explanation; maximum 1.5 pt. deduction for each letter)

 

There are various methods we can use to estimate the reliability/precision of a psychological test. We can estimate reliability using the test–retest method, the alternate-forms method, and/or the internal-consistency method. We can also estimate scorer reliability. There is not always just one correct way to estimate reliability/precision. Depending on the test, we might use one or more methods to gather as much evidence of reliability/precision as possible. Review four testing scenarios and state whether each method for estimating reliability/precision can be used or not (YES or NO). Then, briefly state why you chose that/those method(s). Be sure to cite why some methods are not appropriate. Finally, state whether scorer reliability is important for that test (YES or NO).

** Note that the split-half method is not included as an option. Though split-half is easy to calculate, the other methods for calculating internal consistency are more precise. **

 

a) An instructor has designed a comprehensive math exam for students entering community college. The exam contains multiple-choice questions that measure a student’s ability to read formulas, carry out math calculations, and solve word problems. To be sure students sitting next to each other will not have the same test, the instructor will be making two different tests to measure the same constructs. The tests are heterogenous. Since students may score higher on the second administration purely because they have taken the test one time already, when gathering evidence of reliability/precision, the instructor can give the exam only once. However, the instructor needs to know how reliable the test scores are.

i) Test-Retest Method:

ii) Alternate-Forms Method:

iii) Internal Consistency Method – KR-20:

iv) Internal Consistency Method – coefficient alpha:

v) Should we examine scorer reliability?

 

b) An HR professional wants to assess employee attitudes about quality of work life. She wants to be sure that her self-designed instrument is reliable. Her instrument contains 20 statements that employees will rate from 1 to 5. The HR professional has designed the instrument to be homogeneous.

i) Test-Retest Method:

ii) Alternate-Forms Method:

iii) Internal Consistency Method – KR-20:

iv) Internal Consistency Method – coefficient alpha:

v) Should we examine scorer reliability?

 

c) An Industrial and Organizational (I–O) psychology practitioner designed two parallel promotion tests for firefighters. Both tests required two fire chiefs to observe firefighters completing job-related activities (e.g., use and maintenance of safety equipment). Each firefighter was required to take two parallel tests rated by the same fire chiefs.

i) Test-Retest Method:

ii) Alternate-Forms Method:

iii) Internal Consistency Method – KR-20:

iv) Internal Consistency Method – coefficient alpha:

v) Should we examine scorer reliability?

 

d) A test developer is constructing a measure of critical thinking. The instrument consists of a number of anagrams and riddles—problems for which answers are not readily apparent until solved. The test score depends on the percentage of questions the test taker solved correctly.

i) Test-Retest Method:

ii) Alternate-Forms Method:

iii) Internal Consistency Method – KR-20:

iv) Internal Consistency Method – coefficient alpha:

v) Should we examine scorer reliability?