Test Evaluation Paper Assignment Instructions Overview The purpose of the Test Evaluation Paper Assignment is to evaluate the psychometric properties of a psychological assessment of your choice. Fo

Test Evaluation Paper Assignment Instructions Overview The purpose of the Test Evaluation Paper Assignment is to evaluate the psychometric properties of a psychological assessment of your choice. Fo
16PF Adolescent Personality Questionnaire Acronym: APQ Authors: Schuerger, J. M. Publication Date: 2001. Publisher Information: PSI Services LLC, 611 N. Brand Blvd., 10th Floor, Glendale, CA, 91203, [email protected], psionline.com Source: B. S. Plake, J. C. Impara, & R. A. Spies (Eds.), The fifteenth mental measurements yearbook. 2003. Reviewers: Reynolds, William M.; Whiston, Susan C. Purpose: Designed to ‘measure normal personality of adolescents, problem-solving abilities, and preferred work activities,’ and to identify problems in areas known to be problematic to adolescents. Test Category: Personality. Population: Ages 11–22. Scores: 21 normal personality scales: Primary Personality Factor Scales (Warmth, Reasoning, Emotional Stability, Dominance, Liveliness, Rule-Consciousness, Social Boldness, Sensitivity, Vigilance, Abstractedness, Privateness, Apprehension, Openness to Change, Self-Reliance, Perfectionism, Tension), Global Factor Scales (Extraversion, Anxiety, Tough-Mindedness, Independence, Self-Control); plus a ranking of Work Activity Preferences (Manual, Scientific, Artistic, Helping, Sales/Management, and Procedural), Personal Discomfort (Discouragement, Worry, Poor Body Image, Overall Discomfort), ‘Getting in Trouble’ (Anger or Aggression, Problems with Authority, Alcohol or Drugs, Overall Trouble), Context (Home or School), Coping/Managing Difficulty, Impression Management, Missing Responses, Central Responses, Predicted Grade Point Average Administration: Group or individual. Time: (54–65) minutes (untimed). Price Data: Available from publisher. Comments: Computerized scoring and interpretive reports available via test publisher’s online and software platforms (APQ Guidance Report and APQ Psychological Report); optional Life’s Difficulties section provides an opportunity for the youth to indicate particular problems in areas known to be problematic for adolescents, making the APQ appropriate for screening and for introducing sensitive topics in a counseling setting Cross References: For reviews by William M. Reynolds and by Susan C. Whiston and Jennifer C. Bouwkamp, see 15:230. Published Test Description: 16PF Adolescent Personality Questionnaire. Purpose: Designed to “measure normal personality of adolescents, problem-solving abilities, and preferred work activities,” and to identify problems in areas known to be problematic to adolescents. Population: Ages 11-22. Publication Date: 2001. Acronym: APQ. Scores: 21 normal personality scales: Primary Personality Factor Scales (Warmth, Reasoning, Emotional Stability, Dominance, Liveliness, Rule-Consciousness, Social Boldness, Sensitivity, Vigilance, Abstractedness, Privateness, Apprehension, Openness to Change, Self-Reliance, Perfectionism, Tension), Global Factor Scales (Extraversion, Anxiety, Tough-Mindedness, Independence, Self-Control); plus a ranking of Work Activity Preferences (Manual, Scientific, Artistic, Helping, Sales/Management, and Procedural), Personal Discomfort (Discouragement, Worry, Poor Body Image, Overall Discomfort), “Getting in Trouble” (Anger or Aggression, Problems with Authority, Alcohol or Drugs, Overall Trouble), Context (Home or School), Coping/Managing Difficulty, Impression Management, Missing Responses, Central Responses, Predicted Grade Point Average; 10 life’s difficulties scales: Discouragement, Worry, Poor Body Image, Overall Discomfort, Anger or Aggression, Problems with Authority, Alcohol or Drugs, Overall Trouble, Problems at Home, Problems at School. Administration: Group or individual. Price Data: Available from publisher. Time: (65) minutes (untimed). Comments: Computerized scoring and interpretive reports available (APQ Guidance Report and APQ Psychological Report); optional Life’s Difficulties section provides an opportunity for the youth to indicate particular problems in areas known to be problematic for adolescents, making the APQ appropriate for screening and for introducing sensitive topics in a counseling setting. Author: J. M. Schuerger. Publisher: Institute for Personality and Ability Testing, Inc. (IPAT). Language: English Accession Number: test.2502 Translate Full Text: Choose Language 16PF Adolescent Personality Questionnaire Full Text Review of the 16PF Adolescent Personality Questionnaire by WILLIAM M. REYNOLDS, Professor, Department of Psychology, Humboldt State University, Arcata, CA: DESCRIPTION. The 16PF Adolescent Personality Questionnaire (APQ) is described by its author as appropriate for adolescents ages 11 or 12 to 22 years. There are 147 items on the 16 normal personality (Primary Factor) scales, 15 items on the Work Activities scales, and 43 items on the Life’s Difficulties scales. The manual states that the APQ has a reading grade level of around 5.5, although a check indicates that some of the APQ items are written at an eighth and ninth grade reading level. The APQ evaluates most of the same personality domains as the 14-factor High School Personality Questionnaire (HSPQ; Cattell, Cattell, & Johns, 1984) except for Withdrawal and Demandingness. To make the APQ consistent with the 16PF Questionnaire, which is designed for adults, the author included the four factors of: Vigilance, Abstractedness, Privateness, and Openness to Change that are found on the 16PF Questionnaire. These four along with factors (scales) of Warmth, Reasoning, Emotional Stability, Dominance, Liveliness, Rule-Consciousness, Social Boldness, Sensitivity, Apprehension, Self-Reliance, Perfectionism, and Tension constitute the 16 APQ Primary Factors. Weighted scores on these scales are combined to form five Global Factors of Extraversion, Anxiety, Tough-Mindedness, Independence, and Self-Control. The normal personality scales have between 8 and 12 items each, with 10 of the factors consisting of 8 items each. The normal personality scale of Reasoning is actually a brief measure of general cognitive ability rather than personality, although personality (concrete versus abstract) is inferred from this scale. In addition to the scales of normal personality, further changes included the addition of the Work Activities component that is reported to measure aspects of Holland’s (1973) six career types in the form of: Manual, Scientific, Artistic, Helping, Sales/Management, and Procedural scales. The Life’s Difficulties section includes three Personal Discomfort scales (Discouragement, Worry, and Poor Body Image, as well as a total Overall Discomfort scale), three Getting Into Trouble scales (Anger or Aggression, Problems with Authority, and Alcohol or Drugs, as well as a total Overall Trouble score), two Context Scales (Home, School), and a seven-item Coping scale. There are also several validity indicators, including Impression Management, Central Responses (selection of the middle option), and Missing Responses. Sten scores (standard ten) that are based on a scale of 1 to 10 with a mean of 5.5 and a standard deviation of 2 points are provided for the Global and Primary Factors, with endorsement counts for the Work Activities scales, and percentile ranks for the Life’s Difficulties scales. The test manual suggests that, on average, 15-year-olds can complete the APQ in less than 90 minutes. The items are presented in a 21-page test booklet that can be a bit daunting for some adolescents. Item format varies, with most of the normal personality and Life’s Difficulties items using a true/?/false format, with a number of these items including additional descriptors to clarify the meaning of items. The 12 Reasoning items use a three-alternative, multiple-choice format, and include verbal analogies, opposites, and other problems typically found on verbal aptitude scales. The Work Activities scale consists of six statements arranged across 15 items using a forced-choice format, with each of the six statements representing one of Holland’s career orientation dimensions. Scores on these scales are the number of times the dimension was endorsed. The manual describes the various personality factors measured by the APQ and the characteristics of individuals with high and low scores on each factor. Many of the descriptions read more like adults than characteristics of adolescent personality. The Life’s Difficulties scales are briefly described, with examples of items provided to illustrate content. Following these descriptions, the manual provides a substantial section on interpretation of APQ scales. The APQ is scored by computer, either by sending the answer sheet into the publisher for scoring and generating an interpretive report, or scoring only using a scoring program diskette. This latter program generates a list of scores for each of the scales. There are two score reports available from the publisher: the APQ Guidance Report, which consists of scores for the 16 Personality Factors and the Work Activities scales, and the APQ Psychological Report, which also includes the Life’s Difficulties scales. The computer scoring provides two forms of the report, one for the adolescent and another for the clinician. The adolescent form, which provides scores on the Global Factors and Work Activities, is written at comprehension level beyond that of many adolescents. The clinician report provides a summary of the Testing Indices, Global and Primary Factors (sten scores and graphic illustration of scores with low and high score descriptions for each scale), scores on the Work Activities scales, and percentile ranks for scores on the Life’s Difficulties scales and the Coping scale. An interpretive summary across the APQ domains is also provided. DEVELOPMENT. The development of the APQ involved several iterations of item development, rewriting, and field testing. The author provides a brief description of the number of items on these forms and approximate sample sizes. The manual indicates that some of this data may be obtained from the author. Approximately 16% of the items on the normal personality scales were from the HSPQ, 28% from the adult 16PF Fifth Edition, 19% from the author of the APQ, and the remainder written by students and IPAT staff. The final standardization version of the APQ included 199 items. Scale selection was based on a factor analysis (analysis type not specified) with a Promax rotation of item parcels (two or three related items combined) that resulted in a viable 16-factor solution in which two of the anticipated factors, Emotional Stability and Apprehension, loaded on the same factor. The author notes that a subsequent analysis using a 15-factor solution “was very satisfactory” and it was decided to split the Emotional Stability and Apprehension factor into two factors to make it consistent with the 16PF Questionnaire. PSYCHOMETRIC INFORMATION. Standardization. The standardization sample varied for different components of the APQ. For the 16 normal personality factors (Primary Factors) and Work Activities scales, the standardization sample consisted of 1,460 adolescents ages 11 to 22 years. A subsample of 410 adolescents also completed the Life’s Difficulties scales, although roughly half of these adolescents (n = 213) were from clinical settings (outpatients, educational referrals, and drug treatment). Norms for the APQ Primary Factors, Work Activities, and Life’s Difficulties scales were based on the nonclinical sample of adolescents. In this manner it can be estimated that approximately 1,247 adolescents were the basis for the Primary Factors and Work Activities norms, with the Life’s Difficulties scale norms based on approximately 197 adolescents. The majority of the 1,460 adolescents in the development sample (88%) were between the ages of 15 and 18 years, with 47% male and 53% female, with 81% Caucasian, 6% African American, 5% Hispanic, and small percentages of various other ethnicities. The majority of this sample (67%) was from the north central United States. Sample characteristics (age, gender, etc.) of the specific normative groups (1,247 and 197) are not provided. Reliability. Reliability data are presented in the form of internal consistency reliability (coefficient alpha) for all scales, with a test-retest reliability study reported for the Work Activities scales with a sample of 14 “late adolescents and young adults.” The manual claims to present equivalent form reliability with a 3-week interval for the normal personality scales as shown by correlations between the APQ Primary Factors and corresponding scales on the 16PF and HSPQ in a sample of 107 “late adolescents.” Internal consistency reliability of the 16 Primary Factors ranged from .64 to .83, with a median reliability of .72 and only one reliability coefficient above .76. The internal consistency reliability coefficients for the Work Activities scales range from .46 to .75 with a median reliability coefficient of .62. However, as the author correctly notes, these coefficients are problematic because the items on these scales are ipsative. The test-retest reliabilities (several weeks interval between administrations) of the Work Activities scales were similar, ranging from .53 to .74, with a median of .62, suggesting low reliability for these scales. Internal consistency reliabilities of the Life’s Difficulties scales are low for most scales, with coefficients ranging from .52 to .65 for the three Personal Discomfort scales, .66 to .77 for the Getting into Trouble scales, .60 and .42 for the Home and School scales, respectively and .74 for the Coping scale. Reliability is not reported for the eight-item Impression Management Scale. Validity. Validity data are presented in a somewhat haphazard manner, with a mix of results from previous development versions of the APQ interspersed among validation data for the current version. Given that the primary use of the APQ is the assessment of normal personality, one would expect the majority of validity evidence for this new measure to focus on the 16 Primary Factor scales. Unfortunately, there are few studies/data to support validity. The first demonstration of validity of the 16 Primary Factors was a correlational study of these factors and those on the adult 16 PF in what appears to be a college sample (average age of 19 years, 70% women, n = 107). Because the APQ includes items from the 16PF Questionnaire, it is not surprising to find moderate correlations between .53 and .81. In a study of 30 adolescents of unknown age and other characteristics who also completed the HSPQ (also a source for APQ items), correlations of .17 to .74 were found between same factor scales, with half of these coefficients below .50. Additional evidence for validity of the Primary Factors is suggested in the manual by correlations between these scales and adolescents’ grade point average (GPA) in four studies. Two of these studies were done with an early research version of the APQ that differed in items from the current version. Across the four studies, relatively low correlations were reported between Primary Factors and GPA, providing limited support for the validity of a measure of normal personality. The exception to this is the Reasoning scale, which consists of scholastic ability type items. In the two studies that used the final form of the APQ, the correlations between the Reasoning scale and self-reported GPA or counselors’ ratings of GPA were .41 and .31, respectively, which are relatively low. Because the items on the Reasoning scale are based on academic ability, unless corrected or normed by age (or grade) these correlations are difficult to interpret. The results of the GPA studies and the 16PF Questionnaire and HSPQ are the extent of the validity evidence for the normal personality scales. Several studies are reported for the validity of the Work Activities Scales, some conducted prior to the addition of these scales to the APQ, using either the normative sample or a large sample of “pre-medical college students.” These studies included various other vocational measures. In most of these studies, low correlations between Work Activities scales and related measures were reported. Validity data for the Life’s Difficulties are presented with a sample of 44 older adolescents and young adults (college students?) who were administered the MMPI and the Personality Assessment Screener (Morey, 1997). Unfortunately, this study was conducted with a development version of the Life’s Difficulties scales, which had items and scales that are different from the current version. COMMENTARY. The 16PF APQ is one of few measures that purports to measure personality in adolescents as opposed to psychopathology. The APQ manual leaves out a great deal of information, particularly as it relates to the psychometric characteristics of this measure. The standardization sample, although adequate for adolescents ages 15 to 18, is limited for those below and above this age range. The normative sample is inadequate in size and description for the Life’s Difficulties Scales. Without a description of the age and gender composition of the approximately 197 who constitute the norm group, it is difficult to interpret the percentile ranks derived from these scales. The manual is missing some useful information. There are no data pertaining to the means and standard deviations of the APQ scales for the standardization sample. This would be useful when interpreting the results of the Life’s Difficulties scales for which only percentile ranks are provided as scores. Noticeably missing from the reliability section is information on the standard error of measurement for the various scales. There are numerous references to other data in the author’s possession or outcomes with other samples that are supportive of reliability or validity, but are not described in the manual. The lack of complete data and tables is problematic, as is the suggestion of going to the author for more information. The reliability of the APQ normal personality scales is low to moderate, but higher than previously reported for the HSPQ. Reliabilities for the other APQ scales are mostly low. The validity data for the APQ with adolescents below age 18 are limited. There is no systematic presentation of forms of validity (construct, criterion, convergent/discriminant, etc.) for the normal personality scales, and validity data for the other domains (e.g. Work, Life’s Difficulties) are also limited, with many of the reported correlations providing little support for validity. SUMMARY. The 16PF APQ draws on a rich historical base, namely the High School Personality Questionnaire and the 16PF Questionnaire, with approximately 44% of its items from these two measures. In addition to the 16 normal personality factors, the APQ provides enhanced content and user value by including two brief additional measures, one of occupational preference and the other a brief screen of adolescent psychopathology. This is a commendable combination of assessment domains, particularly for adolescents. However, because of low reliability and limited evidence for validity of these latter two components, care should be taken in their use and interpretation. The relatively modest levels of reliability reported may not preclude the use of the APQ in some counseling settings where clinical decisions are typically not based upon such results. The 16PF APQ is one of the few measures designed to assess “personality” of adolescents. A similar measure, the Millon Adolescent Personality Inventory (Millon, 1982), has its own set of problems (Reynolds & Sattler, 2001) and cannot be recommended over the 16PF APQ. In keeping consistent with the previous perspectives of personality that were operationalized by the HSPQ and 16PF Questionnaire, the APQ may have kept itself from a more natural evolution to a more contemporary perspective of personality, such as that reflected by the big five. It may be that further research will lend greater support for the use of this measure. REVIEWER’S REFERENCES Cattell, R. B., Cattell, M. D., & Johns, E. (1984). Manual and norms for the High School Personality Questionnaire. Champaign, IL: Institute for Personality and Ability Testing. Holland, J. L. (1973). Making vocational choices: A theory of careers. Englewood Cliffs, NJ: Prentice-Hall. Morey, L. C. (1997). Personality Assessment Screener. Odessa, FL: Psychological Assessment Resources. Reynolds, W. M., & Sattler, J. M. (2001). Assessment of behavioral, social and emotional competencies in children and adolescents. In J. M. Sattler (Ed.), Assessment of children: Behavioral and clinical applications, (4TH Ed., pp. 163-188) Costa Mesa, CA: Jerome Sattler, Pub. Review of the 16PF Adolescent Personality Questionnaire by SUSAN C. WHISTON, Professor of Counseling and Educational Psychology, and JENNIFER C. BOUWKAMP, Doctoral Student Counseling Psychology, Indiana University, Bloomington, IN: DESCRIPTION. The 16PF Adolescent Personality Questionnaire (APQ) is a 205-item instrument designed to measure aspects of normal personality in adolescents ages 12 to 22 years. This instrument was designed for use in situations where personality is relevant, such as educational adjustment, personal or social difficulty, and passage through developmental tasks. The APQ includes four sections: (a) Personal Style, (b) Problem Solving, (c) Work Activity Preferences, and (d) Life’s Difficulties (optional items concerning matters known to be difficult for adolescents). The first three sections provide valuable information to professionals who counsel adolescents related to personal and family issues and to those who assist adolescents in academic and career decision making. The APQ is also designed to indicate learning styles and provide information that can be used in developing Individualized Education Programs. When the Life’s Difficulties section is administered with the three other sections, the APQ is appropriate for screening and addressing sensitive topics in the counseling situation, although results are not intended for diagnosis. The APQ is easy to administer with all four sections contained in one booklet, and administration time is around 65 minutes for all 205 items. Students indicate their responses on an answer sheet where each item has two or three choices depending on the section. The Life’s Difficulties section is set off from the other three by a blank page indicating that the student is not to go on unless directed to do so by the test administrator. Both computerized scoring and hand scoring using a disk are available. The results using either method are organized into four areas: (a) Testing Indices, (b) Normal Personality and Ability, (c) Work Activities, and (d) Life’s Difficulties (if administered). Testing Indices includes Impression Management, Missing Responses, and Central Responses. Normal Personality and Abilities includes the Global Factors of Extraversion, Anxiety, Tough-Mindedness, Independence, and Self-Control, under which the Primary Factors of Warmth, Reasoning, Emotional Stability, Dominance, Liveliness, Rule-Consciousness, Social Boldness, Sensitivity, Vigilance, Abstractedness, Privateness, Apprehension, Openness to Change, Self-Reliance, Perfectionism, and Tension are discussed. The Work Activities provides information ranking the young person’s preferences for occupational types using John Holland’s (1973) typology of Manual, Scientific, Artistic, Helping, Sales/Management, and Procedural. Life’s Difficulties supplies information on Personal Discomfort (Discouragement, Worry, Poor Body Image, and Overall Discomfort), “Getting in Trouble” (Anger or Aggression, Problems with Authority, Alcohol or Drugs, and Overall Trouble), the Context in which difficulties are experienced (Home, and School), and Coping (Social Competence, Task Competence, Problem Solving, Utilizing One’s Social Network, Reason of Strong Values, and Attitude Change). Two computerized reports, with separate sections of feedback for the adolescent and for the professional, are also available from The Institute for Personality and Ability Testing (IPAT): (a) the APQ Guidance Report and (b) the APQ Psychological Report. Both reports provide scores on the normal personality factors and their related Global Factors, as well as the Work Activities scales, and some administrative indices. The Guidance Report does not include information on the Life’s Difficulties section and is designed for school counselors or qualified teachers in developing academic or therapeutic strategies. On the other hand, the Psychological Report has the same three sections and the results of the Life’s Difficulties items. This report is more appropriate for clinical applications and is based on items addressing sexual matters, violent feelings and actions, serious worry and despondency, and other similar thoughts and behaviors. DEVELOPMENT. The APQ is one measure in the family of tests originally authored by Raymond B. Cattell and colleagues beginning with the Sixteen Personality Factor Questionnaire (16PF; Cattell, Eber, & Tatsuoka, 1970) for adults. The APQ specifically evolved from the High School Personality Questionnaire (HSPQ; Cattell, Cattell, & Johns, 1984). Cattell used what is often called the domain sampling method, where one starts with a defined domain (e.g., personality) and seeks to identify the number and nature of a relatively few variables that capture the meaning of that domain. Cattell and colleagues would write items to represent the domain of interest and then use factor analysis to discern the number and nature of the variables that define the structure of the domain. Following this method throughout the research phases of the APQ, there were three sequential forms of the instrument: (a) a 240-item research version, (b) a 264-item research version, and (c) a 284-item standardization version. Before creation of the final version of the APQ, items in the Personality and Ability section had been through several revisions and extensive testing. In the final factor analysis, items were merged into “parcels” of two or three items each, according to item correlations and the manifest content of the items. For the Work Activities section, the items were written to reflect Holland’s six-trait model, and factor analyses were performed with a mix of adolescent and adult clients from various settings. The problem content areas for the Life’s Difficulties section were based on a review of common content scales among existing pathology instruments, as well as a review of the then current Diagnostic and Statistical Manual (American Psychiatric Association, 1987). Data for this section were collected with only 80 normal high school students from Florida and Cleveland, 37 adolescents referred to a mental health center in Illinois, and 29 adolescents referred to school psychologists for a variety of problems. The final review and revision of items on the Life’s Difficulties section was conducted by IPAT staff and practitioners familiar with adolescent psychology. TECHNICAL. There were a total of 1,460 young persons involved in the final standardization sample. Of these, 1,050 completed only the Normal Personality and Ability, and Work Activities sections, and another 410 also completed the Life’s Difficulties section. Of these 410 young people, 213 were in clinical settings. In age, roughly 6% were between 11 and 14; 88% between 15 and 18; and another 6% between 19 and 21. There was close to an even split on gender, with 47% male and 53% female. Concerning geographic representation, the preponderance of the sample (67%) was from the north-central section of the United States. Of the other participants, 14% were from the south-central U.S., 7% from the northeast, and 6% from both the southeast and west. No data pertaining to race or ethnicity were mentioned in the description of the standardization sample. Coefficient alphas for the Normal Personality and Ability section vary from a low of .64 (Openness to Change) to a high of .83 for (Social Boldness), a pattern that matches that of the 16PF Fifth Edition. The average over the 16 factors was .72, with the Global Factors having higher reliabilities than the Primary Factors. Test-retest reliabilities (interval of 1 week) ranged from a low of .44 for Reasoning to a high of .95 for Abstractedness. The Global Factors averaged .91. Stability coefficients for the Work Activities section over a period of 1 week averaged .79 over the six scales, whereas test-retest coefficients for the Life’s Difficulties section (interval of 1 week) averaged .77. Scale validity of the APQ Normal Personality and Ability section was examined through correlations with school achievement. Four sets of data were available to address the issue of personality and school grades with the updated instrument. In particular, the relationship between the Reasoning factor and grades, which one would expect to be high, had a wide range of correlation coefficients across four studies: .27, .77, .41, and .31 (all significant at the .05 level). Most validity data on the Work Activities section were from the research prior to its use as part of the APQ and were not discussed in the administration manual provided. The conclusion reached by the authors was that the Work Activities results are similar to those obtained from longer instruments and with similar external validity evidence. However, no compelling evidence was provided to support the use of the Work Activities section of the APQ over these other instruments. Much of the validation evidence for the Life’s Difficulties section involved correlating it with the scales of the Minnesota Multiphasic Personality Inventory (MMPI; Hathaway & McKinley, 1943). In our opinion, stronger evidence could have been provided by correlation of the APQ with the adolescent version of the MMPI (MMPI-A) or at least the more recent MMPI-2. Nevertheless, obtained coefficients were low with an average of .29. Additional evidence included correlations with three other external criteria resulting in coefficients of .41 with the Child Behavior Checklist (Achenbach, 1991), .27 with psychologists’ reports, and .23 with counselors’ reports. These comparatively low coefficients do not indicate a strong relationship between the problem behaviors assessed by the APQ and other measures of child and adolescent difficulties. COMMENTARY AND SUMMARY. Overall, the 16PF Adolescent Personality Questionnaire appears to be a decent measure for obtaining a “snapshot” look at the typical American adolescent. Furthermore, the instrument may provide the professional with insights into how to work successfully with specific individuals. However, there are some questions remaining about its factor structure as no strong argument or compelling explanation is provided for the selection and inclusion of the three distinct scales of Normal Personality and Ability, Work Activities, and Life’s Difficulties. Some of the reliability coefficients for the Primary Factors are low and indicate that a clinician should be cautious in interpreting these results. With regard to adolescent career exploration, one may consider using instruments that incorporate the more current and widely used version of Holland’s (1985, 1997) typology. Although the six types in the 1973 version are similar to the more current version, a significant amount of career information and resources use the more current version rather than the terms Holland used almost 30 years ago. Similarly, although the Life’s Difficulties section is well-intentioned as an indicator of adolescent troubles, much of the validation evidence seems somewhat outdated and needs to be more substantial before we could recommend extensive use. A strength of the APQ is that it purports to measure the “Big Five” factors of personality for adolescents; however, its limitation indicates the instrument should be used cautiously and with other clinical measures. REVIEWERS’ REFERENCES Achenbach, T. M. (1991). Manual for the Child Behavior Checklist/4-18 and 1991 profile. Burlington, VT: Department of Psychiatry, University of Vermont. American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders (3rd ed., rev.). Washington, DC: Author. Cattell, R. B., Cattell, M. D., & Johns, E. (1984). Manual and norms for the High School Personality Questionnaire. Champaign, IL: Institute for Personality and Ability Testing, Inc. Cattell, R. B., Eber, H. W., & Tatsuoka, M. M. (1970). Handbook for the Sixteen Personality Factor Questionnaire (16PF). Champaign, IL: Institute for Personality and Ability Testing, Inc. Hathaway, S. R., & McKinley, J. C. (1943). The Minnesota Multiphasic Personality Inventory. Minneapolis: University of Minnesota. Holland, J. L. (1973). Making vocational choices: A theory of careers. Englewood Cliffs, NJ: Prentice-Hall. Holland, J. L. (1985). Making vocational choices: A theory of vocational personalities and work environments (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall. Holland, J. L. (1997). Making vocational choices: A theory of vocational personalities and work environments (3rd ed.). Englewood Cliffs, NJ: Prentice-Hall.
Test Evaluation Paper Assignment Instructions Overview The purpose of the Test Evaluation Paper Assignment is to evaluate the psychometric properties of a psychological assessment of your choice. Fo
Personality Assessment Inventory–Adolescent Acronym: PAI-A Authors: Morey, Leslie C. Publication Date: 1990-2007. Publisher Information: Psychological Assessment Resources, Inc., 16204 N. Florida Avenue, Lutz, FL, 33549-8119, [email protected], www.parinc.com Source: R. A. Spies, J. F. Carlson, & K. F. Geisinger (Eds.), The eighteenth mental measurements yearbook. 2010. Reviewers: Kade, H. Dennis; Sandoval, Jonathan Purpose: An ‘objective test of personality designed to provide information on critical client variables in professional settings.’ Test Category: Personality. Population: Ages 12-18. Scores: 53: Somatic Complaints (Conversion, Somatization, Health Concerns, Total), Anxiety (Cognitive, Affective, Physiological, Total), Anxiety-Related Disorders (Obsessive-Compulsive, Phobias, Traumatic Stress, Total), Depression (Cognitive, Affective, Physiological, Total), Mania (Activity Level, Grandiosity, Irritability, Total), Paranoia (Hypervigilance, Persecution, Resentment, Total), Schizophrenia (Psychotic Experiences, Social Detachment, Thought Disorder, Total), Borderline Features (Affective Instability, Identity Problems, Negative Relationships, Self-Harm, Total), Antisocial Features (Antisocial behaviors, Egocentricity, Stimulus-Seeking, Total), Alcohol Problems, Drug Problems, Aggression (Aggressive Attitude, Verbal Aggression, Physical Aggression, Total), Suicidal Ideation, Stress, Nonsupport, Treatment Rejection, Dominance, Warmth, Inconsistency, Infrequency, Negative Impression, Positive Impression. Administration: Group. Time: 45(55) minutes. Price Data: 2021: $520 per introductory kit including professional manual (2007, 190 pages), 2 hardcover reusable item booklets, 2 administration folios, 25 hand-scored answer sheets, 25 profile forms-adolescent, and 25 critical items forms-adolescent; $123 per professional manual (print or digital); $59 per hardcover reusable item booklet; $70 per 10 softcover reusable item booklets; $93 per 25 hand-scored answer sheets; $65 per 25 profile forms-adolescent; $56 per 25 critical items forms-adolescent. Comments: Designed to complement its parent instrument, the Personality Assessment Inventory (18:93) Cross References: For reviews by H. Dennis Kade and Jonathan Sandoval, see 18:92. Special Editions: Spanish forms available. Published Test Description: Personality Assessment Inventory-Adolescent. Purpose: An “objective test of personality designed to provide information on critical client variables in professional settings.” Population: Ages 12-18. Publication Dates: 1990-2007. Acronym: PAI-A. Scores, 53: Somatic Complaints (Conversion, Somatization, Health Concerns, Total), Anxiety (Cognitive, Affective, Physiological, Total), Anxiety-Related Disorders (Obsessive-Compulsive, Phobias, Traumatic Stress, Total), Depression (Cognitive, Affective, Physiological, Total), Mania (Activity Level, Grandiosity, Irritability, Total), Paranoia (Hypervigilance, Persecution, Resentment, Total), Schizophrenia (Psychotic Experiences, Social Detachment, Thought Disorder, Total), Borderline Features (Affective Instability, Identity Problems, Negative Relationships, Self-Harm, Total), Antisocial Features (Antisocial Behaviors, Egocentricity, Stimulus-Seeking, Total), Alcohol Problems, Drug Problems, Aggression (Aggressive Attitude, Verbal Aggression, Physical Aggression, Total), Suicidal Ideation, Stress, Nonsupport, Treatment Rejection, Dominance, Warmth, Inconsistency, Infrequency, Negative Impression, Positive Impression. Administration: Group. Price Data, 2008: $295 per complete kit including professional manual (2007, 190 pages), 2 reusable item booklets, 2 administration folios, 25 hand-scored answer sheets, 25 profile forms-adolescent, 25 critical items forms-adolescent, and 1 professional report service answer sheet in a soft-sided attaché case; $32 per reusable item booklet; $34 per 10 soft cover item booklets; $48 per 25 hand-scored answer sheets; $30 per 25 profile forms-adolescent; $35 per 25 critical items forms-adolescent; $70 per professional manual. Time: 45(55) minutes. Comments: Designed to complement its parent instrument, the Personality Assessment Inventory. Authors: Leslie C. Morey. Publisher: Psychological Assessment Resources, Inc. Language: English Accession Number: test.3061 Translate Full Text: Choose Language Personality Assessment Inventory–Adolescent Full Text Review of the Personality Assessment Inventory-Adolescent by H. DENNIS KADE, Director of Developmental & Behavioral Health, Norfolk Department Of Public Health and Tidewater Child Development Clinic and Adjunct Assistant Professor of Psychology, Old Dominion University, Norfolk, VA: DESCRIPTION. The Personality Assessment Inventory-Adolescent (PAI-A) is a self-report personality questionnaire for clinical assessment at ages 12 to 18 years. It was developed as an extension of the adult Personality Assessment Inventory (PAI) for adolescents. The manual states that the PAI-A is not appropriate for comprehensive assessment of normal personality domains. Analysis of the test instructions and items shows reading comprehension at a 4.0 grade level is required. The Flesch Reading Ease score is 80.9, suggesting a fifth grade level. This is within the easy range, perhaps harder to read than a comic but easier than a teen magazine. The test manual indicates there are 264 items that take adolescents 30-45 minutes to complete and examiners 10-15 minutes to score, yielding 22 nonoverlapping scales. Each item is answered on a 4-point scale ranging from false/not at all true, to slightly true, mainly true, or very true. The written instructions for the adolescent are succinct, but contain no example item. The question booklet and answer sheet items are arranged in columns. Breaks in one do not correlate with breaks in the other, but the answer sheet layout facilitates scoring without keys. Alternating colors are used to help the adolescent visually separate the rows on the answer sheet. A critical item sheet facilitates inspection of 17 answers with space for notes from subsequent queries. With no age or gender differences, only one profile sheet is needed to convert raw scale and subscale scores to T-scores. Reference to normative tables in the manual is unnecessary. The examiner’s manual is well organized with a detailed table of contents that facilitates its use as a reference. Hand score and optical scan score answer sheets are available; the latter are also used for the publisher’s mail-in scoring service. Computer-based administration, scoring, and interpretive report software is available. Though its purchase price is more than the PAI-A test kit itself, additional per-use fees are charged only for on-screen administrations. Hand scoring is quite straightforward and a scoring example is included in the manual. Guidelines for interpretation included in the manual extend beyond single scales at different elevations to suggest profiles based on cluster analysis. Unfortunately, no example clinical cases are supplied that might illustrate integration of the interpretive guidelines. The manual states that PAI-A interpretation requires training in psychometric assessment and psychopathology. The four validity scales included in the PAI-A support decisions about the validity of an individual’s protocol. There are 10 pairs of items in the Inconsistency scale; completely random responding produced an average T-score of 82 on this scale and scores above 77T render the protocol invalid. The 7 items of the Infrequency scale were rarely endorsed in the normal and clinical samples; random responding produced an average T-score of 82, and scores above 78T indicate invalid protocols. The 8 items of the Negative Impression scale were infrequently endorsed in the clinical and particularly in the normal samples; random responding produces an average T-score of 77, and a sample instructed to malinger averaged 108T. Scores above 84T indicate invalid protocols. The 8 items of the Positive Impression scale were endorsed most frequently by a sample instructed to respond with positive impression enhancement (average score of 68T), least often by clinical cases, with the normal sample falling in between. Scores above 71T indicate invalid protocols. The manual states that protocols with more than 13 unanswered items are problematic and scales or subscales with more than 19% unanswered should not be interpreted. DEVELOPMENT. Both the structure and most of the items from the PAI were retained in the PAI-A. Thus, constructs relate to the diagnostic categories of mental disorder and the experience of symptoms and other factors relevant to treatment (e.g., Aggression, Suicidal Ideation, Stress, Nonsupport, Treatment Rejection) that occur across the life span. A few items had to be reworded to make them more age-appropriate (e.g., changing “work” to “school”), but new items were not introduced because the PAI-A was not intended to assess psychopathology unique to adolescents. Pilot testing used 275 clinical cases who were 65% male, 54% from forensic (juvenile detention) settings, 26.7% from inpatient mental health, and 22.8% from outpatient mental health. Internal consistency and mean interitem correlations within scales were used to eliminate 80 items. One item was omitted from scoring on the Infrequency scale because adolescents endorsed it too often. The PAI-A item pairs used to score the Inconsistency scale were selected empirically, rather than using the PAI items. TECHNICAL. The test manual provides information on the PAI-A’s empirical structure such as scale intercorrelations, exploratory factor analyses, confirmatory factor analyses of subscales, and cluster analyses. Normative data for the U.S. Census-matched standardization sample and (for comparison purposes) the clinical sample are included in appendices, as well as data on item means and standard deviations in both samples, scale and subscale correlations with the validation measures in the community sample, and frequency distributions of clinical scale codetypes across nine diagnoses plus the entire clinical sample. T-scores are based on a community sample of 707 students in junior and senior high school and college who were between the ages of 12 and 18 years from 21 states stratified by race/ethnicity and gender within age bands to match the 2003 U.S. Census. Analysis showed only two scales had more than 5% of their variance accounted for by ethnicity, gender, or age: higher scores for males on Antisocial Features and higher scores for non-Caucasians on Paranoia. These differences are about 5 T-Score points. Data were also gathered on 1,160 representative clinical cases from 78 sites: outpatient mental health (49.7%), juvenile/correctional (33.3%), inpatient mental health (12.7%) and other settings. The clinical sample data were used to generate a clinical “skyline” on the profile form that exceeds 98% or is 2 standard deviations above the mean. Evidence is provided that the most common diagnoses in treatment are represented, but there are fewer than 5% with an anxiety disorder. Standard error of measurement values for scales in the community sample average 4.68 T-score units (range of 3.3 to 5.9) and subscales average 4.94 (range of 3.42 to 6.42). Coefficient alpha internal consistency values are reported in community and clinical samples for scales and subscales. Internal consistency for scales (except Inconsistency and Infrequency) averaged .79 for the community standardization sample and .80 for the clinical sample. Except for Dominance in the clinical sample, all scale values meet or exceed the .70 traditionally considered acceptable for research use, but only Aggression in the community sample and Suicidal Ideation in the clinical sample reach the .90 standard for making decisions about individuals. Only 13 of the 31 subscales meet or exceed .70 in the community sample and 18 in the clinical sample; none reach the .90 standard. These results are not surprising given the brevity of the scales and subscales. The test author is to be commended for addressing the stability of profiles, though the only datum reported was a median inverse correlation of .84 for the relative position of scale scores within the profiles of test-retest cases. The latter 100 participants were reassessed to yield an average test-retest stability coefficient of .78 for the scales (except Inconsistency and Infrequency) after a mean of 18 days (range from 9 to 35 days). Six validity studies with clinical and community samples ranging from 77 to 1,160 are reported that provide a wealth of support for criterion-related validity. Only clinical diagnosis and symptom checklist were used in the largest sample, but other samples used multiple measures of personality and psychopathology: Minnesota Multiphasic Personality Inventory-Adolescent (MMPI-A), Adolescent Psychopathology Scales (APS), Personality Inventory for Youth (PIY), NEO Five-Factor Inventory (NEO-FFI), Symptom Assessment-45 (SA-45), College Adjustment Scales (CAS), Clinical Assessment of Depression (CAD), Adolescent Anger Rating Scale (AARS), Beck Depression Inventory (BDI), and State-Trait Anxiety Inventory (STAI). Exploratory factor analysis using a principal components approach with orthogonal rotation produced four factors accounting for 41%, 10%, 9%, and 6% of the variance with the first interpreted as representing psychological distress. COMMENTARY. Like other tests linked to diagnostic nosology, the adequacy of the theoretical model behind the PAI-A is limited by the adequacy of the diagnostic model itself. The test’s focus on core characteristics of diagnoses without directly attempting to replicate all diagnostic criteria may prove advantageous if more peripheral criteria change over time or are unimportant to decisions made by the user. As the manual points out, research continues to evolve on some diagnoses such as bipolar mood disorder in adolescents, so that there is no current gold standard by which to judge the PAI-A for some disorders. It would be useful if more information was presented showing the PAI-A’s power to discriminate constructs and diagnoses, such as depression/mood disorders and anxiety/anxiety disorders. In the clinical sample, less than 4% had diagnoses of posttraumatic stress disorder or mania/hypomania, and less than 2% had borderline personality disorder (the only Axis II diagnosis tabulated). Current clinical practice with adolescents often requires a differential between these and other diagnoses, particularly when serving a population with a history of early maltreatment and trauma. Future editions of the manual would also benefit from data on the scales’ sensitivity to treatment effects, the relationship of PAI-A scores to diagnoses made by research criteria, gender/racial/ethnic/cultural differences by diagnosis such as males versus females with Attention Deficit Hyperactivity Disorder, and on profile stability in adolescents 18 years of age and older who complete the PAI after the PAI-A. SUMMARY. The PAI-A achieves its stated purpose of providing a parallel form that extends the PAI to adolescents. More information on discriminative validity and on the ability to detect more subtle denial and malingering is needed. The PAI-A provides an attractive option to the growing number of other self-report measures available for this age group in clinical decision-making (many summarized in Sattler & Hoge, 2006; extensive reviews of several are in Reynolds & Kamphaus, 2004). Self-report measures that parallel parent and teacher ratings typically lack validity scales. The Millon Adolescent Personality Inventory has significant shortcomings (Sattler & Hoge, 2006), and the other Millon scales for this age group have only clinical norms. The PAI-A has the advantage of ease and economy of scoring over the MMPI-A, another downward extension of an adult test of psychopathology. Also, the MMPI-A takes 60-90 minutes to complete and a seventh grade reading level is recommended, but many items are even more difficult (Sattler & Hoge, 2006). The PIY might be considered as an alternative to the PAI-A when empirical scales rather than diagnosis-based scales are acceptable (see review by Sattler & Hoge, 2006). If not, then the APS is a good alternative, but clinical experience has shown the APS has difficulty detecting cases of sophisticated denial. When there is an anticipated need to reevaluate an adolescent over the age of 18, then a clear advantage to the PAI-A is allowing future testing to be based on a very similar measure, the PAI. REVIEWER’S REFERENCES Sattler, J. M., & Hoge, R. D. (2006). Assessment of children: Behavioral, social and clinical applications (5th ed.) San Diego: Jerome M. Sattler. Reynolds, C. R., & Kamphaus, R. W. (Eds.). (2003). Handbook of psychological and educational assessment of children: Personality, behavior and context (2nd ed.). New York: Guilford Press. Review of the Personality Assessment Inventory-Adolescent by JONATHAN SANDOVAL, Professor of Education, University of the Pacific, Stockton, CA: DESCRIPTION. The Personality Assessment Inventory-Adolescent (PAI-A) is an extension to 12- through 18-year-olds of the Personality Assessment Inventory (Morey, 1991). Both measures may be used with 18-year-olds. The self-report personality measure consists of 264 items that generate 4 Validity scales, 11 Clinical scales, 5 Treatment Consideration scales, and 2 Interpersonal scales. Nine of the 11 Clinical scales and the Aggression Treatment Consideration scales have from three to four subscales based on 5 to 6 items each. Other scales typically are derived from about 8 items. In total there are 22 scales and 31 subscales expressed as linear T scores. On the PAI-A, a respondent indicates if a statement is False, Not At All True; Slightly True; Mainly True; or Very True about himself or herself. The intent is to measure constructs relevant to the presence of mental disorder. It is not intended to be a general measure of personality in normal individuals. The materials include a professional manual, an item booklet, hand-scored and optical scan answer sheets, a Critical Items form, and a Profile form. Computer administration and scoring options are available from the test publisher. The Critical Items form consists of 17 items from the PAI-A suggesting behavior or psychopathology demanding immediate attention, such as Suicidal Ideation. The form permits a follow-up interview to explore the critical response. The profile form permits the examination of patterns of scores across the 22 areas, particularly 10 clusters of modal scores identified by statistical analysis that are associated with particular groups such as substance abusers. The test author states that the interpretation of profiles and test score patterns must be done by qualified and experienced clinical professionals. The examiner’s manual is straightforward and provides information about administration considerations, administration procedures, scoring procedures and options, and test feedback to test-takers. Included is a chapter on interpretation that discusses the rationale and construct underlying each scale and subscale and possible meanings of high scores. The PAI-A is not intended to be used mechanically to provide a definitive diagnosis but is rather to provide information relevant to clinical diagnosis, treatment planning, and screening for psychopathology. The information from the PAI-A should supplement information from multiple sources such as case histories, clinical interviews, and other mental tests. DEVELOPMENT. The test author designed the PAI-A to retain the structure and most of the item content of the adult inventory. A few items were reworded to fit adolescent experience. Items were carefully selected based on 15 criteria involving expert opinion, discriminant validity, reading level (fourth grade), positive and negative response sets, and item statistics such as differential item functioning. A total of 344 items were pilot tested with 275 adolescents with mental health diagnoses. Some 80 items with different statistical characteristics for adolescents and adults or other problems were eliminated from the standardization version. TECHNICAL. Norms for the scores were based on 707 adolescents obtained from sites in 21 states in the U.S. The norm sample closely matched the U.S. Census data from 2003 with regard to gender, age, race/ethnicity, but not geographical region, although all regions were represented as well as urban and rural areas. Participants were recruited in a variety of ways including through schools in targeted areas. The sample was not random, but purposive, and data were collected on 1,032 adolescents to create the demographic match. Norms are not broken down by age, ethnicity, or gender, as observed group differences were typically within the range of measurement error and account for less than 5% of the variance in scores. Normative data of a different sort were obtained from a clinical sample of 1,160 adolescents identified by clinicians working in 78 institutions such as inpatient or outpatient mental health settings and juvenile correctional settings as having a mental illness or emotional disturbance. The clinical sample was 58.4 % male and 72.3% Caucasian, 19.8% African American, 4.5% Hispanic, and 4.4% other with a pattern of diagnoses comparable to those found in another large-scale study, with the exception that depression was somewhat overrepresented and anxiety disorder underrepresented, compared to the other study. Evidence of internal consistency and stability of the PAI-A indicates the test has reasonable reliability. Coefficient alpha estimates for the normative sample scores on the 20 substantive scales range from .70 for the Positive Impression Scale to .90 for Aggression with a mean of .79. A similar pattern of alpha values was found for the clinical sample, with remarkable consistency across ages, genders, and ethnicities. The values for the subscales, based on fewer items, was lower, ranging from .47 for Anxiety-Related Disorders-Phobias to .85 Anxiety-Related Disorders-Traumatic Stress, with a mean of .69 for the normative sample. Again, the same pattern and levels of alpha was true of the clinical sample. Test-retest stability over a 9- to 35-day interval based on 100 adolescents in the normative sample for the substantive scales ranged from .65 for Positive Impression to .89 for Somatic Complaints with a mean of .78. Coefficients for the subscales ranged from .59 for Mania-Irritability to .88 for Aggression-Physical Aggression with a mean of .76. The manual also reports stability of PAI-A profiles over time. The test author provides validity information based on a number of studies correlating the PAI-A scores with clinician ratings and with related scores on other commonly used diagnostic measures such as the Minnesota Multiphasic Personality Inventory-Adolescent and the Beck Depression Inventory. Also examined were differences in performance between the normative and clinical samples and the level of performance of the clinical sample on particular scales. The validity scales of Inconsistency, Infrequency, Negative Impression, and Positive Impression were also studied by computer modeling and experimentation with groups of adolescents instructed to “fake bad” or “fake good.” In general, correlations with scales from other measures were in the expected moderate range of .40 to .70. The highest correlations were found for measures of Anxiety and Depression. More modest correlations were found for the Suicidal Ideation, Stress, Treatment Rejection, and Dominance Scales of the PAI-A and parallel measures. In general, patterns of scores of groups from the clinical sample were consistent with their identified disorder. An impressive amount of validity data from studies by the test author and others is provided in the manual and indicates the measures are as valid as other comparable tests. Studies of the internal structure of the PAI-A include an examination of scale intercorrelations, exploratory and confirmatory factor analysis, and cluster analysis. The pattern of intercorrelations is similar for the standardization sample and the clinical sample and related scales have moderate to high correlations within expected domains such as Anxiety, Antisocial Behavior, and Psychosis. A factor analysis of the scales in both samples yielded four orthogonal factors related to internalization, externalization, substance abuse, and social alienation. These factors are consistent with the findings on other measures. The confirmatory factor analysis validated the subscale structures of the PAI-A scales indicating excellent goodness of fit between the subscales and their assigned scales. The manual discusses the method of cluster analysis used to identify 10 profiles on the PAI-A, and to the extent that these profiles make sense, provides some support for the validity of the test. COMMENTARY. Given the vague description of how the normative and clinical samples were obtained, it is difficult to know what biases may have crept into the norms. This is not a critical issue, but should be taken into account in interpreting the test. In any case, because of the developmental variability of adolescents, care should be taken to consider the measurement error around an obtained score. Otherwise, the test was carefully developed using modern psychometric techniques. The issue of test-fairness for minority groups was considered in test development, but more validity work in this area would be welcome. Clearly the test shows promise in clinical applications and may be considered a viable option to other traditional self-report measures for adolescents. The fourth grade reading level and somewhat fewer self-report items are an advantage in soliciting cooperation. SUMMARY. The PAI-A is a well-designed downward extension of the PAI. This self-report inventory, with good measures for detecting uninterpretable results, should aid in clinical diagnosis when used in conjunction with information from other sources. The norms, reliability, and validity information justify the careful and judicious use of the test by qualified clinicians. REVIEWER’S REFERENCE Morey, L. C. (1991). Personality Assessment Inventory professional manual. Odessa, FL: Psychological Assessment Resources.