|
|  |
What is an AP college comparability study?
In an AP college comparability study, a portion of an AP Exam is given to college students enrolled in introductory-level courses comparable to the AP course in the subject. The college faculty members administering these exams evaluate the performance of their students on the comparability study exam and then report the results to psychometricians at ETS, who analyze the data.
The primary purposes of the comparability studies are to maintain congruence between college-level and AP assessment standards and to improve the validity of AP Exams and courses. The results of these studies are used when setting AP grades in order to ensure that AP standards are comparable to or higher than the grading standards used by college faculty in the corresponding college courses. All AP subjects conduct college comparability studies periodically.
Details about the 2007 AP English Language and Composition study:
- College students at participating colleges took a shortened version of the 2007 AP English Language and Composition Exam. Two 55-minute exams were used. Each exam had a different multiple-choice set consisting of 14 questions on a twentieth-century nonfiction passage; both exams had the same 40-minute essay question, question 3. The study was offered at the end of the fall 2006 and spring 2007 semesters to students enrolled in freshman composition courses.
- Instructors graded their students’ exams on an A–F scale, applying the standards they use for their college course. The instructors reported these grades and the grades the students earned for the college course.
- The students’ multiple-choice answer sheets were scanned at ETS. The students’ essays were scored at the 2007 AP English Language and Composition Reading, using the scoring guide for question 3.
- The data from the college students’ scores were used at the 2007 grade-setting session.
Who participated in the AP English Language and Composition study?
Approximately 700 students from 26 colleges and universities (listed below) participated in the study. The number of students per institution ranged from 9 to 58.
| Arizona State University |
Tulane University |
| Ball State University |
University of California, Berkeley |
| Baylor University |
University of California, Los Angeles |
| Bowling Green State University |
University of Central Florida |
| Brigham Young University |
University of Georgia, Athens |
| California State University, Los Angeles |
University of Iowa |
| California State University, Northridge |
University of Mary Washington |
| DePaul University |
University of Pittsburgh at Greensburg |
| Fisk University |
University of Wisconsin, Milwaukee |
| Harvard University |
Utah State University |
| Manhattanville College |
Wartburg College |
| Smith College |
Wittenberg University |
| Tennessee Technological University |
Yale University |
What did the college instructors say about the college comparability study exam?
The college instructors who administered the study exam responded to a questionnaire about the exam administration and the exam content. The responses from the questionnaire included the following feedback:
- Instructors reported that their students took the exam seriously and tried to do their best on it (instructors had been asked to count the comparability study for some part of the students’ course grades).
- Most instructors thought that the essay question on the exam was “very appropriate” for their course.
- Opinion on the multiple-choice questions was more divided, with a few instructors calling it “very appropriate,” and most saying it was “somewhat appropriate.”
- In response to a question asking what course content was not included on the comparability study exam, several instructors mentioned “writing as a process.”
How did the performance of the college students compare with that of the AP English Language students?
After the multiple-choice answers were scanned and the essay responses were scored at the Reading, the scores of the college students on the comparability study exam were translated to the 150-point composite-score scale used for the AP English Language and Composition Exam.
| College Students |
AP Students |
| Exam Grade |
Average Score |
AP Grade |
Minimum Composite Score |
Average Score |
| A |
97 |
5 |
110 |
118 |
| B |
80 |
4 |
94 |
101 |
| C |
67 |
3 |
74 |
83 |
| D |
58 |
2 |
48 |
62 |
| F |
n/a* |
1 |
0 |
35 |
*Cases of college students to whom professors assigned an F for either the exam or the course were removed from the analysis.
How was the comparability study used in grade setting?
At the AP English Language and Composition Exam grade-setting session, the Chief Reader, College Board staff, and ETS staff reviewed the individual college data to identify any “suspect” data—results involving students who were not motivated; instances of poor testing conditions (noise, interruptions, etc.); a poor match between comparability exam content and course content; low correlation between student course grade and comparability exam grade assigned by instructor; and so forth. Information from the comparability study was then added to the other information available at the grade-setting session: equating information from the multiple-choice section of the exam and the Chief Reader’s assessment of the free-response section (difficulty of the questions, rigor of the scoring, ability level of the students).
What factors influenced the 2007 grade distributions?
The college data showed that the current AP standards based on equating were approximately where the top four colleges’ scores were for the cut points between the grades of 5 and 4, between the grades of 4 and 3, and between the grades of 3 and 2. Therefore, no adjustments to these cut points were necessary. Based on data from the comparability study, the cut point between the grades of 2 and 1 was raised from 44 to 48 (on the 150-point scale). Equating information based on previously used multiple-choice questions showed that the 2007 AP English Language and Composition students were better prepared than the 2006 students.
All these factors entered into the final grade distribution for 2007.
Percentage of students earning each AP grade in 2007:
| AP Grade |
Percentage |
| 5 |
9.2% |
| 4 |
18.4% |
| 3 |
31.4% |
| 2 |
30.3% |
| 1 |
10.8% |
College/AP comparisons based on 2007 cut scores:
- AP students who earned scores of 5 on the AP Exam had higher average composite scores than did the college students who earned an A from their instructor on the comparability exam.
- AP students who earned scores of 4 on the AP Exam had higher average composite scores than did the college students who earned a B from their instructor on the comparability exam.
- AP students who earned scores of 3 on the AP Exam had higher average composite scores than did the college students who earned a C from their instructor on the comparability exam.
- AP students who earned scores of 2 on the AP Exam had higher average composite scores than did the college students who earned a D from their instructor on the comparability exam.
- The minimum composite scores for AP grades of 5, 4, and 3 are higher, respectively, than the average scores of college students earning comparability exam grades of A, B, and C.
Conclusion
The study suggests that the standards that have been used for determining grades on the AP English Language and Composition Exam are comparable to the standards used at a sample of selective colleges and are more demanding than those used at many respected colleges.
|
Course Home Pages