A Comparison Study of the Ohio Proficiency Test Results between Fourth-Grade String Pullout Students and Those of Matched Ability

Michael D. Wallick, Hamilton (Ohio) City Schools

From Journal of Research in Music Education, Summer 1998, Vol. 46, No. 2, pp. 239-247. The purpose of this study was to examine the effects of a pullout string program on student achievement in the writing, reading, mathematics, and citizenship sections of the Ohio Proficiency Test. One hundred forty-eight fourth-grade string students and 148 fourth-grade nonstring students from a southwestern Ohio city school district were ability-matched according to their performance on the verbal section of the Cognitive Abilities Test. The scores of the Ohio Proficiency Test were then recorded and compared. This study involved a two-group static-group comparison design. A two-sample independent t-test analysis was used to determine if there was a significant difference between the achievement scores of the string students who were excused from class twice a week for 30 minutes and the matched group of nonstring students who remained in class. It was hypothesized that there would be no significant difference between the two matched groups. The results revealed a significant difference in favor of the string students’ achievement in reading and citizenship, with no significant difference between the two matched groups in the writing and mathematics sections of the Ohio Proficiency Test.

One of the difficulties educators encounter when scheduling at the elementary school level is providing an instrumental music program that is most effective for instruction in music and least disruptive to classroom learning. Performing groups such as band and orchestra must very often be scheduled as “pullouts” that require special attention from the administration (Cowden & Klotman, 1991, p. 203). The problems associated with scheduling pullouts can create frustrations on the part of teachers and administrators. English (1984), Superin- tendent of Northport-East Union Free School District, Long Island, New York, defines pullouts as:

the practice of withdrawing certain elementary school students from their self-contained classroom, usually for special instruction, field trips, assemblies or school projects. The rest of the class stays with the regular teacher, who presumably does something worthwhile until the whole class is together again. Pull-outs have become almost a nightmare for many elementary school principals, who view the practice as a kind of pernicious anemia that attacks whole-class instruction time. Once pull-outs take hold in a school, there appears to be no end to them, and no way to rid the instructional program of their debilitating impact (p. 32).

Hennessey (1984), a sixth-grade classroom teacher, asserts in her article “Pull-outs Disrupt Class Teaching” that:

In the final analysis, students still must know how to read, do math, have attained writing and thinking skills, understand basic scientific concepts, and have an awareness of the world, in order to function in our changing world. With the current state of the pull-out syndrome, such proficiency is in severe jeopardy as eager classroom teachers wait for students to return to the classroom so that the teaching process can resume (p. 18).

Instrumental music class can become frustrating to elementary teachers because only a limited number of students leave their classroom at a given time, and that time can vary from day to day (Bessom, 1969). And in recent years the instrumental music pullout has become one of many. The federal mandates for special instruction in reading via Title I, PL 94-142, special education students, gifted and talented programs, and inclusion have all contributed to the growing tensions between segments of the instructional staff as they compete for student time (English, 1984). Four problems commonly associated with pullout programs are that:

1.    They are disruptive to whole-class teaching.
2.    There is missed instructional time and time on task.
3.    Pullouts create teacher frustration and friction between staff members.
4.    Students in pullout programs are believed to suffer academically.

The focus on the effects of pullouts on academic performance has been magnified by new state mandates for proficiency testing. In March of 1995, all fourth-grade students in the state of Ohio were required to take the first Ohio Proficiency Test (OPT) provided by the Ohio Department of Education (ODE, 1995). This test was mandated by Section 3301.0710 for the Revised Code as amended by Amended Substitute House Bill 55, enacted in the spring of 1992 (ODE, 1995). The OPT is designed to measure reading, writing, mathematics, and citizenship achievement. Teachers and administrators in the Hamilton City Schools were concerned that fourth-grade students who left the regular classroom for string instruction may have suffered negative effects in their OPT achievement.

The purpose of this study was to examine what effects the strings pullout program in the Hamilton City Schools had on fourth-grade students’ OPT performance. This study compared the academic achievement of string students (SS) and matched-ability nonstring students (NS), as measured by the OPT. The following four questions were examined:

1. Is there a significant difference between SS and NS of matched ability in their achievement on the writing section of the OPT?
2. Is there a significant difference between SS and NS of matched ability in their achievement on the reading section of the OPT?
3. Is there a significant difference between SS and NS of matched ability in their achievement on the mathematics section of the OPT?
4. Is there a significant difference between SS and NS of matched ability in their achievement on the citizenship section of the OPT?

 

Review of Literature

Proficiency testing has recently been mandated for fourth-grade students in the state of Ohio. No studies were found that measured the effects of pullout programs on proficiency test performance in Ohio or any other state. A few studies did analyze and compare the academic achievement of instrumental music pullout students with students of matched ability who remained in the classroom.

Groff (1963) matched sixth-grade instrumental students (IS) and noninstrumental students (NI) by sex, IQ, and academic teacher. IS were excused from their regular classroom once a week for 30 minutes. The achievement results of each group were measured by the Iowa Test of Basic Skills. There was no significant difference between IS and NI in academic achievement.

A study was developed by Robitaille and O’Neil (1981a) to investigate if IS in the fifth grade scored significantly higher in academic achievement than NI. Students were tested in reading, mathematics, and total language achievement. The findings revealed that IS did score significantly higher than NI. In a follow-up study, Robitaille and O’Neil (1981b) matched the abilities of IS and NI by IQ. The same procedures from the previous research study were used to test student achievement. This comparison showed no significant difference between the two matched groups.

Kvet (1985) studied 175 matched IS and NI students from four different geographic and socioeconomic areas of a southern Ohio metropolis. Sixth-grade students were matched according to race, sex, IQ, classroom teacher, and cumulative achievement test-scores in reading, mathematics, and language arts. The results showed no significant difference between IS who were excused from class for music instruction and NI who remained in the classroom.

It was surprising that more literature was not available concerning pullout programs and academic achievement considering the widespread use of pullouts for scheduling instrumental music in elementary schools (Kvet, 1985). T
he state of Ohio has instituted proficiency testing in the fourth grade since the completion of Kvet’s 1985 study. This new study was conducted in order to reexamine the effects of music pullout programs on student achievement when measured by the OPT.

 

Methodology

Research Design. This study employed a two-group static-group comparison design. The data collected were scores from the Cognitive Abilities Test (Riverside, 1986) and the OPT. One hundred and forty-eight SS and 148 NS were matched according to their verbal scores on the Cognitive Abilities Test (COGAT). The COGAT verbal scores were chosen as the matching instrument between SS and NS because tests of verbal IQ are more predictive of school success than nonverbal IQ measures (Bowers, Steffy, & Tate, 1988) and because the verbal battery of the COGAT is significantly correlated with success in school (Thorndike & Hagen, 1986).

Students were paired with peers within the same building but not necessarily the same classroom. After they were matched by ability as measured by their COGAT verbal scores, their writing, reading, mathematics, and citizenship scores on the OPT were recorded to be analyzed by a two-sample, matched-means t-test (Dillehay, Graham, & Mercer, 1992).

Sample. Hamilton, Ohio, is a midwestern city of 61,400 just outside the greater Cincinnati metropolitan area. The public school system serves 9,900 students from diverse cultural and socioeconomic backgrounds. The city is predominantly blue-collar middle-class, but also has white-collar professionals, working class poor, and residents on governmental economic assistance.

Seven hundred and fifty-six fourth-grade students were required to take the OPT. The total sample of 296 SS and NS was drawn from 12 of the 13 elementary buildings in the Hamilton City Schools (one building in this district does not start SS in the fourth grade, owing to time limitations on string-staff schedules and the transience of this building’s student population). SS were excused from their regular classrooms at the same time twice a week for 30 minutes. No buildings used a rotating schedule.

Instrument. The OPT was the instrument used to compare SS and NS academic achievement. This test was designed by the Ohio Depart-ment of Education to measure learner outcomes in writing, reading, mathematics, and citizenship. Scores of 4 and above in writing and 200 and above in reading, mathematics, and citizenship were considered to indicate that students were performing at standard. Those who did not perform at standard were recommended for intervention the following year (ODE, 1995). A student may fail to achieve standard on all four sections of the OPT and still pass from fourth grade to fifth grade. This test is designed to identify students who need intervention in specific subjects in the fifth grade.

Data Collection. All fourth-grade SS participating in the city-wide String Festival of April, 1995, were considered possible research subjects. SS and NS who did not take the OPT and the COGAT in the same building were eliminated from the study. This criterion affected two SS and four NS students, making the total sample 296. First, SS verbal scores on the COGAT were recorded, and then these students were paired with NS whose COGAT verbal scores were identical. When no exact match could be made between the verbal scores of SS and NS, the next closest verbal score, plus or minus 7, was used to make a pairing. If two or more NS had the same verbal scores as a string student, the NS COGAT quantitative score that was closest to the string student’s COGAT quantitative score was matched (the quantitative section of the COGAT requires students to use problem-solving skills to analyze quantitative relationships, number series, and equations). This resulted in two groups of 148 SS and NS, matched by ability and performance on the COGAT.

Analysis. A two-sample, independent t-test was computed on STAT-STAR (MacDougall & Stevens, 1995) to determine if a statistically significant difference existed between proficiency achievement of SS and that of the matched group of NS. The t-test, mean, SD, and p value of each matched group were then calculated and compared (Dillehey et al, 1992). It was determined that .05 would serve as the alpha level of significance testing.

 

Findings

In two of the four sections (writing and mathematics) it was found that no statistically significant difference existed between SS and NS. However, in the other two sections (reading and citizenship) there was a statistically significant difference in favor of SS (see Table 1).

The mean scores of SS in every section of the OPT were higher than those of matched-ability NS. A comparison of the standard deviation showed that SS had less variance in the reading, mathematics, and citizenship sections of the OPT, whereas NS performed more consistently in writing.

Table 2 indicates the percentage of students who performed at or above standard and the percentage who passed all four sections of the OPT.

Equal percentages of SS and NS performed at standard on the writing section of the OPT. However, SS had a higher percentage performing at standard in reading, mathematics, and citizenship compared to NS. The greatest discrepancy existed between the percentages of students who passed the mathematics section of the OPT. SS surpassed NS in performing at standard by eleven percentage points. The table also shows that 68% of SS were able to perform at standard on all four sections of the OPT, while only 58% of NS passed all four sections.

 

Conclusions

The evidence is compelling that SS who were pulled out of the regular academic classroom for string instruction did not suffer negative effects in their academic performance, as measured by the OPT.

 

Table 1
Results of SS (n = 148) and NS (n = 148) Proficiency Achievement on OPT

Proficiency section	Mean	SD	t	Prob.
Writing
SS	5.05	1.43
NS	4.85	1.29	1.270	.20512
Reading
SS	229.5	23.73
NS	223.2	26.22	2.127*	.03429*
Mathematics
SS	214.8	24.83
NS	211.8	27.34	0.998	.31907
Citizenship
SS	231.3	23.71
NS	224.8	31.59	2.003*	.04604*

SS = string students who were excused from class 30 minutes twice a week.
NS = matched-ability nonstring students who remained in the classroom.
OPT = Ohio Proficiency Test

* Significant at p < .05.

Furthermore, students who were excused for string instruction scored significantly higher in the reading and citizenship sections of the OPT than students who remained in the classroom. The research in this study supports the following answers to the four questions originally posed:

1. There was no significant difference between SS and NS in writing achievement on the OPT.
2. There was a significant difference between SS and NS in reading achievement on the OPT in favor of SS.
3. There was no significant difference between SS and NS in mathematical achievement on the OPT.
4. There was a significant difference between SS and NS in citizenship achievement on the OPT in favor of the SS.

 

Implications

It may seem paradoxical that academic achievement did not suffer any negative effects and may possibly have been enhanced when students left the regular classroom for their string instruction. Many variables could have caused this effect, and the empirical evidence necessary to draw specific conclusions from this sample of students lies beyond the reach of this study.

 

Table 2
Percentages of Students Performing at Standard on OPT

	SS	NS
% at and above 200/citizenship	93%	87%
% at and above 200/reading	89%	87%
% at and above 4.0/writing	85%	85%
% at and above 200/mathematics	76%	65%
% who passed all four sections	68%	58%

OPT = Ohio Proficiency Test
SS = string students who were excused from class 30 minutes twice a week.
NS = matched-ability nonstring students who remained in the classroom.

4.0 is considered performing at standard in writing.
200 is considered performing at standard in reading, mathematics and citizenship.

One explanation for string students’ scoring significantly higher in the reading and citizenship sections of the OPT could be that both sections require good reading comprehension and critical thinking skills. To perform well on both, one must also have a good English vocabulary. In the citizenship section of the OPT, the students are asked basic questions concerning the three branches of government and Ohio’s economy, and they are required to demonstrate skills in map-reading, interpreting graphs, and analyzing charts.

Like reading words, reading music involves interpreting abstract symbols and translating them into physical and cognitive expressions. The skills involved in reading musical symbols may transfer to those required for comprehending linguistic symbols and interpreting maps, graphs, and charts. Studies need to be developed that would examine if there is a correlation among these cognitive activities.

When string students are excused from their classrooms for string class, they are not leaving instruction. They are moving to another classroom in a different area of the building. The concepts taught in strings go far beyond pitch and rhythm. For example, a student must understand fractions and their relationships to each other in order to manipulate rhythm. The student who has trouble understanding the abstract concept that a half is twice one quarter may comprehend the concrete example of his or her bow’s moving twice as far on half notes as quarter notes. The musician reads abstract concepts from the page and then translates them into concrete phenomena that involve time and space.

The mathematics section of the OPT showed the greatest difference between SS and NS in the percentage of children who performed at standard. Seventy-six percent of the string students achieved proficiency versus 65% of the matched-ability students. However, this statistic could be misleading and the result of matching students primarily by their verbal scores on the COGAT. The quantitative scores were considered only when verbal scores of multiple NS exactly matched a single SS score. The quantitative scores of both SS and NS became the tie- breaker in deciding which individuals to match.

One of the limitations of this study is the inability to assess the impact of different classroom teachers and school atmospheres on string students’ learning. In schools where strings participation is encouraged, students may have less difficulty performing at standard on the OPT as a result of the cooperation of the classroom teacher.

It would be safe to conclude that the problems associated with music pullouts will continue to concern educators in the foreseeable future. Of the difficulties commonly associated with pullout programs, teachers’ concerns that students who are pulled out for instrumental music suffer academically may be unfounded fears. If the results of this study can help to demonstrate this, then the other tensions and frustrations associated with pullout programs may be alleviated.

Hennessey’s statement quoted earlier that “eager classroom teachers wait for [pullout] students to return to the classroom so that the teaching process can resume” seems to imply that the teaching process stops while pullout students are engaged in their various out-of-classroom activities. If this is true, research needs to compare the academic achievement of NI students who attend schools that use pullout scheduling for instrumental music with a matched sample of NI students in schools that do not use pullout scheduling. The scope of research could be broadened to compare the effects of all special pullout programs, including gifted and remedial coursework that requires pullout scheduling. If some teachers cease instruction when pullout students are removed from the classroom, there may be a relationship between the number of pullout programs implemented within a school building and the academic achievement of that building’s student population.

Follow-up studies need to be developed that will continue to measure the impact of pullouts on the OPT. A science section was added to the OPT in 1997. Performance standards in the writing, reading, mathematics, and citizenship sections will be raised in 1999 (ODE, 1995). Studies will need to be conducted to continue monitoring pullouts and student achievement on the OPT as the ODE raises these proficiency standards.

 

References

Bessom, M. E. (1969). Supervising the successful school music program. West Nyack, NY: Parker Publishing.

Bowers, P. G., Steffy, R., & Tate, E. (1988). Comparison of the effects of IQ control methods on memory and naming speed predictors of reading disability. Reading Research Quarterly, 23, 304-319.

Cowden, R. L., & Klotman, R. H. (1991). Administration and supervision of music. New York: Schirmer Books.

Dillehay, J., Graham, G., & Mercer, A. (1992). Basic analysis for research in education (pp. 204-211). Dayton, OH: Wright State University.

English, F. (1984, May). Pull-outs: How much do they erode whole-class teaching? Principal, pp. 32-36.

Groff, F. H. (1963). Effects on academic achievement of excusing elementary school pupils from classes to study instrumental music. Dissertation Abstracts, 25, 5014-5015. (University Microfilms No. 64-3536).

Hennesessy, G. S. (1984, September). Pull-outs disrupt class teaching. Instructor, 94, p. 18.

Kvet, E. J. (1985). Excusing elementary school students from regular classroom activities for the study of instrumental music: The effect on sixth grade reading, language, and mathematics achievement. Journal of Research in Music Education, 32, 45-54.

MacDougall, F. M., & Stevens, E. I. (1995). Stat-Star (3rd ed.) [computer program]. Palm Harbor, FL: Academy Software Inc.

Ohio Department of Education. (1995). The Ohio fourth grade proficiency test. Monterey, CA: CTB McGraw-Hill.

Ohio Department of Education. (1995). Ohio’s statewide testing program: Rules for proficiency testing. Columbus, OH: Author.

Riverside Publishing. (1986). Cognitive abilities test. Chicago: Author.

Robitaille, J. P., & O’Neal, S. (1981a). Why instrumental music in elementary schools? Phi Delta Kappan, 63, 213.

Robitaille, J. P., & O’Neal, S. (1981b). Why instrumental music in the elementary schools? Unpublished manuscript (available from the Albuquerque Public Schools, Albuquerque, NM).

Thorndike, R. L., & Hagen, E. (1986). Cognitive abilities test: Examiner’s manual form 4. Chicago: Riverside. Submitted April 17, 1996; accepted April 30, 1997.