The ubiquitous multiple choice answer sheet. Photo by COCOEN.
Pretty much no one likes standardized tests. The concept is nothing new, of course – the New York State Regent Exam dates back to Civil War times. A century and a half later, the implementation of No Child Left Behind (NCLB) hinged all of the federal government’s reward and punishment on a school’s “Adequate Yearly Progress” (AYP), a now-infamous composite measure of school performance primarily based on test scores. In the decade since, test-bashing has become something akin to a national pastime, and folks are acting out.
Earlier this year teachers in Seattle flat out refused to administer mandated state exams, claiming that the tests were a misuse of precious school resources, unfairly used as part of teacher evaluations, and an inaccurate indication of student learning. And Seattle isn’t alone. The organization United Opt Out National has assembled a state-by-state guide for opting your kids out of testing, claiming, “high-stakes testing is destructive to ALL children, educators, communities…and the democratic principles which underlie the purposes of public education.”
Let’s say they’re right and standardized tests have got to go. What would be a scalable alternative? One possible solution percolating amongst education reformers may surprise you: portfolios. The practice of assessing learning with portfolios has deep roots in the arts world, visual arts and creative writing especially. Could portfolios save our public school students from a life of drill-and-kill?
The Mechanics
A portfolio is a collection of individual work samples (some of which may have been graded previously), assessed as a whole. It’s a way of combining disparate items into one aggregate assessment demonstrating the application of skills and concepts learned in a classroom setting. Portfolios can be either summative or formative in structure. A summative portfolio focuses on the product or end result of the student’s learning such as, for example, a digital recording of a final performance, a scientific lab report, or a final series of photographs. A formative assessment takes into account the student’s process of learning and can include works-in-progress or evidence of the effort leading to the final product. This type of portfolio might include an actor’s annotated script, the shape and light charcoal studies for a still life painting, or math problem demonstrating the steps in between question and answer.
There are three key elements of assessing learning with portfolios:
- Clearly defined skills and/or knowledge to be assessed,
- Work samples, determined either by the student or the teacher or both, and
- A rubric with which to score collection of work with clear descriptors for each level of success, usually using a point system, and ideally made available to students before portfolios are submitted. (Examples can be found here.)
Sometimes the items included are accompanied by written reflection on the process of creating the final product or the intention behind the work. To demonstrate content knowledge, more traditional academic writing may be included as well.
Part of a Larger Movement
Portfolios are one assessment tool under the larger umbrella of an emerging mode of student evaluation called performance assessment. According to Beyond Basic Skills: The Role of Performance Assessment in Achieving 21st Century Standards of Learning by Linda Darling-Hammond and Frank Adamson:
For many people, performance assessment is most easily defined by what it is not: specifically, it is not multiple choice testing. In a performance assessment, rather than choosing among pre-determined options, students must construct an answer, produce a product, or perform an activity.
Using performance assessment, a student might be asked to write a letter to the editor about a historical event from a specific point of view, draw a series of electrical circuits explaining how changes in configurations would affect the flow of electricity, or demonstrate the ability to use a map by actually navigating.
Performance assessment is rising in popularity as the Common Core State Standards inch closer to full implementation. These new learning standards, adopted by 45 states, four U.S. territories, and the District of Columbia, require “a greater focus on critical thinking, synthesis and analysis, problem solving, communication, media and technology.” States including Maine, New Hampshire, New York, and Ohio are starting to adopt performance assessment systems—which reveal students’ ability to apply information, not just to remember and regurgitate it—to meet the new standards.
Not all performance assessment techniques include portfolios. But portfolios (whether arts-specific or not) are an important piece of the performance assessment puzzle, and learning from the arts’ experience with portfolios could be useful as performance assessment reform initiatives move forward in schools and districts across the country.
Portfolio Assessment in Practice
Some schools and districts are already making use of portfolio assessment. The Beacon School, a public high school in New York City, has been celebrated as a national model of portfolio assessment since it opened in 1993. From the beginning, Beacon leaders wanted to assess their students using methods similar to those employed in graduate schools. Students assembled a portfolio of long-term projects and representative samples from all of their classes – science, history, English, foreign languages – and defended their work to a faculty panel.
By the end of its first decade, Beacon’s plans for portfolio assessment had been somewhat derailed. In the late 1990s New York State began to require students to pass the state’s Regents Exam to graduate. This new requirement meant time was diverted from labs and projects to test preparation. In his 2004 reporting on the Beacon School, writer Jay Matthews explained, “even the most ardent advocates [of portfolios] have acknowledged that samples of student work cannot compete with the ability of standardized testing to quickly and cheaply determine the overall performance of a school district.”
The school still maintains as much of its original assessment strategy as possible. Its website stresses the dual priorities of the school and the state: “Beacon offers a dynamic, inquiry-based curriculum for all students that exceeds standards set by the New York State Regents. Technology and arts are infused throughout the college preparatory curriculum. Each year students must present performance-based projects to panels of teachers, and pass New State Regents tests and community service to graduate.”
School districts in Tennessee are using portfolio assessment for a different purpose that harkens back to its arts-based roots and combines student evaluation with teacher evaluation. In Tennessee’s teacher evaluation system, 35% of a teacher’s employment review is based on test scores. This is a problem for the 70% of teachers in most schools who do not teach a state-tested subject, but portfolios may offer the solution. As reported back in 2012 in The Commercial Appeal newspaper, “[The head of Memphis City Schools arts education, Dru Davison,] and 40 Memphis art teachers wrote a four-page rubric for what peer reviewers should see in the student work for each of 40 art disciplines, from marching band to jazz band.” Each teacher chooses samples of his or her students’ work to form the teacher’s review portfolio, which is then assessed by a blind peer group based on the pre-determined rubrics. The Tennessee Fine Arts Growth Measures System, as it’s called, is in the pilot phase in Memphis City Schools and is starting to garner some national attention. Participation is voluntary for Tennessee school districts. More than ten are participating this school year, up from three last year.
Laura D. Goe, a research scientist at the Educational Testing Service, told EdWeek about the initiative, “Tennessee has the right idea in promoting this effort to achieve some rigor and comparability in a set of content that is difficult to measure…To me, it is a model for where we want to ultimately go [with teacher evaluation], and where I think we will go in most subjects.”
Challenges
The case studies in Tennessee and the Beacon School are intriguing and reflect well on portfolios as a school-wide, district-wide, and maybe even statewide option for evaluating students and even teachers. Could portfolio assessment become a core mechanism for measuring student learning, on the scale of standardized tests?
Statewide and national testing systems depend on reliable, valid results. Reporting on The Beacon School, Jay Matthews wrote, “the argument between advocates of standardized tests and advocates of portfolios usually ends with each side saying it cannot trust the results produced by the other.” For portfolio assessment, it is often the problem of subjectivity that causes concern among test supporters.
Standardized tests, with so-called selected response questions such as multiple choice or true/false, don’t need to be graded by humans. Such questions can arguably be biased, and there is the possibility of human error in the setting of the machines and the handling of the scoring sheets, but the grading is never subjective thanks to grading machines, which also make the process comparatively faster and less expensive. By contrast, portfolios must be graded by humans and grading between raters or even the same raters at different times can be inconsistent. A good rubric and rigorous training can eliminate some personal bias, but not all (see page 22 of Beyond Basic Skills for more information). This problem is a big one if portfolios are to be adopted on a large scale. The ability to reliably compare standardized tests makes it possible to identify outliers among schools, districts, and states, to learn from overachievers, and support underachievers. If the assessment itself and the grading method are not the same for all students, the evaluation won’t be useful for these purposes.
Fortunately, there are large-scale testing systems that already deal with this problem. The Advanced Placement exams taken by high school students in advanced classes in subjects like history and English, for example, are composed of a series of essay questions of various lengths. Students take the exams and then their teachers ship their answer booklets off to be graded by trained “readers.” The writing portion of the SAT is another example. The volume of exams to be graded by each trained reader means that for a handful of SAT essays, factual errors may be overlooked as readers are rewarded for speed. In such cases, students end up demonstrating their knowledge of the grading system rather than their writing ability. While there may be issues with the quality of grading vs. quantity of exams, the fact that the SAT has been accepted for years as a satisfactory (if far from perfect) method of judging students’ college readiness should mean that the subjectivity challenge of grading portfolios is nothing new or prohibitive.
In education, it all comes down to implementation. If a student’s portfolio is filled with work samples that aren’t authentic demonstrations of knowledge and skills learned, it is no better an indicator of learning than a long chain of multiple-choice questions with memorized answers. For portfolio assessment to succeed as a nationwide option for student evaluation, appropriate learning goals must be set with rigorous and specific rubrics. Teachers must also be well trained in administering and scoring assessments, and students well prepared. It’s a novel concept to some, but if portfolios continue to spread as a viable, scalable assessment method, we might emerge from the era of crushing accountability into a new age – one in which testing has a positive effect on learning.