Reflective Evaluation: How Can Museums Change Teens—and Vice Versa? Part 1

Over the past four years, I have worked with hundreds of Milwaukee-area teens who love art, and who, over their time in teen programs at the Milwaukee Art Museum, grow to love museums as well.

I have always had a sense that my students grow over their time at the Museum. This year, though, to really study that growth, we designed our longstanding Satellite High School Program as a year-long experience to explore exactly how weekly sessions at an art museum might change the thinking of our teen participants. To that end, our program outcome for students was that they would show an increased ability to reflect upon their own experiences and performance.

This means I’ve been thinking a lot about evaluation: How do we show there was a change? Years ago, I thought evaluation was more or less a prickly, black-and-white, necessary evil that forced me to use altogether too much math. But over the past two years, I’ve come around to believe evaluation is completely the opposite (though math is still important!). Evaluation is a grey area—much like teaching and interpretation—and we as educators need to use multiple methods in order to get a fuller picture of what’s going on with our students. And further, these methods can be tools to help our teaching, improving programs and our impact on students.

In the end, I found I needed to use reflective practice myself to understand how my students were changing, and to explore and experiment with a number of different methods for articulating their growth. In this post, I’ll share a few of the methods we used in the Satellite High School Program this year to explore how our teen interns changed through reflective practice.

First…What is Satellite?

The Satellite High School Program is a year-long internship for sixteen teens ages 16 to 18 from diverse high schools all over the Milwaukee area. Once a week after school, they come together at the Museum and explore how art can be made relevant to our lives today. They participate in “object studies” (hour-long discussions on a single work of art), behind-the-scenes career talks with staff, and resume-writing workshops, and they also mentor elementary school students in tours of the permanent collection.

Teens create a final project that has a real-world impact on the Museum. They choose a work of art in the Museum Collection, research it, and form their own interpretation of the piece. In past years, students have created responses in visual art, writing, or performance. This year, the students used iPads to create videos on their work of art, explaining what the work means to them and how it changed their thinking or art practice. You’ll see a few of those videos later in this series of posts.

Let’s start with the core evaluation method we used for the program. We were lucky to work with one of our teen program funders, the Milwaukee Public Schools Partnership for the Arts & Humanities, and the University of Wisconsin-Milwaukee’s Center for Urban Initiatives and Research (CUIR) to develop the outcome above and to establish a tool to measure it.

Core Evaluation

We settled on one-on-one interviews, doing a “pre” interview on the first days of the program in October and a “post” interview on the final days of the program in May. Each student was privately asked the same set of questions in the pre- and post-interviews, meant to get at their ability to reflect on their experiences in the program. I scored each interview on a rubric that measured level of detail in their responses, and then we compared their pre-program score to their post-program score to see if they had improved.

At the end, every student did improve in their ability to reflect—their answers got significantly more detailed. As someone whose default is to be a more qualitative thinker, it was rewarding to use the rubric to see their interviews as data, in a quantitative, more tangible way.

But as helpful as this was, it’s still just one method of evaluation. Being able to explain in detail is certainly one aspect of successfully being able to reflect. But as I listened to their responses, and thought about what I had seen in the students over the course of the whole year, I realized there is much more to reflecting than just detail. Their responses used stronger vocabulary, they expressed sophisticated ideas, and they asked more and deeper questions. How could I articulate that kind of change?

Continued in Reflective Evaluation: How Can Museums Change Teens—and Vice Versa? Part 2, coming soon.