Students drop a bomb at the #gafesummit Vancouver: “We don’t like rubrics.”

Update: Here’s a Storify story with as much of the conversation on Twitter as I can capture:

rubrics

student’s note from the panel discussion

This year we assembled a panel of students at the Google Apps for Education (GAFE) Vancouver Summit we hosted at Mulgrave School. The thinking was, it’s about time we talked to our customers, so to speak.

We had 5 high school students students from three local schools sitting round the table blue-sky-ing the future school. I’ll be posting more of their thoughts later but I thought I’d start with this bomb: the students hate rubrics!

Rubrics are the bread and butter of assessment these days. The IB Diploma program, for example, depends on them. One student did say that on the upside, rubrics provide some security against teacher bias (real or perceived.) But, all of the kids said, rubrics feel too standardized: “We want something more personal than a number or letter that refers to some canned comment.”

Good rubrics are built by students and teachers, collaboratively, so this comment from the kids might mean we’re not doing a good enough job consulting the major stakeholders in the assessment. My sense was, however, that this was not the problem and that students were touching on something much deeper. As Yong Zhao said in the summit keynote, so long as outcomes–whether employable skills or university entrance requirements–are prescribed by an external body, schooling becomes an exercise in squeezing all students through the “sausage-making machine.” It’s an impersonal activity, by definition, because in this structure we are not cultivating personal talents but rather bending and shaping people to meet some externally determined standard.

Zhao keynote

 

For an outstanding example of what cultivating talent looks like, watch this video of Benjamin Zander teaching a 15-year old student to play cello. It remains the best teaching I’ve  seen. “You can give an “A” to anybody,” says Zander. Listen to him explain what great teaching and assessment look like at about 5:45 on.

I recall reading somewhere of a study that suggested that rubrics did not actual provide any more objectivity or consistency that good teacher judgement: the same student essays were given to two different groups of teachers with one group using a rubric and the second, their good judgement. The study reported no statistically significant difference in the scoring. If anyone can get their hands on this or similar research I’d be grateful.

Also, five students are admittedly a ridiculous small sample size. I think I need to at least send out a quick poll to increase the numbers and draw on a more representative sample.

12 Comments

  1. Reply

    Who else from BC is using Google Apps for Education? Any “higher ed” users? I would like to speak to someone who has experience using them in the university realm.

  2. Reply
    Kern May 8, 2014

    Count my students in if you’re looking for a larger sampling.

  3. Reply

    […] This year we assembled a panel of students at the Google Apps for Education (GAFE) Vancouver Summit we hosted at Mulgrave School. The thinking was, it's about time we talked to our customers, so to…  […]

  4. Reply

    Hi Brad,

    Students want feedback, so the question is, how useful are rubrics for giving feedback to students?

    Based on the number of times I have clarified language in the IB rubrics, I’d say that those particular rubrics are not very good at giving students actionable, personal, specific feedback that they can use.

    I get a bit nervous when I hear teachers talking about how they just know how well students have done from personal experience. Personal experienced is very biased. It’s personal after all!

    As one example of bias (there are millions), consider the recent research about how resumes are interpreted by heads of law firms. A team of researchers created some fake resumes, and then sent them out to heads of law firms, and asked the lawyers to evaluate the resumes. The trick is, although the resumes were the same, the applicant race was different. When the race was given as black, the heads of the law firms consistently evaluated the candidates lower, and found more errors in the candidates resumes than if the race was given as white. These people would almost certainly not consider themselves racist, but they have bias.

    Bias is everywhere. Pretty people (on average) get higher grades than people who are not pretty. Tall people are consistently rated higher on aspects of leadership.

    Rubrics are an attempt to mitigate the effects of bias. If everyone is using the same rubric, and takes the time to norm what the rubric means with other people using the same rubric, the effect of bias will be made smaller than otherwise (aside from any bias in the rubric itself).

    I would use a rubric to evaluate performance, but I don’t know if I would use it to give feedback necessarily. Maybe there are rubrics out there which are written in student friendly language and are able to be used to give actionable, personal, specific feedback, but I haven’t met them.

    David

  5. Reply

    These kids are spot on. Rubrics are limiting. The one use I ever had for them was to provide students with criteria I was looking for when assessing assignments. Then I realized that I could just provide the criteria and not be constrained by the rubric, or suggest to my students that there was a “ceiling” on what was an appropriate effort for a particular work product. You should really check out Alifie Kohn on rubrics: http://www.alfiekohn.org/teaching/rubrics.htm

  6. Reply
    Braddo May 9, 2014

    I hear you, David Wees. And the students acknowledged as much. I think we have a few questions we need to investigate further before we can go very much farther:

    It could be that rubrics are inherently great but are not used effectively by teachers. The best one’s I’ve used have been collaboratively developed collaboratively with students.

    It could be that rubrics are good at generating feedback for the individual but not the person (see Do you Teach Persons or Individuals? http://wp.me/p2Orok-ne ) and our students are speaking as persons.

    It could be we need to think about where rubrics are used: Rubric or no, I find myself wondering about the value of any summative assessment. Right now, I’m having a hard time seeing them as anything other than a tool for gathering marks.I don’t see any instructional value in them.

    And if we’re using rubrics in formative assessments, well then I’m leaning towards Kohn (thanks for the article David Knuffke) who argues that rubrics are inherently constraining and, what bothers me especially, ignore the inherent messiness of learning. They are, in short, reductionist.

  7. Reply

    I guess rubrics can be limiting, but they don’t need to be. And I was thinking about rubrics in the sense that I have used them – which is something you provide in advance, to help students better understand what you are looking for. In this sense, any clarity is much appreciated (I think).

    One proviso – I am speaking about university courses, where the instructions are sometimes incredibly vague (“write an essay on…”) and the feedback is catastrophically poor (“could have been better argued, C+”).

    A rubric – and even the work it takes to develop one – is a great way to focus the mind of the instructor and force clarity rather than obscurity.

    …r

  8. Reply
    Braddo May 9, 2014

    Thanks Richard. So, maybe the perceived utility of a rubric depends on context. How can that be with a tool that is supposed to be neutral/objective?

    What are your thoughts on Kohn’s article?

  9. Reply

    […] Students drop a bomb at the #gafesummit Vancouver: “We don’t like rubrics.” | A Stick in the S… […]

  10. Reply

    Rubrics do involve thinking. It could be a case of thinking being harder than not thinking. Furthermore, thinking for yourself is nicer than being forced to think, which may or may not be so nice. How to get students to own the rubric thinking, is possibly a question that could help us along with our use of rubrics.

  11. Reply

    […] “ This year we assembled a panel of students at the Google Apps for Education (GAFE) Vancouver Summit we hosted at Mulgrave School. The thinking was, it's about time we talked to our customers, so to…”  […]

  12. Reply

    […] perform or how well we deliver a program? For a great illustration of the difference see the Benjamin Zander video after the jump (this is one of the best examples of great teaching I’ve ever seen and well […]

Leave a Reply