How to assess writing? With rubrics, duh! (Sara Weigle)

So as discussed in the prior post the LARC has had webinars about assessment in the past. There was also a TOEFL Spain presentation on the different ways to assess homework.

Well this time I had the change to listen to Susan Weigle discuss assessing writing. You can read more about her and the webinar here: http://larc.sdsu.edu/webinar-sara-weigle/ She is a published and respected teacher, but this was her first webinar, and it showed. I am not honestly sure I would recommend watching the recording to anyone, but you can find it linked above. AGAIN Sara is very qualified, but the webinar appeared to be a bit too much. She was often overwhelmed and honestly I struggled to get through the presentation. Participants as well seemed to be clueing out and asking questions that she had already answered. No worries, everyone struggles their first time and I am sure she will improve with each webinar.

So, she started by going into the basics of writing, authentic writing tasks versus the tasks where we use writing to check grammar or vocab comprehension. For me this was a reminder that whenever possible we should try to keep our assignments as authentic as possible (kinda hard given I am currently teaching academic writing, but still).

Then she discussed Hamp-Lyons definition of a writing test:

1. Test takers must write one piece of continuous text
2. Test takers are given a prompt or instructions on what to write
3. Writing is evaluated by at least one trained grader
4. Graders judge using a rubric or set scale
5. Judgments are given as numbers or grades

How can writing be assessed without being tested?
Webinar participants suggested logs, journals, timed writings, chat forums, portfolios, essays, fb group discussions, etc I would actually argue that many of these fall in the definition of the writing test per above, but I suppose it depends on if a rubric is used or not (I use them with every assignment I give for consistency). Sara goes to specifically define a test as an in class timed writing, compared to take home essays.

Then she discusses the components of a test, the task (what they should write) and the scale (how we grade it).

She gave us the topics (http://www2.gsu.edu/~wwwrtp/topics.htm) that University students have to write on to prove they are university level writers. Most of the webinar agreed that for high school or EFL learners the topics were fine but they were a bit too broad for an academic timed writing.

To avoid having a task that doesn’t meet up with the students’ (or teacher’s) needs it is important to remember the principles for task design.

1. Clarity Students need to understand what they are doing (a colleague of mine discussed an exam the other day where she asked students to write on one of the topics in the box below. She also reminded them to use good grammar and watch their spelling. Several students wrote on why using spelling and grammar is appropriate. This is the result of unclear topics(or students who don’t pay attention) )
2. Validity It should be a writing that is relevant.
3. Reliable the task should elicit writing samples that can be scored consistently
4. Interest the task should be interesting to writers and readers (not another opinion paper on abortion PLEASE)

OK, so now we have our pretty task, but how do we grade it? Well, RUBRICS!
I liked one ”pro” about Rubrics that she mentioned: It is important to, “make implicit assumptions and goals explicit” this lets students better prepare since they know exactly what the teacher wants. As a teacher we may think it is OBVIOUS we want out students to use cause and effect linkers as we have studied them for the past 3 months, but if you don’t specifically say it (and let the students know) I promise you most students will not assume this.

Why else are rubrics amazing? You can share them with your department for some consistency amoung all the classes, and when the students see them they cannot claim favoritism or “the teacher doesn’t like me” they are specifically given their grades with what they didn’t or did do.
Now the type of rubric that I do is analytic which means I score on several dimensions. This is normally easier for new teachers (I guess I fit that stereotype) and in gives students a clearer idea of where they need to focus more energy next time.

• For example a paragraph could be graded on mechanics, transition words, organization and format. All of these are totaled for a final grade.

This is compared to holistic where the paragraph is grades on one “standard”:

• Example: The paragraph is grammatically correct, with good transition words and is organized. 90pts
• The paragraph is grammatically OK, with some transition words and is organized. 75pts
• Etc.

DESIGNING RUBRICS

1. Define according to the topic, process and the product
2. Determine the components that you are interested in
3. Decide on what TYPE of rubric you want
4. Define components
5. Decide on number of score levels (generaly 4.5)
6. Write descriptions for each score level. Look at existing rubrics for ideas!

I am always up for a good online tool and this time she have us: http://rubistar.4teachers.org/ I tend to do my rubrics by “hand” but I do think that some assignments could be OK with an automatically generated

Brief sidenote: she did throw in her two cents on peer grading (that the grader usually benefits more than the gradee). Which I have always felt so it was nice to hear :)