User:Benjamin Mako Hill/Assessment

In assessment, my goal is to provide very clear goals and paths for achieving them. I do not "grade on a curve" and strive not to compare students to each other. If every student fulfills the requirements I lay out in my syllabus and this rubric, I would be thrilled to give every student in a course a 4.0.

Comprehensive Rubric

This rubric describes prototypical students in terms of the attitudes and behaviors important for successful learning. It does not represent any specific criteria for evaluation or assigning grades.

The 4.0 Student - An Outstanding Student

• Participation: 4.0 students are reliable and engaged participants. Their commitment to the class resembles that of the teacher.
• Preparation: 4.0 students are prepared for class. They always read assignments fully. Their attention to detail is such that they occasionally catch the teacher in a mistake. They always participate in class.
• Curiosity: 4.0 students show interest in the class and in the subject. They look up or dig out what they don't understand. They often ask interesting questions or make thoughtful comments.
• Retention: 4.0 students have retentive minds. They are able to connect past learning with the present. They bring a background with them to class.
• Attitude: 4.0 students have a winning attitude. They have both the determination and the self-discipline necessary for success. They show initiative. They do things they have not been told to do.
• Results: 4.0 students make high grades on work in courses. Their work is a pleasure to grade.

The 3.0 Student - A Good Student

• Participation: 3.0 students participate most of the time. Academics sometimes compete with other priorities.
• Preparation: 3.0 students are usually prepared for class. They try to participate in class discussions.
• Curiosity: 3.0 students have some interest in the subject and ask questions when they do not understand.
• Retention: 3.0 students will frequently make connections among different ideas in the course and occasionally with other ideas from outside.
• Attitude: 3.0 students desire to master the course material. They are active participants. They occasionally show initiative and seek out additional topics related to the course.
• Results: 3.0 students usually improve over the duration of the course with increasing grades on coursework as they master the material and become more efficient in their work.

The 2.0 Student - A Fair Student

• Participation: 2.0 students often fail to participate effectively. Too often they put other priorities ahead of academic work.
• Curiosity: 2.0 students ask few questions and show little interest in course readings and class discussions.
• Preparation: 2.0 students prepare their assignments consistently but in a perfunctory manner. Their work may be sloppy or careless. At times, it is incomplete or late.
• Attitude: 2.0 students are not visibly committed to the class. They participate without enthusiasm. Their body language often expresses boredom.
• Results: 2.0 students obtain mediocre or inconsistent results on tests. They have some concept of what is going on but clearly have not mastered the material.

The 1.0 Student - A Student in Difficulty

• Participation: 1.0 students frequently fail to participate, sometimes a majority of the time. When they miss class, they often fail to find out what was covered in class or even what work was assigned.
• Curiosity: 1.0 students rarely ask questions and often hope not to be noticed during class discussions.
• Preparation: 1.0 students prepare their work in a slipshod fashion. Sometimes, they miss assignments and fail to follow directions. Work is submitted late.
• Attitude: 1.0 students are uncommitted to the class. They may be in the course only because it is required or because the other alternatives are worse. They are frequently bored by the class and show it. They have poor study habits and try to minimize their study time in the course.
• Results: 1.0 students demonstrate little understanding of course material on papers, class work, and exams. They fail to complete many assignments and rarely participate in class discussions unless forced to do so.

Writing Rubric

In formal papers, I will always ask you to connect something you have experience or knowledge about to the material and concepts we have covered in the course. Although specific details or prompts might vary, all writing assignments in my class are structured in this way.

A successful paper will present your topic of interest and demonstrate that you understand and have read, learned, and engaged deeply with the course material. A "4.0" paper will tell a compelling story and will engage with, and improve upon, the course material to teach an audience that includes me, and your classmates, and other students taking this class in future years, how to take advantage of course material. The very best papers will give us all a new understanding of some aspect of course material and change the way I teach some portion of this course in the future.

• 4.0—Excellent: Writing demonstrates impressive understanding of readings, discussions, themes, and ideas. Written work is fluid, clear, analytical, well-organized, and grammatically polished. Reasoning and logic are well-grounded, and examples are precise.
• 3.0—Good: Work demonstrates a thorough and solid understanding of readings, discussions, themes, and ideas. Written work is clear and competent but is somewhat general, a bit vague, or otherwise lacking in precision. While analytical, writing presents more description than analysis. Arguments are solid but not thoroughly original or polished.
• 2.0—Fair: Work demonstrates a somewhat fragmented understanding of readings, discussions, themes, and ideas. Shows acquaintance with readings and ideas but not intellectual engagement. Written work is choppy and argument somewhat difficult to follow, examples are vague or irrelevant, and ideas are imprecise. Work veers toward underdeveloped ideas, off-topic sources or examples, personal anecdotes, creative writing, memoirs, etc.
• 1.0—Unsatisfactory: Work demonstrates little understanding or even acquaintance with readings, discussions, themes, and ideas. Written work is choppy, fractured, and unclear. This argument follows little logical development, or work presents little discernible argument whatsoever.
• 0.0—Failure / Unacceptable: Work does not demonstrate understanding of topics, ideas, and readings. This is also the grade for work not submitted and plagiarized work.

If you need help improving your writing, the Oodegard Writing & Research Center has many resources that can help.

Case Discussion

Many of my courses rely heavily on the case study method. In these courses, your primary form of homework will be preparation for case discussion each day of class.

A standard "case" usually involves reading an example—perhaps up to 20-35 pages of background about an organization or group facing an ambiguous or difficult challenge. I will mark certain readings as "[Cases]" in the syllabus and expect you to read these particularly closely. It is important to realize that I will not summarize case material in class, and I will not cover it in the lecture. I expect everyone in class to have read it, and we will jump in and start discussing it.

Cases ask students to put themselves in the positions of individuals facing difficult situations to tease out the tensions and forces at play in the case and to construct — through group discussion — the broader lessons and takeaways. Cases are a wonderful way to connect the sometimes abstract concepts taught in many academic courses to real examples of the type of ambiguous situations you will likely encounter in your career. Generally speaking, there are no right or wrong answers.

Cold calling in cases

Cases rely roughly on the Socratic method where instructors teaching cases cold call on students—i.e., instructors call on people without asking for volunteers first. I will be doing this in each class.

Because I understand that cold calling can be terrifying for some students, I will circulate a list of questions similar to those I will ask alongside the weekly announcements (i.e., at least six days in advance). Although it is a good idea to write out answers to these questions in advance, I will not be collecting these answers. You are welcome to work with other students to brainstorm possible answers. Although I may also ask questions that I do not distribute beforehand, they will typically be in the same vein as these questions.

I have written a computer program that will generate a random list of students each day. I will use this list to randomly cold call students in the class. To maintain balance in discussions, the program will try to ensure that everybody is cold-called a similar number of times during the quarter by weighting in favor of people who have been called upon fewer times in the past. Although there is always some chance that you will called upon next, you will become less likely to be called upon relative to your classmates each time you are called upon.

Rubric for case discussion answers

Each time you are called upon randomly, I will assess your preparedness based on how you answer. I tend to do these assessments generously, but I don't treat this as a "gimme" either. The rubric I will use for evaluating each answer you give is as follows:

• Engagement: Do you respond in a way that demonstrates that you have been following and engaged with the case discussion?
• Preparedness: Does your answer demonstrate that you have prepared for the case? Have you done the reading?
• Fluency: Are you able to refer to relevant course concepts from the lecture and the non-case material in framing your answers or opinions? Can you engage in synthesis using the material we've covered?

For every question answered during the year, I will assess readiness and participation as "GOOD", "SATISFACTORY", or "POOR", "NO MEANINGFUL ANSWER". These correspond to a 4.0, a 3.0, 2.0, and a 0.0 on the UW 4.0 undergraduate grade scale. I am generous and, in the past, the large majority of answers (~90%) have been assessed as GOOD.

Opting out of case discussions

If you don't want to be included in the cold call list for any reason (e.g., you cannot attend a class), you must tell a member of the teaching team in advance by filling out a simple Google form that asks for two things: (1) your UW student number and (2) the date you would like to be removed from the cold call list. You must fill this out by a specified time before class begins or I will not be able to incorporate it into the program that selects names. The lead time will typically be 1-2 hours and will depend on my schedule. I will also record whether you were not present in class when your name was called. If you fail to use the form and are called upon but are not around to answer, your case discussion grade will be lowered.

If you need to step out briefly during a remote class, just let the teaching team know in some ways so that we can call on you only when you're back.

As described below, the amount that you participate in case discussions will affect your case grade. Of course, being present will directly affect whether or not you are available to participate at a level similar to your classmates.

As per UW policy, no part of your grade will be determined by attendance. In other words, attending class while opting out or not answering when called will have the same effect on your grade---regardless of whether you are absent or present. Of course, if you do not attend class often, it will be difficult or impossible for you to engage in case discussions at the same level as your classmates.

Overall case discussion assessment

Final grades are computed at the end of the final case using the following algorithm:

1. First, I will compute a case grade that is the average of all the questions they have answered using the rubric described above.
2. Next, I will subtract 0.2 points on the UW grade scale for every day that a person did not answer a question when called upon (i.e., presumably because people were absent from class without having used the opt-out form). This is the baseline case grade.
3. Finally, I ensure that students are never penalized for a relative lack of participation in cold call discussions simply because they were "unlucky." I do this by computing the median number of questions students were asked and then sorting students into three groups:
   1. First, I identify students who were asked at least the median number of questions. Their course grade is simply their baseline case grade.
   2. Second, I identify the subset of remaining students who were asked fewer than the median number of questions simply because they were "unlucky" (i.e., were they asked fewer questions not because they had opted out of participation using the class form more often than their classmates). All these students' grades are also the baseline case grade.
   3. For the remaining students who participated less than their classmates because they had opted out of doing so, I assess students with no credit (0 points) for the difference between the median and the number of questions the student was asked. For example, if the class median were 3 questions and a student asked only two questions despite being "luckier" than their classmates (i.e., they opted out so much that even my weighting algorithm couldn't make up for things), I would assess students as a zero for 1/3 of their case participation grade and provide them with the baseline case grade for the two questions they did answer. In the past, the only students in this category have opted out of many case discussions.

Participation Rubric

Some of my smaller classes, mostly graduate courses, grade on participation. Participation is one of the most subjective activities to assess. Hence, you should ask yourself: am I consistently making a positive contribution and impression on the instructor and other students? If the answer to this question is "yes", you are probably doing just fine.

As a rubric, an excellent student satisfies all of these criteria:

1. Participation: Are you an active participant in class?

   Although I will not penalize you for absence, it is simply a fact that if you aren't present, you can't participate—even if you are absent for a good reason.

2. Preparation: Do you read the assignments fully with attention to detail? Do you note relevant ideas, questions, or current events in class and online?

   For example, you might forward a news story to the class with a question for discussion.

3. Engagement: Do you make at least one excellent contribution (e.g., insight or question) to each class without monopolizing discussion? (see the section on participation balance below). Do you give active nonverbal and verbal feedback? Do you refer to other students by name and react to their contributions?
4. Activity: Do you fully engage in group exercises? Do you follow up on open questions and share your findings with the class?

Maintaining Participation Balance

I do not assess participation in terms of how much you speak in class. Indeed, I will assess students as lower if they routinely dominate the conversation to the detriment of the conversation. A useful rule of thumb is to be wary of speaking three times before everyone has had a chance and make sure you make at least one good contribution.

In any group, there will be those who speak more and those who speak less; this might be because of differences in personality, language fluency, or culture. For instance, some people like to think before they speak, and some believe that interaction should be rapid and assertive. I want everyone to participate, and I believe it's worthwhile to achieve balance in classroom discussions.

When I was a student, I tended to dominate conversation. My friend Joseph Reagle shared two strategies that I've found helpful:

1. In classes where I was excited about the topic, I tried to be mindful of how much I spoke when I realized others had interesting things to say but were not as quick to speak. We are often uncomfortable with a little silence, including teachers, and speak to fill the void. However, teaching and facilitation guides recommend that we be open to such spaces: take a couple of breaths or even say, “take two minutes to think about this.” So I began a practice of pacing myself, limiting myself to three really good responses in class, and then making sure others have had time before jumping in — if at all — to contribute.
2. In classes where I was less motivated, I found that I could still usually come up with one good comment or question that nobody else raised. In this way, I could still make a contribution to class — and lessen my chance of being cold-called.

Joseph calls these two techniques the rule of three and one for balanced discussion.

Additionally, you can be a skillful communicator by encouraging balanced discussion. For instance, notice if a person or group hasn't said much. Without putting anyone on the spot, ask them a question or respond to something they said. (Use people's names!) If the class is a small seminar, you might say you'd like to hear from someone who hasn't spoken yet or ask the group to pause so as to collect their thoughts.

Credit for these rubrics

Much of this is borrowed or adapted from Joseph Reagle from these pages:

• Achieving Balance in Group Discussion
• Participation
• Writing: Rubric Dimensions
• Assessment: Comprehensive Rubric
• Kaylea Champion's modified version of this rubric.

Reagle kindly makes his material available for anybody who wants to reuse and share under the Creative Commons Attribution Non-Commercial Share-Alike License, and I happily distribute my own modifications under those terms as well.