Editing User:Benjamin Mako Hill/Assessment

In assessment, my goal is to provide very clear goals and paths for achieving them. I do not &quot;grade on a curve&quot; and I will not compare students to each other. If every student fulfills the requirements I lay out in my syllabus and in this rubric, I would be thrilled to give every student in a course a 4.0.

== Comprehensive Rubric ==

This "Rubric" offers a description of prototypical students in terms of the attitudes and behaviors that are important for successful learning. It does '''not''' represent any specific criteria for evaluation or assigning grades. 

=== The 4.0 Student - An Outstanding Student ===

* Participation: 4.0 students are reliable and engaged participants. Their commitment to the class resembles that of the teacher.
* Preparation: 4.0 students are prepared for class. They always read assignments fully. Their attention to detail is such that they occasionally catch the teacher in a mistake. They always participate in class.
* Curiosity: 4.0 students show interest in the class and in the subject. They look up or dig out what they don't understand. They often ask interesting questions or make thoughtful comments.
* Retention: 4.0 students have retentive minds. They are able to connect past learning with the present. They bring a background with them to class.
* Attitude: 4.0 students have a winning attitude. They have both the determination and the self-discipline necessary for success. They show initiative. They do things they have not been told to do.
* Results: 4.0 students make high grades on work in courses. Their work is a pleasure to grade.

=== The 3.0 Student - A Good Student ===

* Participation: 3.0 students participate most of the time. Academics sometimes compete with other priorities.
* Preparation: 3.0 students are usually prepared for class. They try to participate in class discussion.
* Curiosity: 3.0 students have some interest in the subject and ask questions when they do not understand.
* Retention: 3.0 students will frequently make connections among different ideas in the course and occasionally with other ideas from outside.
* Attitude: 3.0 students desire to master the course material. They are active participants. They occasionally show initiative and seek out additional topics related to the course.
* Results: 3.0 students usually improve over the duration of the course with increasing grades on course work as they master the material and become more efficient in their work.

=== The 2.0 Student - A Fair Student ===

* Participation: 2.0 students often fail to participate effectively. Too often they put other priorities ahead of academic work.
* Curiosity: 2.0 students ask few questions and show little interest in course readings and class discussion.
* Preparation: 2.0 students prepare their assignments consistently but in perfunctory manner. Their work may be sloppy or careless. At times, it is incomplete or late.
* Attitude: 2.0 students are not visibly committed to the class. They participate without enthusiasm. Their body language often expresses boredom.
* Results: 2.0 students obtain mediocre or inconsistent results on tests. They have some concept of what is going on but clearly have not mastered the material.

=== The 1.0 Student - A Student in Difficulty ===

* Participation: 1.0 students frequently fail to participate, sometimes a majority of the time. When they miss class, they often fail to find out what was covered in class or even what work was assigned.
* Curiosity: 1.0 students rarely ask questions and often hope not to be noticed during class discussion.
* Preparation: 1.0 students prepare their work in a slipshod fashion. Sometimes they miss assignments and fail to follow directions on others. Work is submitted late.
* Attitude: 1.0 students are uncommitted to the class. They may be in the course only because it is required or because the other alternatives are worse. They are frequently bored by the class and show it. They have poor study habits and try to minimize their study time in the course.
* Results: 1.0 students demonstrate little understanding of course material on papers, class work, and exams. They fail to complete many assignments and rarely participate in class discussions unless forced to do so.

== Writing Rubric ==

In formal papers, I will always ask you to connect something you have experience or knowledge about to the material and concepts we have covered in the course. Although specific details or prompts might vary, all writing assignments in my class are structured in this way.

A successful paper will both present your topic of interest and demonstrate that you understand and have read, learned, and engaged with the course material deeply. A &quot;4.0&quot; paper will tell a compelling story and will engage with, and improve upon, the course material to teach an audience that includes me, and your classmates, and other students taking this class in future years, how to take advantage of course material. The very best papers will give us all a new understanding of some aspect of course material and change the way I teach some portion of this course in the future.

* '''4.0—Excellent:''' Writing demonstrates impressive understanding of readings, discussions, themes and ideas. Written work is fluid, clear, analytical, well-organized and grammatically polished. Reasoning and logic are well-grounded and examples precise.
* '''3.0—Good:''' Work demonstrates a thorough and solid understanding of readings, discussions, themes and ideas. Written work is clear and competent, but is somewhat general, a bit vague, or otherwise lacking in precision. While analytical, writing presents more description than analysis. Arguments are solid but not thoroughly original or polished.
* '''2.0—Fair:''' Work demonstrates a somewhat fragmented understanding of readings, discussions, themes and ideas. Shows acquaintance with readings and ideas, but not intellectual engagement. Written work is choppy and argument somewhat difficult to follow, examples are vague or irrelevant, and ideas are imprecise. Work veers toward underdeveloped ideas, off-topic sources or examples, personal anecdotes, creative writing, memoir, etc.
* '''1.0—Unsatisfactory:''' Work demonstrates little understanding or even acquaintance with readings, discussions, themes and ideas. Written work is choppy, fractured and unclear. Argument follows little logical development, or work presents little discernible argument whatsoever.
* '''0.0—Failure / Unacceptable:''' Work does not demonstrate understanding of topics, ideas and readings. This is also the grade for work not submitted and plagiarized work.

If you need help improving your writing, the [https://depts.washington.edu/owrc/ Oodegard Writing &amp; Research Center] has many resources that can help.

Many of my courses rely heavily on the case study method. In these courses, your primary form of homework will be '''preparation for case discussion''' each day of class.

A standard "case" usually involves reading an example—perhaps up to 20-35 pages of background about an organization or group facing an ambiguous or difficult challenge. I will mark certain readings as "[Cases]" in the syllabus and I will expect you to read these particularly closely. It is important to realize that '''I will not summarize case material in class and I will not cover it in lecture'''. I expect everyone in class to have read it and we will jump in and start discussing it.

Cases ask students to put themselves in the positions of individuals facing difficult situations to tease out the tensions and forces at play in the case and to construct — through group discussion — the broader lessons and takeaways. Cases are a wonderful way to connect the sometimes abstract concepts taught in many academic courses to real examples of the type of ambiguous situations that you will likely encounter in your career. Generally speaking, there are not right and wrong answers in cases.

==== Cold Calling ====

Cases rely roughly on the [[:wikipedia:Socratic method|socratic method]] where instructors teaching cases cold call on students—i.e., instructors call on people ''without'' asking for volunteers first. I will be doing this in each class.

Because I understand that cold calling can be terrifying for some students, I will be circulating a list of questions we will alongside the weekly announcements (i.e., at least 6 days in advance). I will only cold call to ask students for which you have time to prepare your answers. Although it is a very good idea to write out answers to these questions in advance, we will not be collecting these answers. You are welcome to work with other students to brainstorm possible answers. Although I may also ask questions that I do not distribute ahead of time, I will never cold call when asking these questions.

I have written a computer program that will generate a random list of students each day and I will use this list to '''randomly''' cold call students in the class. To try to maintain balance in discussions, the program will try to ensure that everybody is cold called a similar number of times during the quarter by ''weighting'' in favor of people who have been called upon fewer times in the past. Although there is there always some chance that you will called upon next, you will become less likely to be called upon relative to your classmates each time you are called upon.

==== Rubric for case discussion answers ====

Each time you are called upon randomly, I will assess your preparedness based on how you answer. I tend to do these assessments generously but I don't treat this as a "gimme" either. The rubic I will use for evaluating each answer you give is:

* '''Engagement:''' Do you respond in a way that makes it clear that you have been following and engaged with the case discussion?
* '''Preparedness:''' Does your answer demonstrate that you have prepared for the case? Have you clearly done the reading?
* '''Fluency:''' Are you able to refer to relevant course concepts from lecture and the non-case material in framing your answers or opinions. Can you engage in synthesis using material we've covered?

For every question answered during the year, I will assess readiness and participation as "GOOD", "SATISFACTORY", or "POOR", "NO MEANINGFUL ANSWER". These correspond to a 4.0, a 3.0, 2.0, and a 0.0 on the UW 4.0 undergraduate grade scale. I am generous and, in the past, the large majority of answers (~90%) have been assessed as GOOD.

==== Absence from class ====

Although no part of your grade will be determined by attendance, attendance is important. Of course, if you do not attend class, it will be difficult for you to engage in case discussion at the same level of your classmates. My cold calling algorithm will do everything it can to balance the number of questions asked of each students even if some folks are in class more often than others by calling on folks more when they are present. In the past, every students who attended the large majority of classes were able to participate on the same level of their classmates and had full credit for their case discussion assessment. 

If you cannot attend a class, you '''must tell us in advance''' by filling out a simple Google form that asks for two things: (1) your UW student number and (2) the date you will be absent from class. You must fill this out '''one hour before class begins''' or I will not be able to incorporate it into the program that select names. I will also record absences based on whether you were not present in class when your name was called. If you fail to use the form and are cold called but are not around to answer, your case discussion grade will be lowered.

==== Overall case discussion grade ====
<!--
I also record absences based on whether (a) students reported being absent or (b) were not present in class when their name was called. I will not penalize students for absences but I will use this information to compute the number of questions that each student was present (i.e., in the room for) because doing so reflects the proportion of questions you were prepared to answer questions for.

Next, I rank students in terms of the proportion of the questions they answered that they were present for. This provides a measure of the degree to which students were either lucky/unlucky or (more likely) "helped" by the weighting algorithm. We'll refer to getting asked a higher proportion of questions one is present for, relatively to classmates, as "luck" below.
-->

Final grades for are computed at the end of the final case using the following algorithm:

# I compute the median number of questions that students were asked. Because the distribution is rather narrow, a majority of students in the class in the past were asked at least this many questions (typically around 2/3 of the class).
# I will take everybody who has been answered the median number of questions or more and assess their grade to be the mean question assessment of the questions they were asked minus 0.2 points for every day they were absent from class when called upon (i.e., because they didn't answer when called upon despite being presenting the chat channel for remote classes or because they did not record themselves as absent for a face-to-face class).
# Next, I identify the subset of remaining students who were asked fewer than the median number of questions and identify those that were simply "unlucky" (i.e., were they asked fewer questions ''not'' because they absent more often than their classmates). For all these  students, we compute their grade in the same as described in (2) so that these students are in no way penalized.   
# For any remaining students, I compute average scores for as per (2) for any questions but assess students with no credit (0 points) for each question below the median. For example, if the class median were 3 questions and a student asked only 2 questions despite being "luckier" than their classmates (i.e., they missed so much class and even the weighting algorithm couldn't adjust things), I would assess students as a zero for 1/3 of their case participation grade and provide them with the assessment as per (2) for the two questions they did answer. In the past, the only students in this category have missed an extremely large number of class sessions.

In the past, I have needed to modify the design of case discussions. For example, I have had to move synchronous case discussions into asynchronous conversations conducted online. In these cases, I will communicate an alternate system for discussion and assessment and use it to apply a proportional amount of individuals case discussion grade in the course.

== Participation Rubric ==

Some of my smaller classes, mostly graduate courses, grade on participation. Participation is one of the most subjective activities to assess. Hence, you should ask yourself: am I consistently making a positive contribution and impression on the instructor and other students? If the answer to this question is "yes", you are probably doing just fine. 

As a rubric, an excellent student satisfies all of these criteria:

<ol style="list-style-type: decimal;">
<li><p>'''Participation:''' ''Are you an active participant in class?''</p>
<p>
Although I will not penalize for absence, it is simply a fact that if you aren't present, you can't participate—even if you are absent for a good reason.</p></li>
<li><p>'''Preparation:''' ''Do you read the assignments fully with attention to detail?'' ''Do you note relevant ideas, questions, or current events in class and online?''</p>
<p>For example, you might forward a news story to the class with a question for discussion.</p></li>
<li>'''Engagement:''' ''Do you make at least one excellent contribution (e.g., insight or question) to each class without monopolizing discussion?'' (see section on participation balance below). ''Do you give active nonverbal and verbal feedback?'' ''Do you refer to other students by name and react to their contributions?''</li>
<li>'''Activity:''' ''Do you fully engage in group exercises?'' ''Do you follow up on open questions and share your findings with the class?''</li></ol>

=== Maintaining Participation Balance ===

I do not assess participation in terms of how much you speak in class. Indeed, I will assess students as lower if they routinely dominate conversation to the detriment of conversation. A useful rule of thumb is to ''be wary of speaking three times before everyone has had a chance and make sure you make at least one good contribution.''

In any group there will be those who speak more and those who speak less; this might be because of differences in personality, language fluency, or culture. For instance, some people like to carefully think before they speak and some believe that interaction should be rapid and assertive. I want everyone to participate and I believe it's worthwhile to achieve balance in classroom discussion.

When I was a student, I tended to dominate conversation. My friend Joseph Reagle shared two strategies that I've found helpful:

# In classes where I was excited about the topic, I tried to be mindful of how much I spoke when I realized others had interesting things to say but were not as quick to speak. We are often uncomfortable with a little silence, including teachers, and speak to fill the void. However, teaching and facilitation guides recommend that we be open to such spaces: take a couple of breaths, or even say “take two minutes to think about this.” So I began a practice of pacing myself, limiting myself to three really good responses in class, and then make sure others have had time before jumping in — if at all — to contribute.
# In classes where I was less motivated, I found that if I could still usually come up with one good comment or question that nobody else raised. In this way, I could still make a contribution to class — and lessen my chance of being cold called.

Joseph calls these two techniques the ''rule of three and one for balanced discussion''.

Additionally, you can be a skillful communicator by encouraging balanced discussion. For instance, notice if a person or group is hasn't said much. Without putting anyone on the spot, ask them a question or respond to something they said. (Use people's names!) Or, say you'd like to hear from someone who hasn't spoken yet, or ask the group to pause so as to collect their thoughts.

== Credit for these rubrics ==

Much of this is borrowed or adapted from Joseph Reagle from these pages:

* [http://reagle.org/joseph/zwiki/Teaching/Best_Practices/Learning/Balance_in_Discussion.html Achieving Balance in Group Discussion]
* [http://reagle.org/joseph/zwiki/Teaching/Assessment/Participation.html Participation]
* [http://reagle.org/joseph/zwiki/Teaching/Assessment/Writing.html Writing: Rubric Dimensions]
* [http://reagle.org/joseph/zwiki/Teaching/Assessment.html Assessment: Comprehensive Rubric]

Reagle kindly makes his material available for anybody that wants to reuse and share under the [http://creativecommons.org/licenses/by-nc-sa/4.0/ Creative Commons Attribution Non-Commercial Share-Alike License] and I happily distribute my own modifications under those terms as well.