Editing User:Benjamin Mako Hill/Assessment

From CommunityData

Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.

The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.

Latest revision Your text
Line 1: Line 1:
In assessment, my goal is to provide very clear goals and paths for achieving them. I do not "grade on a curve" and I will not compare students to each other. If every student fulfills the requirements I lay out in my syllabus and in this rubric, I would be thrilled to give every student in a course a 4.0.
In assessment, my goal is to provide very clear goals and paths for achieving them. I do not "grade on a curve" and I will not compare students to each other. If every student fulfills the requirements I lay out in my syllabus and in this rubric, I would be thrilled to give every student in a course a 4.0.


== Comprehensive Rubric ==
= Comprehensive Rubric =


This "Rubric" offers a description of prototypical students in terms of the attitudes and behaviors that are important for successful learning. It does '''not''' represent any specific criteria for evaluation or assigning grades.  
This "Rubric" offers a description of prototypical students in terms of the attitudes and behaviors that are important for successful learning. It does '''not''' represent any specific criteria for evaluation or assigning grades.  


=== The 4.0 Student - An Outstanding Student ===
== The 4.0 Student - An Outstanding Student ==


* Participation: 4.0 students are reliable and engaged participants. Their commitment to the class resembles that of the teacher.
* Participation: 4.0 students are reliable and engaged participants. Their commitment to the class resembles that of the teacher.
Line 14: Line 14:
* Results: 4.0 students make high grades on work in courses. Their work is a pleasure to grade.
* Results: 4.0 students make high grades on work in courses. Their work is a pleasure to grade.


=== The 3.0 Student - A Good Student ===
== The 3.0 Student - A Good Student ==


* Participation: 3.0 students participate most of the time. Academics sometimes compete with other priorities.
* Participation: 3.0 students participate most of the time. Academics sometimes compete with other priorities.
Line 23: Line 23:
* Results: 3.0 students usually improve over the duration of the course with increasing grades on course work as they master the material and become more efficient in their work.
* Results: 3.0 students usually improve over the duration of the course with increasing grades on course work as they master the material and become more efficient in their work.


=== The 2.0 Student - A Fair Student ===
== The 2.0 Student - A Fair Student ==


* Participation: 2.0 students often fail to participate effectively. Too often they put other priorities ahead of academic work.
* Participation: 2.0 students often fail to participate effectively. Too often they put other priorities ahead of academic work.
Line 31: Line 31:
* Results: 2.0 students obtain mediocre or inconsistent results on tests. They have some concept of what is going on but clearly have not mastered the material.
* Results: 2.0 students obtain mediocre or inconsistent results on tests. They have some concept of what is going on but clearly have not mastered the material.


=== The 1.0 Student - A Student in Difficulty ===
== The 1.0 Student - A Student in Difficulty ==


* Participation: 1.0 students frequently fail to participate, sometimes a majority of the time. When they miss class, they often fail to find out what was covered in class or even what work was assigned.
* Participation: 1.0 students frequently fail to participate, sometimes a majority of the time. When they miss class, they often fail to find out what was covered in class or even what work was assigned.
Line 39: Line 39:
* Results: 1.0 students demonstrate little understanding of course material on papers, class work, and exams. They fail to complete many assignments and rarely participate in class discussions unless forced to do so.
* Results: 1.0 students demonstrate little understanding of course material on papers, class work, and exams. They fail to complete many assignments and rarely participate in class discussions unless forced to do so.


== Writing Rubric ==
= Writing Rubric =


In formal papers, I will always ask you to connect something you have experience or knowledge about to the material and concepts we have covered in the course. Although specific details or prompts might vary, all writing assignments in my class are structured in this way.
In formal papers, I will always ask you to connect something you have experience or knowledge about to course material.


A successful paper will both present your topic of interest and demonstrate that you understand and have read, learned, and engaged with the course material deeply. A "4.0" paper will tell a compelling story and will engage with, and improve upon, the course material to teach an audience that includes me, and your classmates, and other students taking this class in future years, how to take advantage of course material. The very best papers will give us all a new understanding of some aspect of course material and change the way I teach some portion of this course in the future.
A successful paper will both present your topic of interest and demonstrate that you understand and have read, learned, and engaged with the course material deeply. A "4.0" paper will tell a compelling story and will engage with, and improve upon, the course material to teach an audience that includes me, and your classmates, and other students taking this class in future years, how to take advantage of course material. The very best papers will give us all a new understanding of some aspect of course material and change the way I teach some portion of this course in the future.
Line 53: Line 53:
If you need help improving your writing, the [https://depts.washington.edu/owrc/ Oodegard Writing & Research Center] has many resources that can help.
If you need help improving your writing, the [https://depts.washington.edu/owrc/ Oodegard Writing & Research Center] has many resources that can help.


== Case Discussion ==
= Participation Rubric =


Many of my courses rely heavily on the case study method. In these courses, your primary form of homework will be '''preparation for case discussion''' each day of class.
An excellent student satisfies all of these criteria. Also, participation is one of the most subjective activities to assess. Hence, you should ask yourself: am I consistently making a positive contribution and impression on the instructor and other students?
 
A standard "case" usually involves reading an example—perhaps up to 20-35 pages of background about an organization or group facing an ambiguous or difficult challenge. I will mark certain readings as "[Cases]" in the syllabus and I will expect you to read these particularly closely. It is important to realize that '''I will not summarize case material in class and I will not cover it in lecture'''. I expect everyone in class to have read it and we will jump in and start discussing it.
 
Cases ask students to put themselves in the positions of individuals facing difficult situations to tease out the tensions and forces at play in the case and to construct — through group discussion — the broader lessons and takeaways. Cases are a wonderful way to connect the sometimes abstract concepts taught in many academic courses to real examples of the type of ambiguous situations that you will likely encounter in your career. Generally speaking, there are not right and wrong answers in cases.
 
=== Cold calling in cases ===
 
Cases rely roughly on the [[:wikipedia:Socratic method|socratic method]] where instructors teaching cases cold call on students—i.e., instructors call on people ''without'' asking for volunteers first. I will be doing this in each class.
 
Because I understand that cold calling can be terrifying for some students, I will be circulating a list of questions we will alongside the weekly announcements (i.e., at least 6 days in advance). I will only cold call to ask students for which you have time to prepare your answers. Although it is a very good idea to write out answers to these questions in advance, we will not be collecting these answers. You are welcome to work with other students to brainstorm possible answers. Although I may also ask questions that I do not distribute ahead of time, I will never cold call when asking these questions.
 
I have written a computer program that will generate a random list of students each day and I will use this list to '''randomly''' cold call students in the class. To try to maintain balance in discussions, the program will try to ensure that everybody is cold called a similar number of times during the quarter by ''weighting'' in favor of people who have been called upon fewer times in the past. Although there is there always some chance that you will called upon next, you will become less likely to be called upon relative to your classmates each time you are called upon.
 
=== Rubric for case discussion answers ===
 
Each time you are called upon randomly, I will assess your preparedness based on how you answer. I tend to do these assessments generously but I don't treat this as a "gimme" either. The rubic I will use for evaluating each answer you give is:
 
* '''Engagement:''' Do you respond in a way that makes it clear that you have been following and engaged with the case discussion?
* '''Preparedness:''' Does your answer demonstrate that you have prepared for the case? Have you clearly done the reading?
* '''Fluency:''' Are you able to refer to relevant course concepts from lecture and the non-case material in framing your answers or opinions. Can you engage in synthesis using material we've covered?
 
For every question answered during the year, I will assess readiness and participation as "GOOD", "SATISFACTORY", or "POOR", "NO MEANINGFUL ANSWER". These correspond to a 4.0, a 3.0, 2.0, and a 0.0 on the UW 4.0 undergraduate grade scale. I am generous and, in the past, the large majority of answers (~90%) have been assessed as GOOD.
 
=== Absence from class ===
 
Although no part of your grade will be determined by attendance, attendance is important. Of course, if you do not attend class, it will be difficult for you to engage in case discussion at the same level of your classmates. My cold calling algorithm will do everything it can to balance the number of questions asked of each students even if some folks are in class more often than others by calling on folks more when they are present. In the past, every students who attended the large majority of classes were able to participate on the same level of their classmates and had full credit for their case discussion assessment.
 
If you cannot attend a class, you '''must tell us in advance''' by filling out a simple Google form that asks for two things: (1) your UW student number and (2) the date you will be absent from class. You must fill this out '''one hour before class begins''' or I will not be able to incorporate it into the program that select names. I will also record absences based on whether you were not present in class when your name was called. If you fail to use the form and are cold called but are not around to answer, your case discussion grade will be lowered.
 
=== Overall case discussion grade ===
<!--
I also record absences based on whether (a) students reported being absent or (b) were not present in class when their name was called. I will not penalize students for absences but I will use this information to compute the number of questions that each student was present (i.e., in the room for) because doing so reflects the proportion of questions you were prepared to answer questions for.
 
Next, I rank students in terms of the proportion of the questions they answered that they were present for. This provides a measure of the degree to which students were either lucky/unlucky or (more likely) "helped" by the weighting algorithm. We'll refer to getting asked a higher proportion of questions one is present for, relatively to classmates, as "luck" below.
-->
 
Final grades for are computed at the end of the final case using the following algorithm:
 
# I compute the median number of questions that students were asked. Because the distribution is rather narrow, a majority of students in the class in the past were asked at least this many questions (typically around 2/3 of the class).
# I will take everybody who has been answered the median number of questions or more and assess their grade to be the mean question assessment of the questions they were asked minus 0.2 points for every day they were absent from class when called upon (i.e., because they didn't answer when called upon despite being presenting the chat channel for remote classes or because they did not record themselves as absent for a face-to-face class).
# Next, I identify the subset of remaining students who were asked fewer than the median number of questions and identify those that were simply "unlucky" (i.e., were they asked fewer questions ''not'' because they absent more often than their classmates). For all these  students, we compute their grade in the same as described in (2) so that these students are in no way penalized. 
# For any remaining students, I compute average scores for as per (2) for any questions but assess students with no credit (0 points) for each question below the median. For example, if the class median were 3 questions and a student asked only 2 questions despite being "luckier" than their classmates (i.e., they missed so much class and even the weighting algorithm couldn't adjust things), I would assess students as a zero for 1/3 of their case participation grade and provide them with the assessment as per (2) for the two questions they did answer. In the past, the only students in this category have missed an extremely large number of class sessions.
 
In the past, I have needed to modify the design of case discussions. For example, I have had to move synchronous case discussions into asynchronous conversations conducted online. In these cases, I will communicate an alternate system for discussion and assessment and use it to apply a proportional amount of individuals case discussion grade in the course.
 
== Participation Rubric ==
 
Some of my smaller classes, mostly graduate courses, grade on participation. Participation is one of the most subjective activities to assess. Hence, you should ask yourself: am I consistently making a positive contribution and impression on the instructor and other students? If the answer to this question is "yes", you are probably doing just fine.
 
As a rubric, an excellent student satisfies all of these criteria:


<ol style="list-style-type: decimal;">
<ol style="list-style-type: decimal;">
<li><p>'''Participation:''' ''Are you an active participant in class?''</p>
<li><p>'''Attendance:''' ''Do you punctually attend most all classes?''</p>
<p>
<p>
Although I will not penalize for absence, it is simply a fact that if you aren't present, you can't participate—even if you are absent for a good reason.</p></li>
Even if your absence is excused: if you aren't present, you can't participate.</p></li>
<li><p>'''Preparation:''' ''Do you read the assignments fully with attention to detail?'' ''Do you note relevant ideas, questions, or current events in class and online?''</p>
<li><p>'''Preparation:''' ''Do you read the assignments fully with attention to detail?'' ''Do you note relevant ideas, questions, or current events in class and online?''</p>
<p>For example, you might forward a news story to the class with a question for discussion.</p></li>
<p>For example, you might forward a news story to the class with a question for discussion.</p></li>
<li>'''Engagement:''' ''Do you make at least one excellent contribution (e.g., insight or question) to each class without monopolizing discussion?'' (see section on participation balance below). ''Do you give active nonverbal and verbal feedback?'' ''Do you refer to other students by name and react to their contributions?''</li>
<li>'''Participation:''' ''Do you make at least one excellent contribution (e.g., insight or question) to each class without monopolizing discussion?'' (see section on participation balance below). ''Do you give active nonverbal and verbal feedback?'' ''Do you refer to other students by name and react to their contributions?''</li>
<li>'''Activity:''' ''Do you fully engage in group exercises?'' ''Do you follow up on open questions and share your findings with the class?''</li></ol>
<li>'''Activity:''' ''Do you fully engage in group exercises?'' ''Do you follow up on open questions and share your findings with the class?''</li></ol>


=== Maintaining Participation Balance ===
== Maintaining Participation Balance ==


I do not assess participation in terms of how much you speak in class. Indeed, I will assess students as lower if they routinely dominate conversation to the detriment of conversation. A useful rule of thumb is to ''be wary of speaking three times before everyone has had a chance and make sure you make at least one good contribution.''
In a nutshell: ''Be wary of speaking three times before everyone has had a chance and make sure you make at least one good contribution.''


In any group there will be those who speak more and those who speak less; this might be because of differences in personality, language fluency, or culture. For instance, some people like to carefully think before they speak and some believe that interaction should be rapid and assertive. I want everyone to participate and I believe it's worthwhile to achieve balance in classroom discussion.
In any group there will be those who speak more and those who speak less; this might be because of differences in personality, language fluency, or culture. For instance, some people like to carefully think before they speak and some believe that interaction should be rapid and assertive. I want everyone to participate and I believe it's worthwhile to achieve balance in classroom discussion.
Line 125: Line 75:


# In classes where I was excited about the topic, I tried to be mindful of how much I spoke when I realized others had interesting things to say but were not as quick to speak. We are often uncomfortable with a little silence, including teachers, and speak to fill the void. However, teaching and facilitation guides recommend that we be open to such spaces: take a couple of breaths, or even say “take two minutes to think about this.” So I began a practice of pacing myself, limiting myself to three really good responses in class, and then make sure others have had time before jumping in — if at all — to contribute.
# In classes where I was excited about the topic, I tried to be mindful of how much I spoke when I realized others had interesting things to say but were not as quick to speak. We are often uncomfortable with a little silence, including teachers, and speak to fill the void. However, teaching and facilitation guides recommend that we be open to such spaces: take a couple of breaths, or even say “take two minutes to think about this.” So I began a practice of pacing myself, limiting myself to three really good responses in class, and then make sure others have had time before jumping in — if at all — to contribute.
# In classes where I was less motivated, I found that if I could still usually come up with one good comment or question that nobody else raised. In this way, I could still make a contribution to class — and lessen my chance of being cold called.
# In classes where I was less motivated, I found that if I could still usually come up with one good comment or question that nobody else raised. In thi way, I could still make a contribution to class — and lessen my chance of being cold called.


Joseph calls these two techniques the ''rule of three and one for balanced discussion''.
Joseph calls these two techniques the ''rule of three and one for balanced discussion''.
Line 131: Line 81:
Additionally, you can be a skillful communicator by encouraging balanced discussion. For instance, notice if a person or group is hasn't said much. Without putting anyone on the spot, ask them a question or respond to something they said. (Use people's names!) Or, say you'd like to hear from someone who hasn't spoken yet, or ask the group to pause so as to collect their thoughts.
Additionally, you can be a skillful communicator by encouraging balanced discussion. For instance, notice if a person or group is hasn't said much. Without putting anyone on the spot, ask them a question or respond to something they said. (Use people's names!) Or, say you'd like to hear from someone who hasn't spoken yet, or ask the group to pause so as to collect their thoughts.


== Credit for these rubrics ==
= Credit for this Rubric =


Much of this is borrowed or adapted from Joseph Reagle from these pages:
Much of this is borrowed or adapted from Joseph Reagle from these pages:
Please note that all contributions to CommunityData are considered to be released under the Attribution-Share Alike 3.0 Unported (see CommunityData:Copyrights for details). If you do not want your writing to be edited mercilessly and redistributed at will, then do not submit it here.
You are also promising us that you wrote this yourself, or copied it from a public domain or similar free resource. Do not submit copyrighted work without permission!

To protect the wiki against automated edit spam, we kindly ask you to solve the following CAPTCHA:

Cancel Editing help (opens in new window)
Retrieved from "https://wiki.communitydata.science/User:Benjamin_Mako_Hill/Assessment"
Attribution-Share Alike 3.0 Unported
Powered by MediaWiki