Human Centered Data Science (Fall 2018)/Schedule: Difference between revisions

From CommunityData
Line 257: Line 257:
* Neff, G., Tanweer, A., Fiore-Gartland, B., & Osburn, L. (2017). Critique and Contribute: A Practice-Based Framework for Improving Critical Data Studies and Data Science. Big Data, 5(2), 85–97. https://doi.org/10.1089/big.2016.0050
* Neff, G., Tanweer, A., Fiore-Gartland, B., & Osburn, L. (2017). Critique and Contribute: A Practice-Based Framework for Improving Critical Data Studies and Data Science. Big Data, 5(2), 85–97. https://doi.org/10.1089/big.2016.0050
* Lilly C. Irani and M. Six Silberman. 2013. ''[https://escholarship.org/content/qt10c125z3/qt10c125z3.pdf Turkopticon: interrupting worker invisibility in amazon mechanical turk]''. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). DOI: https://doi.org/10.1145/2470654.2470742
* Lilly C. Irani and M. Six Silberman. 2013. ''[https://escholarship.org/content/qt10c125z3/qt10c125z3.pdf Turkopticon: interrupting worker invisibility in amazon mechanical turk]''. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). DOI: https://doi.org/10.1145/2470654.2470742
* Shilad Sen, Margaret E. Giesel, Rebecca Gold, Benjamin Hillmann, Matt Lesicko, Samuel Naden, Jesse Russell, Zixiao (Ken) Wang, and Brent Hecht. 2015. ''[http://www-users.cs.umn.edu/~bhecht/publications/goldstandards_CSCW2015.pdf Turkers, Scholars, "Arafat" and "Peace": Cultural Communities and Algorithmic Gold Standards]''. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15). DOI: http://dx.doi.org/10.1145/2675133.2675285
* Bivens, R. and Haimson, O.L. 2016. ''[http://journals.sagepub.com/doi/pdf/10.1177/2056305116672486 Baking Gender Into Social Media Design: How Platforms Shape Categories for Users and Advertisers]''. Social Media + Society. 2, 4 (2016), 205630511667248. DOI:https://doi.org/10.1177/2056305116672486.  
<!-- * Wanda J. Orlikowski. 1992. ''[https://dspace.mit.edu/bitstream/handle/1721.1/2412/SWP-3428-27000158-CCSTR-134.pdf%3Bjsessionid%3D89CCB8F0923C0235DB2902AA40C25E28?sequence%3D1 Learning from Notes: organizational issues in groupware implementation]''. In Proceedings of the 1992 ACM conference on Computer-supported cooperative work (CSCW '92). DOI=http://dx.doi.org/10.1145/143457.143549 -->
* Schlesinger, A. et al. 2017. ''[http://arischlesinger.com/wp-content/uploads/2017/03/chi2017-schlesinger-intersectionality.pdf Intersectional HCI: Engaging Identity through Gender, Race, and Class].'' Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI ’17. (2017), 5412–5427. DOI:https://doi.org/10.1145/3025453.3025766.
 


<br/>
<br/>

Revision as of 00:19, 17 September 2018

This page is a work in progress.


Week 1: September 27

Day 1 plan


Introduction to Human Centered Data Science
What is data science? What is human centered? What is human centered data science?
Assignments due
Agenda
  • Syllabus review
  • Pre-course survey results
  • What do we mean by data science?
  • What do we mean by human centered?
  • How does human centered design relate to data science?
  • Looking ahead: Week 2 assignments and topics


Readings assigned
  • Read: Barocas, Solan and Nissenbaum, Helen. Big Data's End Run around Anonymity and Consent. In Privacy, Big Data, and the Public Good. 2014. (PDF on Canvas)
Homework assigned
  • Reading reflection
Resources




Week 2: October 4

Day 2 plan


Ethical considerations
privacy, informed consent and user treatment


Assignments due
  • Week 1 reading reflection
Agenda
  • Intro to assignment 1: Data Curation
  • A brief history of research ethics
  • Guest lecture: Javier Salido and Mark van Hollebeke, "A Practitioners View of Privacy & Data Protection"
  • Guest lecture: Javier Salido, "Differential Privacy"
  • Contextual Integrity in data science
  • Week 2 reading reflection


Readings assigned
Homework assigned
  • Reading reflection
Resources




Week 3: October 11

Day 3 plan


Reproducibility and Accountability
data curation, preservation, documentation, and archiving; best practices for open scientific research
Assignments due
  • Week 2 reading reflection
Agenda
  • Six Provocations for Big Data: Review & Reflections
  • A primer on copyright, licensing, and hosting for code and data
  • Introduction to replicability, reproducibility, and open research
  • Reproducibility case study: fivethirtyeight.com
  • Group activity: assessing reproducibility in data journalism
  • Overview of Assignment 1: Data curation


Readings assigned
Homework assigned
Examples of well-documented open research projects
Examples of not-so-well documented open research projects
Other resources





Week 4: October 18

Day 4 plan


Interrogating datasets
bias in data; best practices for selecting, describing, and implementing training data


Assignments due
  • Reading reflection
  • A1: Data curation
Agenda
  • Final project: Goal, timeline, and deliverables.
  • Overview of assignment 2: Bias in data
  • Reading reflections review
  • Sources of bias in datasets
  • Introduction to assignment 2: Bias in data
  • Sources of bias in data collection and processing
  • In-class exercise: assessing bias in training data


Readings assigned
  • Read: Duarte, N., Llanso, E., & Loup, A. (2018). Mixed Messages? The Limits of Automated Social Media Content Analysis. Proceedings of the 1st Conference on Fairness, Accountability and Transparency, 81, 106. PDF: http://proceedings.mlr.press/v81/duarte18a.html
  • Read: Bender, E. M., & Friedman, B. (2018). Data Statements for NLP: Toward Mitigating System Bias and Enabling Better Science. T0 appear in Transactions of the ACL. PDF: https://openreview.net/forum?id=By4oPeX9f


Homework assigned
  • Reading reflection
  • A2: Bias in data


Resources




Week 5: October 25

Day 5 plan


Introduction to mixed-methods research
Big data vs thick data; qualitative research in data science


Assignments due
  • Reading reflection


Agenda
  • Assignment 1 review & reflection
  • Week 4 reading reflection discussion
  • Survey of qualitative research methods
  • Mixed-methods case study #1: The Wikipedia Gender Gap: causes & consequences
  • In-class activity: Automated Gender Recognition scenarios
  • Introduction to ethnography
  • Ethnographic research case study: Structured data on Wikimedia Commons
  • Introduction to crowdwork
  • Overview of Assignment 3: Crowdwork ethnography


Readings assigned
Homework assigned


Resources




Week 6: November 1

Day 6 plan


Interrogating algorithms
algorithmic transparency and accountability; methods and contexts for algorithmic audits
Assignments due
  • Reading reflection
  • A2: Bias in data
Agenda
  • Assignment 1 review & reflection
  • Week 4 reading reflection discussion
  • Survey of qualitative research methods
  • Mixed-methods case study #1: The Wikipedia Gender Gap: causes & consequences
  • In-class activity: Automated Gender Recognition scenarios
  • Introduction to ethnography
  • Ethnographic research case study: Structured data on Wikimedia Commons
  • Introduction to crowdwork
  • Overview of Assignment 3: Crowdwork ethnography


Readings assigned
Homework assigned
  • Reading reflection


Resources





Week 7: November 8

Day 7 plan

Critical approaches to data science
power, data & society, ethics of crowdwork


Assignments due
  • Reading reflection


Agenda
  • Guest lecture: Rochelle LaPlante


Readings assigned (read both, reflect on one)
Homework assigned


Resources





Week 8: November 15

Day 8 plan


Human-centered algorithm design
human-centered methods for designing and evaluating algorithmic systems


Assignments due
  • Reading reflection


Agenda
  • Final project overview & examples
  • Guest Lecture: Kelly Franznick, Blink UX
  • Reading reflections
  • Human-centered algorithm design
  • design process
  • user-driven evaluation
  • design patterns & anti-patterns


Readings assigned
Homework assigned
  • Reading reflection


Resources




Week 9: November 22 (No Class Session)

Day 9 plan

Data science for social good
TBD
Assignments due
  • Reading reflection
  • A4: Crowdwork ethnography
Agenda
  • Reading reflections discussion
  • Feedback on Final Project Plans
  • Guest lecture: Steven Drucker (Microsoft Research)
  • UI patterns & UX considerations for ML/data-driven applications
  • Final project presentation: what to expect
  • In-class activity: final project peer review


Readings assigned
  • Hill, B. M., Dailey, D., Guy, R. T., Lewis, B., Matsuzaki, M., & Morgan, J. T. (2017). Democratizing Data Science: The Community Data Science Workshops and Classes. In N. Jullien, S. A. Matei, & S. P. Goggins (Eds.), Big Data Factories: Scientific Collaborative approaches for virtual community data collection, repurposing, recombining, and dissemination. New York, New York: Springer Nature. [Preprint/Draft PDF]


Homework assigned
  • Reading reflection
Resources





Week 10: November 29

Day 10 plan

Day 10 slides

User experience and big data


Assignments due
  • Reading reflection


Agenda
  • Reading reflections discussion
  • Feedback on Final Project Plans
  • Guest lecture: Steven Drucker (Microsoft Research)
  • UI patterns & UX considerations for ML/data-driven applications
  • Final project presentation: what to expect
  • In-class activity: final project peer review


Readings assigned


Homework assigned
  • Reading reflection
  • A5: Final presentation
Resources




Week 11: December 6

Day 11 plan

Final presentations
course wrap up, presentation of student projects


Assignments due
  • Reading reflection
  • A5: Final presentation


Agenda
  • Student final presentations
  • Course wrap-up


Readings assigned
  • none!
Homework assigned
  • none!
Resources
  • one




Week 12: Finals Week (No Class Session)

  • NO CLASS
  • A6: FINAL PROJECT REPORT DUE BY 11:59PM on Sunday, December 9
  • LATE PROJECT SUBMISSIONS NOT ACCEPTED.