Editing Designing Internet Research (Spring 2022) (section)

== Schedule ==
=== Monday March 28: Introduction ===

'''Resources:'''

* [https://uw.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=9cbb0e8c-3ddf-4fe2-a4b6-ae68005f674b Class video recording]  ''[Available through Canvas]'' — It's mostly just me walking through the syllabus and doesn't include the introductions and such

'''Required Readings:'''

* Agre, Phil. 2004. “Internet Research: For and Against.” In ''Internet Research Annual: Selected Papers from the Association of Internet Researchers Conferences 2000-2002'', edited by Mia Consalvo, Nancy Baym, Jeremy Hunsinger, Klaus Bruhn Jensen, John Logie, Monica Muerero, and Leslie Regan Shade. Vol. 1. New York: Peter Lang. http://polaris.gseis.ucla.edu/pagre/research.html. {{avail-free|http://polaris.gseis.ucla.edu/pagre/research.html}}
* Lazer, David, Alex Pentland, Lada Adamic, Sinan Aral, Albert-Laszlo Barabasi, Devon Brewer, Nicholas Christakis, et al. 2009. “Computational Social Science.” ''Science'' 323 (5915): 721–23. https://doi.org/10.1126/science.1167742. {{avail-uw|https://doi.org/10.1126/science.1167742}}
* Sandvig, Christian, and Eszter Hargittai. 2015. “How to Think about Digital Research.” In Digital Research Confidential: The Secrets of Studying Behavior Online, edited by Eszter Hargittai and Christian Sandvig, 1–28. Cambridge, MA: MIT Press. {{avail-canvas|1=https://canvas.uw.edu/files/90060240/download?download_frd=1}}

'''Optional Reading:'''

* December, John. 1996. “Units of Analysis for Internet Communication.” Journal of Computer-Mediated Communication 1 (4): 0–0. https://doi.org/10.1111/j.1083-6101.1996.tb00173.x. {{avail-free|https://doi.org/10.1111/j.1083-6101.1996.tb00173.x}}

=== Wednesday March 30: Ethics ===

'''Required Readings:'''

* franzke, aline shakti, Anja Bechmann, Michael Zimmer, and Charles M. Ess. 2020. “Internet Research: Ethical Guidelines 3.0.” Association of Internet Researchers. https://aoir.org/reports/ethics3.pdf. {{avail-free|https://aoir.org/reports/ethics3.pdf}}

To frame a conversation around research ethics, lets read this piece:

* Kramer, Adam D. I., Jamie E. Guillory, and Jeffrey T. Hancock. 2014. “Experimental Evidence of Massive-Scale Emotional Contagion through Social Networks.” Proceedings of the National Academy of Sciences 111 (24): 8788–90. https://doi.org/10.1073/pnas.1320040111. {{avail-uw|https://doi.org/10.1073/pnas.1320040111}}

And these pieces that are all vaguely in response to it:

* Carr, Nicholas. 2014. “The Manipulators: Facebook’s Social Engineering Project.” The Los Angeles Review of Books, September 14, 2014. http://lareviewofbooks.org/essay/manipulators-facebooks-social-engineering-project/. {{avail-free|http://lareviewofbooks.org/essay/manipulators-facebooks-social-engineering-project/}}
* [Skim page to get a sense of the backlash] Grimmelmann, James. 2014. “The Facebook Emotional Manipulation Study: Sources.” The Laboratorium (blog). June 30, 2014. http://laboratorium.net/archive/2014/06/30/the_facebook_emotional_manipulation_study_source. {{avail-free|http://laboratorium.net/archive/2014/06/30/the_facebook_emotional_manipulation_study_source}}
* Bernstein, Michael. 2014. “The Destructive Silence of Social Computing Researchers.” Medium (blog). July 7, 2014. https://medium.com/@msbernst/the-destructive-silence-of-social-computing-researchers-9155cdff659. {{avail-free|https://medium.com/@msbernst/the-destructive-silence-of-social-computing-researchers-9155cdff659}}
* Lampe, Clifford. 2014. “Facebook Is Good for Science.” The Chronicle of Higher Education Blogs: The Conversation (blog). July 8, 2014. http://chronicle.com/blogs/conversation/2014/07/08/facebook-is-good-for-science/. {{avail-uw|http://chronicle.com/blogs/conversation/2014/07/08/facebook-is-good-for-science/}}
* Hancock, Jeffrey T. 2018. “The Ethics of Digital Research.” The Oxford Handbook of Networked Communication, April. https://doi.org/10.1093/oxfordhb/9780190460518.013.25. {{avail-uw|https://doi.org/10.1093/oxfordhb/9780190460518.013.25}}

'''Optional Readings:'''

* Department of Health, Education, and Welfare, and National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. 2014. “The Belmont Report. Ethical Principles and Guidelines for the Protection of Human Subjects of Research.” http://www.hhs.gov/ohrp/policy/belmont.html. {{avail-free|http://www.hhs.gov/ohrp/policy/belmont.html}}
* Frankel, Mark S., and Sanyin Siang. 1999. “Ethical and Legal Aspects of Human Subject Research on the Internet.” Workshop Report. Washington, DC: American Association for the Advancement of Science. {{avail-free|https://www.researchgate.net/profile/Sanyin_Siang/publication/268296040_Ethical_and_Legal_Aspects_of_Human_Subjects_Research_on_the_Internet/links/5632302008ae0530378fd404/Ethical-and-Legal-Aspects-of-Human-Subjects-Research-on-the-Internet.pdf}}

=== Monday April 4: Internet Data Collection ===

'''Required Readings:'''

* Mislove, Alan, and Christo Wilson. 2018. “A Practitioner’s Guide to Ethical Web Data Collection.” In The Oxford Handbook of Networked Communication, edited by Brooke Foucault Welles and Sandra González-Bailón. London, UK: Oxford University Press. https://doi.org/10.1093/oxfordhb/9780190460518.001.0001. {{avail-uw|https://doi.org/10.1093/oxfordhb/9780190460518.001.0001}}
* Brügger, Niels. 2018. “Web History and Social Media.” In The SAGE Handbook of Social Media, edited by Jean Burgess, Alice Marwick, and Thomas Poell, 196–212. London, UK: SAGE Publications Ltd. https://doi.org/10.4135/9781473984066. {{avail-uw|https://doi.org/10.4135/9781473984066}}
* Shumate, Michelle, and Matthew S. Weber. 2015. “The Art of Web Crawling for Social Science Research.” In Digital Research Confidential: The Secrets of Studying Behavior Online, edited by Eszter Hargittai and Christian Sandvig, 234–59. Cambridge, MA: The MIT Press. {{avail-canvas|1=https://canvas.uw.edu/files/90060239/download?download_frd=1}}
* Freelon, Deen. 2018. “Computational Research in the Post-API Age.” Political Communication 35 (4): 665–68. https://doi.org/10.1080/10584609.2018.1477506. {{avail-uw|https://doi.org/10.1080/10584609.2018.1477506}}
* '''[Example]''' Graeff, Erhardt, Matt Stempeck, and Ethan Zuckerman. 2014. “The Battle for ‘Trayvon Martin’: Mapping a Media Controversy Online and Off-Line.” First Monday 19 (2). http://firstmonday.org/ojs/index.php/fm/article/view/4947. {{avail-free|http://firstmonday.org/ojs/index.php/fm/article/view/4947}}

'''Optional Readings:'''

* Ankerson, Megan Sapnar. 2015. “Read/Write the Digital Archive: Strategies for Historical Web Research.” In Digital Research Confidential: The Secrets of Studying Behavior Online, edited by Eszter Hargittai and Christian Sandvig, 29–54. Cambridge, MA: MIT Press. {{avail-canvas|1=https://canvas.uw.edu/files/90060241/download?download_frd=1}}
* Spaniol, Marc, Dimitar Denev, Arturas Mazeika, Gerhard Weikum, and Pierre Senellart. 2009. “Data Quality in Web Archiving.” In Proceedings of the 3rd Workshop on Information Credibility on the Web, 19–26. WICOW ’09. New York, NY, USA: ACM. https://doi.org/10.1145/1526993.1526999. {{avail-uw|https://doi.org/10.1145/1526993.1526999}}
* Schneider, Steven M., and Kirsten A. Foot. 2004. “The Web as an Object of Study.” New Media & Society 6 (1): 114–22. https://doi.org/10.1177/1461444804039912. {{avail-uw|https://doi.org/10.1177/1461444804039912}}
* Weber, Matthew S. 2014. “Observing the Web by Understanding the Past: Archival Internet Research.” In Proceedings of the Companion Publication of the 23rd International Conference on World Wide Web Companion, 1031–1036. WWW Companion ’14. Republic and Canton of Geneva, Switzerland: International World Wide Web Conferences Steering Committee. https://doi.org/10.1145/2567948.2579213. {{avail-uw|https://doi.org/10.1145/2567948.2579213}}

'''Optional readings related to the ethics of data collection online:'''

* Amy Bruckman's two 2016 blog posts about researchers violating terms of Service (TOS) while doing academic research: [https://nextbison.wordpress.com/2016/02/26/tos/ Do Researchers Need to Abide by Terms of Service (TOS)? An Answer.] and [https://nextbison.wordpress.com/2016/02/29/tos2/ More on TOS: Maybe Documenting Intent Is Not So Smart]
* [http://www.copyright.gov/legislation/dmca.pdf Digital Millenium Copyright Act] and these explanatory/commentary essays & sites:
** The [https://www.eff.org/ Electronic Frontier Foundation's] [https://www.eff.org/issues/dmca page on the DMCA].
** Templeton, Brad's [http://www.templetons.com/brad/copyright.html A Brief Intro to Copyright] & [http://www.templetons.com/brad/copymyths.html 10 Big Myths about Copyright Explained]
** Sections on Copyright, Privacy, and Social Media in the “Internet Case Digest” of the [http://www.perkinscoie.com/casedigest/ Perkins Coie LLP “Case Digest” site].
* Narayanan, A., and V. Shmatikov. 2008. “Robust De-Anonymization of Large Sparse Datasets.” In IEEE Symposium on Security and Privacy, 2008. SP 2008, 111–25. https://doi.org/10.1109/SP.2008.33. {{avail-uw|https://doi.org/10.1109/SP.2008.33}}

'''Two useful sources of data collection:'''

* [http://www.archiveteam.org/index.php?title=Main_Page Archive Team] is an online community that archives websites. They are a fantastic resource and include many pieces of detailed technical documentation on the practice of engaging in web archiving. For example, here are detailed explanations of [http://www.archiveteam.org/index.php?title=Wget#Mirroring_a_website mirroring a website with GNU wget] which is the piece of free software I usually use to archive websites.
* [https://www.openhumans.org/ OpenHumans] is an online community where people share personal data with each other and with researchers.

=== Wednesday April 6: Digital & Trace Ethnography ===

'''Required Readings:'''

More traditional ethnographic research in online settings:

* Hine, Christine. 2017. “Ethnographies of Online Communities and Social Media: Modes, Varieties, Affordances.” In The SAGE Handbook of Online Research Methods, edited by Nigel G. Fielding, Raymond M. Lee, and Grant Blank, 2 edition, 401–15. London, UK: SAGE. {{avail-canvas|1=https://canvas.uw.edu/files/90060250/download?download_frd=1}}
* [Selections] Jemielniak, Dariusz. 2014. Common Knowledge?: An Ethnography of Wikipedia. Stanford, California: Stanford University Press. ''["Introduction" and "Appendix A: Methodology"] {{avail-canvas|1=https://canvas.uw.edu/files/90060244/download?download_frd=1}}

Material on "Trace" and "network" ethnographies:

* Geiger, R. Stuart, and David Ribes. 2011. “Trace Ethnography: Following Coordination Through Documentary Practices.” In Proceedings of the 2011 44th Hawaii International Conference on System Sciences, 1–10. HICSS ’11. Washington, DC, USA: IEEE Computer Society. https://doi.org/10.1109/HICSS.2011.455. {{avail-canvas|1=https://canvas.uw.edu/files/90060247/download?download_frd=1}}
* Geiger, R. Stuart, and Aaron Halfaker. 2017. “Operationalizing Conflict and Cooperation between Automated Software Agents in Wikipedia: A Replication and Expansion of ‘Even Good Bots Fight.’” Proceedings of the ACM on Human-Computer Interaction 1 (CSCW): 49:1–49:33. https://doi.org/10.1145/3134684. {{avail-uw|https://doi.org/10.1145/3134684}}
* Howard, Philip N. 2002. “Network Ethnography and the Hypermedia Organization: New Media, New Organizations, New Methods.” New Media & Society 4 (4): 550–74. https://doi.org/10.1177/146144402321466813. {{avail-uw|https://doi.org/10.1177/146144402321466813}}

'''Optional Readings:'''

* Hine, Christine. 2000. Virtual Ethnography. London, UK: SAGE Publications. ''[Available from the Instructor]''
:This is the canonical book-length account and ''the'' main citation in this space.
*  Coleman, E. Gabriella. 2010. “Ethnographic Approaches to Digital Media.” Annual Review of Anthropology 39 (1): 487–505. https://doi.org/10.1146/annurev.anthro.012809.104945. {{avail-uw|https://doi.org/10.1146/annurev.anthro.012809.104945}}
* Response by danah boyd To Hine's "Question One: How Can Qualitative Internet Researchers Define the Boundaries of Their Projects?" from Internet Inquiry: Conversations About Method, Annette Markham and Nancy Baym (Eds.), Sage, 2009, pp. 1-32. {{avail-canvas|1=https://canvas.uw.edu/files/90060245/download?download_frd=1}}
:Note: You may also be interest in reading the essay by Hine that boyd is responding to. {{avail-canvas|1=https://canvas.uw.edu/files/90060246/download?download_frd=1}}
* Hjorth, Larissa, Heather Horst, Anne Galloway, and Genevieve Bell, eds. 2016. The Routledge Companion to Digital Ethnography. New York, NY: Routledge. ''[Available from the instructor]''
* Sinanan, Jolynna, and Tom McDonald. 2018. “Ethnography.” In The SAGE Handbook of Social Media, 179–95. 55 City Road: SAGE Publications Ltd. https://doi.org/10.4135/9781473984066. {{avail-uw|https://doi.org/10.4135/9781473984066}}
* Maxwell, Joseph A. 2002. “Understanding and Validity in Qualitative Research.” In The Qualitative Researcher’s Companion, edited by A. M. Huberman and Matthew B. Miles, 37–64. London, UK: SAGE. {{avail-canvas|1=https://canvas.uw.edu/files/90060242/download?download_frd=1}}
* Champion, Kaylea, Nora McDonald, Stephanie Bankes, Joseph Zhang, Rachel Greenstadt, Andrea Forte, and Benjamin Mako Hill. 2019. “A Forensic Qualitative Analysis of Contributions to Wikipedia from Anonymity Seeking Users.” Proceedings of the ACM on Human-Computer Interaction 3 (CSCW): 53:1–53:26. https://doi.org/10.1145/3359155. {{avail-uw|https://doi.org/10.1145/3359155}}

These are all other interesting and/or frequently cited examples of Internet-based ethnographies:

* Geiger, R. Stuart, and David Ribes. 2010. “The Work of Sustaining Order in Wikipedia:The Banning of a Vandal.” In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work, 117–126. CSCW ’10. New York, NY, USA: ACM. https://doi.org/10.1145/1718918.1718941. {{avail-uw|https://doi.org/10.1145/1718918.1718941}} — A trace ethnography and sort of the companion paper/substantive paper for the methods piece included in the required readings above.
* Brotsky, Sarah R., and David Giles. 2007. “Inside the ‘Pro-Ana’ Community: A Covert Online Participant Observation.” Eating Disorders 15 (2): 93–109. https://doi.org/10.1080/10640260701190600. {{avail-uw|https://doi.org/10.1080/10640260701190600}}
:Note: To conduct the study reported in this paper the authors created a used a fake profile in order to observe the psychological support offered to participants.
* Williams, Matthew. 2007. “Avatar Watching: Participant Observation in Graphical Online Environments.” Qualitative Research 7 (1): 5–24. https://doi.org/10.1177/1468794107071408. {{avail-uw|https://doi.org/10.1177/1468794107071408}}
: Note: Fantastic more general introduction but takeaways that are more specifically targeted toward people studying virtual reality type environments with virtual physicality.

Charlie's optional readings (virtual world ethnographies):

* Bainbridge, William Sims. 2010. The Warcraft Civilization: Social Science in a Virtual World. Cambridge, Massachusetts: MIT. [mitpress https://mitpress.mit.edu/books/warcraft-civilization]
* Nardi, Bonnie A. 2009. My Life as a Night Elf Priest: An Anthropological Account of World of Warcraft. Ann Arbor, Michigan: University of Michigan. {{avail-free|https://muse.jhu.edu/book/1093}}
* Pearce, Celia, Tom Boellstorff, and Bonnie A. Nardi. 2011. Communities of Play: Emergent Cultures in Multiplayer Games and Virtual Worlds. The MIT Press. [mitpress https://mitpress.mit.edu/books/communities-play]
* Boellstorff, Tom, Bonnie Nardi, Celia Pearce, T. L. Taylor, and George E. Marcus. 2012. Ethnography and Virtual Worlds: A Handbook of Method. Princeton: Princeton University Press. [https://press.princeton.edu/books/hardcover/9780691149509/ethnography-and-virtual-worlds]

=== Monday April 11: Online Interviewing ===

'''Required Readings:'''

* O’Connor, Henrietta, and Clare Madge. 2017. “Internet-Based Interviewing.” In The SAGE Handbook of Online Research Methods, edited by Nigel G. Fielding, Raymond M. Lee, and Grant Blank, 2 edition, 416–34. London, UK: SAGE. {{avail-canvas|1=https://canvas.uw.edu/files/90060243/download?download_frd=1}}
* Abrams, Katie M ., and Ted J. Gaiser. 2017. “Online Focus Groups.” In The SAGE Handbook of Online Research Methods, edited by Nigel G. Fielding, Raymond M. Lee, and Grant Blank, 2 edition, 435–50. London, UK: SAGE. {{avail-uw|1=https://canvas.uw.edu/files/90060230/download?download_frd=1}}
* Hanna, Paul. 2012. “Using Internet Technologies (Such as Skype) as a Research Medium: A Research Note.” Qualitative Research 12 (2): 239–42. https://doi.org/10.1177/1468794111426607. {{avail-uw|https://doi.org/10.1177/1468794111426607}}
: Note: Short article you can basically skim. Read it quickly so you can cite it later.
* Dowling, Sally. 2012. “Online Asynchronous and Face-to-Face Interviewing: Comparing Methods for Exploring Women’s Experiences of Breastfeeding Long Term.” In Cases in Online Interview Research, edited by Janet Salmons, 277–303. 2455 Teller Road,  Thousand Oaks  California  91320  United States: SAGE Publications, Inc. http://srmo.sagepub.com/view/cases-in-online-interview-research/n11.xml. {{avail-uw|http://srmo.sagepub.com/view/cases-in-online-interview-research/n11.xml}}

'''Optional Readings:'''

* boyd, danah. 2015. “Making Sense of Teen Life: Strategies for Capturing Ethnographic Data in a Networked Era.” In Digital Research Confidential: The Secrets of Studying Behavior Online, edited by Eszter Hargittai and Christian Sandvig. Cambridge, Massachusetts: MIT Press. {{avail-uw|1=https://canvas.uw.edu/files/90060231/download?download_frd=1}} 
: Note: Strongly focused on ethnographic interviews with tons of very specific details. Fantastic article on interviewing, although perhaps a bit weak on Internet-specific advice.
* Markham, Annette N. 1998. “The Shifting Project, The Shifting Self.” In Life Online: Researching Real Experience in Virtual Space, 61–83. Rowman Altamira. ''[Available from instructor]''
: Note: One of the earliest books on online life and one of the earliest attempts to do online interviewing. This is dated, but highlight some important challenge.
* Hutchinson, Emma. 2016. “Digital Methods and Perpetual Reinvention? Asynchronous Interviewing and Photo Elicitation.” In Digital Methods for Social Science: An Interdisciplinary Guide to Research Innovation, edited by Helene Snee, Christine Hine, Yvette Morey, Steven Roberts, and Hayley Watson, 143–56. London: Palgrave Macmillan UK. https://doi.org/10.1057/9781137453662_9. {{avail-uw|https://doi.org/10.1057/9781137453662_9}}
* Hawkins, Janice. 2018. “The Practical Utility and Suitability of Email Interviews in Qualitative Research.” The Qualitative Report 23 (2). https://digitalcommons.odu.edu/nursing_fac_pubs/24. {{avail-free|https://digitalcommons.odu.edu/nursing_fac_pubs/24}}

'''Alternate Accounts:'''

These texts are largely redundant to the required texts above but do provide a different perspective and examples:

*Salmons, Janet. 2014. Qualitative Online Interviews: Strategies, Design, and Skills. SAGE Publications. ''[Preface, TOC, and Chapter 1]'' {{avail-uw|1=https://canvas.uw.edu/files/90060232/download?download_frd=1}}
: This is a book that lays out what claims to be a comprehensive account to online interviewing. I have the book and am happy to loan my copy to anybody in the class that thinks this will be a core part of their research.

'''Optional readings related to the ethics of identify subjects:'''

* Markham, Annette. 2012. “Fabrication as Ethical Practice.” Information, Communication & Society 15 (3): 334–53. https://doi.org/10.1080/1369118X.2011.641993. {{avail-uw|https://doi.org/10.1080/1369118X.2011.641993}}
* Trevisan, Filippo, and Paul Reilly. 2014. “Ethical Dilemmas in Researching Sensitive Issues Online: Lessons from the Study of British Disability Dissent Networks.” Information, Communication & Society 17 (9): 1131–46. https://doi.org/10.1080/1369118X.2014.889188. {{avail-uw|https://doi.org/10.1080/1369118X.2014.889188}}

=== Wednesday April 13: Discourse Analysis ===

'''Required Readings:'''

* Mitra, Ananda. 1999. “Characteristics of the WWW Text: Tracing Discursive Strategies.” Journal of Computer-Mediated Communication 5 (1): 0–0. https://doi.org/10.1111/j.1083-6101.1999.tb00330.x. {{avail-free|https://doi.org/10.1111/j.1083-6101.1999.tb00330.x}}
* Thurlow, Crispin. 2018. “Digital Discourse: Locating Language in New/Social Media.” In The SAGE Handbook of Social Media, edited by Jean Burgess, Alice Marwick, and Thomas Poell, 135–45. London, UK: SAGE. https://doi.org/10.4135/9781473984066. {{avail-uw|https://doi.org/10.4135/9781473984066}}
* Brock, André. 2018. “Critical Technocultural Discourse Analysis.” New Media & Society 20 (3): 1012–30. https://doi.org/10.1177/1461444816677532. {{avail-uw|https://doi.org/10.1177/1461444816677532}}
* Bouvier, Gwen, and David Machin. 2018. “Critical Discourse Analysis and the Challenges and Opportunities of Social Media.” Review of Communication 18 (3): 178–92. https://doi.org/10.1080/15358593.2018.1479881. {{avail-canvas|1=https://canvas.uw.edu/files/90628240/download?download_frd=1}}

'''Optional Readings:'''

* Kaun, Anne. 2010. “Open-Ended Online Diaries: Capturing Life as It Is Narrated.” International Journal of Qualitative Methods 9 (2): 133–48. https://doi.org/10.1177/160940691000900202. {{avail-uw|https://doi.org/10.1177/160940691000900202}}

=== Monday April 18: Content analysis ===

<!-- split this into a pure content analysis something that is more interpretive maybe with LDA? -->

'''Required Readings:'''

* McMillan, Sally J. 2000. “The Microscope and the Moving Target: The Challenge of Applying Content Analysis to the World Wide Web.” Journalism & Mass Communication Quarterly 77 (1): 80–98. https://doi.org/10.1177/107769900007700107. {{avail-uw|https://doi.org/10.1177/107769900007700107}}
* Zamith, Rodrigo, and Seth C. Lewis. 2015. “Content Analysis and the Algorithmic Coder: What Computational Social Science Means for Traditional Modes of Media Analysis.” The Annals of the American Academy of Political and Social Science 659 (1): 307–18. https://doi.org/10.1177/0002716215570576. {{avail-uw|https://doi.org/10.1177/0002716215570576}}
* Grimmer, Justin, and Brandon M. Stewart. 2013. “Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts.” Political Analysis, January, mps028. https://doi.org/10.1093/pan/mps028. {{avail-uw|https://doi.org/10.1093/pan/mps028}}
* DiMaggio, Paul, Manish Nag, and David Blei. 2013. “Exploiting Affinities between Topic Modeling and the Sociological Perspective on Culture: Application to Newspaper Coverage of U.S. Government Arts Funding.” Poetics, Topic Models and the Cultural Sciences, 41 (6): 570–606. https://doi.org/10.1016/j.poetic.2013.08.004. {{avail-uw|https://doi.org/10.1016/j.poetic.2013.08.004}}

'''Optional Readings:'''

* Trilling, Damian, and Jeroen G. F. Jonkman. 2018. “Scaling up Content Analysis.” Communication Methods and Measures 12 (2–3): 158–74. https://doi.org/10.1080/19312458.2018.1447655. {{avail-uw|https://doi.org/10.1080/19312458.2018.1447655}}
*  Leetaru, Kalev Hannes. 2011. Data Mining Methods for the Content Analyst: An Introduction to the Computational Analysis of Content. Routledge Communication Series. New York, NY: Taylor and Francis. ''[[https://ebookcentral.proquest.com/lib/washington/detail.action?docID=1075229 Available through UW libraries]]''.

I'm assuming you have at least a rough familiarity with content analysis as a methodology. If your not as comfortable with this, check out [https://en.wikipedia.org/wiki/Content_analysis content analysis|the content analysis Wikipedia article] to start. These help provide more of a background into content analysis (in general, and online):

* Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, Calif.: Sage Publications. ''[Available from Instructor]''
* Krippendorff, K. (2005). Content analysis: an introduction to its methodology. Thousand Oaks; London; New Delhi: Sage. ''[Available from Instructor]''

Examples of more traditional content analysis using online content:

* Trammell, K. D., Tarkowski, A., Hofmokl, J., & Sapp, A. M. (2006). [http://doi.org/10.1111/j.1083-6101.2006.00032.x Rzeczpospolita blogów (Republic of Blog): Examining Polish Bloggers Through Content Analysis.] Journal of Computer-Mediated Communication, 11(3), 702–722. {{avail-free|http://doi.org/10.1111/j.1083-6101.2006.00032.x}}
* Woolley, J. K., Limperos, A. M., & Oliver, M. B. (2010). [http://doi.org/10.1080/15205436.2010.516864 The 2008 Presidential Election, 2.0: A Content Analysis of User-Generated Political Facebook Groups.] Mass Communication and Society, 13(5), 631–652. {{avail-uw|http://doi.org/10.1080/15205436.2010.516864}}
* Maier, Daniel, A. Waldherr, P. Miltner, G. Wiedemann, A. Niekler, A. Keinert, B. Pfetsch, et al. 2018. “Applying LDA Topic Modeling in Communication Research: Toward a Valid and Reliable Methodology.” Communication Methods and Measures 12 (2–3): 93–118. https://doi.org/10.1080/19312458.2018.1430754. {{avail-uw|https://doi.org/10.1080/19312458.2018.1430754}}

A few other things related to topic modeling and sentiment analysis:

* Barberá, P., Bonneau, R., Egan, P., Jost, J. T., Nagler, J., & Tucker, J. (2014). [http://smapp.nyu.edu/SMAPP_Website_Papers_Articles/leadersAndFollowersMeasuringPolitical.pdf Leaders or Followers? Measuring Political Responsiveness in the US Congress Using Social Media Data.] Presented at the Annual Meeting of the American Political Science Association. {{avail-free|http://smapp.nyu.edu/SMAPP_Website_Papers_Articles/leadersAndFollowersMeasuringPolitical.pdf}}
* Feldman, Ronen. 2013. “Techniques and Applications for Sentiment Analysis.” Communications of the ACM 56 (4): 82–90. https://doi.org/10.1145/2436256.2436274. {{avail-uw|https://doi.org/10.1145/2436256.2436274}}
* Baumer, Eric P. S., David Mimno, Shion Guha, Emily Quan, and Geri K. Gay. 2017. “Comparing Grounded Theory and Topic Modeling: Extreme Divergence or Unlikely Convergence?” Journal of the Association for Information Science and Technology 68 (6): 1397–1410. https://doi.org/10.1002/asi.23786. {{avail-uw|https://doi.org/10.1002/asi.23786}}
* Rudkowsky, Elena, Martin Haselmayer, Matthias Wastian, Marcelo Jenny, Štefan Emrich, and Michael Sedlmair. 2018. “More than Bags of Words: Sentiment Analysis with Word Embeddings.” ''Communication Methods and Measures'' 12 (2–3): 140–57. https://doi.org/10.1080/19312458.2018.1455817. {{avail-uw|https://doi.org/10.1080/19312458.2018.1455817}}

=== Wednesday April 20: Social network analysis ===

'''Required Readings:'''

* Lazer, David. 2018. “Networks and Information Flow.” The Oxford Handbook of Networked Communication, April. https://doi.org/10.1093/oxfordhb/9780190460518.013.2. {{avail-uw|https://doi.org/10.1093/oxfordhb/9780190460518.013.2}}
* Garton, Laura, Caroline Haythornthwaite, and Barry Wellman. 1997. “Studying Online Social Networks.” Journal of Computer-Mediated Communication 3 (1): 0–0. https://doi.org/10.1111/j.1083-6101.1997.tb00062.x. {{avail-free|https://doi.org/10.1111/j.1083-6101.1997.tb00062.x}}
* Mislove, Alan, Massimiliano Marcon, Krishna P. Gummadi, Peter Druschel, and Bobby Bhattacharjee. 2007. “Measurement and Analysis of Online Social Networks.” In Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement, 29–42. IMC ’07. New York, NY, USA: ACM. https://doi.org/10.1145/1298306.1298311. {{avail-uw|https://doi.org/10.1145/1298306.1298311}}
* Shumate, Michelle, and Edward T. Palazzolo. 2010. “Exponential Random Graph (P*) Models as a Method for Social Network Analysis in Communication Research.” Communication Methods and Measures 4 (4): 341–71. https://doi.org/10.1080/19312458.2010.527869. {{avail-canvas|1=https://canvas.uw.edu/files/90060234/download?download_frd=1}}
* Foucault Welles, Brooke, Anthony Vashevko, Nick Bennett, and Noshir Contractor. 2014. “Dynamic Models of Communication in an Online Friendship Network.” Communication Methods and Measures 8 (4): 223–43. https://doi.org/10.1080/19312458.2014.967843. {{avail-canvas|1=https://canvas.uw.edu/files/90060236/download?download_frd=1}}
* Freelon, Deen. 2018. “Partition-Specific Network Analysis of Digital Trace Data.” The Oxford Handbook of Networked Communication, April. https://doi.org/10.1093/oxfordhb/9780190460518.013.3. {{avail-uw|https://doi.org/10.1093/oxfordhb/9780190460518.013.3}}

'''Optional Readings:'''

* Wellman, Barry. 2016. “Challenges in Collecting Personal Network Data: The Nature of Personal Network Analysis - Barry Wellman, 2007.” Field Methods, July. http://journals.sagepub.com/doi/10.1177/1525822X06299133. {{avail-uw|http://journals.sagepub.com/doi/10.1177/1525822X06299133}}
* Yang, Jaewon, and Jure Leskovec. 2015. “Defining and Evaluating Network Communities Based on Ground-Truth.” Knowledge and Information Systems 42 (1): 181–213. https://doi.org/10.1007/s10115-013-0693-z. {{avail-canvas|1=https://canvas.uw.edu/files/90060237/download?download_frd=1}}
* Centola, Damon. 2010. “The Spread of Behavior in an Online Social Network Experiment.” Science 329 (5996): 1194–97. https://doi.org/10.1126/science.1185231.  {{avail-uw|https://doi.org/10.1126/science.1185231}}
* '''[Example]''' Jackson, Sarah J., and Brooke Foucault Welles. 2015. “Hijacking #myNYPD: Social Media Dissent and Networked Counterpublics.” Journal of Communication 65 (6): 932–52. https://doi.org/10.1111/jcom.12185. {{avail-uw|https://doi.org/10.1111/jcom.12185}}

'''Network datasets:'''

* [https://snap.stanford.edu/data/ Stanford Large Network Dataset Collection] which contains a variety of network datasets. Many, but certainly not all, are social networks.

=== Monday April 25: Visual Analysis  ===

'''Required Readings:'''

* Faulkner, Simon, Farida Vis, and Francesco D’Orazio. 2018. “Analysing Social Media Images.” In The SAGE Handbook of Social Media, edited by Jean Burgess, Alice Marwick, and Thomas Poell, 160–78. London, UK: SAGE. https://doi.org/10.4135/9781473984066. {{avail-canvas|1=https://canvas.uw.edu/files/91222224/download?download_frd=1}}
* Casas, Andreu, and Nora Webb Williams. 2019. “Images That Matter: Online Protests and the Mobilizing Role of Pictures.” Political Research Quarterly 72 (2): 360–75. https://doi.org/10.1177/1065912918786805. {{avail-uw|https://doi.org/10.1177/1065912918786805}}
* Casas, Andreu, and Nora Webb Williams. 2017. “Computer Vision for Political Science Research: A Study of Online Protest Images.” In. College Park, PA: Pennsylvania State University. http://andreucasas.com/casas_webb_williams_NewFaces2017_images_as_data.pdf. {{avail-free|http://andreucasas.com/casas_webb_williams_NewFaces2017_images_as_data.pdf}}
* Hochman, Nadav, and Raz Schwartz. 2012. “Visualizing Instagram: Tracing Cultural Visual Rhythms.” In Sixth International AAAI Conference on Weblogs and Social Media. https://pdfs.semanticscholar.org/780d/c7ff86eb36731d5faa043ac635cbae6bbe45.pdf. {{avail-free|https://pdfs.semanticscholar.org/780d/c7ff86eb36731d5faa043ac635cbae6bbe45.pdf}}

'''Optional Readings:'''

* Torralba, Antonio. 2009. “Understanding Visual Scenes.” Tutorial presented at the NIPS, Vancouver, BC, Canada. http://videolectures.net/nips09_torralba_uvs/. {{avail-uw|http://videolectures.net/nips09_torralba_uvs/}}
: Note: This is a two-part (each part is one hour) lecture and tutorial by an expert in computer vision. I strongly recommend watching Part I. I think this gives you a good sense of the nature of the kinds of challenges that were (and still are) facing the field of computer vision and anybody trying to have their computer look at images.
* Hochman, Nadav, and Lev Manovich. 2013. “Zooming into an Instagram City: Reading the Local through Social Media.” First Monday 18 (7). https://firstmonday.org/article/view/4711/3698. {{avail-free|https://firstmonday.org/article/view/4711/3698}}

These five papers are all technical approaches to doing image classification using datasets from Internet-based datasets of images like Flickr, Google Image Search, Google Street View, or Instagram. Each of these describes interesting and challenges technical issues. If you're interested, it would be a great idea to read these to get a sense for the state of the art and what is and isn't possible:

* Jaffe, Alexandar, Mor Naaman, Tamir Tassa, and Marc Davis. 2006. “Generating Summaries and Visualization for Large Collections of Geo-Referenced Photographs.” In Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval, 89–98. MIR ’06. New York, NY, USA: ACM. https://doi.org/10.1145/1178677.1178692. {{avail-uw|https://doi.org/10.1145/1178677.1178692}}
* Simon, Ian, Noah Snavely, and Steven M. Seitz. 2007. “Scene Summarization for Online Image Collections.” In Computer Vision, IEEE International Conference On, 0:1–8. Los Alamitos, CA, USA: IEEE Computer Society. https://doi.org/10.1109/ICCV.2007.4408863. {{avail-free|https://doi.org/10.1109/ICCV.2007.4408863}}
* Crandall, David J., Lars Backstrom, Daniel Huttenlocher, and Jon Kleinberg. 2009. “Mapping the World’s Photos.” In Proceedings of the 18th International Conference on World Wide Web, 761–770. WWW ’09. New York, NY, USA: ACM. https://doi.org/10.1145/1526709.1526812. {{avail-uw|https://doi.org/10.1145/1526709.1526812}}
* San Pedro, Jose, and Stefan Siersdorfer. 2009. “Ranking and Classifying Attractiveness of Photos in Folksonomies.” In Proceedings of the 18th International Conference on World Wide Web, 771–780. WWW ’09. New York, NY, USA: ACM. https://doi.org/10.1145/1526709.1526813. {{avail-uw|https://doi.org/10.1145/1526709.1526813}}
* Doersch, Carl, Saurabh Singh, Abhinav Gupta, Josef Sivic, and Alexei A. Efros. 2012. “What Makes Paris Look like Paris?” ACM Trans. Graph. 31 (4): 101:1–101:9. https://doi.org/10.1145/2185520.2185597. {{avail-uw|https://doi.org/10.1145/2185520.2185597}}

=== Wednesday April 27: Design Research ===

'''Required Readings:'''

* Bernstein, Michael S., Mark S. Ackerman, Ed H. Chi, and Robert C. Miller. 2011. “The Trouble with Social Computing Systems Research.” In ''CHI ’11 Extended Abstracts on Human Factors in Computing Systems'', 389–98. CHI EA ’11. New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/1979742.1979618. {{avail-uw|https://doi.org/10.1145/1979742.1979618}}
* Ackerman, Mark S. 2000. “The Intellectual Challenge of CSCW: The Gap between Social Requirements and Technical Feasibility.” ''Human–Computer Interaction'' 15 (2–3): 179–203. https://doi.org/10.1207/S15327051HCI1523_5. {{avail-uw|https://doi.org/10.1207/S15327051HCI1523_5}}
* Gilbert, Eric. 2012. “Designing Social Translucence over Social Networks.” In ''Proceedings of the SIGCHI Conference on Human Factors in Computing Systems'', 2731–40. CHI ’12. New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/2207676.2208670. {{avail-uw|https://doi.org/10.1145/2207676.2208670}}
* Grevet, Catherine, and Eric Gilbert. 2015. “Piggyback Prototyping: Using Existing, Large-Scale Social Computing Systems to Prototype New Ones.” In ''Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems'', 4047–56. CHI ’15. New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/2702123.2702395. {{avail-uw|https://doi.org/10.1145/2702123.2702395}}

'''Optional Readings:'''

* Olsen, Dan R., Jr. 2007. “Evaluating User Interface Systems Research.” In ''Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology'', 251–58. UIST ’07. New York, NY, USA: ACM. https://doi.org/10.1145/1294211.1294256. {{Avail-uw|https://doi.org/10.1145/1294211.1294256}}
* Grudin, Jonathan. 1988. “Why CSCW Applications Fail: Problems in the Design and Evaluation of Organizational Interfaces.” In ''Proceedings of the 1988 ACM Conference on Computer-Supported Cooperative Work'', 85–93. CSCW ’88. New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/62266.62273. {{avail-uw|https://doi.org/10.1145/62266.62273}}
* Zhang, Amy X., Grant Hugh, and Michael S. Bernstein. 2020. “PolicyKit: Building Governance in Online Communities.” In ''Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology'', 365–78. New York, NY, USA: Association for Computing Machinery. https://doi.org/10.1145/3379337.3415858. {{avail-uw|https://doi.org/10.1145/62266.62273}}

=== Monday May 2: Consulting Day ===

We will not meet together as a group today. Instead, I will schedule one-on-one in-person meetings of an hour with each student individually to catch up with you about your project and to work directly with you to resolve any technical issues you have run into with data collected.

=== Wednesday May 4: Consulting Day ===

We will not meet together as a group today. Instead, I will schedule one-on-one in-person meetings of an hour with each student individually to catch up with you about your project and to work directly with you to resolve any technical issues you have run into with data collected.

=== Monday May 9: Class Cancelled ===

Class was cancelled due to health issues.

=== Wednesday May 11: Experiments ===


'''Required Readings:'''

* Reips, Ulf-Dietrich. 2002. “Standards for Internet-Based Experimenting.” Experimental Psychology 49 (4): 243–56. https://doi.org/10.1026//1618-3169.49.4.243. {{avail-uw|https://doi.org/10.1026//1618-3169.49.4.243}}
* Salganik, Matthew J., Peter Sheridan Dodds, and Duncan J. Watts. 2006. “Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market.” Science 311 (5762): 854–56. https://doi.org/10.1126/science.1121066. {{avail-uw|https://doi.org/10.1126/science.1121066}}
* Hergueux, Jérôme, and Nicolas Jacquemet. 2014. “Social Preferences in the Online Laboratory: A Randomized Experiment.” Experimental Economics 18 (2): 251–83. https://doi.org/10.1007/s10683-014-9400-5. {{avail-uw|https://doi.org/10.1007/s10683-014-9400-5}}
* Rijt, Arnout van de, Soong Moon Kang, Michael Restivo, and Akshay Patil. 2014. “Field Experiments of Success-Breeds-Success Dynamics.” Proceedings of the National Academy of Sciences 111 (19): 6934–39. https://doi.org/10.1073/pnas.1316836111. {{avail-uw|https://doi.org/10.1073/pnas.1316836111}}
* Narayan, Sneha, Nathan TeBlunthuis, Wm Salt Hale, Benjamin Mako Hill, and Aaron Shaw. 2019. “All Talk: How Increasing Interpersonal Communication on Wikis May Not Enhance Productivity.” Proceedings of the ACM on Human-Computer Interaction 3 (CSCW): 101:1–101:19. https://doi.org/10.1145/3359203. {{avail-uw|https://doi.org/10.1145/3359203}}

'''Optional Readings:'''

* Eckles, Dean, Brian Karrer, and Johan Ugander. 2017. “Design and Analysis of Experiments in Networks: Reducing Bias from Interference.” Journal of Causal Inference 5 (1). https://doi.org/10.1515/jci-2015-0021. {{avail-uw|https://doi.org/10.1515/jci-2015-0021}}
: This piece is set as the intersection of networks and experiments. It's very important but is probably too technical to assign for the whole c.ass
* Kohavi, Ron, Alex Deng, Brian Frasca, Toby Walker, Ya Xu, and Nils Pohlmann. 2013. “Online Controlled Experiments at Large Scale.” In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1168–1176. KDD ’13. Chicago, Illinois, USA: Association for Computing Machinery. https://doi.org/10.1145/2487575.2488217. {{avail-uw|https://doi.org/10.1145/2487575.2488217}}
* Reinecke, Katharina, and Krzysztof Z. Gajos. 2015. “LabintheWild: Conducting Large-Scale Online Experiments With Uncompensated Samples.” In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing, 1364–1378. CSCW ’15. New York, NY, USA: ACM. https://doi.org/10.1145/2675133.2675246. {{avail-uw|https://doi.org/10.1145/2675133.2675246}}
* Zhu, Haiyi, Amy Zhang, Jiping He, Robert E. Kraut, and Aniket Kittur. 2013. “Effects of Peer Feedback on Contribution: A Field Experiment in Wikipedia.” In , 2253. ACM Press. https://doi.org/10.1145/2470654.2481311. {{avail-uw|https://doi.org/10.1145/2470654.2481311}}
* Zhang, Xiaoquan (Michael), and Feng Zhu. 2011. “Group Size and Incentives to Contribute: A Natural Experiment at Chinese Wikipedia.” American Economic Review 101 (4): 1601–15. https://doi.org/10.1257/aer.101.4.1601. {{avail-uw|https://doi.org/10.1257/aer.101.4.1601}}
* Weninger, Tim, Thomas James Johnston, and Maria Glenski. 2015. “Random Voting Effects in Social-Digital Spaces: A Case Study of Reddit Post Submissions.” Pp. 293–297 in Proceedings of the 26th ACM Conference on Hypertext & Social Media, HT ’15. Guzelyurt, Northern Cyprus: Association for Computing Machinery.

=== Monday May 16: Surveys ===


'''Required Readings:'''

* Fricker, Jr., Ronald D., and Katja Lozar Manfreda. 2017. “Sampling Methods for Online Surveys.” In The SAGE Handbook of Online Research Methods, edited by Nigel G. Fielding, Raymond M. Lee, and Grant Blank, 2 edition, 162–83. London, UK: SAGE. {{avail-canvas|1=https://canvas.uw.edu/files/90060249/download?download_frd=1}}
* Walejko, Gina. 2009. “Online Survey: Instant Publication, Instant Mistake, All of the Above.” In Research Confidential: Solutions to Problems Most Social Scientists Pretend They Never Have, edited by Eszter Hargittai, 101–21. Ann Arbor, MI: University of Michigan Press. {{avail-canvas|1=https://canvas.uw.edu/files/90060253/download?download_frd=1}}
* Konstan, Joseph A., B. R. Simon Rosser, Michael W. Ross, Jeffrey Stanton, and Weston M. Edwards. 2005. “The Story of Subject Naught: A Cautionary but Optimistic Tale of Internet Survey Research.” Journal of Computer-Mediated Communication 10 (2): 00–00. https://doi.org/10.1111/j.1083-6101.2005.tb00248.x. {{avail-free|https://doi.org/10.1111/j.1083-6101.2005.tb00248.x}}
* Hill, Benjamin Mako, and Aaron Shaw. 2013. “The Wikipedia Gender Gap Revisited: Characterizing Survey Response Bias with Propensity Score Estimation.” PLoS ONE 8 (6): e65782. https://doi.org/10.1371/journal.pone.0065782. {{avail-free|https://doi.org/10.1371/journal.pone.0065782}}
* Salganik, Matthew J., and Karen E. C. Levy. 2015. “Wiki Surveys: Open and Quantifiable Social Data Collection.” PLOS ONE 10 (5): e0123483. https://doi.org/10.1371/journal.pone.0123483. {{avail-free|https://doi.org/10.1371/journal.pone.0123483}}
: Note: [http://www.technologyreview.com/view/531696/inspired-by-wikipedia-social-scientists-create-a-revolution-in-online-surveys/ This journalistic account of the research] may also be useful.
* Alperin, Juan Pablo, Erik Warren Hanson, Kenneth Shores, and Stefanie Haustein. 2017. “Twitter Bot Surveys: A Discrete Choice Experiment to Increase Response Rates.” In Proceedings of the 8th International Conference on Social Media & Society, 1–4. #SMSociety17. Toronto, ON, Canada: Association for Computing Machinery. https://doi.org/10.1145/3097286.3097313. {{avail-uw|https://doi.org/10.1145/3097286.3097313}}

'''Optional Readings:'''

* Van Selm, Martine, and Nicholas W. Jankowski. 2006. “Conducting Online Surveys.” Quality and Quantity 40 (3): 435–56. https://doi.org/10.1007/s11135-005-8081-8. {{avail-uw|https://doi.org/10.1007/s11135-005-8081-8}}
* Vehovar, Vasja, and Katja Lozar Manfreda. 2017. “Overview: Online Surveys.” In The SAGE Handbook of Online Research Methods, edited by Nigel G. Fielding, Raymond M. Lee, and Grant Blank, 2 edition, 143–61. London, UK: SAGE. {{avail-canvas|1=https://canvas.uw.edu/files/90060233/download?download_frd=1}}
* Kaczmirek, Lars. 2017. “Online Survey Software.” In The SAGE Handbook of Online Research Methods, edited by Nigel G. Fielding, Raymond M. Lee, and Grant Blank, 2 edition, 203–19. London, UK: SAGE. {{avail-canvas|1=https://canvas.uw.edu/files/90060248/download?download_frd=1}}
* Toepoel, Vera. 2017. “Online Survey Design.” In The SAGE Handbook of Online Research Methods, edited by Nigel G. Fielding, Raymond M. Lee, and Grant Blank, 2 edition, 184–202. London, UK: SAGE. {{avail-canvas|1=https://canvas.uw.edu/files/90060252/download?download_frd=1}}
* Mavletova, Aigul, and Mick P. Couper. 2014. “Mobile Web Survey Design: Scrolling versus Paging, SMS versus E-Mail Invitations.” Journal of Survey Statistics and Methodology 2 (4): 498–518. https://doi.org/10.1093/jssam/smu015. {{avail-uw|https://doi.org/10.1093/jssam/smu015}}
* Yun, Gi Woong, and Craig W. Trumbo. 2000. “Comparative Response to a Survey Executed by Post, e-Mail, & Web Form.” Journal of Computer-Mediated Communication 6 (1): 0–0. https://doi.org/10.1111/j.1083-6101.2000.tb00112.x. {{avail-free|https://doi.org/10.1111/j.1083-6101.2000.tb00112.x}}
* Hargittai, Eszter, and Chris Karr. 2009. “WAT R U DOIN? Studying the Thumb Generation Using Text Messaging.” In Research Confidential: Solutions to Problems Most Social Scientists Pretend They Never Have, edited by Eszter Hargittai, 192–216. Ann Arbor, MI: University of Michigan Press. {{avail-canvas|1=https://canvas.uw.edu/files/90060251/download?download_frd=1}}

If you don't have a background in survey design, these two have been recommended by our guest speaker as good basic things to read:

* Krosnick, Jon A. 1999. “Maximizing Measurement Quality: Principles of Good Questionnaire Design.” In Measures of Political Attitudes, edited by John P. Robinson, Phillip R. Shaver, and Lawrence S. Wrightsman. New York: Academic Press.
* Krosnick, Jon A. 1999. “Survey Research.” Annual Review of Psychology 50 (1): 537–67. https://doi.org/10.1146/annurev.psych.50.1.537. {{avail-uw|https://doi.org/10.1146/annurev.psych.50.1.537}}

Tools for doing mobile surveys:

* [https://www.rapidsms.org/ RapidSMS]
* [https://www.twilio.com/ Twilio]

=== Wednesday May 18: Digital Trace and Sensor Data ===

'''Required Readings:'''

* Müller, Jörg, Sergi Fàbregues, Elisabeth Anna Guenther, and María José Romano. 2019. “Using Sensors in Organizational Research—Clarifying Rationales and Validation Challenges for Mixed Methods.” ''Frontiers in Psychology'' 10. https://www.frontiersin.org/article/10.3389/fpsyg.2019.01188. {{avail-free|https://www.frontiersin.org/article/10.3389/fpsyg.2019.01188}}
* Eagle, Nathan. 2011. “Mobile Phones as Sensors for Social Research.” In ''The Handbook of Emergent Technologies in Social Research'', 492–521. Oxford, UK: Oxford University Press. {{avail-canvas|1=https://canvas.uw.edu/files/92130482/download?download_frd=1}}
* Jiang, Jie, Riccardo Pozza, Nigel Gilbert, and Klaus Moessner. 2020. “MakeSense: An IoT Testbed for Social Research of Indoor Activities.” ''ACM Transactions on Internet of Things'' 1 (3): 17:1-17:25. https://doi.org/10.1145/3381914. {{avail-free|https://doi.org/10.1145/3381914}}
:'''''Note:''' I'm mostly thinking this is a useful example a sort of home/IoT based approach to sensors. There's a bunch of technical detail on the system here but please skip/skim the detail here.''
* Greshake Tzovaras, Bastian, Misha Angrist, Kevin Arvai, Mairi Dulaney, Vero Estrada-Galiñanes, Beau Gunderson, Tim Head, et al. 2019. “Open Humans: A Platform for Participant-Centered Research and Personal Data Exploration.” ''GigaScience'' 8 (6): giz076. https://doi.org/10.1093/gigascience/giz076. {{avail-free|https://doi.org/10.1093/gigascience/giz076}} 

'''Optional:'''

* Blumenstock, Joshua, Gabriel Cadamuro, and Robert On. 2015. “Predicting Poverty and Wealth from Mobile Phone Metadata.” ''Science'' 350 (6264): 1073–76. https://doi.org/10.1126/science.aac4420. {{avail-uw|https://doi.org/10.1126/science.aac4420}}
* Menchen-Trevino, Ericka. 2018. “Digital Trace Data and Social Research: A Proactive Research Ethics.” Edited by Brooke Foucault Welles and Sandra González-Bailón. In ''The Oxford Handbook of Networked Communication.'' Oxford, UK: Oxford University Press. https://doi.org/10.1093/oxfordhb/9780190460518.013.25. {avail-uw|https://doi.org/10.1093/oxfordhb/9780190460518.013.25}}
* Rom, Adina, Isabel Günther, and Yael Borofsky. 2020. “Using Sensors to Measure Technology Adoption in the Social Sciences.” ''Development Engineering'' 5 (January): 100056. https://doi.org/10.1016/j.deveng.2020.100056. {{avail-free|https://doi.org/10.1016/j.deveng.2020.100056}}
* Steele, Jessica E., Pål Roe Sundsøy, Carla Pezzulo, Victor A. Alegana, Tomas J. Bird, Joshua Blumenstock, Johannes Bjelland, et al. 2017. “Mapping Poverty Using Mobile Phone and Satellite Data.” ''Journal of The Royal Society Interface'' 14 (127): 20160690. https://doi.org/10.1098/rsif.2016.0690. {{avail-free|https://doi.org/10.1098/rsif.2016.0690}}
* Struminskaya, Bella, Peter Lugtig, Florian Keusch, and Jan Karem Höhne. 2020. “Augmenting Surveys with Data from Sensors and Apps: Opportunities and Challenges.” ''Social Science Computer Review'', December, 0894439320979951. https://doi.org/10.1177/0894439320979951. {{Avail-free|https://doi.org/10.1177/0894439320979951}}
* Wiebe, Douglas J., and Chistopher N. Morrison. 2018. “Digital Mapping of Urban Mobility Patterns.” Edited by Brooke Foucault Welles and Sandra González-Bailón. In ''The Oxford Handbook of Networked Communication.'' Oxford, UK: Oxford University Press. https://doi.org/10.1093/oxfordhb/9780190460518.013.25. {{avail-uw|https://doi.org/10.1093/oxfordhb/9780190460518.013.25}}

=== Monday May 23: Consulting Day ===

We will not meet together as a group today. Instead, I will schedule one-on-one in-person meetings of an hour with each student individually to catch up with you about your project and to work directly with you to resolve any technical issues you have run into with data collected.

=== Wednesday May 25: Consulting Day  ===

We will not meet together as a group today. Instead, I will schedule one-on-one in-person meetings of an hour with each student individually to catch up with you about your project and to work directly with you to resolve any technical issues you have run into with data collected.

=== Monday May 30: NO CLASS for Memorial Day ===
=== Wednesday June 1: Virtual Final Presentations ===

The plan for final presentations is as follows:

# Everybody should '''record and a share copy of their final presentation by Thursday June 2nd 11:59pm'''. 
# Everybody should '''view everybody else's presentations and give them feedback by Sunday June 5th 11:59pm'''.

I've pushed these deadlines back because my own travel schedule means I'm not likely to be able to review these before Friday June 3rd. I'm going to try to get everybody feedback by then.

==== Presentation content and form ====

Your projects are different stages so there be variation in terms of what is presented. That said, I expect nearly everyone will present one of two kinds of presentations:

# An overview and summary of your final project in its current state so that your classmates and I can give you feedback that is useful for your final written project due a week later. Present your research questions and context and walk us through the key deliverables and your current progress. Emphasize your methods since this is what we will be best positioned to provide you feedback. on If you have specific things you want feedback on, please communicate this during your talk and/or on Discord.
# If your project is a complete paper, you might want to instead do a full research presentation like what you would give at a conference. This would be fine as well.

'''Each presentation should between 8-12 minutes and absolutely not longer than 15m.''' I expect most people will use slides but walking through a posters could work too. I'm open/flexible and you're welcome to be creative.

==== Recording and sharing your presentation ====

My suggestion is that everybody share their presentation by placing a link to a video recording directly in the <code>#final-presentations</code> channel on Discord. Just create a new message in the channel.

There are many ways to record your presentation. Here are some ideas:

* Probably the easiest way is just to join the a Zoom room using your UW institutional Zoom account, sharing your screen, and recording it. If you ensure that you've enabled public link-sharing, you should be able to link directly to the Zoom recording. 
* Record using [https://obsproject.com/ Open Broadcasting Software (OBS)] which is used by lots of streamers.
* Try any number of other options (I put [[Online Communities (UW COM481 Winter 2022)/Final presentations#Presentation_Format:_Video_Pitches|a list]] together earlier this year).

Besides sharing directly from Zoom, you can share your file with Dropbox, Google Drive, an non-searchable Youtube video, etc.

==== Feedback ====

Once the videos are uploaded, everybody should watch every video and then provide feedback on Discord:

* My expectations is that everybody will write feedback to every classmates for 10-15 minutes.
* To leave feedback, leave it in [https://support.discord.com/hc/en-us/articles/4403205878423-Threads-FAQ a Discord thread] associated with each videos. The threads will be listed underneath the channel in the channel listing sidebar. If the thread doesn't exist yet, you can just mouse over the message in the main channel and create a thread. Let's name them something like "Mako's Presentation"

There will be 8 presentations (there is one 2-person project) so this will work out to a maximum of 2 hours watching videos and about 2 hours leaving feedback. Since I had planned to do two classes for final presentations, this works out about right. I understand that you'll have more feedback to give to some folks than others but do try to keep this time target in mind and do try to give feedback to everybody.

<!-- 
=== Crowdsourcing, Digital Labor Markets, and Human Computation ===

:'''Note:''' I've marked things as '''[Required]''' below if they are required because I thought it made more sense to keep the topics groups of articles below intact.

MTurk documentation and guidelines:

* '''[Required]''' [https://docs.aws.amazon.com/AWSMechTurk/latest/RequesterUI/Introduction.html Amazon Mechanical Turk Requester UI Guide] — ''Skim, but make sure you're ready to submit tasks.''
* '''[Required]''' [https://mturkpublic.s3.amazonaws.com/docs/MTURK_BP.pdf Amazon Mechanical Turk Best Practices Guide] — ''Skim, but make sure you're ready to submit tasks.''
* '''[Required]''' Shaw, Aaron. 2015. “Hired Hands and Dubious Guesses: Adventures in Crowdsourced Data Collection.” In Digital Research Confidential: The Secrets of Studying Behavior Online, edited by Eszter Hargittai and Christian Sandvig. The MIT Press. {{forthcoming}} 
* '''[Required]''' [https://blog.mturk.com/tutorials/home Tutorials Posted on the MTurk blog] — ''Skim and browse and pay attention to things that are like what you'd like to do in the class session.''
* '''[Required]''' [https://wearedynamo.fandom.com/wiki/Guidelines_for_Academic_Requesters Guidelines for Academic Requesters] and [https://wearedynamo.fandom.com/wiki/Basics_of_how_to_be_a_good_requester Basics of How to be a good Requester] from the ''We Are Dynamo'' — These sets of guidelines were created by Turkers as part of an effort to engage in collective actions and organizer of Turkers run by Niloufar Saleh in the paper below.
* Mason, Winter, and Siddharth Suri. 2011. “Conducting Behavioral Research on Amazon’s Mechanical Turk.” Behavior Research Methods 44 (1): 1–23. https://doi.org/10.3758/s13428-011-0124-6. {{avail-uw|https://doi.org/10.3758/s13428-011-0124-6}} — ''Dated but still somewhat useful.''

Overviews of MTurk and issues of data quality:

* Horton, John J., David G. Rand, and Richard J. Zeckhauser. 2011. “The Online Laboratory: Conducting Experiments in a Real Labor Market.” Experimental Economics 14 (3): 399–425. https://doi.org/10.1007/s10683-011-9273-9. {{avail-uw|https://doi.org/10.1007/s10683-011-9273-9}}
* Buhrmester, Michael, Tracy Kwang, and Samuel D. Gosling. 2011. “Amazon’s Mechanical Turk: A New Source of Inexpensive, yet High-Quality, Data?” Perspectives on Psychological Science, February. https://doi.org/10.1177/1745691610393980. {{avail-uw|https://doi.org/10.1177/1745691610393980}}
* Casler, Krista, Lydia Bickel, and Elizabeth Hackett. 2013. “Separate but Equal? A Comparison of Participants and Data Gathered via Amazon’s MTurk, Social Media, and Face-to-Face Behavioral Testing.” Computers in Human Behavior 29 (6): 2156–60. https://doi.org/10.1016/j.chb.2013.05.009. {{avail-uw|https://doi.org/10.1016/j.chb.2013.05.009}}
* '''[Required]''' Weinberg, Jill, Jeremy Freese, and David McElhattan. 2014. “Comparing Data Characteristics and Results of an Online Factorial Survey between a Population-Based and a Crowdsource-Recruited Sample.” Sociological Science 1: 292–310. https://doi.org/10.15195/v1.a19. {{avail-free|https://doi.org/10.15195/v1.a19}}
* Kees, Jeremy, Christopher Berry, Scot Burton, and Kim Sheehan. 2017. “An Analysis of Data Quality: Professional Panels, Student Subject Pools, and Amazon’s Mechanical Turk.” Journal of Advertising 46 (1): 141–55. https://doi.org/10.1080/00913367.2016.1269304. {{avail-uw|https://doi.org/10.1080/00913367.2016.1269304}}
* '''[Required]''' Kennedy, Ryan, Scott Clifford, Tyler Burleigh, Ryan Jewell, and Philip Waggoner. 2018. “The Shape of and Solutions to the MTurk Quality Crisis.” SSRN Scholarly Paper ID 3272468. Rochester, NY: Social Science Research Network. https://papers.ssrn.com/abstract=3272468. {{avail-free|https://papers.ssrn.com/abstract=3272468}}

Culture and work conditions for Turkers:

* Irani, Lilly. 2015. “The Cultural Work of Microwork.” New Media & Society 17 (5): 720–39. https://doi.org/10.1177/1461444813511926. {{avail-uw|https://doi.org/10.1177/1461444813511926}}
* Kittur, Aniket, Jeffrey V. Nickerson, Michael Bernstein, Elizabeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease, and John Horton. 2013. “The Future of Crowd Work.” In Proceedings of the 2013 Conference on Computer Supported Cooperative Work, 1301–1318. CSCW ’13. San Antonio, Texas, USA: Association for Computing Machinery. https://doi.org/10.1145/2441776.2441923. {{avail-uw|https://doi.org/10.1145/2441776.2441923}}  {{avail-free|http://hci.stanford.edu/publications/2013/CrowdWork/futureofcrowdwork-cscw2013.pdf}}
* Gray, Mary L., Siddharth Suri, Syed Shoaib Ali, and Deepti Kulkarni. 2016. “The Crowd Is a Collaborative Network.” In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing, 134–147. CSCW ’16. San Francisco, California, USA: Association for Computing Machinery. https://doi.org/10.1145/2818048.2819942. {{avail-uw|https://doi.org/10.1145/2818048.2819942}}
* '''[Required]''' Semuels, Alana. 2018. “The Internet Is Enabling a New Kind of Poorly Paid Hell.” The Atlantic. January 23, 2018. https://www.theatlantic.com/business/archive/2018/01/amazon-mechanical-turk/551192/. {{avail-free|https://www.theatlantic.com/business/archive/2018/01/amazon-mechanical-turk/551192/}}

Systems to approve Turker experiences:

* Salehi, Niloufar, Lilly C. Irani, Michael S. Bernstein, Ali Alkhatib, Eva Ogbe, Kristy Milland, and Clickhappier. 2015. “We Are Dynamo: Overcoming Stalling and Friction in Collective Action for Crowd Workers.” In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 1621–1630. CHI ’15. Seoul, Republic of Korea: Association for Computing Machinery. https://doi.org/10.1145/2702123.2702508. {{avail-uw|https://doi.org/10.1145/2702123.2702508}}
* Irani, Lilly C., and M. Six Silberman. 2013. “Turkopticon: Interrupting Worker Invisibility in Amazon Mechanical Turk.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 611–620. CHI ’13. Paris, France: Association for Computing Machinery. https://doi.org/10.1145/2470654.2470742. {{avail-uw|https://doi.org/10.1145/2470654.2470742}}

'''Assignments to complete before class:'''

The first task is to complete a task a crowd worker:

* '''If you are a US citizen:''' Sign up as a worker on MTurk. Find and complete at least 5 "hits" as a worker on [http://mturk.com Amazon Mechanical Turk]. Note that to do this you will need to create a ''worker'' account on Mturk.
* '''If you are not a US citizen or if you cannot sign up on MTurk for some other reason:''' Complete at least 3-4 classification tasks in at least 2 different [https://www.zooniverse.org/projects Zooniverse projects] of your choice. Also, complete at least one "study" in [https://www.labinthewild.org/ Lab in the Wild] 
* In either case: Record (write down) details and notes about your tasks: What did you do? Who was the requester? What could you was the purpose of the task (as best you could tell)? What was the experience like? What research applications can you (not) imagine for this kind of system? 

The second task is to get ready to launch a task as a requestor. We will design and launch tasks in class but I want you to do the following ahead of time:

* Create a "requester" account on [http://mturk.com Amazon Mechnical Turk]. Doing so may require up top 48 hours to be approved so please do that immediately so you have it ready to go in class.
* Put money onto your requestor account to pay workers. A $5 budget should be sufficient for our class. They should take any payment that Amazon does.
* Think of at least one small classification or coding task (e.g., of Tweets, images, etc) and one human subjects data collection tasks like a survey, a survey experiment, etc, that you would like to run. You will have a budget of $5 to run the task!
* If running this task will involve some data (e.g., a set of images or URLs, a set of Tweets, etc), collect that material in a spreadsheet before class. If it will involve a survey, create your survey in a Google Form and/or a Survey Monkey or Qualtrics survey before class.

=== Hyperlink Networks {{tentative}} ===

* Park, Han Woo. 2003. “Hyperlink Network Analysis: A New Method for the Study of Social Structure on the Web.” Connections 25 (1): 49–61. ''[[https://canvas.uw.edu/files/61256175/download?download_frd=1 Available through Canvas]]''
* González-Bailón, Sandra. 2009. “Opening the Black Box of Link Formation: Social Factors Underlying the Structure of the Web.” Social Networks 31 (4): 271–80. https://doi.org/10.1016/j.socnet.2009.07.003. {{avail-uw|https://doi.org/10.1016/j.socnet.2009.07.003}}
* '''[Example]''' Elgin, Dallas J. 2015. “Utilizing Hyperlink Network Analysis to Examine Climate Change Supporters and Opponents.” Review of Policy Research 32 (2): 226–45. https://doi.org/10.1111/ropr.12118. {{avail-uw|https://doi.org/10.1111/ropr.12118}}

'''Optional readings:'''

* Jackson, Michele H. 1997. “Assessing the Structure of Communication on the World Wide Web.” Journal of Computer-Mediated Communication 3 (1). https://doi.org/10.1111/j.1083-6101.1997.tb00063.x. {{avail-free|https://doi.org/10.1111/j.1083-6101.1997.tb00063.x}}
* Ackland, Robert. 2016. “WWW Hyperlink Networks.” Lecture Slides presented at the SOCR8006 Online Research Methods, Canberra, Australia. http://vosonlab.net/papers/ACSPRISummer2017/Lecture_Hyperlink_Networks.pdf. {{avail-free|http://vosonlab.net/papers/ACSPRISummer2017/Lecture_Hyperlink_Networks.pdf}}
* Lusher, Dean, and Robert Ackland. 2011. “A Relational Hyperlink Analysis of an Online Social Movement.” Journal of Social Software 12 (5). https://www.cmu.edu/joss/content/articles/volume12/Lusher/. {{avail-free|https://www.cmu.edu/joss/content/articles/volume12/Lusher/}}
* '''[Example]''' Shumate, Michelle, and Lori Dewitt. 2008. “The North/South Divide in NGO Hyperlink Networks.” Journal of Computer-Mediated Communication 13 (2): 405–28. https://doi.org/10.1111/j.1083-6101.2008.00402.x. {{avail-free|https://doi.org/10.1111/j.1083-6101.2008.00402.x}}

'''Tools for collecting hyperlink network data:'''

* [http://www.govcom.org/Issuecrawler_instructions.htm Issue Crawler] — network mapping software by the Govcom.org Foundation, Amsterdam in a group run by Richard Rogers
* [http://vosonlab.net/VOSON Virtual Observatory for the Study of Online Networks (VOSON)] — "web-based software incorporating web mining, data visualisation, and traditional empirical social science methods (e.g. social network analysis, SNA). Text analysis, dataset manipulation and visualisation, and social network analysis (SNA) are available within an integrated environment."

-->