Editing Designing Internet Research (Winter 2020)
From CommunityData
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 21: | Line 21: | ||
* Describe particular challenges and threats to research validity associated with each method. | * Describe particular challenges and threats to research validity associated with each method. | ||
* For at least one method, be able to provide a detailed description of a research project and feel comfortable embarking on a formative study using this methodology. | * For at least one method, be able to provide a detailed description of a research project and feel comfortable embarking on a formative study using this methodology. | ||
* Given a manuscript (e.g., in the context of a request for peer review), be able to evaluate | * Given a manuscript (e.g., in the context of a request for peer review), be able to evaluate a Internet-based study in terms of its use its methodological choices. | ||
* Use a modern programming language (like Python) to collect a dataset from a web API like those published by Twitter, Reddit, or Wikipedia. | * Use a modern programming language (like Python) to collect a dataset from a web API like those published by Twitter, Reddit, or Wikipedia. | ||
Line 154: | Line 154: | ||
=== Participation === | === Participation === | ||
The course relies heavily on participation and discussion. It is important to realize that we will not summarize reading in class and I will not cover it in lecture. I expect you all to have read it and we will jump in and start discussing it. The "Participation Rubric" section of [ | The course relies heavily on participation and discussion. It is important to realize that we will not summarize reading in class and I will not cover it in lecture. I expect you all to have read it and we will jump in and start discussing it. The "Participation Rubric" section of [https://mako.cc/teaching/assessment.html my detailed page on assessment] gives the rubric I will use in evaluating participation. | ||
=== Assessment === | === Assessment === | ||
I have put together a very detailed page that describes [[ | I have put together a very detailed page that describes [[Assessment|the way I approach assessment and grading]]—both in general and in this course. Please read it carefully I will assign grades for each of following items on the UW 4.0 grade scale according to the weights below: | ||
* Participation: 30% | * Participation: 30% | ||
Line 336: | Line 336: | ||
More traditional ethnographic research in online settings: | More traditional ethnographic research in online settings: | ||
* Hines, Christine. 2017. “Ethnographies of Online Communities and Social Media: Modes, Varieties, Affordances.” In The SAGE Handbook of Online Research Methods, edited by Nigel G. Fielding, Raymond M. Lee, and Grant Blank, 2 edition, 401–15. London, UK: SAGE. | * Hines, Christine. 2017. “Ethnographies of Online Communities and Social Media: Modes, Varieties, Affordances.” In The SAGE Handbook of Online Research Methods, edited by Nigel G. Fielding, Raymond M. Lee, and Grant Blank, 2 edition, 401–15. London, UK: SAGE. | ||
* [Selections] Jemielniak, Dariusz. 2014. Common Knowledge?: An Ethnography of Wikipedia. Stanford, California: Stanford University Press. | * [Selections] Jemielniak, Dariusz. 2014. Common Knowledge?: An Ethnography of Wikipedia. Stanford, California: Stanford University Press. [https://canvas.uw.edu/files/36067463/download?download_frd=1 "Introduction" and "Appendix A: Methodology."] ''[Available in Canvas]'' | ||
Material on "Trace" and "network" ethnographies: | Material on "Trace" and "network" ethnographies: | ||
* Geiger, R. Stuart, and David Ribes. 2011. “Trace Ethnography: Following Coordination Through Documentary Practices.” In Proceedings of the 2011 44th Hawaii International Conference on System Sciences, 1–10. HICSS ’11. Washington, DC, USA: IEEE Computer Society. https://doi.org/10.1109/HICSS.2011.455. | * Geiger, R. Stuart, and David Ribes. 2011. “Trace Ethnography: Following Coordination Through Documentary Practices.” In Proceedings of the 2011 44th Hawaii International Conference on System Sciences, 1–10. HICSS ’11. Washington, DC, USA: IEEE Computer Society. https://doi.org/10.1109/HICSS.2011.455. | ||
* Geiger, R. Stuart, and Aaron Halfaker. 2017. “Operationalizing Conflict and Cooperation between Automated Software Agents in Wikipedia: A Replication and Expansion of ‘Even Good Bots Fight.’” Proceedings of the ACM on Human-Computer Interaction 1 (CSCW): 49:1–49:33. https://doi.org/10.1145/3134684. | * Geiger, R. Stuart, and Aaron Halfaker. 2017. “Operationalizing Conflict and Cooperation between Automated Software Agents in Wikipedia: A Replication and Expansion of ‘Even Good Bots Fight.’” Proceedings of the ACM on Human-Computer Interaction 1 (CSCW): 49:1–49:33. https://doi.org/10.1145/3134684. | ||
* Howard, Philip N. 2002. “Network Ethnography and the Hypermedia Organization: New Media, New Organizations, New Methods.” New Media & Society 4 (4): 550–74. https://doi.org/10.1177/146144402321466813. | * Howard, Philip N. 2002. “Network Ethnography and the Hypermedia Organization: New Media, New Organizations, New Methods.” New Media & Society 4 (4): 550–74. https://doi.org/10.1177/146144402321466813. | ||
'''Optional Readings:''' | '''Optional Readings:''' | ||
Line 349: | Line 349: | ||
* Hine, Christine. 2000. Virtual Ethnography. London, UK: SAGE Publications. ''[Available from the Instructor]'' | * Hine, Christine. 2000. Virtual Ethnography. London, UK: SAGE Publications. ''[Available from the Instructor]'' | ||
:This is the canonical book-length account and ''the'' main citation in this space. | :This is the canonical book-length account and ''the'' main citation in this space. | ||
* Coleman, E. Gabriella. 2010. “Ethnographic Approaches to Digital Media.” Annual Review of Anthropology 39 (1): 487–505. https://doi.org/10.1146/annurev.anthro.012809.104945. | * Coleman, E. Gabriella. 2010. “Ethnographic Approaches to Digital Media.” Annual Review of Anthropology 39 (1): 487–505. https://doi.org/10.1146/annurev.anthro.012809.104945. | ||
* [https://canvas.uw.edu/files/36079509/download?download_frd=1 Response by danah boyd To Hine's "Question One: How Can Qualitative Internet Researchers Define the Boundaries of Their Projects?"] from Internet Inquiry: Conversations About Method, Annette Markham and Nancy Baym (Eds.), Sage, 2009, pp. 1-32. ''[Available in Canvas]'' | |||
:Note: You may also be interest in reading | :Note: You may also be interest in reading [https://canvas.uw.edu/files/36079510/download?download_frd=1 the essay by Hine that boyd is responding to]. ''[Available in Canvas]'' | ||
* Hjorth, Larissa, Heather Horst, Anne Galloway, and Genevieve Bell, eds. 2016. The Routledge Companion to Digital Ethnography. New York, NY: Routledge. | * Hjorth, Larissa, Heather Horst, Anne Galloway, and Genevieve Bell, eds. 2016. The Routledge Companion to Digital Ethnography. New York, NY: Routledge. | ||
* Sinanan, Jolynna, and Tom McDonald. 2018. “Ethnography.” In The SAGE Handbook of Social Media, 179–95. 55 City Road: SAGE Publications Ltd. https://doi.org/10.4135/9781473984066. | * Sinanan, Jolynna, and Tom McDonald. 2018. “Ethnography.” In The SAGE Handbook of Social Media, 179–95. 55 City Road: SAGE Publications Ltd. https://doi.org/10.4135/9781473984066. | ||
* Maxwell, | |||
* Maxwell, J. A. 2002. “Understanding and Validity in Qualitative Research.” In The Qualitative Researcher’s Companion, edited by A. M. Huberman and Matthew B. Miles, 37–64. London, UK: SAGE. ''[Available in Canvas]'' | |||
These are all other interesting and/or frequently cited examples of Internet-based ethnographies: | These are all other interesting and/or frequently cited examples of Internet-based ethnographies: | ||
* Geiger, R. Stuart, and David Ribes. 2010. “The Work of Sustaining Order in Wikipedia:The Banning of a Vandal.” In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work, 117–126. CSCW ’10. New York, NY, USA: ACM. https://doi.org/10.1145/1718918.1718941. | * Geiger, R. Stuart, and David Ribes. 2010. “The Work of Sustaining Order in Wikipedia:The Banning of a Vandal.” In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work, 117–126. CSCW ’10. New York, NY, USA: ACM. https://doi.org/10.1145/1718918.1718941. — A trace ethnography. | ||
* Brotsky, Sarah R., and David Giles. 2007. “Inside the ‘Pro-Ana’ Community: A Covert Online Participant Observation.” Eating Disorders 15 (2): 93–109. https://doi.org/10.1080/10640260701190600. | * Brotsky, Sarah R., and David Giles. 2007. “Inside the ‘Pro-Ana’ Community: A Covert Online Participant Observation.” Eating Disorders 15 (2): 93–109. https://doi.org/10.1080/10640260701190600. | ||
:Note: To conduct the study reported in this paper the authors created a used a fake profile in order to observe the psychological support offered to participants. | :Note: To conduct the study reported in this paper the authors created a used a fake profile in order to observe the psychological support offered to participants. | ||
* Williams, Matthew. 2007. “Avatar Watching: Participant Observation in Graphical Online Environments.” Qualitative Research 7 (1): 5–24. https://doi.org/10.1177/1468794107071408. | * Williams, Matthew. 2007. “Avatar Watching: Participant Observation in Graphical Online Environments.” Qualitative Research 7 (1): 5–24. https://doi.org/10.1177/1468794107071408. | ||
: Note: Fantastic more general introduction but takeaways that are more specifically targeted toward people studying virtual reality type environments with virtual physicality. | : Note: Fantastic more general introduction but takeaways that are more specifically targeted toward people studying virtual reality type environments with virtual physicality. | ||
==== Part II: Online Interviewing ==== | ==== Part II: Online Interviewing ==== | ||
Line 376: | Line 369: | ||
'''Required Readings:''' | '''Required Readings:''' | ||
* O’Connor, | * O’Connor, H., Madge, C., Shaw, R., & Wellens, J. (2008). [http://srmo.sagepub.com/view/the-sage-handbook-of-online-research-methods/n15.xml Internet-based Interviewing]. In N. G. Fielding, R. M. Lee, & G. Blank (Eds.), The SAGE Handbook of Online Research Methods (pp. 271–289). London, UK: SAGE Publications, Ltd. ''[Available through UW Libraries]'' | ||
* Stewart, K., & Williams, M. (2005). [http://doi.org/10.1177/1468794105056916 Researching online populations: the use of online focus groups for social research.] Qualitative Research, 5(4), 395–416. | |||
* | * Hanna, P. (2012). [http://doi.org/10.1177/1468794111426607 Using internet technologies (such as Skype) as a research medium: a research note.] Qualitative Research, 12(2), 239–242. ''[Available through UW Libraries]'' | ||
: Note: Short article you can basically skim. Read it quickly so you can cite it later. | : Note: Short article you can basically skim. Read it quickly so you can cite it later. | ||
* Dowling, | * Dowling, S. (2012). [http://srmo.sagepub.com/view/cases-in-online-interview-research/n11.xml Online Asynchronous and Face-to-Face Interviewing: Comparing Methods for Exploring Women’s Experiences of Breastfeeding Long Term]. In Salmons, J. (Ed.), Cases in Online Interview Research (pp. 277–303). 2455 Teller Road, Thousand Oaks California 91320 United States: SAGE Publications, Inc. ''[Available through UW Libraries]'' | ||
'''Optional Readings:''' | '''Optional Readings:''' | ||
* boyd, danah. 2015. | * boyd, danah. (2015). [https://canvas.uw.edu/files/36133652/download?download_frd=1 Making sense of teen life: Strategies for capturing ethnographic data in a networked era.] In E. Hargittai & C. Sandvig (Eds.), Digital Research Confidential: The Secrets of Studying Behavior Online. Cambridge, Massachusetts: The MIT Press. ''[Available in Canvas]'' | ||
: Note: Strongly focused on | : Note: Strongly focused on enthnographic interviews with tons of very specific details. Fantastic article on interviewing, although perhaps a bit weak on Internet specific advice. | ||
* Markham, Annette | * Markham, Annette (1998), "[https://canvas.uw.edu/files/36133806/download?download_frd=1 The Shifting Project, the Shifting Self]," from Life Online, Altamira Press, 1998, pp. 61-84. ''[Available in Canvas]'' | ||
: Note: One of the earliest books on online life and one of the earliest attempts to do online interviewing. This is dated, but highlight some important challenge. | : Note: One of the earliest books on online life and one of the earliest attempts to do online interviewing. This is dated, but highlight some important challenge. | ||
* | * Stromer-Galley, Jennifer (2003), "[https://canvas.uw.edu/files/36133838/download?download_frd=1 Depth Interviews for the Study of Motives and Perceptions of Internet Use]," International Communication Association, San Diego, May. ''[Available in Canvas]'' | ||
* | : Note: Start reading on page 8 on "The Internet and the Interview". The beginning is a theoretical argument that's not really relevant to this class.* Chou, C. (2001). [http://online.liebertpub.com/doi/abs/10.1089/109493101753235160 Internet heavy use and addiction among Taiwanese college students: an online interview study.] CyberPsychology & Behavior, 4(5), 573-585. ''[Available through UW Libraries]'' | ||
'''Alternate Accounts:''' | '''Alternate Accounts:''' | ||
Line 395: | Line 388: | ||
These texts are largely redundant to the required texts above but do provide a different perspective and examples: | These texts are largely redundant to the required texts above but do provide a different perspective and examples: | ||
*Salmons, | * Salmons, J. (2014). Qualitative Online Interviews: Strategies, Design, and Skills. SAGE Publications. | ||
: | : This is a book that lays out what claims to be a comprehensive account to online interviewing. Take a quick through the [https://canvas.uw.edu/files/36133582/download?download_frd=1 preface and table of contents] and read [https://canvas.uw.edu/files/36133581/download?download_frd=1 Chapter 1]. ''[Both Available in Canvas.]'' | ||
: I have the book and am happy to loan my copy to anybody in the class that thinks this will be a core part of their research. | |||
* Morgan, David L. and Bojana Lobe, "[https://canvas.uw.edu/files/36133861/download?download_frd=1 Online focus groups]," Ch. 9 in HET. ''[Available in Canvas]'' | |||
* Gaiser, T. J. (2008). [http://srmo.sagepub.com/view/the-sage-handbook-of-online-research-methods/n16.xml Online Focus Groups]. In N. G. Fielding, R. M. Lee, & G. Blank (Eds.), The SAGE Handbook of Online Research Methods (pp. 290–307). London, UK: SAGE Publications, Ltd. ''[Available through UW Libraries]'' | |||
'''Optional readings related to the ethics of identify subjects:''' | '''Optional readings related to the ethics of identify subjects:''' | ||
* Markham, Annette. 2012. “Fabrication as Ethical Practice.” Information, Communication & Society 15 (3): 334–53. https://doi.org/10.1080/1369118X.2011.641993. | * Markham, Annette. 2012. “Fabrication as Ethical Practice.” Information, Communication & Society 15 (3): 334–53. https://doi.org/10.1080/1369118X.2011.641993. | ||
* Trevisan, Filippo, and Paul Reilly. 2014. “Ethical Dilemmas in Researching Sensitive Issues Online: Lessons from the Study of British Disability Dissent Networks.” Information, Communication & Society 17 (9): 1131–46. https://doi.org/10.1080/1369118X.2014.889188. | * Narayanan, A., and V. Shmatikov. 2008. “Robust De-Anonymization of Large Sparse Datasets.” In IEEE Symposium on Security and Privacy, 2008. SP 2008, 111–25. https://doi.org/10.1109/SP.2008.33. | ||
* Trevisan, Filippo, and Paul Reilly. 2014. “Ethical Dilemmas in Researching Sensitive Issues Online: Lessons from the Study of British Disability Dissent Networks.” Information, Communication & Society 17 (9): 1131–46. https://doi.org/10.1080/1369118X.2014.889188. | |||
* Bruckman, A., Luther, K., & Fiesler, C. (2015). [http://blog.kurtluther.com/pdf/bruckman_real_names.pdf When Should We Use Real Names in Published Accounts of Internet Research?] In Digital Research Confidential: The Secrets of Studying Behavior Online (pp. 243–259). Cambridge, Massachusetts: MIT Press. | |||
=== Week 4: Saturday February 1: CDSW Session 2 === | === Week 4: Saturday February 1: CDSW Session 2 === | ||
As | As description in the section on technical skills above, I expect everybody who is not comfortable with at least basic programming and data collection to attend the [[Community Data Science Workshops (Spring 2020)]] which I am running concurrently with this class. | ||
This session will run from | This session will run from 10am-4pm. Details on the [[CDSW Spring 2020]] page. | ||
=== Week 5: Tuesday February 4: (I) Surveys and (II) Experiments === | === Week 5: Tuesday February 4: (I) Surveys and (II) Experiments === | ||
Line 414: | Line 412: | ||
'''Required Readings:''' | '''Required Readings:''' | ||
* | * Van Selm, Martine & Nicholas Jankowski (2006), "[http://doi.org/10.1007/s11135-005-8081-8 Conducting Online Surveys]," Quality and Quantity, 40: 435-456. ''[Available through UW Libraries]'' | ||
* Walejko, Gina, "[https://canvas.uw.edu/files/36334690/download?download_frd=1 Online survey: Instant publication, instant mistake, all of the above]," from Research Confidential: Solutions to Problems Most Research Scientists Pretend They Never Have, University of Michigan Press, 2009, pp. 101-121. ''[Available in Canvas]'' | |||
* | * Joseph A. Konstan, B. R. Simon Rosser, Michael W. Ross, Jeffrey Stanton, & Weston M. Edwards, “[http://onlinelibrary.wiley.com/doi/10.1111/j.1083-6101.2005.tb00248.x/full The Story of Subject Naught: A Cautionary but Optimistic Tale of Internet Survey Research],” Journal of Computer-Mediated Communication, V.10, N. 2, January 2005. ''[Free Online]'' | ||
* Hill, B. M., & Shaw, A. (2013). [http://dx.doi.org/10.1371/journal.pone.0065782 The Wikipedia Gender Gap Revisited: Characterizing Survey Response Bias with Propensity Score Estimation]. PLoS ONE, 8(6), e65782. ''[Free Online]'' | |||
* Salganik, M. J., & Levy, K. E. C. (2015). [http://doi.org/10.1371/journal.pone.0123483 Wiki Surveys: Open and Quantifiable Social Data Collection]. PLOS ONE, 10(5), e0123483. ''[Free Online]'' | |||
: Note: [http://www.technologyreview.com/view/531696/inspired-by-wikipedia-social-scientists-create-a-revolution-in-online-surveys/ This journalistic account of the research] may also be useful. | : Note: [http://www.technologyreview.com/view/531696/inspired-by-wikipedia-social-scientists-create-a-revolution-in-online-surveys/ This journalistic account of the research] may also be useful. | ||
'''Optional Readings:''' | '''Optional Readings:''' | ||
If you don't have a background in survey design, these two have been recommended by our guest speaker as good basic things to read: | If you don't have a background in survey design, these two have been recommended by our guest speaker as good basic things to read: | ||
* Krosnick, | * Krosnick, J. A. (1999). [http://doi.org/10.1146/annurev.psych.50.1.537 Survey Research]. Annual Review of Psychology, 50(1), 537–567. ''[Available through UW Libraries]'' | ||
* Krosnick, J. A. (1999). Maximizing measurement quality: Principles of good questionnaire design. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of Political Attitudes. New York: Academic Press. | |||
These are other texts on the subject that you might find useful: | |||
* [https:// | * Dal, Michael, "[https://canvas.uw.edu/files/36335114/download?download_frd=1 Online data collection and data analysis using emergent technologies]," Ch. 12 in HET. ''[Available in Canvas]'' | ||
* [https:// | * Smith, Tom W. and John Sokolowski, "[https://canvas.uw.edu/files/36335113/download?download_frd=1 The use of audiovisuals in surveys]," Ch. 19 in HET. ''[Available in Canvas]'' | ||
* Kellock, Anne, et. al. "[https://canvas.uw.edu/files/36335148/download?download_frd=1 Using technology and the experience sampling method to understand real life]," Ch. 24 from HET. ''[Available in Canvas]'' | |||
* Yun, Gi Woong and Craig Trumbo, "[http://onlinelibrary.wiley.com/doi/10.1111/j.1083-6101.2000.tb00112.x/abstract Comparative Response to a Survey Executed by Post, E-mail and Web Form]," Journal of Computer-Mediated Communication, V.6, N.1, September, 2000. ''[Free Online]'' | |||
* Hargittai, Eszter, and Chris Karr, "[https://canvas.uw.edu/files/36334928/download?download_frd=1 WAT R U DOIN? Studying the Thumb Generation Using Text Messaging]," from Research Confidential: Solutions to Problems Most Research Scientists Pretend They Never Have, University of Michigan Press, 2009, pp. 192-216. ''[Available in Canvas]'' | |||
* Wright, Kevin, "[http://onlinelibrary.wiley.com/doi/10.1111/j.1083-6101.2005.tb00259.x/abstract Researching Internet-Based Populations: Advantages and Disadvantages of Online Survey Research, Online Questionnaire Authoring Software Packages, and Web Survey Services]," Journal of Computer-Mediated Communication, V. 10, N. 3, April 2005. ''[Free Online]'' | |||
==== Part II: Experiments ==== | ==== Part II: Experiments ==== | ||
Line 446: | Line 439: | ||
'''Required Readings:''' | '''Required Readings:''' | ||
* Reips, | * Reips, U.-D. (2002). [http://doi.org/10.1026//1618-3169.49.4.243 Standards for Internet-based experimenting]. Experimental Psychology, 49(4), 243–256. [[http://iscience.deusto.es/wp-content/uploads/2010/04/ulf27.pdf Alternate Link]] | ||
* | * Hergueux, J., & Jacquemet, N. (2014). [http://doi.org/10.1007/s10683-014-9400-5 Social preferences in the online laboratory: a randomized experiment]. Experimental Economics, 18(2), 251–283. ''[Available through UW Libraries]'' | ||
* | * Salganik, M. J., Dodds, P. S., & Watts, D. J. (2006). [http://doi.org/10.1126/science.1121066 Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market]. Science, 311(5762), 854–856. ''[Available through UW Libraries]'' | ||
* Rijt, | * Rijt, A. van de, Kang, S. M., Restivo, M., & Patil, A. (2014). [http://doi.org/10.1073/pnas.1316836111 Field experiments of success-breeds-success dynamics]. Proceedings of the National Academy of Sciences, 111(19), 6934–6939. ''[Available through UW Libraries]'' [[http://www.akshaynpatil.com/papers/success.pdf Alternative Link]] | ||
* | * Bond, R. M., Fariss, C. J., Jones, J. J., Kramer, A. D. I., Marlow, C., Settle, J. E., & Fowler, J. H. (2012). [http://doi.org/10.1038/nature11421 A 61-million-person experiment in social influence and political mobilization]. Nature, 489(7415), 295–298. ''[Available through UW Libraries]'' | ||
'''Optional Readings:''' | '''Optional Readings:''' | ||
* | * Zhu, H., Zhang, A., He, J., Kraut, R., & Kittur, A. (2013). [http://doi.org/10.1145/2470654.2481311 Effects of Peer Feedback on Contribution: A Field Experiment in Wikipedia]. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Paris, France: ACM. ''[Available through UW Libraries]'' | ||
: This | * Restivo, M., & van de Rijt, A. (2012). [http://dx.doi.org/10.1371/journal.pone.0034358 Experimental Study of Informal Rewards in Peer Production]. PLoS ONE, 7(3), e34358. ''[Free Online]'' | ||
* | : This is really just a more in-depth version of the experiments in the Restivo and van de Rijt article described above. | ||
* Restivo, M., & van de Rijt, A. (0). [http://doi.org/10.1080/1369118X.2014.888459 No praise without effort: experimental evidence on how rewards affect Wikipedia’s contributor community]. Information, Communication & Society, 0(0), 1–12. ''[Available through UW Libraries]'' | |||
* | : Note: This piece is, more or less, a continuation of the Restivo and van de Rijt piece included above but it is longer and goes into much more depth on at least one of the important theoretical issues. | ||
* | * Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014). [http://doi.org/10.1073/pnas.1320040111 Experimental evidence of massive-scale emotional contagion through social networks]. Proceedings of the National Academy of Sciences, 111(24), 8788–8790. ''[Available through UW Libraries]'' | ||
: Note: We've already read but I'd like to discuss it again. | |||
* Cosley, D., Frankowski, D., Terveen, L., & Riedl, J. (2007). [http://doi.org/10.1145/1216295.1216309 SuggestBot: Using Intelligent Task Routing to Help People Find Work in Wikipedia]. In Proceedings of the 12th International Conference on Intelligent User Interfaces (pp. 32–41). New York, NY, USA: ACM. ''[Available through UW Libraries]'' | |||
* Reinecke, K., & Gajos, K. Z. (2015). [http://doi.org/10.1145/2675133.2675246 LabintheWild: Conducting Large-Scale Online Experiments With Uncompensated Samples]. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (pp. 1364–1378). New York, NY, USA: ACM. ''[Available through UW Libraries]'' | |||
=== Week 6: Tuesday February 11: Crowdsourcing | === Week 6: Tuesday February 11: (I) Crowdsourcing (II) Discourse Analysis === | ||
==== Part I: (I) Crowdsourced Data Analysis and Experimentation ==== | |||
'''Assignment:''' | |||
* Find and complete at least 2 "hits" as a worker on [http://mturk.com Amazon Mechnical Turk]. Note that to do this you will need to create a ''worker'' account on Mturk. | |||
** Record (write down) details and notes about your tasks: What did you do? Who was the requester? What could you was the purpose of the task (as best you could tell)? What was the experience like? What research applications can you (not) imagine for this kind of system? | |||
* Design and deploy a small-scale research task on Mturk. Note that to do this, you will need to create a ''requester'' account on Mturk. Be sure to allow some time to get the task design the way you want it! Some ideas for study designs you might do: | |||
** A small survey. | |||
** Classification of texts or images (e.g., label tweets, pictures, or comments from a discussion thread). | |||
** A small experiment (e.g., you can do a survey where you insert ''different'' images and ask the same set of questions. Check out the [https://requester.mturk.com/help/getting_started.html Mturk requester getting started guide] | |||
* Prepare to share details of your small-scale research task in class, including results (they will come fast). | |||
''Note:'' In terms of running your task, it will cost real money and you have to put money on your Amazon account yourself. You've each got a $3 budget. Please use your credit card to put $3 on your account right away. I will pay each of you $3 in cash next week to reimburse you for the cost of running the experiment. | |||
'''Required Readings:''' | |||
* | * [https://docs.aws.amazon.com/AWSMechTurk/latest/RequesterUI/Introduction.html Amazon Mechanical Turk Requester UI Guide] ''[Free Online]'' | ||
* [https://mturkpublic.s3.amazonaws.com/docs/MTURK_BP.pdf Amazon Mechanical Turk Best Practices Guide]. ''[Free Online]'' | |||
* | * Weinberg, J., Freese, J., & McElhattan, D. (2014). [https://www.sociologicalscience.com/articles-vol1-19-292/ Comparing Data Characteristics and Results of an Online Factorial Survey between a Population-Based and a Crowdsource-Recruited Sample]. Sociological Science, 1, 292–310. ''[Free Online]'' | ||
* Shaw, A. (2015). [https://canvas.uw.edu/files/36419326/download?download_frd=1 Hired Hands and Dubious Guesses: Adventures in Crowdsourced Data Collection]. In E. Hargittai & C. Sandvig (Eds.), Digital Research Confidential: The Secrets of Studying Behavior Online. The MIT Press. ''[Available in Canvas]'' | |||
* | |||
'''Optional Readings:''' | |||
* | * Gray, M. L., Suri, S., Ali, S. S., & Kulkarni, D. (2016). [http://sidsuri.com/Publications_files/collab_paper21.pdf The Crowd is a Collaborative Network]. Proceedings of Computer-Supported Cooperative Work. ''[Free Online]'' | ||
* Kittur et al. (2013). [http://hci.stanford.edu/publications/2013/CrowdWork/futureofcrowdwork-cscw2013.pdf The Future of Crowd Work]. Proceedings of Computer-Supported Cooperative Work. ''[Free Online]'' | |||
'''Resources:''' | |||
* [http://www.mturk-tracker.com/ Mturk Tracker] | |||
==== Part II: Discourse Analysis ==== | |||
''' | '''Required Readings:''' | ||
Narrative Analysis: | |||
* | * Mitra, A. (1999). [http://doi.org/10.1111/j.1083-6101.1999.tb00330.x Characteristics of the WWW Text: Tracing Discursive Strategies]. Journal of Computer-Mediated Communication, 5(1), 0–0. ''[Free Online]'' | ||
* | * Kaun, Anne (2010), "[http://ejournals.library.ualberta.ca/index.php/IJQM/article/view/7165 Open-Ended Online Diaries: Capturing Life as it is Narrated]," International Journal of Qualitative Methods, Vol. 9 Issue 2, p133-148. ''[Free Online]'' | ||
Visual Analysis: | |||
* | * Hochman, N., & Schwartz, R. (2012). [https://www.aaai.org/ocs/index.php/ICWSM/ICWSM12/paper/view/4782 Visualizing Instagram: Tracing Cultural Visual Rhythms]. In Sixth International AAAI Conference on Weblogs and Social Media. ''[Available through UW Libraries]'' | ||
* | * Hochman, N., & Manovich, L. (2013). [http://firstmonday.org/ojs/index.php/fm/article/viewArticle/4711/ Zooming into an Instagram City: Reading the local through social media]. First Monday, 18(7). ''[Free Online]'' | ||
'''Optional Readings:''' | |||
Narrative Analysis: | |||
* Gubrium, Aline and K.C. Nat Turner, "[https://canvas.uw.edu/files/36418703/download?download_frd=1 Digital storytelling as an emergent method for social research and practice]," Ch. 21 in HET. | |||
Visual Analysis: | |||
* Newbold, Curtis, 2013, "[http://thevisualcommunicationguy.com/2015/01/12/how-to-do-a-visual-analysis-a-five-step-process/ How to Do a Visual Analysis (A 5-Step Process)]". ''[Free Online]'' | |||
: Note: Although I'm not a fan of infograpraphics as a genre, I suppose it makes sense that visual communication people would put together a pretty good one! If you're already familiar with visual analysis from the rhetorical tradition, there's not going to be a lot new here. If this is new for you, this will help you frame and understand the other readings. | |||
* Torralba, A. (2009). [http://videolectures.net/nips09_torralba_uvs/ Understanding Visual Scenes]. Tutorial presented at the NIPS, Vancouver, BC, Canada. Part I. ''[Free Online]'' | |||
: Note: This is a two part (each part is one hour) lecture and tutorial by a expert in computer vision. I strongly recommend watching Part I. I think this gives you a good sense of the nature of the kinds of challenges that were (and still are) facing the field of computer vision and anybody trying to have their computer look at images. | |||
These five paper are all technical approaches to doing image classification using datasets from Internet-based datasets of images like Flickr, Google Image Search, Google Street View, or Instagram. Each of these describes interesting and challenges technical issues. If you're interested, it would be a great idea to read these to get a sense for the state of the art and what is and isn't possible: | |||
* Jaffe, A., Naaman, M., Tassa, T., & Davis, M. (2006). [http://doi.org/10.1145/1178677.1178692 Generating Summaries and Visualization for Large Collections of Geo-referenced Photographs]. In Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval (pp. 89–98). New York, NY, USA: ACM. ''[Available through UW Libraries]'' | |||
* Simon, I., Snavely, N., & Seitz, S. M. (2007). [http://doi.org/10.1109/ICCV.2007.4408863 Scene Summarization for Online Image Collections]. In Computer Vision, IEEE International Conference on (Vol. 0, pp. 1–8). Los Alamitos, CA, USA: IEEE Computer Society. ''[Free Online]'' | |||
* Crandall, D. J., Backstrom, L., Huttenlocher, D., & Kleinberg, J. (2009). [http://doi.org/10.1145/1526709.1526812 Mapping the World’s Photos]. In Proceedings of the 18th International Conference on World Wide Web (pp. 761–770). New York, NY, USA: ACM. ''[Available through UW Libraries]'' | |||
* San Pedro, J., & Siersdorfer, S. (2009). [http://doi.org/10.1145/1526709.1526813 Ranking and Classifying Attractiveness of Photos in Folksonomies]. In Proceedings of the 18th International Conference on World Wide Web (pp. 771–780). New York, NY, USA: ACM. ''[Available through UW Libraries]'' | |||
* Doersch, C., Singh, S., Gupta, A., Sivic, J., & Efros, A. A. (2012). [http://doi.org/10.1145/2185520.2185597 What Makes Paris Look Like Paris?] ACM Trans. Graph., 31(4), 101:1–101:9. ''[Available through UW Libraries]'' | |||
Discourse Analysis: | |||
* | * Honeycutt, Courtenay (2005), “[http://onlinelibrary.wiley.com/enhanced/doi/10.1111/j.1083-6101.2005.tb00240.x Hazing as a process of boundary maintenance in an online community]”, Journal of Computer-Mediated Communication, 10(2). [Available through UW Libraries] | ||
:Note: Combines quantitative and qualitative computer-mediated discourse analysis methods.* | |||
=== Week 6: Saturday February 15: CDSW Session 3 === | |||
As description in the section on technical skills above, I expect everybody who is not comfortable with at least basic programming and data collection to attend the [[Community Data Science Workshops (Spring 2020)]] which I am running concurrently with this class. | |||
This session will run from 9am-3pm. Details on the [[CDSW Spring 2020]] page. | |||
=== Week 7: Tuesday February 18: Consulting Week (i.e., no group meeting) === | |||
During this week, we not meet together. Instead, I will schedule one-on-one in person meetings of an hour with each student individually to catch up with you about your project and to work directly with you to resolve any technical issues you have run into with data collection, etc. | |||
During this week, we | |||
=== Week 8: Tuesday February 25: Consulting Week (i.e., no group meeting) === | |||
During this week, we not meet together. Instead, I will schedule one-on-one in person meetings of an hour with each student individually to catch up with you about your project and to work directly with you to resolve any technical issues you have run into with data collect | |||
=== Week 9: Tuesday March 3: (I) Design Research and (II) Digital Trace and Sensor Data === | |||
==== Part I: Design Research ==== | ==== Part I: Design Research ==== | ||
Today we'll have a guest visitor — [http://www.andresmh.com/ Andrés Monroy-Hernández] who is director of HCI research at SNAP and formerly from [http://fuse.microsoft.com/ Microsoft | Today we'll have a guest visitor — [http://www.andresmh.com/ Andrés Monroy-Hernández] who is director of HCI research at SNAP and formerly from [http://fuse.microsoft.com/ Microsoft Resarch's FUSE labs]. Andrés is affiliate faculty in the Department of Communication and Department of Human-Centered Design and Engineering at UW. Monroy-Hernández research involves studying people by designing and building systems. He's built a number of very large and successful socio-technical systems as part of his research. In his graduate work, he build the [http://scratch.mit.edu/ Scratch Online Community] which is now used by more than 10 million people. | ||
I've asked him to come and talk to us about design research as a process. As a result, it will be helpful to read about two projects he has worked on recently that he will talked to us about. Those projects are called NewsPad and Eventful. | I've asked him to come and talk to us about design research as a process. As a result, it will be helpful to read about two projects he has worked on recently that he will talked to us about. Those projects are called NewsPad and Eventful. | ||
Line 586: | Line 561: | ||
* Eagle, Nathan, "[https://canvas.uw.edu/files/36870285/download?download_frd=1 Mobile phones as sensors for social research]," Ch. 22 in HET. | * Eagle, Nathan, "[https://canvas.uw.edu/files/36870285/download?download_frd=1 Mobile phones as sensors for social research]," Ch. 22 in HET. | ||
* Visser, Albertine | * Visser, Albertine and Ingrid Mulder, "[https://canvas.uw.edu/files/36870283/download?download_frd=1 Emergent technologies for assessing social feelings and experiences]," Ch. 16 in HET. | ||
* de Haan, Geert, et. al., "[https://canvas.uw.edu/files/36870284/download?download_frd=1 Bringing the research lab into everyday life: Exploiting sensitive environments to acquire data for social research]," Ch. 23 in HET. | * de Haan, Geert, et. al., "[https://canvas.uw.edu/files/36870284/download?download_frd=1 Bringing the research lab into everyday life: Exploiting sensitive environments to acquire data for social research]," Ch. 23 in HET. | ||
* Fowler, Chris, et. al., "[https://canvas.uw.edu/files/36870282/download?download_frd=1 Living laboratories: Social research applications and evaluations]," Ch. 27 in HET. | * Fowler, Chris, et. al., "[https://canvas.uw.edu/files/36870282/download?download_frd=1 Living laboratories: Social research applications and evaluations]," Ch. 27 in HET. | ||
* Holohan, Anne, et. al., "[https://canvas.uw.edu/files/36870280/download?download_frd=1 The digital home: A new locus of social science research]," Ch. 28 in HET. | * Holohan, Anne, et. al., "[https://canvas.uw.edu/files/36870280/download?download_frd=1 The digital home: A new locus of social science research]," Ch. 28 in HET. | ||
=== Week 10: Tuesday March 10: Final Presentations === | |||
== Administrative Notes == | == Administrative Notes == | ||
=== Your Presence in Class === | === Your Presence in Class === | ||
As detailed in [[#Participation|the section on participation]] and in [[ | As detailed in [[#Participation|the section on participation]] and in [[Teaching Assessment|my page on assessment]], class participation is a critical way that I will assess learning in the class. Obviously, you must be in class in order to participate. If you need to miss class for any reason, please contact me ahead of time (email is best). In the event of an absence, you are responsible for obtaining class notes, handouts, assignments, etc. | ||
=== Office Hours === | === Office Hours === | ||
I will hold office | I will hold office Hours on '''Thursdays 1-2pm''' in [https://uw.edu/maps/?cmu Communications (CMU) 333]. In addition to my scheduled office hours, I am generally in [[Community Data Science Lab (UW)|my lab in CMU 306]]. Feel free to stop by at any time or to contact me to arrange a time to meet. | ||
=== Religious Accommodations === | === Religious Accommodations === | ||
Washington state law requires that UW develop a policy for | Washington state law requires that UW develop a policy for accommodation of student absences or significant hardship due to reasons of faith or conscience, or for organized religious activities. The UW’s policy, including more information about how to request an accommodation, is available at [https://registrar.washington.edu/staffandfaculty/religious-accommodations-policy/ Religious Accommodations Policy]. Accommodations must be requested within the first two weeks of this course using the [https://registrar.washington.edu/students/religious-accommodations-request/ Religious Accommodations Request form]. | ||
=== Student Conduct === | === Student Conduct === | ||
Line 612: | Line 586: | ||
Safety | Safety | ||
Call SafeCampus at 206-685-7233 anytime–no matter where you work or study–to anonymously discuss safety and well-being concerns for yourself or others. SafeCampus’s team of caring professionals will provide individualized support while discussing short- and long-term solutions and connecting you with additional resources when requested. | Call SafeCampus at 206-685-7233 anytime–no matter where you work or study–to anonymously discuss safety and well-being concerns for yourself or others. SafeCampus’s team of caring professionals will provide individualized support, while discussing short- and long-term solutions and connecting you with additional resources when requested. | ||
=== Academic Dishonesty === | === Academic Dishonesty === | ||
Line 620: | Line 594: | ||
=== Disability Resources === | === Disability Resources === | ||
If you have already established accommodations with Disability Resources for Students (DRS), please communicate your approved accommodations to | If you have already established accommodations with Disability Resources for Students (DRS), please communicate your approved accommodations to uw at your earliest convenience so we can discuss your needs in this course. | ||
If you have not yet established services through DRS, but have a temporary health condition or permanent disability that requires accommodations (conditions include but not limited to; mental health, attention-related, learning, vision, hearing, physical or health impacts), you are welcome to contact DRS at 206-543-8924 or uwdrs@uw.edu or disability.uw.edu. DRS offers resources and coordinates reasonable accommodations for students with disabilities and/or temporary health conditions. Reasonable accommodations are established through an interactive process between you, your instructor(s) and DRS. It is the policy and practice of the University of Washington to create inclusive and accessible learning environments consistent with federal and state law. | If you have not yet established services through DRS, but have a temporary health condition or permanent disability that requires accommodations (conditions include but not limited to; mental health, attention-related, learning, vision, hearing, physical or health impacts), you are welcome to contact DRS at 206-543-8924 or uwdrs@uw.edu or disability.uw.edu. DRS offers resources and coordinates reasonable accommodations for students with disabilities and/or temporary health conditions. Reasonable accommodations are established through an interactive process between you, your instructor(s) and DRS. It is the policy and practice of the University of Washington to create inclusive and accessible learning environments consistent with federal and state law. | ||
Line 630: | Line 604: | ||
== Credit and Notes == | == Credit and Notes == | ||
This will be | This will be third time I have taught this course at UW in its current form. This syllabuses draws heavily from previous versions. Syllabuses from earlier classes can be found online at: | ||
* [[Internet Research Methods (Spring 2016)]] | * [[Internet Research Methods (Spring 2016)]] | ||
* [https://mako.cc/teaching/2015/internet_research/ Internet Research Methods (Spring 2015)] | * [https://mako.cc/teaching/2015/internet_research/ Internet Research Methods (Spring 2015)] | ||
This syllabus was inspired by and borrows with permission from, a | This syllabus was inspired by, and borrows with permission from, a syallbus from an earlier version of this class taught by [http://www.com.washington.edu/foot/ Kirsten Foot]. Professor Foot last taught the course in Spring 2014. |