Statistics and Statistical Programming (Winter 2017): Difference between revisions

From CommunityData
Line 13: Line 13:
== Overview and Learning Objectives ==
== Overview and Learning Objectives ==


What new lines of inquiry and approaches to social research are made possible and necessary by the Internet? In what ways have established research methods been affected by the Internet? How does the Internet challenge established methods of social research? How are researchers responding to these challenges?
This course is the second course in a two-quarter quantitative methods sequence in the University of Washington's Department of Communication MA/PhD program. The first course ('''COM 520''') is a first introduction to quantitative social science in communication and focuses primarily on what you might think of the "soft skills" associated with doing social science: the conceptualization, operationalization of quantifiable variables and the design of quantitative analyses. Although the course introduces some univariate and bivariate statistics at the end, this is done using the tools that students already know (like Microsoft Excel or similar spreadsheet software). This class assumes that you have taken COM 520 and that you understand what is involved in describing and testing social scientific theories with data.


These are some of the key questions we will explore in this course. The course will focus on assessing the incorporation of Internet tools in established and emergent methods of social research, the adaptation of social research methods to study online phenomena, and the development of new methods and tools that correspond with the particular capacities and characteristics of the Internet. The readings will include both descriptions of Internet-related research methods with an eye to introducing skills and examples of studies that use them. The legal and ethical aspects of Internet research will receive ongoing consideration throughout the course. The purpose of this course is to help prepare students to design high quality research projects that use the Internet to study online communicative, social, cultural, and political phenomena.
This course ('''COM 521''') is focused on technical skill-building and aims to be a get-your-hands-dirty introduction to statistics and statistical programming. The point of the course is to give you to the technical and mathematical tools to carry out your own analysis. Through the process, it also seeks to help you become more sophisticated consumers of quantitative research.
 
Although we'll be doing some math (and a lot of arithmetic) in the course, this is not a math class.  We're not going to cover the math behind the techniques we'll be covering and I'm not going to be doing proofs on the board.  Instead, the class is unapologetically focused on ''the application of statistic methodology''. In that sense, the goal of the is course is to create ''informed consumers'' of quantitative methodology, not producers of new types of methods.
 
This course does not seek to be your final statistics course. I took 12 different statistics and math courses in graduate school and I wish I had done more. What it does seek to do is give you a solid basis on which to build statistical knowledge. Anyone who finishes this class should feel comfortable moving on to take advance classes in CSSS and to start building toward a Concentration in Statistics in Communication certificate.
 
We'll cover theses basic statistical techniques: t-tests; chi-squared tests; ANOVA, MANOVA, and related methods; linear regression; logistic regression.


I will consider the course a complete success if every student is able to do all of these things at the end of the quarter:
I will consider the course a complete success if every student is able to do all of these things at the end of the quarter:


* Discuss and compare distinct types of Internet research including: web archiving; textual analysis; ethnography; interviews; network analyses of social and hyperlink networks; analysis of digital trace data, traditional, natural, and field experiments; design research; interviewing; survey research; design research; and narrative and visual analyses.
* Carry out a complete analysis of a quantitative research project, start to finish. This means you will all:
* Describe particular challenges and threats to research validity associated with each method.
** Design a study.
* For at least one method, be able to provide a detailed description of a research project and feel comfortable embarking on a formative study using this methodology.
** Find, collect, or build a quantitative dataset.
* Given a manuscript (e.g., in the context of a request for peer review), be able to evaluate a Internet-based study in terms of its use its methodological choices.
** Test a hypotheses about relationships between two or more variables using your dataset.
* Use a modern programming language (e.g., Python) to collect a dataset from a web API like the APIs from Twitter and Wikipedia.
** Report your findings in a short paper and a short presentation.
** Provide code and data for your analysis in a way that makes your work replicable by other researchers.
* Read, modify, and create short programs in the GNU R statistical programming language.
* Feel comfortable reading papers that use basic statistical techniques.
* Feel comfortable and prepared enrolling in future statistics courses in CSSS.


== Note About This Syllabus ==
== Note About This Syllabus ==

Revision as of 04:27, 26 December 2016

Designing Internet Research
COM521 - Department of Communication, University of Washington
Instructor: Benjamin Mako Hill (University of Washington)
Course Websites:
Course Catalog Description:
Discusses complexities in quantitative research on communication. Focus on multivariate data design and analysis, including multiple and logistic regression, ANOVA and MANOVA, and factor analysis.

Overview and Learning Objectives

This course is the second course in a two-quarter quantitative methods sequence in the University of Washington's Department of Communication MA/PhD program. The first course (COM 520) is a first introduction to quantitative social science in communication and focuses primarily on what you might think of the "soft skills" associated with doing social science: the conceptualization, operationalization of quantifiable variables and the design of quantitative analyses. Although the course introduces some univariate and bivariate statistics at the end, this is done using the tools that students already know (like Microsoft Excel or similar spreadsheet software). This class assumes that you have taken COM 520 and that you understand what is involved in describing and testing social scientific theories with data.

This course (COM 521) is focused on technical skill-building and aims to be a get-your-hands-dirty introduction to statistics and statistical programming. The point of the course is to give you to the technical and mathematical tools to carry out your own analysis. Through the process, it also seeks to help you become more sophisticated consumers of quantitative research.

Although we'll be doing some math (and a lot of arithmetic) in the course, this is not a math class. We're not going to cover the math behind the techniques we'll be covering and I'm not going to be doing proofs on the board. Instead, the class is unapologetically focused on the application of statistic methodology. In that sense, the goal of the is course is to create informed consumers of quantitative methodology, not producers of new types of methods.

This course does not seek to be your final statistics course. I took 12 different statistics and math courses in graduate school and I wish I had done more. What it does seek to do is give you a solid basis on which to build statistical knowledge. Anyone who finishes this class should feel comfortable moving on to take advance classes in CSSS and to start building toward a Concentration in Statistics in Communication certificate.

We'll cover theses basic statistical techniques: t-tests; chi-squared tests; ANOVA, MANOVA, and related methods; linear regression; logistic regression.

I will consider the course a complete success if every student is able to do all of these things at the end of the quarter:

  • Carry out a complete analysis of a quantitative research project, start to finish. This means you will all:
    • Design a study.
    • Find, collect, or build a quantitative dataset.
    • Test a hypotheses about relationships between two or more variables using your dataset.
    • Report your findings in a short paper and a short presentation.
    • Provide code and data for your analysis in a way that makes your work replicable by other researchers.
  • Read, modify, and create short programs in the GNU R statistical programming language.
  • Feel comfortable reading papers that use basic statistical techniques.
  • Feel comfortable and prepared enrolling in future statistics courses in CSSS.

Note About This Syllabus

You should expect this syllabus to be a dynamic document and you will notice that there are a few places marked "To Be Determined." Although the core expectations for this class are fixed, the details of readings and assignments will shift. As a result, there are three important things to keep in mind:

  1. Although details on this syllabus will change, I will not change readings or assignments less than one week before they are due. If I don't fill in a "To Be Determined" one week before it's due, it is dropped. If you plan to read more than one week ahead, contact me first.
  2. Closely monitor your email or the announcements section on the course website on Canvas. When I make changes, these changes will be recorded in the history of this page so that you can track what has changed and I will summarize these changes in an announcement on Canvas that will be emailed to everybody in the class.
  3. I will ask the class for voluntary anonymous feedback frequently — especially toward the beginning of the quarter. Please let me know what is working and what can be improved. In the past, I have made many adjustments based on this feedback.

Books

This class has no textbook and I am not requiring you to buy any books for this class. That said, several required readings and many suggested readings, will come from several excellent books which you might want consider adding to your library:

These books include:

Assignments

The assignments in this class are designed to give you an opportunity to try your hand at using the conceptual material taught in the class. There will be no exams or quizzes. Unless otherwise noted, all assignments are due at the end of the day (i.e., 11:59pm on the day they are due).

Research Project

As a demonstration of your learning in this course, you will design a plan for an internet research project and will, if possible, also collect (at least) an initial sample of a dataset that you will use to complete the project.

The genre of the paper you can produce can one of the following three things:

  1. A draft of a manuscript for submission to a conference or journal.
  2. A proposal for funding (e.g., for submission for the NSF for a graduate student fellowship).
  3. A draft of the methods chapter of your dissertation.

In any the three paths, I expect you take this opportunity to produce a document that will further your to academic career outside of the class.

Project Identification

Due Date
April 10
Maximum paper length
500 words (~1-2 page)
Deliverables
Turn in in Canvas

Early on, I want you to identify your final project. Your proposal should be short and can be either paragraphs or bullets. It should include the following things:

  • The genre of the project and a short description of how it fits into your career trajectory.
  • A one paragraph abstract of the proposed study and research question, theory, community, and/or groups you plan to study.
  • A short description of the type of data you plan to collect as part of your final project.

Final Project

Outline Due Date
May 8
Maximum outline length
2 pages
Paper Due Date
June 12
Maximum outline length
6000 words (~20 pages)
Presentation Date
June 2
All Deliverables
Turn in in Canvas

Because the emphasis in this class is on methods and because I'm not an expert in each of your areas or fields, I'm happy to assume that your paper, proposal, or thesis chapter has already established the relevance and significance of your study and has a comprehensive literature review, well-grounded conceptual approach, and compelling reason why this research is so important. Instead of providing all of this details, instead feel free to start with a brief summary of the purpose and importance of this research, and an introduction of your research questions or hypotheses. If your provide more detail, that's fine, but I won't give you detailed feedback on this parts.

The final paper should include:

  • a statement of the purpose, central focus, relevance and significance of this research;
  • a description of the specific Internet application(s) and/or environment(s) and/or objects to be studied and employed in the research;
  • key research questions or hypotheses;
  • operationalization of key concepts;
  • a description and rationale of the specific method(s), (if more than one method will be used, explain how the methods will produce complementary findings);
  • a description of the step-by-step plan for data collection;
  • description and rationale of the level(s), unit(s) and process of analysis (if more than one kind of data are generated, explain how each kind will be analyzed individually and/or comparatively);
  • an explanation of how these analyses will enable you to answer the RQs
  • a sample instrument (as appropriate);
  • a sample dataset and description of a formative analysis you have completed;
  • a description of actual or anticipated results and any potential problems with their interpretation;
  • a plan for publishing/disseminating the findings from this research
  • a summary of technical, ethical, human subjects and legal issues that may be encountered in this research, and how you will address them;
  • a schedule (using specific dates) and proposed budget.

I also expect each student to begin data for your project (i.e., using the technical skills you learn in the class) and describe your progress in this regard this in your paper. If collecting data for a proposed project is impractical (e.g., because of IRB applications, funding, etc) I would love for you to engage in the collection of public dataset as part of a pilot or formative study. If this is not feasible or useful, we can discuss other options.

I have a strong preference for you to write this paper individually but I'm open to the idea that you may want to work with others in the class.

Participation

The course relies heavily on participation and discussion. It is important to realize that we will not summarize reading in class and I will not cover it in lecture. I expect you all to have read it and we will jump in and start discussing it. The "Participation Rubric" section of my detailed page on assessment gives the rubric I will use in evaluating participation.

Grading

I have put together a very detailed page that describes grading rubric I will be using in this course. Please read it carefully I will assign grades for each of following items on the UW 4.0 grade scale according to the weights below:

  • Participation: 25%
  • Presentation of method/approach: 15%
  • Proposal identification: 5%
  • Final paper outline: 5%
  • Final Presentation: 10%
  • Final Paper: 40%

Schedule

Week 1: Monday March 28: Introduction and Framing

Resources:

Required Readings:

Optional Reading:

Week 1: Wednesday March 30: Ethics

Resources:

Required Readings:

Optional Readings:

Week 2: Monday April 4: NO CLASS

Week 2: Wednesday April 6: Web Archiving

Facilitator: Janny

Required Readings:

Optional Readings:

Week 2: Friday April 8: CDSW Session 0

As description in the section on technical skills above, I expect everybody who is not comfortable with at least basic programming and data collection to attend the Community Data Science Workshops (Spring 2016) which I am running concurrently with this class.

This session will run from 6-9pm and is the only session which can probably be missed. Please do contact me, however, if you will not be able to attend it.

Week 2: Saturday April 9: CDSW Session 1

As description in the section on technical skills above, I expect everybody who is not comfortable with at least basic programming and data collection to attend the Community Data Science Workshops (Spring 2016) which I am running concurrently with this class.

This session will run from 9am-3pm. Details on the CDSW Spring 2016 page.

Week 3: Monday April 11: Textual Analyses

Facilitator: Adam

Required Readings:

Optional Readings:

I'm assuming you have at least a rough familiarity with content analysis as a methodology. If your not as comfortable with this, check out the Wikipedia article to start. These help provide more of a background into content analysis (in general, and online):

  • Van Selm, Martine & Jankowski, Nick, (2005) "Content Analysis of Internet-Based Documents." Unpublished Manuscript. [Available in Canvas]
  • Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, Calif.: Sage Publications. [Available from Instructor]
  • Krippendorff, K. (2005). Content analysis: an introduction to its methodology. Thousand Oaks; London; New Delhi: Sage. [Available from Instructor]

Examples of more traditional content analysis using online content:

Another example of topic modeling, but from political science:

Week 3: Wednesday April 13: Digital Ethnography & Trace Ethnography

Session Coordinator: Nate

Required Readings:

Optional Readings:

Note: You may also be interest in reading the essay by Hine that boyd is responding to. [Available in Canvas]

This is the canonical book-length account and the main citation in this space:

  • Hine, C. (2000). Virtual ethnography. London, UK: SAGE Publications. [Available from Instructor]

These are all other interesting and/or frequently cited examples of Internet-based ethnographies:

Note: Dodgeball is a mobile social network system (MSNS) that allows groups of friends to connect and meet up via mobile phone. The author employed participant observation in order to understand norms of interaction in the MSNS "space".
Note: To conduct the study reported in this paper the authors created a used a fake profile in order to observe the psychological support offered to participants.
Note: Fantastic more general introduction but takeaways that are more specifically targetted toward people studying virtual reality type environments with virtual physicality.

Apropos of class discussion:

Week 4: Monday April 18: Online Interviews

Facilitator: Julia

Required Readings:

Note: Short article you can basically skim. Read it quickly so you can cite it later.

Optional Readings:

Note: Strongly focused on enthnographic interviews with tons of very specific details. Fantastic article on interviewing, although perhaps a bit weak on Internet specific advice.
Note: One of the earliest books on online life and one of the earliest attempts to do online interviewing. This is dated, but highlight some important challenge.
Note: Start reading on page 8 on "The Internet and the Interview". The beginning is a theoretical argument that's not really relevant to this class.* Chou, C. (2001). Internet heavy use and addiction among Taiwanese college students: an online interview study. CyberPsychology & Behavior, 4(5), 573-585. [Available through UW Libraries]

Alternate Accounts:

These texts are largely redundant to the required texts above but do provide a different perspective and examples:

  • Salmons, J. (2014). Qualitative Online Interviews: Strategies, Design, and Skills. SAGE Publications.
This is a book that lays out what claims to be a comprehensive account to online interviewing. Take a quick through the preface and table of contents and read Chapter 1. [Both Available in Canvas.]
I have the book and am happy to loan my copy to anybody in the class that thinks this will be a core part of their research.
  • Morgan, David L. and Bojana Lobe, "Online focus groups," Ch. 9 in HET. [Available in Canvas]
  • Gaiser, T. J. (2008). Online Focus Groups. In N. G. Fielding, R. M. Lee, & G. Blank (Eds.), The SAGE Handbook of Online Research Methods (pp. 290–307). London, UK: SAGE Publications, Ltd. [Available through UW Libraries]

Week 4: Wednesday April 20: Social Network Analysis

Faciliator: Mengjun

Required Readings:

Week 4: Saturday April 23: CDSW Session 2

As description in the section on technical skills above, I expect everybody who is not comfortable with at least basic programming and data collection to attend the Community Data Science Workshops (Spring 2016) which I am running concurrently with this class.

This session will run from 10am-4pm. Details on the CDSW Spring 2016 page.

Week 5: Monday April 25: Experiments

Facilitator: Emma

Required Readings:

Optional Readings:

This is really just a more in-depth version of the experiments in the Restivo and van de Rijt article described above.
Note: This piece is, more or less, a continuation of the Restivo and van de Rijt piece included above but it is longer and goes into much more depth on at least one of the important theoretical issues.
Note: We've already read but I'd like to discuss it again.

Week 5: Wednesday April 27: Surveys

Facilitator: Ben

Required Readings:

Note: This journalistic account of the research may also be useful.

Optional Readings:

If you don't have a background in survey design, these two have been recommended by our guest speaker as good basic things to read:

  • Krosnick, J. A. (1999). Survey Research. Annual Review of Psychology, 50(1), 537–567. [Available through UW Libraries]
  • Krosnick, J. A. (1999). Maximizing measurement quality: Principles of good questionnaire design. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of Political Attitudes. New York: Academic Press.

These are other texts on the subject that you might find useful:

Week 6: Monday May 2: Narrative, Discourse and Visual Analysis

Facilitator: Liang

Required Readings:

Narrative Analysis:

Visual Analysis:

Optional Readings:

Narrative Analysis:

Visual Analysis:

Note: Although I'm not a fan of infograpraphics as a genre, I suppose it makes sense that visual communication people would put together a pretty good one! If you're already familiar with visual analysis from the rhetorical tradition, there's not going to be a lot new here. If this is new for you, this will help you frame and understand the other readings.
Note: This is a two part (each part is one hour) lecture and tutorial by a expert in computer vision. I strongly recommend watching Part I. I think this gives you a good sense of the nature of the kinds of challenges that were (and still are) facing the field of computer vision and anybody trying to have their computer look at images.

These five paper are all technical approaches to doing image classification using datasets from Internet-based datasets of images like Flickr, Google Image Search, Google Street View, or Instagram. Each of these describes interesting and challenges technical issues. If you're interested, it would be a great idea to read these to get a sense for the state of the art and what is and isn't possible:

Discourse Analysis:

Note: Combines quantitative and qualitative computer-mediated discourse analysis methods.*

Week 6: Wednesday May 4: Crowdsourced Data Analysis and Experimentation

Assignment:

  • Find and complete at least 2 "hits" as a worker on Amazon Mechnical Turk. Note that to do this you will need to create a worker account on Mturk.
    • Record (write down) details and notes about your tasks: What did you do? Who was the requester? What could you was the purpose of the task (as best you could tell)? What was the experience like? What research applications can you (not) imagine for this kind of system?
  • Design and deploy a small-scale research task on Mturk. Note that to do this, you will need to create a requester account on Mturk. Be sure to allow some time to get the task design the way you want it! Some ideas for study designs you might do:
    • A small survey.
    • Classification of texts or images (e.g., label tweets, pictures, or comments from a discussion thread).
    • A small experiment (e.g., you can do a survey where you insert different images and ask the same set of questions. Check out the Mturk requester getting started guide
  • Prepare to share details of your small-scale research task in class, including results (they will come fast).

Note: In terms of running your task, it will cost real money and you have to put money on your Amazon account yourself. You've each got a $3 budget. Please use your credit card to put $3 on your account right away. I will pay each of you $3 in cash next week to reimburse you for the cost of running the experiment.

Required Readings:

Optional Readings:

Resources:

Week 6: Saturday May 7: CDSW Session 3

As description in the section on technical skills above, I expect everybody who is not comfortable with at least basic programming and data collection to attend the Community Data Science Workshops (Spring 2016) which I am running concurrently with this class.

This session will run from 9am-3pm. Details on the CDSW Spring 2016 page.

Week 7: Monday May 9: Consulting Week (i.e., no group meeting)

During this week, we not meet together. Instead, I will schedule one-on-one in person meetings of an hour with each student individually to catch up with you about your project and to work directly with you to resolve any technical issues you have run into with data collection, etc.

Week 7: Wednesday May 11: Consulting Week (i.e., no group meeting)

Week 8: Monday May 16: Consulting Week (i.e., no group meeting)

During this week, we not meet together. Instead, I will schedule one-on-one in person meetings of an hour with each student individually to catch up with you about your project and to work directly with you to resolve any technical issues you have run into with data collect

Week 8: Wednesday May 18: Consulting Week (i.e., no group meeting)

Week 9: Monday May 23: Design Research

Today we'll have a guest visitor — Andrés Monroy-Hernández from Microsoft Resarch's FUSE labs and affiliate faculty in the Department of Communication and Department of Human-Centered Design and Engineering at UW. Monroy-Hernández research involves studying people by designing and building systems. He's built a number of very large and successful socio-technical systems as part of his research. In his graduate work, he build the Scratch Online Community which is now used by more than 10 million people.

I've asked him to come and talk to us about design research as a process. As a result, it will be helpful to read about two projects he has worked on recently that he will talked to us about. Those projects are called NewsPad and Eventful.

Required Readings:

Week 9: Wednesday May 25: Digital Trace and Sensor Data

Required Readings:

Read any 2 of these 4 chapters from the Handbook of Emerging Technology in Social Research:

Week 10: Monday May 30: Final Presentations

Week 10: Wednesday June 1: Final Presentations

Not Covered: Hyperlink Networks

  • Jackson, Michele, (1997), "Assessing the Structure of the Communication on the World Wide Web," Journal of Computer-Mediated Communication, V. 3, N. 1, June, 1997.
  • Jackson, Michele, (2011), "What Should Researchers Infer From Links on an Organization's Site?", blog post at http://assett.colorado.edu/jackson/what-should-researchers-infer-from-links-on-an-organizations-site/
  • Olesen, Thomas (2004), "The Transnational Zapatista Solidarity Network: An Infrastructure Analysis," Global Networks, 4(1):89-107 [Although this article uses the term infrastructure analysis, the method employed is best described as a hyperlink network analysis.]

Administrative Notes

Attendance

As detailed in my page on assessment, attendance in class is expected of all participants. If you need to miss class for any reason, please contact me ahead of time (email is best). Multiple unexplained absences will likely result in a lower grade or (in extreme circumstances) a failing grade. In the event of an absence, you are responsible for obtaining class notes, handouts, assignments, etc.

Office Hours

I will not hold regular office hours. In general, I will be available to meet after class. Please contact me on email to arrange a meeting then or at another time.

Accommodations

In general, if you have an issue, such as needing an accommodation for a religious obligation or learning disability, speak with me before it affects your performance; afterward it is too late. Do not ask for favors; instead, offer proposals that show initiative and a willingness to work.

To request academic accommodations due to a disability please contact Disability Resources for Students, 448 Schmitz, 206-543-8924/V, 206-5430-8925/TTY. If you have a letter from Disability Resources for Students indicating that you have a disability that requires academic accommodations, please present the letter to me so we can discuss the accommodations that you might need for the class. I am happy to work with you to maximize your learning experience.

Academic Misconduct

I am committed to upholding the academic standards of the University of Washington’s Student Conduct Code. If I suspect a student violation of that code, I will first engage in a conversation with that student about my concerns.

If we cannot successfully resolve a suspected case of academic misconduct through our conversations, I will refer the situation to the department of communication advising office who can then work with the COM Chair to seek further input and if necessary, move the case up through the College.

While evidence of academic misconduct may result in a lower grade, I will not unilaterally lower a grade without addressing the issue with you first through the process outlined above.

Credit and Notes

This syllabus was inspired by, and borrows with permission from, a syallbus from an earlier version of this class taught by Kirsten Foot in Spring 2014.

Retrieved from "https://wiki.communitydata.science/index.php?title=Statistics_and_Statistical_Programming_(Winter_2017)&oldid=88061"
Attribution-Share Alike 3.0 Unported
Powered by MediaWiki