The Wikipedia Adventure: Difference between revisions

From CommunityData
 
(12 intermediate revisions by 2 users not shown)
Line 13: Line 13:
== Current TO DO list ==
== Current TO DO list ==
* '''introduction'''
* '''introduction'''
* '''Background and prior work'''
** Continue to improve gamification subsection (mako)
* '''System design'''
** Use Webstrates paper (UIST 2015) as a model (mako)
* '''Study 1:'''
* '''Study 2:'''
* '''Discussion & Conclusion'''
** make sure connections to framing are clear
* '''Throughout'''
** Full read-throughs for tone, consistency, clear contributions.
** Proofreading
== Recent changes log ==
'''These have been addressed, but will likely still benefit from further attention so they're here for archival purposes'''
* '''introduction'''
* compress the introduction. reduce redundancy (as)
** focus on two study framing
** focus on two study framing
** Change framing to show that two studies were always part of the plan
** Change framing to show that two studies were always part of the plan
Line 19: Line 37:
*** challenges of deploying new systems in a massive community with passionate, experienced members
*** challenges of deploying new systems in a massive community with passionate, experienced members
*** large-scale user survey and invitation-based field experiment under realistic conditions provide more useful evidence (than hallway testing or small-scale usability studies).
*** large-scale user survey and invitation-based field experiment under realistic conditions provide more useful evidence (than hallway testing or small-scale usability studies).
* '''Background and prior work'''
** Continue to improve gamification subsection (mako)
* '''System design'''
** Use Webstrates paper (UIST 2015) as a model (mako)
* '''Study 1:'''
* '''Study 1:'''
** replace bar charts w new visualizations (sn)
** replace bar charts w new visualizations (sn)
Line 32: Line 46:
** restrict main analysis to last 100 days of sampling (sn)
** restrict main analysis to last 100 days of sampling (sn)
*** revise '''all''' reported statistics to reflect this
*** revise '''all''' reported statistics to reflect this
* '''Discussion & Conclusion''' (sn, jm)
** include boxplots (w scatter) of dependent variables for all treatment and control units.
** Make sure to address big contributions
** Include Mann-Whitney tests for distribution shift
** Note that most studies on gamified systems don't do quantitative impact assessment, highlight this as a contribution of our study
* '''Discussion & Conclusion'''  
** Make sure to address big contributions.
** Elaborate explanations of null effect.
** Note that most studies on gamified systems don't do quantitative impact assessment, highlight this as a contribution of our  
 
* '''Throughout'''
* '''Throughout'''
* re-built refs-processed and eliminated missing citation errors (as)
* change language around Teahouse (jm)
** Add citations currently in text
** Add citations currently in text
** Remove excess uses of passive voice
** Remove excess uses of passive voice
 
** Add keywords and ACM categories.
== Recent changes log ==
 
* change language around Teahouse


==Timeline for CSCW 2017 submission==
==Timeline for CSCW 2017 submission==


===Thursday, May 19th===
* Refine organization of Study 1 findings (sn) [Done]
* Analyze differential attrition in Study 2 (sn) [Done]
* Add overarching discussion & conclusion sections (sn, jm) [Done]
**make sure to address big contributions
* Fill in as many "TO DO" items as possible in the doc (sn) [Done]
** Refine prior work section on gamification (Mako)
* Incorporate UIST model paper system evaluation stuff suggested by Mako
** Use Webstrates paper (UIST 2015) as a model.
* Refine Study 2 Discussion (Jmo, Sneha) [Done]
** Explain why TWA didn't alter behavior
* get extracted data from the bar charts and redo graphics (sn)
===Monday, May 23rd===
* Add more descriptive info on sample and experiment in Study 2 (sn, as)
* insert new graphics (sn)
* add and fix citations (sn)
* Emphasize hypotheses in study 2 (sn)
* change language around how we talk about Teahouse (Jm, sn)
* highlight contribution to gamification literature (jm, sn)
* do multiple passes through the whole paper, editing language, grammar and everything else (sn, as, whoever else wants to)
* do multiple passes through the whole paper, editing language, grammar and everything else (sn, as, whoever else wants to)
* share this with folks peer reviewing CSCW paper (sn)
* share this with folks peer reviewing CSCW paper (sn)
===Friday, May 20th===
* read it a hundred times, make sure the title works
* read it a hundred times, make sure the title works


===Friday, May 27th===
===Friday, May 27th===
* submit final version! (though earlier is better)
* submit final version! (though earlier is better)
== Post-submission TO-DO list ==
* Reproducibility
** Create RData files for all datasets, models, results, etc.
** Convert paper to .rnw (knitr).
** Create communitydata (wiki?) page to host reproducibility content
** Make sure git repository is up to date and all files have READMEs
* Study 2 analysis
** Report full models w all model fit stats etc. (presumably we cut this down for the CHI note. No need for CSCW).
** Collect additional covariates and include them in the models
*** user data via API (gender, edit count pre-inclusion in the study, etc.)
*** added measures should enhance precision of estimates (even smaller SEs!)
** Convert boxplots to a faceted grid in ggplot2.

Latest revision as of 12:27, 27 May 2016

Project page for The Wikipedia Adventure paper co-authored by Sneha Narayan, Jake Orlowitz, Jonathan Morgan, Mako Hill, and Aaron Shaw.


Current Status[edit]

We are in the process of revising this for submission to CSCW 2017. The timeline and milestones for submission are living on this page (below).

Resources[edit]

  • We're writing on sharelatex. The project lives here.
  • Bibliography is currently stored in a shared zotero directory. Aaron and Mako can share access if needed.


Current TO DO list[edit]

  • introduction
  • Background and prior work
    • Continue to improve gamification subsection (mako)
  • System design
    • Use Webstrates paper (UIST 2015) as a model (mako)
  • Study 1:
  • Study 2:
  • Discussion & Conclusion
    • make sure connections to framing are clear
  • Throughout
    • Full read-throughs for tone, consistency, clear contributions.
    • Proofreading

Recent changes log[edit]

These have been addressed, but will likely still benefit from further attention so they're here for archival purposes

  • introduction
  • compress the introduction. reduce redundancy (as)
    • focus on two study framing
    • Change framing to show that two studies were always part of the plan
    • Incorporate idea that large scale field testing is important
    • emphasize hypotheses
      • challenges of deploying new systems in a massive community with passionate, experienced members
      • large-scale user survey and invitation-based field experiment under realistic conditions provide more useful evidence (than hallway testing or small-scale usability studies).
  • Study 1:
    • replace bar charts w new visualizations (sn)
  • Study 2:
    • emphasize hypotheses
    • add details of random assignment (sn)
    • add more descriptive information about the sample and experiment
    • incorporate randomization check (compare days in study across treatment & control)
    • restrict main analysis to last 100 days of sampling (sn)
      • revise all reported statistics to reflect this
    • include boxplots (w scatter) of dependent variables for all treatment and control units.
    • Include Mann-Whitney tests for distribution shift
  • Discussion & Conclusion
    • Make sure to address big contributions.
    • Elaborate explanations of null effect.
    • Note that most studies on gamified systems don't do quantitative impact assessment, highlight this as a contribution of our
  • Throughout
  • re-built refs-processed and eliminated missing citation errors (as)
  • change language around Teahouse (jm)
    • Add citations currently in text
    • Remove excess uses of passive voice
    • Add keywords and ACM categories.

Timeline for CSCW 2017 submission[edit]

  • do multiple passes through the whole paper, editing language, grammar and everything else (sn, as, whoever else wants to)
  • share this with folks peer reviewing CSCW paper (sn)
  • read it a hundred times, make sure the title works

Friday, May 27th[edit]

  • submit final version! (though earlier is better)

Post-submission TO-DO list[edit]

  • Reproducibility
    • Create RData files for all datasets, models, results, etc.
    • Convert paper to .rnw (knitr).
    • Create communitydata (wiki?) page to host reproducibility content
    • Make sure git repository is up to date and all files have READMEs
  • Study 2 analysis
    • Report full models w all model fit stats etc. (presumably we cut this down for the CHI note. No need for CSCW).
    • Collect additional covariates and include them in the models
      • user data via API (gender, edit count pre-inclusion in the study, etc.)
      • added measures should enhance precision of estimates (even smaller SEs!)
    • Convert boxplots to a faceted grid in ggplot2.