The Wikipedia Adventure: Difference between revisions
From CommunityData
| (32 intermediate revisions by 2 users not shown) | |||
| Line 11: | Line 11: | ||
== | == Current TO DO list == | ||
* '''introduction''' | |||
* '''Background and prior work''' | |||
** Continue to improve gamification subsection (mako) | |||
* '''System design''' | |||
** Use Webstrates paper (UIST 2015) as a model (mako) | |||
* '''Study 1:''' | |||
* '''Study 2:''' | |||
* '''Discussion & Conclusion''' | |||
** make sure connections to framing are clear | |||
* '''Throughout''' | |||
** Full read-throughs for tone, consistency, clear contributions. | |||
** Proofreading | |||
=== | == Recent changes log == | ||
'''These have been addressed, but will likely still benefit from further attention so they're here for archival purposes''' | |||
* '''introduction''' | |||
* | * compress the introduction. reduce redundancy (as) | ||
* | ** focus on two study framing | ||
** | ** Change framing to show that two studies were always part of the plan | ||
** | ** Incorporate idea that large scale field testing is important | ||
** emphasize hypotheses | |||
*** challenges of deploying new systems in a massive community with passionate, experienced members | *** challenges of deploying new systems in a massive community with passionate, experienced members | ||
*** large-scale user survey and invitation-based field experiment under realistic conditions provide more useful evidence (than hallway testing or small-scale usability studies). | *** large-scale user survey and invitation-based field experiment under realistic conditions provide more useful evidence (than hallway testing or small-scale usability studies). | ||
* | * '''Study 1:''' | ||
** replace bar charts w new visualizations (sn) | |||
* '''Study 2:''' | |||
** emphasize hypotheses | |||
** add details of random assignment (sn) | |||
** add more descriptive information about the sample and experiment | |||
** incorporate randomization check (compare days in study across treatment & control) | |||
** restrict main analysis to last 100 days of sampling (sn) | |||
*** revise '''all''' reported statistics to reflect this | |||
** include boxplots (w scatter) of dependent variables for all treatment and control units. | |||
** Include Mann-Whitney tests for distribution shift | |||
* '''Discussion & Conclusion''' | |||
** Make sure to address big contributions. | |||
** Elaborate explanations of null effect. | |||
** Note that most studies on gamified systems don't do quantitative impact assessment, highlight this as a contribution of our | |||
* '''Throughout''' | |||
* | * re-built refs-processed and eliminated missing citation errors (as) | ||
* | * change language around Teahouse (jm) | ||
* | ** Add citations currently in text | ||
* | ** Remove excess uses of passive voice | ||
* | ** Add keywords and ACM categories. | ||
** | |||
** | |||
=== | ==Timeline for CSCW 2017 submission== | ||
* do multiple passes through the whole paper, editing language, grammar and everything else (sn, as, whoever else wants to) | |||
* share this with folks peer reviewing CSCW paper (sn) | |||
* read it a hundred times, make sure the title works | * read it a hundred times, make sure the title works | ||
| Line 55: | Line 69: | ||
* submit final version! (though earlier is better) | * submit final version! (though earlier is better) | ||
== TO DO list == | == Post-submission TO-DO list == | ||
* | * Reproducibility | ||
** | ** Create RData files for all datasets, models, results, etc. | ||
** | ** Convert paper to .rnw (knitr). | ||
** | ** Create communitydata (wiki?) page to host reproducibility content | ||
* | ** Make sure git repository is up to date and all files have READMEs | ||
* Study 2 analysis | |||
** Report full models w all model fit stats etc. (presumably we cut this down for the CHI note. No need for CSCW). | |||
** Collect additional covariates and include them in the models | |||
*** user data via API (gender, edit count pre-inclusion in the study, etc.) | |||
*** added measures should enhance precision of estimates (even smaller SEs!) | |||
** Convert boxplots to a faceted grid in ggplot2. | |||
Latest revision as of 14:27, 27 May 2016
Project page for The Wikipedia Adventure paper co-authored by Sneha Narayan, Jake Orlowitz, Jonathan Morgan, Mako Hill, and Aaron Shaw.
Current Status[edit]
We are in the process of revising this for submission to CSCW 2017. The timeline and milestones for submission are living on this page (below).
Resources[edit]
- We're writing on sharelatex. The project lives here.
- Bibliography is currently stored in a shared zotero directory. Aaron and Mako can share access if needed.
Current TO DO list[edit]
- introduction
- Background and prior work
- Continue to improve gamification subsection (mako)
- System design
- Use Webstrates paper (UIST 2015) as a model (mako)
- Study 1:
- Study 2:
- Discussion & Conclusion
- make sure connections to framing are clear
- Throughout
- Full read-throughs for tone, consistency, clear contributions.
- Proofreading
Recent changes log[edit]
These have been addressed, but will likely still benefit from further attention so they're here for archival purposes
- introduction
- compress the introduction. reduce redundancy (as)
- focus on two study framing
- Change framing to show that two studies were always part of the plan
- Incorporate idea that large scale field testing is important
- emphasize hypotheses
- challenges of deploying new systems in a massive community with passionate, experienced members
- large-scale user survey and invitation-based field experiment under realistic conditions provide more useful evidence (than hallway testing or small-scale usability studies).
- Study 1:
- replace bar charts w new visualizations (sn)
- Study 2:
- emphasize hypotheses
- add details of random assignment (sn)
- add more descriptive information about the sample and experiment
- incorporate randomization check (compare days in study across treatment & control)
- restrict main analysis to last 100 days of sampling (sn)
- revise all reported statistics to reflect this
- include boxplots (w scatter) of dependent variables for all treatment and control units.
- Include Mann-Whitney tests for distribution shift
- Discussion & Conclusion
- Make sure to address big contributions.
- Elaborate explanations of null effect.
- Note that most studies on gamified systems don't do quantitative impact assessment, highlight this as a contribution of our
- Throughout
- re-built refs-processed and eliminated missing citation errors (as)
- change language around Teahouse (jm)
- Add citations currently in text
- Remove excess uses of passive voice
- Add keywords and ACM categories.
Timeline for CSCW 2017 submission[edit]
- do multiple passes through the whole paper, editing language, grammar and everything else (sn, as, whoever else wants to)
- share this with folks peer reviewing CSCW paper (sn)
- read it a hundred times, make sure the title works
Friday, May 27th[edit]
- submit final version! (though earlier is better)
Post-submission TO-DO list[edit]
- Reproducibility
- Create RData files for all datasets, models, results, etc.
- Convert paper to .rnw (knitr).
- Create communitydata (wiki?) page to host reproducibility content
- Make sure git repository is up to date and all files have READMEs
- Study 2 analysis
- Report full models w all model fit stats etc. (presumably we cut this down for the CHI note. No need for CSCW).
- Collect additional covariates and include them in the models
- user data via API (gender, edit count pre-inclusion in the study, etc.)
- added measures should enhance precision of estimates (even smaller SEs!)
- Convert boxplots to a faceted grid in ggplot2.
