Editing CommunityData:Automating and Streamlining Walkthrough
From CommunityData
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
Welcome to the | Welcome to the Automation and Streamlining Walkthrough! | ||
This guide steps you through why and how you might like to adopt some of our tips and tricks around automating and streamlining your research workflow. This guide is opinionated---you might not find all of it useful---but this advice is built from lived experience: these are strategies CDSC members use to keep away from certain kinds of annoyances, traps, and mistakes. Some of what's described here will be a '''lot''' easier if you have completed the [[CommunityData:Onboarding Checklist]]. CDSC members will want to make sure they have a fresh copy of the ''' | This guide steps you through why and how you might like to adopt some of our tips and tricks around automating and streamlining your research workflow. This guide is opinionated---you might not find all of it useful---but this advice is built from lived experience: these are strategies CDSC members use to keep away from certain kinds of annoyances, traps, and mistakes. Some of what's described here will be a '''lot''' easier if you have completed the [[CommunityData:Onboarding Checklist]]. CDSC members will want to make sure they have a fresh copy of the '''cdsc_examples''' git repository. | ||
=== Staying Organized === | === Staying Organized === | ||
The kiddie cartoon version of the scientific method describes a linear and rather sterile process from hypothesis to experiment to | The kiddie cartoon version of the scientific method describes a linear and rather sterile process from hypothesis to experiment to insight -- the reality looks a lot messier. We rummage around, scratch our heads, wander down dark alleys, think and re-think, scrape, crunch, gather....and then we look around at all the beautiful mess we've made and try to turn it into a paper: write and re-write, submit, revise, re-submit, re-revise, re-resubmit --- and then maybe a year or two later, we're announcing, releasing, publishing and presenting. Keeping track of the weird and wild ride can be tremendously helpful when it comes time to write it up and tell the world what you've learned. | ||
# '''Take notes''' for yourself as you go along | # '''Take notes''' for yourself as you go along -- think of it as keeping a lab notebook. Use whatever works for you, but make sure it is: | ||
## Searchable | ## Searchable | ||
## Backed up | ## Backed up | ||
## Filled with all kinds of context. Copy in commands, URLs, wild ideas, things you tried and why | ## Filled with all kinds of context. Copy in commands, URLs, wild ideas, things you tried and why. | ||
# Don't let data-related details fall through the cracks. [[CommunityData:Keeping Track of Metadata | Keep track of metadata]] because future you will forget. | # Don't let data-related details fall through the cracks. [[CommunityData:Keeping Track of Metadata | Keep track of metadata]] because future you will forget. | ||
=== Automating === | === Automating === | ||
Automation can be extremely helpful, but it's an investment. You will not regret time spent on modest automation, in particular if you do computational work. You never want to be in the position of copy-pasting from R into LaTeX or Word: it is error prone and when you later find yourself needing to revise, you might not remember where you got that number from ....... instead, what you want is some automation magic, so that every time you run your R code, your new data and fresh visualizations land in your Overleaf. Example code for making this work is in the '''cdsc_examples/ | Automation can be extremely helpful, but it's an investment. You will not regret time spent on modest automation, in particular if you do computational work. You never want to be in the position of copy-pasting from R into LaTeX or Word: it is error prone and when you later find yourself needing to revise, you might not remember where you got that number from ....... instead, what you want is some automation magic, so that every time you run your R code, your new data and fresh visualizations land in your Overleaf. Example code for making this work is in the '''cdsc_examples/R_examples/automation''' git repository. If you want to learn even more, there's [[Knitr_tutorial| a little tutorial on Knitr]], and here's [[CommunityData:Knitr | a more expanded guide]]. | ||
# [[CommunityData:Build_papers | Set up this marvelous R - Overleaf automation]] | # [[CommunityData:Build_papers | Set up this marvelous R - Overleaf automation]] | ||
Line 33: | Line 31: | ||
Revise and resubmit makes papers better but can be a time crunch and if you have co-authors, it can get hard to tell what's been done or what certain revisions were trying to accomplish. When you get reviews back that ask you to make changes, you can save yourself a lot of annoyance if you follow this four-part method: | Revise and resubmit makes papers better but can be a time crunch and if you have co-authors, it can get hard to tell what's been done or what certain revisions were trying to accomplish. When you get reviews back that ask you to make changes, you can save yourself a lot of annoyance if you follow this four-part method: | ||
# Keep a copy of your original submission, including the main.tex file (if you are using .Rtex file because you followed the above advice about automation....please note that the main.tex file is hidden in Overleaf -- click the little 'document' icon next to 'Recompile' (mouseover has a tooltip that says 'Logs and output files' -- then 'Other logs and files' at the bottom right -- note that the output.bbl file is there too, that's also useful because you'll want it for your upload to arXiv if you post a preprint). You will need this for the LaTeX Diff below | # Keep a copy of your original submission, including the main.tex file (if you are using .Rtex file because you followed the above advice about automation....please note that the main.tex file is hidden in Overleaf -- click the little 'document' icon next to 'Recompile' (mouseover has a tooltip that says 'Logs and output files' -- then 'Other logs and files' at the bottom right -- note that the output.bbl file is there too, that's also useful because you'll want it for your upload to arXiv if you post a preprint). You will need this for the LaTeX Diff below. | ||
# Paste all reviews into a Google Sheet | # Paste all your reviews into a Google Sheet -- [https://docs.google.com/spreadsheets/d/1ZMMKI3HQ6bOTRT0oF5ohBtVLqo3ye65J5WJ-rn6WxsM/edit?usp=sharing feel free to use this as a template] | ||
# | # Also paste all your reviews into a response to reviewers letter in the same Overleaf project where you wrote the paper. Place them below the '''/end{document}''' line so it won't show up.. As you revise to address the issue, move the comment up into the letter, quote it and then write what you did. It might feel a skosh redundant with the Google Sheet, and it is ... think of it as double-entry bookkeeping to make sure nothing gets dropped. | ||
# Show your work: some venues require (and reviewers often appreciate) a document that shows what was changed. If you use Word, that means turning on Track Changes. If you use LaTeX, [[CommunityData:LaTex Diff | latexdiff is a way to achieve this]]. Latexdiff is available on [[CommunityData:Kibo]] | # Show your work: some venues require (and reviewers often appreciate) a document that shows what was changed. If you use Word, that means turning on Track Changes. If you use LaTeX, [[CommunityData:LaTex Diff | latexdiff is a way to achieve this]]. Latexdiff is available on [[CommunityData:Kibo]]. | ||
Please note that the '''cdsc_example''' repository contains examples of past revision processes. | Please note that the '''cdsc_example''' repository contains examples of past revision processes. |