CommunityData:Automating and Streamlining Walkthrough

Welcome to the Automation and Streamlining Walkthrough!

This guide steps you through why and how you might like to adopt some of our tips and tricks around automating and streamlining your research workflow. Our questions and methods lead to a couple of challenges and complexities. These are strategies we use to keep away from certain kinds of annoyances, traps, and mistakes. Some of what's described here will be a *lot* easier if you have completed the CommunityData:Onboarding Checklist. CDSC members will want to make sure they have a fresh copy of the cdsc_examples git repository.

Staying Organized

The kid cartoon version of the scientific method describes a linear and rather sterile process from hypothesis to experiment to insight -- the reality looks a lot messier. We rummage around, scratch our heads, wander down dark alleys, think and re-think, scrape, crunch, gather....and then we look around at all the beautiful mess we've made and try to turn it into a paper: write and re-write, submit, revise, re-submit, re-revise, re-resubmit --- and then maybe a year or two later, we're announcing, releasing, publishing and presenting. Keeping track of the weird and wild ride can be tremendously helpful.

CommunityData:Keeping Track of Metadata

Automating Updates

Automation can be extremely helpful, but it's an investment. You will not regret time spent on modest automation, in particular if you do computational work. You never want to be in the position of copy-pasting from R into LaTeX or Word.

We have an overall guide for setting up this automation.

Here's a little tutorial on Knitr, and here's a more expanded guide.

Building from Prior Efforts

CommunityData:Zotero CommunityData:TeX — Installing our LaTeX templates CommunityData:LaTex Diff — For an R+R, it's often helpful to create a PDF that shows the changes made. Here's one way to do that. Latexdiff is available on CommunityData:Kibo.