Editing CommunityData:Git (section)

== Using git-annex to manage large files in git ==

{{note}} This is still experimental, and may go away. Don't put files in it without a backup.

=== Getting Set Up ===

Git is not a very good system for managing large files, which is a problem for us, since we often have large data files. Enter [https://git-annex.branchable.com/walkthrough/ git-annex], a system that works in tandem with git and lets you store large files (but avoids using git as the data store). Our gitolite installation supports git-annex. To start using git-annex, install git-annex locally in your computer. Most GNU/Linux distributions have git-annex packages. If you're on a Mac, in Terminal.app, try the instructions from Homebrew: https://formulae.brew.sh/formula/git-annex. 

=== Setting Up Your Repo To Use Annex ===
Then, in your existing git repository execute the following initialization command:
  
<source lang='bash'>
$ git annex init
</source>

This needs to be done only once. To add a file, in your repository, run the following commands:

<source lang='bash'>
$ mkdir data
$ cp ~/largedata.csv.bz2 data/
</source>

You should encrypt the file if the data is not public. You can use GNU Privacy Guard to do the encryption, and have all your collaborators as recipients for the file. Once encrypted, execute the following commands to include and push the file to the server.

<source lang='bash'>
$ git annex add data/largedata.csv.bz2.gpg
$ git commit -m "Added data file"
$ git push --all
$ git annex copy --to origin
</source>

=== Using an Existing Annex Repo ===

Once these commands are successful, your collaborators should be able to get the file with the following command (assuming that they have already run <code>git annex init</code>):

<source lang='bash'>
$ git annex get data/largedata.csv.bz2.gpg
</source>

Once you've encrypted non-public data, git-annex is easy to use using the webapp. 
<source lang='bash'>
$ git annex webapp
</source>