Editing CommunityData:Git

== Install Git==

To get started, you will need to '''install git'''. Doing so requires different steps depending on your operating system. Basic instructions available from [https://git-scm.com/downloads the Git website]. 

Mac OS users should check out [https://www.moncefbelyamani.com/how-to-install-xcode-homebrew-git-rvm-ruby-on-mac/ this tutorial], which recommends also setting up the "Command Line Tools" as well as the "homebrew" package manager.

== Configuring Git for submodules ==

Once you've installed git, there are some configuration options which will make your life much easier. You can set them globally with the following commands:

 git config --global alias.spull '!__git_spull() { git pull "$@" && git submodule sync --recursive && git submodule update --init --recursive; }; __git_spull'
 git config --global status.submoduleSummary true

These two commands will ensure that git works a little better with submodules. Submodules are essentially git repositories that are buried inside other git repositories. For example, the <code>wikiresearch</code> repository currently uses the <code>RCommunityData</code> repository as a submodule. If you're working in a repository like this, you'll want to use <code>git spull</code> instead of just <code>git pull</code> which will also check for and pull changes made in any of your submodules.

== Gitolite Server ==

We have a private git server which uses [http://gitolite.com/gitolite/index.html gitolite] to manage permissions for git repositories.

=== Dependencies ===

Once you've got git installed, you will also need a [https://help.github.com/articles/generating-ssh-keys/ public SSH key]. You can send your public key (usually ~/.ssh/id_rsa.pub) to a current administrator (see the list of administrators below on this page), and they can add you as a new user.

=== Cloning a repository ===

"Cloning" a repository downloads the files, as well as the history, of a repository. It also creates a new git instance in that directory, so that you can commit changes to the code.

To clone a repository, run the following command:

  git clone --recursive git@code.communitydata.science:''REPOSITORY_NAME''

Note that you need to use this SSH syntax rather than the [https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols git protocol] (e.g., <code>git://code.communitydata.science/repo_name</code>), which doesn't have write permissions.

=== Creating a new repository ===

To create a new repository, you will need to have admin rights. Currently, the administrators are Kaylea, Nate, Jeremy, Aaron, Mako, Sayamindu, and Jim. If you'd like to be an administrator, you should contact one of them!

=== Using git-annex to manage large files in git ===

{{note}} This is still experimental, and may go away. Don't put files in it without a backup.

Git is not a very good system for managing large files, which is a problem for us, since we often have large data files. Enter [https://git-annex.branchable.com/walkthrough/ git-annex], a system that works in tandem with git and lets you store large files (but avoids using git as the data store). Our gitolite installation supports git-annex. To start using git-annex, install git-annex locally in your computer. Most GNU/Linux distributions has git-annex packages. Then, in your existing git repository execute the following initialization command:
  
<source lang='bash'>
$ git annex init
</source>

This needs to be done only once. To add a file, in your repository, run the following commands:

<source lang='bash'>
$ mkdir data
$ cp ~/largedata.csv.bz2 data/
</source>

You should encrypt the file if the data is not public. You can use GNU Privacy Guard to do the encryption, and have all your collaborators as recipients for the file. Once encrypted, execute the following commands to include and push the file to the server.

<source lang='bash'>
$ git annex add data/largedata.csv.bz2.gpg
$ git commit -m "Added data file"
$ git push --all
$ git annex copy --to origin
</source>
Once these commands are successful, your collaborators should be able to get the file with the following command (assuming that they have already run <code>git annex init</code>):

<source lang='bash'>
$ git annex get data/largedata.csv.bz2.gpg
</source>

Once you've encrypted non-public data, git-annex is easy to use using the webapp. 
<source lang='bash'>
$ git annex webapp
</source>

=== Details for Administrators ===
==== Creating new repositories ====

If you are all already administrator, this describes how you will create a new repository.

First, you will need to clone the gitolite-admin repository

 $ git clone git@code.communitydata.science:gitolite-admin

And then edit the file <code>conf/gitolite.conf</code>. To add a new project, simply create a new entry at the bottom of the file.

For example,
 
<source lang='apache'>
repo foo
    RW+ = aaron mako
    R   = jdfoote
</source>

would create a new repository at git@code.communitydata.science:foo with aaron and mako as admins, and give jdfoote read-only access *once this file was saved, committed, and pushed*.

You could then go to wherever the files are that you would like to track, and add this repository as a remote, like so:
<source lang='bash'>
$ cd foo
$ git init
$ git remote add origin git@code.communitydata.science:foo
$ git add ./ # Adding everything to be tracked in git
$ git commit 
$ git push --set-upstream origin master
</source>

If this project already exists in git, then it's even easier. Just change the remote, and push it.

<source lang='bash'>
$ git remote set-url origin git@code.communitydata.science:foo
$ git push
</source>

==== Adding new users ====

To add new users, simply add their public key to the <code>keydir/</code> directory, renamed as <code>username.pub</code>. The persons username (as called in the <code>code/gitolite.conf</code> file) will be whatever the username in the filename above is.