Editing CommunityData:TACC (section)

== TACC Filesystems ==

TACC users will work with 4 distributed filesystems:

* '''Home''' is persistent, not fast, and backed up.  Each user has a 15GB allocation. This is a good place to install software, code, and configuration. You can refer to your home directory in scripts using the <code>$HOME</code> environment variable. You can quickly navigate to your home directory using the <code>cdh</code> alias.
* '''Work''' is persistent and fast. Each ''project'' has a 1TB allocation. This is a good place to store data that you are actively working with and need to persist. Use the <code>$WORK</code> environment variable and the <code>cdw</code> alias.
* '''CORRAL''' is persistent, slower, but larger. Also, it is inaccessible from compute nodes. So you need to copy data from corral to work or scratch before working with it. The environment variable is <code>$CORRAL</code>
* '''Scratch''' is fast and unlimited, but unaccessed files are automatically removed after 10 days.  Use it for intermediate stages in a data analysis pipeline or for data that is too large for '''Work'''. The environment variable is <code>$SCRATCH</code> and the command is <code>cds</code>.

=== Note on set-up ===

When you run scripts, many temporary files, the logging, etc. will store to <code>$HOME</code> but you will run out of space and things will crash. Set it up to save to <code>$SCRATCH</code> instead, as it is larger but not persistent. So far, Sohyeon has had to do the follow:

<code>
export XDG_CACHE_HOME=$SCRATCH/.cache
export IPYTHONDIR=$SCRATCH/.ipython
export DUCKDB_TMPDIR=$SCRATCH/tmp
export SLURM_STATE_SAVE_LOCATION=$SCRATCH/.slurm
</code>