Editing CommunityData:Klone (section)

== Creating a apptainer container == 

Our goal is to write a apptainer definition file that will install the software that we want to work with.  The definition file contains instructions for building a more reproducible environment.  For example,  the file <code>cdsc_base.def</code> contains instructions for installing an environment based on debian 11 (bullseye). Once we have the definition file, we just have to run:

'''NOTE:''' For some reason building a container doesn't work on the <code>/gscratch</code> filesystem.  Instead build containers on the <code>/mmfs1</code> filesystem and then copy them to their eventual homes on <code>/gscratch</code>. 

<syntaxhighlight language='bash'>
 apptainer build --fakeroot cdsc_base.sif cdsc_base.def
</syntaxhighlight>

On a klone compute node to create the apptainer container <code>cdsc_base.sif</code>.  This can take quite awhile to run as it downloads and installs a lot of software!

You can start a shell in the container using:

<syntaxhighlight language='bash'>
apptainer shell cdsc_base.sif
</syntaxhighlight>

You can also just execute a single command using:

<syntaxhighlight language='bash'>
apptainer exec cdsc_base.sif echo "my command"
</syntaxhighlight>

Sandbox containers don't seem to work consistently.  It's better to just update the definition file and rebuild the container. It's a hassle, but it works. 
<strike>
The <code>.sif</code> container is immutable, but you can modify it by converting it to a sandbox. 

<syntaxhighlight language='bash'>
apptainer build --sandbox cdsc_base_sandbox cdsc_base.sif
</syntaxhighlight>

You might run into trouble with exceeding space in your temporary file path. If you do, run 
<syntaxhighlight language='bash'>
sudo export APPTAINER_TMPDIR=/my/large/tmp
sudo export APPTAINER_CACHEDIR=/my/large/apt_cache
sudo export APPTAINER_LOCALCACHEDIR=/my/large/apt_cache
</syntaxhighlight>
before running the build.

For developing a container it's useful to use a <code>sandbox</code> container, which is mutable
so you can continue installing software on it. However, you should add your changes to the definition file so you can build immutable containers that are as reproducible as possible. 

The <code>cdsc_base_sandbox</code> is mutable, so we can continue working on that environment and installing more software as we like.  We just need to build it as a <code>.sif</code> file to use it on klone.  It's also possible to convert the container back into sandbox mode and then modify non-root parts of the container on klone, but this requires running the container in a way that makes the outside klone system invisible! This is useful for installing R or Python packages in userspace within the container.  It's not that useful for working with data outside of the container.
</strike>
So in summary, the workflow is:

# Develop a definition file (<code>cdsc_base.dev</code>) to setup your desired environment.
# Keep the definition file up to date with any modifications you make to the container in sandbox mode so your container is reproducible. 
# Run programs in the container to work with files outside of it (possibly including other packages, allowing us to use debian to bootstrap klone-optimzed binaries).
# If you want to work on you local machine you can use the same definition file to install the container on your local machine.