Editing CommunityData:Hyak software installation
From CommunityData
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 55: | Line 55: | ||
{{note}} If you are using <code>screen</code> to run and manage your builds, keep in mind that <code>screen</code> [https://superuser.com/a/235773 drops a few environment variables] such as <code>LD_LIBRARY_PATH</code>, which may mess up your build process. You should check that all the relevant environment variables are set before starting your build. | {{note}} If you are using <code>screen</code> to run and manage your builds, keep in mind that <code>screen</code> [https://superuser.com/a/235773 drops a few environment variables] such as <code>LD_LIBRARY_PATH</code>, which may mess up your build process. You should check that all the relevant environment variables are set before starting your build. | ||
The first step toward installing and making available a custom module (in this case, | The first step toward installing and making available a custom module (in this case, R 3.5.0) is to spin up the build node, download R, compile it with a specific prefix, and install it. | ||
<source lang='bash'> | <source lang='bash'> | ||
$ build_machine | $ build_machine | ||
$ wget https:// | $ module load contrib/texlive/2017 # loads the texlive module that is helpful for generating R documentation | ||
$ tar | $ module load contrib/openblas/0.2.20 # loads the openblas library, which speeds up some R operations significantly | ||
$ cd | $ wget https://cran.r-project.org/src/base/R-3/R-3.5.0.tar.gz | ||
$ ./configure --prefix=/gscratch/comdata/modules/sw/ | $ tar xzvf R-3.5.0.tar.gz | ||
$ cd R-3.5.0 | |||
$ ./configure --prefix=/gscratch/comdata/modules/sw/R/3.5.0 --without-x --enable-R-shlib --with-lapack --with-blas="-L/sw/contrib/openblas/0.2.20/lib -lopenblas" | |||
$ make | $ make | ||
$ make install | $ make install | ||
</source> | </source> | ||
The <code>--prefix</code> option to <code>./configure</code> tells the build scripts that | The <code>--prefix</code> option to <code>./configure</code> tells the build scripts that R is going to be installed in <code>/gscratch/comdata/modules/sw/R/3.5.0</code>. This follows a convention that we picked—software in modules should go into <code>/gscratch/comdata/modules/sw/{SOFTWARE_NAME}/{SOFTWARE_VERSION}</code>. The <code>--prefix</code> option is the most important flag for <code>./configure</code>—any other flag or option will be specific to the software being installed. | ||
The second step is to write a <code>modulefile</code>. This contains the metadata about our module. Edit the file <code>/gscratch/comdata/modules/modulefiles/ | The second step is to write a <code>modulefile</code>. This contains the metadata about our module. Edit the file <code>/gscratch/comdata/modules/modulefiles/R/3.5.0</code> to contain the following | ||
<source lang='tcl'> | <source lang='tcl'> | ||
Line 75: | Line 77: | ||
## | ## | ||
proc ModulesHelp { } { | proc ModulesHelp { } { | ||
puts stderr "\tModule providing | puts stderr "\tModule providing R 3.5.0." | ||
} | } | ||
module-whatis "Module providing | module-whatis "Module providing R 3.5.0." | ||
prepend-path PATH /gscratch/comdata/modules/sw/ | module load contrib/openblas/0.2.20 # this loads any module dependencies | ||
prepend-path MANPATH /gscratch/comdata/modules/sw/ | prepend-path PATH /gscratch/comdata/modules/sw/R/3.5.0/bin | ||
prepend-path MANPATH /gscratch/comdata/modules/sw/R/3.5.0/share/man | |||
</source> | </source> | ||
Note that the filename follows a similar convention as <code>--prefix</code> earlier (<code>/gscratch/comdata/modules/modulefiles/{SOFTWARE_NAME}/{SOFTWARE_VERSION}</code>). This file sets up the <code>PATH</code> | Note that the filename follows a similar convention as <code>--prefix</code> earlier (<code>/gscratch/comdata/modules/modulefiles/{SOFTWARE_NAME}/{SOFTWARE_VERSION}</code>). This file sets up the <code>PATH</code> and <code>MANPATH</code> environment variables appropriately so that the specified version of R can be accessed and run as needed. There are many more directives that can go into the <code>modulefile</code>—see <code>man modulefile</code> for details on those directives. | ||
Once this file is written out, the <code>module avail</code> command should list <code> | Once this file is written out, the <code>module avail</code> command should list <code>R/3.5.0</code> as an available module. This is because the module system is set up to look inside <code>/gscratch/comdata/modules/modulefiles</code> for module files, thanks to the <code>MODULEPATH</code> variable that is set through <code>.bashrc</code>. The command <code>module load R/3.5.0</code> should make R available and ready for use. To avoid running <code>module load R/3.5.0</code> whenever you log in, you can add the command at the end of your <code>.bashrc</code> file (after the section that sets <code>MODULEPATH</code>). |