Editing CommunityData:Hyak Datasets (section)

=== Install Arrow for R ===

If you want to use Arrow in R and your R on Hyak doesn't already have Arrow installed, follow these steps. On computers not running CentOS you'll probably be fine just running <code>install.packages("arrow")</code>.  These instructions are derived from this debugging session on the [https://issues.apache.org/jira/browse/ARROW-9303 Arrow bug tracker].

First you need to load a modern cmake and set some environment variables.

<syntaxhighlight lang='bash'>
module load cmake/3.11.2

export ARROW_WITH_LZ4=ON; export ARROW_WITH_ZSTD=ON; export ARROW_WITH_BZ2=ON; export ARROW_WITH_GZIP=ON; export ARROW_WITH_LZ4_FRAME=ON; export ARROW_WITH_SNAPPY=ON; export ARROW_WITH_LZO=ON; ARROW_WITH_BROTLI=ON;
 
export LIBARROW_MINIMAL=FALSE
</syntaxhighlight>


Now, start R and '''download''' (not install!) the <code>arrow</code> package. 

<syntaxhighlight lang='R'>
download.packages("arrow",destdir='.')
</syntaxhighlight>

Now, you need to unpack <code>arrow_0.17.1.tar.gz</code> and edit <code>arrow/inst/build_arrow_static.sh</code>.

<syntaxhighlight lang='bash'>
tar xvzf arrow_0.17.1.tar.gz
nano arrow/inst/build_arrow_static.sh
</syntaxhighlight>

In <code>build_arrow_static.sh</code>, modify the value of <code>DARROW_DEPENDENCY_SOURCE</code> and set it to <code>BUNDLED</code>.

<syntaxhighlight lang='bash'>
# build_arrow_static.sh
...
-DARROW_DEPENDENCY_SOURCE=BUNDLED
...
</syntaxhighlight>

Finally, go back into R and finish installing arrow.

<syntaxhighlight lang='R'>

install.packages("arrow",repos=NULL)

</syntaxhighlight>