Editing CommunityData:Hyak Spark
From CommunityData
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 294: | Line 294: | ||
</syntaxhighlight> | </syntaxhighlight> | ||
This will setup the cluster. | This will setup the cluster. Take note of the node that is assigned to be the master, and use that information to set your $SPARK_MASTER environment variable, for example <code>export SPARK_MASTER="n0650"</code>. The program <code> spark-submit </code> submits your script to the running Spark cluster. | ||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> |