Editing CommunityData:Hyak Spark
From CommunityData
Warning: You are not logged in. Your IP address will be publicly visible if you make any edits. If you log in or create an account, your edits will be attributed to your username, along with other benefits.
The edit can be undone. Please check the comparison below to verify that this is what you want to do, and then publish the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 452: | Line 452: | ||
spark java.io.IOException: No subfolder can be created in . | spark java.io.IOException: No subfolder can be created in . | ||
the University of Google told me it was a disk space issue. | the University of Google told me it was a disk space issue. Re-running the same job with a 4-node cluster instead of 2-node cluster seemed to fix the problem, which is kinda strange since all my code uses a big shared disk. | ||
=== Slurm kills my job! Encountering Memory Limits === | === Slurm kills my job! Encountering Memory Limits === |