CommunityData:Hyak migration (mox to klone)

CONTEXT: Hyak is multiple supercomputers. We have allocations on the two extant supercomputers: mox (old) and klone (new). Mox is shutting down very soon. The systems appear similar but /nothing is shared/ between them. Files and software on mox is not on klone. Everyone reading this should have received multiple emails about the mox shutdown. Tomorrow id the deadline for moving/saving files on mox.

Mox is winding down in Fall 2024, and each person will need to move everything they want to keep to a personal machine, Hyak klone, or to condo. This is a good opportunity for some spring cleaning and file hygiene.

For directories you are responsible for:

1. Wherever possible, delete any files you no longer need; this includes duplicate files, temp files, data that is stored elsewhere or can be easily retrieved, etc. The `du` command will likely be useful here.
2. tar and compress your directory.
3. Take it off of mox. If it's going to klone, you can do the below. If it's going to condo, there should be a directory somewhere called COLD_STAGING or something similar that you can place it in.

Migrating files

Mako has now copied everything that was left in the shared CDSC space on mox (mox:/grscratch/comdata/) into a staging space on klone (klone:/gscratch/comdata/MOX_INCOMING/) or into cold storage if it was for a long-dormant project (see 5 below).

Here's what you need to do. #1 is URGENT (MUST HAPPEN BEFORE AUGUST 1):

1. Copy your homedirectory over from mox if you want anything in it.

Log into mox and figure out if there is anything in your home directory you want to keep. Since space is pretty constrained in hyak home directories, the answer is likely no, but you might want to check. If you want to just backup the whole thing, you can do something like what I did below (changing "makohill" to your username, of course):

  mkdir /gscratch/scrubbed/makohill
  any_machine

  # wait for machine to be checked out, then run the following commands from the compute node
  cd /usr/lusers
  XZ_DEFAULTS="-T0" tar Jcvf /gscratch/scrubbed/makohill-mox_homedirectory-20240730.tar.xz makohill
  exit

  # wait to return to the login node
  rsync -P /gscratch/scrubbed/makohill-mox_homedirectory-20240730.tar.xz  klone1.hyak.uw.edu:

  # now log into klone, verify that everything is fine, and then delete the file from /gscratch/scrubbed on mox

2. CRITICAL BEFORE YOU USE KLONE (but not urgent)

Please verify that you have set up your environment on klone. If you did it on mox, you need to follow the instructions on CommunityData:Klone#Setting_up_your_Environment.

Even if you did these before on mox, YOU MUST DO IT AGAIN ON KLONE. Not doing this leads to problems including issues with files that are not group readable/writable which have been a huge headache for me the last weeks. Please do this and ask on IRC if you run into trouble.

3.

If you "left" files on mox, those are now in klone:/gscratch/comdata/MOX_INCOMING

I will be merging those into main filesystem on klone over the next couple weeks. Once you have completed (2), you can help! The obvious thing to do is to move files from /gscratch/comdata/MOX_INCOMING/users/<YOUR USERNAME> into /gscratch/comdata/users/<YOUR USERNAME>