CommunityData:Hyak migration (mox to klone): Difference between revisions

Revision as of 23:15, 30 July 2024

This page includes resources that are useful for migration of things on Hyak mox: from Ikt to Mox (completed in June 2020) in Hyak; and the winding down of mox in 2024.

Ikt to Mox (2020) home directories

You're responsible for moving your home directories from Ikt to Mox. To do so, follow these instructions:

1. on IKT: prepare your homedirectory

First, go through and delete the things you don't want in your home directory. the program ncdu is installed on ikt and it will show you detailed information of what is taking up space. go through and recursively delete thing you don't want. if you want to archive things on cold storage, you can do that too by putting compressed tar files in /com/STAGING-cold_storage (we're strongly encouarged to ensure that each file is more than 100MB).

There are likely many hidden files as well. You can see those with "ls -a".

2. on IKT: make a tarball of your home directory

You will need to make a tarball of your home directory and store it in /com/STAGING-bbcp

Note: change every instance of "<YOURUWNETID>" to your NetID (e.g., "makohill") before running the instructions below

$ ssh login2.hyak.uw.edu
$ cd /usr/lusers
$ tar cvf /com/STAGING-bbcp/<YOURUWNETID>_ikt_homedir.tar <YOURUWNETID>

3. on IKT: copy the file over to your new homedirectory on MOX

$ cd /com/STAGING-bbcp
$ hyakbbcp <YOURUWNETID>_ikt_homedir.tar mox1.hyak.uw.edu:/usr/lusers/<YOURUWNETID>/

4. on MOX: unpack you new data

Run the following:

$ cd /usr/lusers/<YOURUWNETID>
$ tar xvf <YOURUWNETID>_ikt_homedir.tar --transform 's/^<YOURUWNETID>/ikt_homedir/'
$ rm ~/<YOURUWNETID>_ikt_homedir.tar

This will create a new directory called ikt_homedir in my home directory on mox. I'll then go through and move out the things from ikt_homedir into ../ your new home dirctory. Or just leave them there if you want.

Winding down mox in 2024

Mox is winding down in Fall 2024, and each person will need to move everything they want to keep to a personal machine, Hyak klone, or to condo. This is a good opportunity for some spring cleaning and file hygiene.

For directories you are responsible for: (1) Wherever possible, delete any files you no longer need; this includes duplicate files, temp files, data that is stored elsewhere or can be easily retrieved, etc. The `du` command will likely be useful here. (2) Zip and compress your directory. (3) Take it off of mox. If it's going to klone, you can do the below. If it's going to condo, there should be a directory somewhere called COLD_STAGING or something similar that you can place it in.

Migrating files

From mox to klone

   mox$ rsync myfile klone.hyak.uw.edu:/gscratch/comdata/users/YOUR_ID/YOUR_DIR
   mox$ rsync -r mydir klone.hyak.uw.edu:/gscratch/comdata/users/YOUR_ID/YOUR_DIR

It will ask you to ssh connect to klone

@@ Line 42: / Line 42: @@
 This will create a new directory called <code>ikt_homedir</code> in my home directory on mox. I'll then go through and move out the things from <code>ikt_homedir</code> into <code>../</code> your new home dirctory.
 Or just leave them there if you want.
-== Moving files from ikt to mox ==
-You can copy files at high speed without a password between the Hyak systems using commands like the ones below (instructions from the [http://wiki.cac.washington.edu/display/hyakusers/Hyak+mox+Overview Hyak documentation]).
-'''From ikt to mox'''
-    ikt1$ hyakbbcp myfile mox1.hyak.uw.edu:/gscratch/comdata/users/YOUR_ID/YOUR_DIR
-    ikt1$ hyakbbcp -r mydirectory mox1.hyak.uw.edu:/gscratch/comdata/users/YOUR_DIR
-'''From mox to ikt'''
-    mox1$ hyakbbcp myfile ikt1.hyak.uw.edu:/com/users/YOUR_DIR
-    mox1$ hyakbbcp -r mydirectory ikt1.hyak.uw.edu:/com/users/YOUR_DIR
 == Winding down mox in 2024 ==