Difference between revisions of "Migration"

From HPC users
Jump to navigationJump to search
(Created page with "== Introduction == As you know, we will get a new HPC cluster this year. This also means that CARL and EDDY will be shutdown at some point (no date decided yet) and all users have to migrate to the new system. Here you will find information for this transition, which will be updated regularly as we go along. == Data Migration == The cluster CARL and EDDY have serveral file systems for storing data. The file systems <tt>$DATA</tt> and <tt>$OFFSITE</tt> will be availab...")
 
Line 7: Line 7:
The cluster CARL and EDDY have serveral file systems for storing data. The file systems <tt>$DATA</tt> and <tt>$OFFSITE</tt> will be available on the new cluster as before, so there is no need to migrate any data here. The directory <tt>$HOME</tt> will be different on the new cluster, however we will provide access to your current <tt>$HOME</tt> so you can transfer files as needed.  
The cluster CARL and EDDY have serveral file systems for storing data. The file systems <tt>$DATA</tt> and <tt>$OFFSITE</tt> will be available on the new cluster as before, so there is no need to migrate any data here. The directory <tt>$HOME</tt> will be different on the new cluster, however we will provide access to your current <tt>$HOME</tt> so you can transfer files as needed.  


The filesystem <tt>$WORK</tt> will also be completely new, and access to the current <tt>$WORK</tt> will only be possible through the old cluster or via SMB. This access will also be limited in time, although we cannot give an exact date for this now. We strongly recommend that you transfer any files you want to keep after CARL and EDDY are shutdown to <tt>$DATA</tt> or <tt>$OFFSITE</tt> as soon as possible. Keep in mind that transfering data is done at rate of about 1TB/hour.
The filesystem <tt>$WORK</tt> will also be completely new, and access to the current <tt>$WORK</tt> will only be possible through the old cluster or via SMB. This access will also be limited in time, details on this will follow. While you should have enough time to migrate your recent results from the current <tt>$WORK</tt>, we strongly recommend that you transfer all files that you not actively need for computations and that you want to keep after CARL and EDDY are shutdown to <tt>$DATA</tt> or <tt>$OFFSITE</tt> as soon as possible. Files that are no longer needed should be deleted from <tt>$WORK</tt> but keep in mind that there is no way to recover a file once it is deleted (this is another reason why files should not kept on <tt>$WORK</tt> for long-term storage).
 
Keep in mind that transfering data can be done at rate of about 1TB/hour, but this bandwidth is shared between all users. You should only transfer those files that you will need to keep and clean up your files before the transfer. You can get more in information on how to move data files to <tt>$DATA</tt> or <tt>$OFFSITE</tt> [[File system and Data Management#Compressing_and_Moving_Files_to_$DATA_or_$OFFSITE|here]].

Revision as of 13:34, 19 January 2023

Introduction

As you know, we will get a new HPC cluster this year. This also means that CARL and EDDY will be shutdown at some point (no date decided yet) and all users have to migrate to the new system. Here you will find information for this transition, which will be updated regularly as we go along.

Data Migration

The cluster CARL and EDDY have serveral file systems for storing data. The file systems $DATA and $OFFSITE will be available on the new cluster as before, so there is no need to migrate any data here. The directory $HOME will be different on the new cluster, however we will provide access to your current $HOME so you can transfer files as needed.

The filesystem $WORK will also be completely new, and access to the current $WORK will only be possible through the old cluster or via SMB. This access will also be limited in time, details on this will follow. While you should have enough time to migrate your recent results from the current $WORK, we strongly recommend that you transfer all files that you not actively need for computations and that you want to keep after CARL and EDDY are shutdown to $DATA or $OFFSITE as soon as possible. Files that are no longer needed should be deleted from $WORK but keep in mind that there is no way to recover a file once it is deleted (this is another reason why files should not kept on $WORK for long-term storage).

Keep in mind that transfering data can be done at rate of about 1TB/hour, but this bandwidth is shared between all users. You should only transfer those files that you will need to keep and clean up your files before the transfer. You can get more in information on how to move data files to $DATA or $OFFSITE here.