Difference between revisions of "Migration"
Schwietzer (talk | contribs) (No changes to content) |
|||
Line 1: | Line 1: | ||
== Introduction == | == Introduction == | ||
As you know, we will get a new HPC cluster this year. This also means that CARL and EDDY will be | As you know, we will get a new HPC cluster this year. This also means that CARL and EDDY will be shut down at some point (no date decided yet) and all users have to migrate to the new system. Here you will find information for this transition, which will be updated regularly as we go along. | ||
== Data Migration == | == Data Migration == | ||
The cluster CARL and EDDY have | The cluster CARL and EDDY have several file systems for storing data. The file systems <tt>$DATA</tt> and <tt>$OFFSITE</tt> will be available on the new cluster as before, so there is no need to migrate any data here. The directory <tt>$HOME</tt> will be different on the new cluster, however, we will provide access to your current <tt>$HOME</tt> so you can transfer files as needed. | ||
The filesystem <tt>$WORK</tt> will also be completely new, and access to the current <tt>$WORK</tt> will only be possible through the old cluster or via SMB. This access will also be limited in time, details on this will follow. While you should have enough time to migrate your recent results from the current <tt>$WORK</tt>, we strongly recommend that you transfer all files that you not actively need for computations and that you want to keep after CARL and EDDY are | The filesystem <tt>$WORK</tt> will also be completely new, and access to the current <tt>$WORK</tt> will only be possible through the old cluster or via SMB. This access will also be limited in time, details on this will follow. While you should have enough time to migrate your recent results from the current <tt>$WORK</tt>, we strongly recommend that you transfer all files that you do not actively need for computations and that you want to keep after CARL and EDDY are shut down to <tt>$DATA</tt> or <tt>$OFFSITE</tt> as soon as possible. Files that are no longer needed should be deleted from <tt>$WORK</tt> but keep in mind that there is no way to recover a file once it is deleted (this is another reason why files should not be kept on <tt>$WORK</tt> for long-term storage). | ||
Keep in mind that | Keep in mind that transferring data can be done at a rate of about 1TB/hour, but this bandwidth is shared between all users. You should only transfer those files that you will need to keep and clean up your files before the transfer. You can get more information on how to move data files to <tt>$DATA</tt> or <tt>$OFFSITE</tt> [[File system and Data Management#Compressing_and_Moving_Files_to_$DATA_or_$OFFSITE|here]]. |
Latest revision as of 09:36, 23 January 2023
Introduction
As you know, we will get a new HPC cluster this year. This also means that CARL and EDDY will be shut down at some point (no date decided yet) and all users have to migrate to the new system. Here you will find information for this transition, which will be updated regularly as we go along.
Data Migration
The cluster CARL and EDDY have several file systems for storing data. The file systems $DATA and $OFFSITE will be available on the new cluster as before, so there is no need to migrate any data here. The directory $HOME will be different on the new cluster, however, we will provide access to your current $HOME so you can transfer files as needed.
The filesystem $WORK will also be completely new, and access to the current $WORK will only be possible through the old cluster or via SMB. This access will also be limited in time, details on this will follow. While you should have enough time to migrate your recent results from the current $WORK, we strongly recommend that you transfer all files that you do not actively need for computations and that you want to keep after CARL and EDDY are shut down to $DATA or $OFFSITE as soon as possible. Files that are no longer needed should be deleted from $WORK but keep in mind that there is no way to recover a file once it is deleted (this is another reason why files should not be kept on $WORK for long-term storage).
Keep in mind that transferring data can be done at a rate of about 1TB/hour, but this bandwidth is shared between all users. You should only transfer those files that you will need to keep and clean up your files before the transfer. You can get more information on how to move data files to $DATA or $OFFSITE here.