Difference between revisions of "Local Mounting of File Systems"

From HPC users
Jump to navigationJump to search
 
(20 intermediate revisions by 2 users not shown)
Line 21: Line 21:


== Shares under Linux ==
== Shares under Linux ==
<br/>
 
'''------ UPDATE ------''' <br/>
Since 24/04/2019 we made some changes on our file system. To make the reconfiguration easier for you, the changes will be '''''highlighted''''' for the next few days. <br/>
'''--------------------------''' <br/>
<br/> <br/> <br/>
If you are using a Linux computer, the HOME and WORK directory of CARL and EDDY can be mount with the command
If you are using a Linux computer, the HOME and WORK directory of CARL and EDDY can be mount with the command


   sudo mount -t cifs -o workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>  //'''''smb'''''.uni-oldenburg.de/hpc'''''_'''''home <mount_point4home>
   sudo mount -t cifs -o workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>  //smb.uni-oldenburg.de/hpc_home <mount_point4home>


and
and


   sudo mount -t cifs -o workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>  //'''''smb'''''.uni-oldenburg.de/'''''hpc_'''''data <mount_point4data>
   sudo mount -t cifs -o workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>  //smb.uni-oldenburg.de/hpc_data <mount_point4data>
   sudo mount -t cifs -o workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>  //'''''smb.'''''hpc.uni-oldenburg.de/'''''hpc_'''''work <mount_point4work>
   sudo mount -t cifs -o workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>  //smb.hpc.uni-oldenburg.de/hpc_work <mount_point4work>


'''<user>''' has to be replaced by the username you are using on CARL and EDDY. The place holders '''<mount_point4home>''' or similar ones have to be replaced by an valid, empty directory on your system. The variables '''<linux_username>''' and '''<linux_group>''' have to be replaced by the user- and group name under Linux if on the local machine the user and the group ID/name are different to the user and group ID on the cluster. When executing the command, ''sudo'' will first ask for your password of your local machine. After that, ''mount'' will ask for your password of CARL or EDDY.
'''<user>''' has to be replaced by the username you are using on CARL and EDDY. The place holders '''<mount_point4home>''' or similar ones have to be replaced by an valid, empty directory on your system. The variables '''<linux_username>''' and '''<linux_group>''' have to be replaced by the user- and group name under Linux if on the local machine the user and the group ID/name are different to the user and group ID on the cluster. When executing the command, ''sudo'' will first ask for your password of your local machine. After that, ''mount'' will ask for your password of CARL or EDDY.
Line 39: Line 35:
Alternatively one can extend the file ''/etc/fstab'' by the lines
Alternatively one can extend the file ''/etc/fstab'' by the lines


   //'''''smb.'''''uni-oldenburg.de/hpc'''_'''home <mount_point4home> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users
   //smb.uni-oldenburg.de/hpc_home <mount_point4home> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users
   //'''''smb.'''''uni-oldenburg.de/'''''hpc_'''''data <mount_point4data> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users
   //smb.uni-oldenburg.de/hpc_data <mount_point4data> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users
   //'''''smb.'''''hpc.uni-oldenburg.de/'''''hpc_'''''work <mount_point4work> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users
   //smb.hpc.uni-oldenburg.de/hpc_work <mount_point4work> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users
   '''''//smb.uni-oldenburg.de/hpc_offsite  <mount_point4offsite> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users'''''
   //smb.hpc.uni-oldenburg.de/<group_name>  <mount_point4group> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users
  //smb.uni-oldenburg.de/hpc_offsite  <mount_point4offsite> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users


and then mount the devices by
and then mount the devices by
Line 83: Line 80:


== Mounting the Directory on MacOS ==
== Mounting the Directory on MacOS ==
To mount your directories on MacOS, you can use the ''Finder'' function: <br/>
To mount your directories on MacOS, you can use the ''Finder'' function or just press <tt>command + K</tt>: <br/>
Select ''Go'' and ''Connect to Server'' <br/>
Select ''Go'' and ''Connect to Server'' <br/>
Type in the following address: <br/>
Type in one of the following addresses depending on which directory you want to mount: <br/>
  smb://ces.hpc.uni-oldenburg.de
  smb://smb.uni-oldenburg.de/hpc_home
smb://smb.uni-oldenburg.de/hpc_data
smb://smb.hpc.uni-oldenburg.de/hpc_work
... and type in your credentials. <br/>
... and type in your credentials. <br/>
Now your cluster-sided directories are linked within your Finder.
Now your cluster-sided directories are linked within your Finder.


Please keep in mind that the mount connection can only establish successfully when it is performed from within the universitie's network. That being said, you don't have to do this physically from within the university building, but you can also accomplish the same result if you are connected to our network via [[Login#VPN:_Internet_.2F_From_Home | VPN]]
== Mounting a Directory on Nextcloud ==
{{warningbox|
Whenever you configure your HPC shares with NextCloud you must ''always'' fill in the <tt>share</tt> box with a subfolder! Otherwise, NextCloud indexes every single file of the connected share, which leads to possibly millions of files being regularly read out by the cloud service and thus overloading the file system!
}}
Besides locally mounting your shares, you can also mount your shares on Nextcloud. This way, you will have access to your research files from all over the world through the [https:cs.uol.de/ | Nextcloud web interface]. You need to perform the following steps for this:


== Mounting a Directory on Nextcloud ==
# If you are using the cloud for the first time, you have to activate your cloud at cs.uol.de. Your university login data is all you need.
Besides locally mounting your shares, you can also mount your shares on Nextcloud. This way, you will have access to your research files from all over the world. First of all, you have to activate your cloud at sc.uol.de. Your university login data is all you need. <br/>
# Login to your Nextcloud account, go to settings by clicking your user symbol at the upper right corner and select '''Settings'''.
After setting up your Nextcloud account, go to settings by clicking your user symbol at the upper right corner and select ''Settings''.
# Now change to '''External storages''' and select '''SMB / CIFS''' from the drop-down menu to '''Add storage'''.  
Now change to ''External storages'' where you add a new storage '''SMB / CIFS'''.  
# Now, you fill in the fields as follows (see picture):
Now, you fill in the fields as follows:<br/>
#:[[Image:Nextcloud_HPC.png|700px]]
[[Image:Nextcloud_HPC.png|700px]]
#* choose a '''Folder name''', e.g. <tt>HPC_HOME</tt> for <tt>$HOME</tt>, this will be the name of the folder under which you will find it your cloud.
* $DATA
#* Enter <tt>smb.uni-oldenburg.de</tt> in the '''Host''' field, except for <tt>$WORK</tt>, in which case you need to use <tt>smb.hpc.uni-oldenburg.de</tt>.
** Enter '''ces.hpc.uni-oldenburg.de''' in the ''Host'' field.
#* Enter the name of the share (see above for details) you want to mount, e.g <tt>hpc_home</tt> for <tt>$HOME</tt> in the '''Share''' field.
** Enter '''data''' in the ''Share'' field.
#* You can use the field '''Remote subfolder''' to only mount a specific subfolder instead of the whole drive. Example subfolder can be <tt>Research</tt> or <tt>Research/Subtopic</tt>.
** You can use the field "Remote subfolder" to include a subfolder instead of the whole group drive. Example subfolder "Research" or "Research/Field1"
#* Enter <tt>w2kroot</tt> as '''Domain'''.
#* Enter your login credentials fpr '''User''' and '''Password'''.
#* Click the check-symbol on the right to complete adn verify the setup. A green check-symbol will indicate success.
#* The three-dots icon can be used to make optional settings (in general not needed).
# After adding the share, you can go back to your files using the navigation in the top left. There you should see the new folder with your HPC files. You can browse and down- or upload files as needed.
 
If you are using a client on your local computer to synchronize with the cloud, you should make sure that the HPC folders are not synchronized (per default this might already be the case). For more information about the cloud, visit the section [https://uol.de/itdienste/services/datenhaltung/cloudstorage/ Cloud-Storage] from our IT services.
===Possible Shares to Connect with Nextcloud ===
The following table shows all possible ways to connect the HPC shares to the NextCloud interface:


*$HOME
<center>
** Enter '''daten.uni-oldenburg.de''' in the ''Host'' field.
{| style="background-color:#eeeeff;" cellpadding="10" border="1" cellspacing="0"
** Enter '''hpchome''' in the ''Share'' field.
|- style="background-color:#ddddff;"
** You can use the field "Remote subfolder" to include a subfolder instead of the whole group drive. Example subfolder "Documents" or "Documents/Research1"
!Host
!Share
!Remote Subfolder
!Domain
|- valign="top"
| smb.uni-oldenburg.de
| hpc_home
| folder on $HOME to mount to
| w2kroot
|- valign="top"
| smb.uni-oldenburg.de
| hpc_data
| folder on $HOMEto mount to
| w2kroot
|-
|- valign="top"
|  smb.uni-oldenburg.de
| hpc_work
| folder on $WORK to mount to
| w2kroot
|-
|  smb.uni-oldenburg.de
| hpc_offsite
| folder on $OFFSITE to mount to
| w2kroot
|-
|  smb.uni-oldenburg.de
| <tt>agyourgroup</tt>
| folder on $GROUP to mount to
| w2kroot
|}
</center>


For more information about the cloud, visit the section [[https://uol.de/itdienste/services/datenhaltung/cloudstorage/ Cloud-Storage]] from our IT services.
Hypothetically, if a user with the sign-in ID <tt>abcd1234</tt> had folders called <tt>my_data</tt> on every single share, and had set up every possible connection to said folders, the following paths would be accessible via NextCloud:
* /user/abcd1234/my_data
* /nfs/data/abcd1234/my_data
* /gss/work/abcd1234/my_data
* /nfs/group/agexample/my_data
* /nfs/offsite/user/abcd1234/my_data

Latest revision as of 13:47, 25 February 2022

Shares under Windows

If you are using a Windows computer, the HOME, DATA and WORK directories of CARL and EDDY can be mounted as network drive by using

\\smb.uni-oldenburg.de\hpc_home
\\smb.uni-oldenburg.de\hpc_data

for HOME and DATA and

\\smb.hpc.uni-oldenburg.de\hpc_work

as addresses WORK, respectively. After that, the username and password of your university account (also used for CARL and EDDY) will be requested to connect this share.

There are two ways, to connect to a network share on Windows (also see web pages of IT Services regarding the L:-drive):

  1. Open the File Explorer and Right-Click on "This PC". Select "Map Network Drive". Choose a letter for the network drive (e.g. V:) and enter the address as explained above. If you are not using your university account to login on your Computer, also tick "Using different credentials". Click "Finish" to proceed. You should be prompted for a user name (see below) and password. After entering both, the network drive should be ready to use. If you have problems, try to google "windows map network drive" for more detailed instructions.
  2. Alternatively, open a command windows (cmd.exe). The command "net use" will show you any connected network drives. Use the command "net use <drive letter> /d /y" to remove any unwanted connection. Replace "<drive letter>" by e.g. "V:". The command "net use <drive letter> <network address> * /user:w2kroot\abcd1234" will prompt you for your password and connect the network drive. Replace "<drive letter>" (e.g. "V:"), "<network address>" as shown above, and "abcd1234" with your user name.

In either case contact hpcsupport@uni-oldenburg.de if you are getting an error message.

Note: The username has to be specified with the domain name, e.g.: "w2kroot\abcd1234"

Shares under Linux

If you are using a Linux computer, the HOME and WORK directory of CARL and EDDY can be mount with the command

 sudo mount -t cifs -o workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>  //smb.uni-oldenburg.de/hpc_home <mount_point4home>

and

 sudo mount -t cifs -o workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>  //smb.uni-oldenburg.de/hpc_data <mount_point4data>
 sudo mount -t cifs -o workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>  //smb.hpc.uni-oldenburg.de/hpc_work <mount_point4work>

<user> has to be replaced by the username you are using on CARL and EDDY. The place holders <mount_point4home> or similar ones have to be replaced by an valid, empty directory on your system. The variables <linux_username> and <linux_group> have to be replaced by the user- and group name under Linux if on the local machine the user and the group ID/name are different to the user and group ID on the cluster. When executing the command, sudo will first ask for your password of your local machine. After that, mount will ask for your password of CARL or EDDY.

Alternatively one can extend the file /etc/fstab by the lines

 //smb.uni-oldenburg.de/hpc_home  <mount_point4home> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users
 //smb.uni-oldenburg.de/hpc_data  <mount_point4data> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users
 //smb.hpc.uni-oldenburg.de/hpc_work  <mount_point4work> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users
 //smb.hpc.uni-oldenburg.de/<group_name>  <mount_point4group> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users
 //smb.uni-oldenburg.de/hpc_offsite  <mount_point4offsite> cifs workgroup=W2KROOT,username=<user>,file_mode=0600,dir_mode=0700,uid=<linux_username>,gid=<linux_group>,noauto,users

and then mount the devices by

 mount <mount_point4home>
 mount <mount_point4data>
 mount <mount_point4work>
 mount <mount_point4offsite>

Here the sudo command is not needed due to the users option.

Note:

  • The mount command without the entry in the /etc/fstab needs root permissions. Thats why you have to add sudo to the mount command.
  • To mount CIFS file systems under Linux usually additional packages have to be installed. E.g. on Ubuntu 12.04LTS the package 'smbfs' and 'smbclient' have to be installed by sudo apt-get install smbfs. For Ubuntu 14.04LTS you'll need 'cifs-utils'. For other Linux distributions please search for 'cifs' or 'smb' in your package manager.
  • The file system has to be mounted after each reboot or shutdown. They are not mounted automatically
  • sudo caches the password for a certain time. So if you call sudo again, sudo doesn't ask for your password again.
  • Mounting DATA and WORK requires SMB version 2.0 or higher (HOME not yet). If you are using an older Linux distribution you may encounter mount error messages. In that case you can try to add the option
vers=3.0
to the mount command or fstab-entry (to the comma-separated list of options). If this does not solve the problem you may have to upgrade your cifs-utils to a newer version.
  • Some versions of Linux (e.g. Ubuntu 16.04 with a recent kernel update) may cause similar problems when mounting HOME, which can be solved by adding vers=3.0 as well.

Temporary Mount of Any Cluster Directory for Linux

Alternatively to mount any directory of the cluster on your local computer to may use sshfs for mounting. For most Ubuntu based systems you will require to install the package sshfs

 sudo apt-get install sshfs

Once this is done you can create a folder that will work as mount-point by

 mkdir myMountDir

Finally by typing

 sshfs abcd1234@carl.hpc.uni-oldenburg.de:/some/directiory/on/the/cluster ~/myMountDir

the directory /some/directiory/on/the/cluster will be mounted temporarily to myMountDir on your local machine.


Mounting the Directory on MacOS

To mount your directories on MacOS, you can use the Finder function or just press command + K:
Select Go and Connect to Server
Type in one of the following addresses depending on which directory you want to mount:

smb://smb.uni-oldenburg.de/hpc_home
smb://smb.uni-oldenburg.de/hpc_data
smb://smb.hpc.uni-oldenburg.de/hpc_work

... and type in your credentials.
Now your cluster-sided directories are linked within your Finder.

Please keep in mind that the mount connection can only establish successfully when it is performed from within the universitie's network. That being said, you don't have to do this physically from within the university building, but you can also accomplish the same result if you are connected to our network via VPN

Mounting a Directory on Nextcloud

Whenever you configure your HPC shares with NextCloud you must always fill in the share box with a subfolder! Otherwise, NextCloud indexes every single file of the connected share, which leads to possibly millions of files being regularly read out by the cloud service and thus overloading the file system!

Besides locally mounting your shares, you can also mount your shares on Nextcloud. This way, you will have access to your research files from all over the world through the [https:cs.uol.de/ | Nextcloud web interface]. You need to perform the following steps for this:

  1. If you are using the cloud for the first time, you have to activate your cloud at cs.uol.de. Your university login data is all you need.
  2. Login to your Nextcloud account, go to settings by clicking your user symbol at the upper right corner and select Settings.
  3. Now change to External storages and select SMB / CIFS from the drop-down menu to Add storage.
  4. Now, you fill in the fields as follows (see picture):
    Nextcloud HPC.png
    • choose a Folder name, e.g. HPC_HOME for $HOME, this will be the name of the folder under which you will find it your cloud.
    • Enter smb.uni-oldenburg.de in the Host field, except for $WORK, in which case you need to use smb.hpc.uni-oldenburg.de.
    • Enter the name of the share (see above for details) you want to mount, e.g hpc_home for $HOME in the Share field.
    • You can use the field Remote subfolder to only mount a specific subfolder instead of the whole drive. Example subfolder can be Research or Research/Subtopic.
    • Enter w2kroot as Domain.
    • Enter your login credentials fpr User and Password.
    • Click the check-symbol on the right to complete adn verify the setup. A green check-symbol will indicate success.
    • The three-dots icon can be used to make optional settings (in general not needed).
  5. After adding the share, you can go back to your files using the navigation in the top left. There you should see the new folder with your HPC files. You can browse and down- or upload files as needed.

If you are using a client on your local computer to synchronize with the cloud, you should make sure that the HPC folders are not synchronized (per default this might already be the case). For more information about the cloud, visit the section Cloud-Storage from our IT services.

Possible Shares to Connect with Nextcloud

The following table shows all possible ways to connect the HPC shares to the NextCloud interface:

Host Share Remote Subfolder Domain
smb.uni-oldenburg.de hpc_home folder on $HOME to mount to w2kroot
smb.uni-oldenburg.de hpc_data folder on $HOMEto mount to w2kroot
smb.uni-oldenburg.de hpc_work folder on $WORK to mount to w2kroot
smb.uni-oldenburg.de hpc_offsite folder on $OFFSITE to mount to w2kroot
smb.uni-oldenburg.de agyourgroup folder on $GROUP to mount to w2kroot

Hypothetically, if a user with the sign-in ID abcd1234 had folders called my_data on every single share, and had set up every possible connection to said folders, the following paths would be accessible via NextCloud:

  • /user/abcd1234/my_data
  • /nfs/data/abcd1234/my_data
  • /gss/work/abcd1234/my_data
  • /nfs/group/agexample/my_data
  • /nfs/offsite/user/abcd1234/my_data