Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Panel
borderColor#9c9fb5
bgColor#fcfcfc
titleColor#fcfcfc
titleBGColor#021D61
borderStylesolid
titlexdisk


Excerpt


Expand
titleQ. Can someone other than a PI manage a group's xdisk?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted


Yes! A PI can add a trusted group member as a delegate by following the instructions in our Research and Class Groups page. Once a group member is added as a delegate, they can manage the group's xdisk allocation on behalf of their PI in the user portal.



Expand
titleQ. Who owns our group's /xdisk?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted


A group's PI owns the /xdisk allocation. By default, your PI has exclusive read/write/execute privileges for the root folder /xdisk/PI.




Expand
titleQ. Can a PI make their /xdisk accessible to their whole group?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted


By default, members of a research group only have write access to their subdirectories under /xdisk/PI. If they so choose, a PI may allow their group members to write directly to /xdisk/PI by running one of the following commands:

CommandResult

$ chmod g+r /xdisk/PI

Group members can see the contents of the directory with ls, but may not access it or make modifications (e.g. add, delete, or edit files/directories)

$ chmod g+rx /xdisk/PI

Group members can access the directory and see files but cannot make modifications (e.g. add, delete, or edit files/directories)

$ chmod g+rwx /xdisk/PI

Group members are granted full read/write/execute privileges.




Expand
titleQ. Where can group members store their files?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted


When an /xdisk allocation is created, a subdirectory is automatically generated for and owned by each individual group member. If the directory /xdisk/PI does not have group privileges, group members may not access the root directory, but may access their individual spaces by:

Code Block
languagebash
themeMidnight
$ cd /xdisk/PI/netid

If a user joins the group after the xdisk was created and /xdisk/PI is not writeable for group members, contact our consultants and they can create one.




Expand
titleQ. A group member's directory isn't in our /xdisk, how can we add it?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted


Typically when an /xdisk allocation is created, it will automatically generate a directory for each group member. In the unlikely event that it doesn't or, more commonly, a group member is added after the allocation has been created, either the PI may manually create a directory or, if the root directory is group writable, the user may create one themselves. If the group's /xdisk does not have full group permissions, the PI may run:

Code Block
languagebash
themeMidnight
$ mkdir /xdisk/PI/netid

then can reach out to our hpc consultants to request an ownership change.




Expand
titleQ. Do we need to request an individual allocation within the /xdisk for each user in our group?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted


No, the full /xdisk allocation is available for every member of the group. It's up to group members to communicate with one another on how they want to utilize the space.




Expand
titleQ. Why am I getting xdisk emails?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted


xdisk is a temporary storage space available to your research group. When it's close to its expiration date, notifications will be sent to all members of your group. For detailed information on xdisk allocations, see Storageour xdisk documentation.




Expand
titleQ. Why am I getting "/xdisk allocations can only be authorized by principal investigators"?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted


xdisks are managed by your group's PI by default. This means if you want to request an xdisk or modify an existing allocation (e.g., extending the time limit or increasing the storage quota), you will need to consult your PI. Your PI may either perform these actions directly or, if they want to delegate xdisk management to a group member, they may do so by following the instructions under Delegating xdisk Group Management Rights.




Expand
titleQ. How can we modify our xdisk allocation?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted


To modify your allocation's time limit or storage quota, your PI can either do so through the Web Portal under the Storage tab, or via the command line. If your PI would like to delegate management rights to a group member, they may follow the instructions under Delegating xdisk Group Management Rights. Once a group member has received management rights, they may manage the allocation through our web portal.




Expand
titleQ. Why am I getting "xdisk: command not found"?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted


If you're getting errors using xdisk commands in a terminal session, check that you are on a login node. If you are on the bastion host (hostname: gatekeeper), are in an interactive session, or are on the filexfer node, you won't be able to check or modify your xdisk. When you are on a login node, your terminal prompt should show the hostname junonia or wentletrap. You can also check your hostname using the command:

Code Block
languagebash
themeMidnight
$ hostname




Expand
titleQ. Why am I getting errors when trying to extend my allocation?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted

If you're trying to extend your group's allocation but are seeing something like:

Code Block
languagebash
themeMidnight
(puma) [netid@junonia ~]$ xdisk -c expire -d 1
invalid request_days: 1

for every value you enter, your xdisk has likely reached its maximum time limit. To check, have a delegate or PI go to portal.hpc.arizona.edu, click Manage XDISK, and look at the box next to Duration. If you see 300, your allocation cannot be extended further.

If your allocation is at it's its limit, you will need to back up your data to external storage (e.g., a local machine, lab server, or cloud service). Once your xdisk has expired been removed (either by expiring or through manual deletion), you can immediately create a new allocation and restore your data. Detailed xdisk information can be found on our our HPC High Performance Storage page. You may also want to look at our page on Transferring Data.



Expand
titleQ. Can we keep our xdisk allocation for more than 300 days?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted

No, once an xdisk has reached its time limit it will expire. It's a good idea to start preparing for this early by making frequent backups and paying attention to xdisk expiration emails. 



Expand
titleQ. What happens when our xdisk allocation expires?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted

Once an xdisk expires, all the associated data are deleted. Deleted data are non-retrievable since HPC is not backed up. It's advised to keep frequent backups of your data on different platforms, for example a local hard drive or a cloud-based service like Google Drive, or (even better) both! Check our Storage documentation for more information on alternative storage offerings.




Expand
titleQ. What's the best way to backup/transfer our data before our xdisk expires?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted


Warning

Free Google Drive storage will be going away in 2023 and we are transitioning to AWS as a Tier 2 storage solution. We will continue to support researchers with Google Drive during this transition. See Tier 2 Storage for information on the new rental storage option.

Before your group's xdisk expires, you'll want to make an external backup of anything you need to keep. External storage options include personal computers, lab servers, external hard drives, or cloud services such as Google Drive or AWS. 

If you're moving large quantities of data, Globus is a great option. We have instructions in our storage Globus documentation for setting up and using this software.

We strongly recommend making archives (.tar, .zip, files etc.) of large directories prior to transferring them off the system. In general, transfer software struggles with moving many small files and performs much more efficiently moving fewer large files. You will get the better transfer speeds (sometimes by orders of magnitude) if you compress your files prior to transferring them. This can be done on our filexfer node which is designed for large file management operations (hostname: filexfer.hpc.arizona.edu). 


Expand
titleQ. Once our xdisk expires, can we request a new one?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted

Yes, a new xdisk may be requested immediately after the old partition expires. Data, however, may not be transferred directly from the old partition to the new one. 



Expand
titleQ. Can a PI have more than one xdisk active at a time?


Panel
borderColor#07105b
bgColor#f6f6f6
titleColor#00084d
borderWidth1
titleBGColor#d9dae5
borderStyledotted

No, only one xdisk may be active per PI at a given time. 




...