Storage
New HPC Documentation Website!
New documentation is coming that will replace our current Confluence website (the one you're viewing right now). We will be sending an announcement on when the site will go live. Interested in taking a peek? Check out this page for the beta version. Note: the URL is likely to change.
New GPUs on Ocelote!
We have recently added 22 new P100 GPUs to Ocelote. Need to request multiple GPUs on a node and you're finding Puma queue times too slow? You can now request two GPUs per node on Ocelote using --gres=gpu:2.
Where Should I Store My Data?
- Data undergoing active analyses should be stored in HPC's local High Performance Storage.
- Large amounts of data not requiring immediate access from our HPC compute nodes can be stored at reasonable rates on our Rental Storage.
- RDAS is a research data service which supports the mounting of SMB shares. The supported operating systems are MacOS, Linux, and Windows. It provides 5TB of free storage.
- Research data not requiring immediate access should be stored in General Research Data Storage (Tier 2).
For example:- Large datasets where only subsets are actively being analyzed
- Results no longer requiring immediate access
- Backups (highly encouraged!)
- Data that require HIPAA-compliance can be stored on Soteria (currently in the pilot phase).
- HPC High Performance Storage — The University’s Research Data Center provides data storage for active analysis on the high-performance computers (HPCs). Your storage is mounted as a filesystem and all the clusters have access to the same filesystems.
- Rental Storage — We offer a rental storage solution on a storage array housed in the Research Data Center. This array is mounted on our data transfer nodes (filexfer.hpc.arizona.edu). Rental storage is not mounted on HPC compute or login nodes.
- Tier 2 Storage — Research Technologies in partnership with UITS has implemented an AWS rental storage solution. Buckets can be requested through the user portal and are intended for archival storage and data not requiring immediate access from the user.
- Research Desktop Attached Storage — R-DAS provides up to 5 TB of no-cost storage capacity for each PI group. Our requirement was to enable our users to easily share data with other research group members. You can treat the allocation as a drive mounted on your local computer. R-DAS is intended for storing open research data, but not controlled or regulated data.
- Google Drive — This offering is deprecated.
Storage Option Summary
| Purpose | Capacity | Cost | Restricted data? | Access | Duration | Backup | |
|---|---|---|---|---|---|---|---|
| Primary Storage | Research data. Supports compute. Directly attached to HPC | /home 50GB /groups 500GB /xdisk 20TB | Free | Not for restricted data | Directly mounted to HPC. Also uses Globus and DTNs | Long term. Aligns with HPC purchase cycle | No |
| R-DAS | Research Desktop Attached Storage - SMB shares | 5 TB | Free | Not for restricted data | Mounted to workstations as shares | Long term | No |
| Rental Storage | Research data. Large datasets. Typically for staging to HPC | Rented per Terabyte per year | Rental rate: $47.35 per TB per year | Not for restricted data | Uses Globus and DTNs. Copy data to Primary | Long term. Aligns with HPC purchase cycle | No |
| Tier 2 | Typically research data. Unused data is archived | 15GB to TB's | Tier-based system. First 1TB of active data and archival data are free. Active data > 1TB is paid. | Not for restricted data | Uses Globus and AWS command line interface | Typically long term since use of Glacier is free and slow | Archival |
| ReData | Research data. Managed by UA Libraries | Quota system | Free | Not for restricted data | Login and fill out fields, then upload | Longer than 10 years | No |
Soteria | Secure data enclave | Individual requests | Free upon qualification | Restricted data; HIPAA, ePHI | HIPAA training required, followed by request process | Long term | No |
| Box | General data | 50GB | Free | Not for restricted data | Browser | Long term | No |
| Google Drive | General data | 15GB | Free. Google rates for amounts > 15GB | Not for restricted data | Browser | Unlimited usage expires March 1, 2023 | No |
NIH Data Management and Sharing Policy
The NIH has issued a new data management and sharing policy, effective January 25, 2023. The university libraries now offers a comprehensive guide for how to navigate these policies and what they mean for you.
What's new about the 2023 NIH Data Management and Sharing Policy?
Previously, the NIH only required grants with $500,000 per year or more in direct costs to provide a brief explanation of how and when data resulting from the grant would be shared.
The 2023 policy is entirely new. Beginning in 2023, ALL grant applications or renewals that generate Scientific Data must now include a robust and detailed plan for how you will manage and share data during the entire funded period. This includes information on data storage, access policies/procedures, preservation, metadata standards, distribution approaches, and more. You must provide this information in a data management and sharing plan (DMSP). The DMSP is similar to what other funders call a data management plan (DMP).
The DMSP will be assessed by NIH Program Staff (though peer reviewers will be able to comment on the proposed data management budget). The Institute, Center, or Office (ICO)-approved plan becomes a Term and Condition of the Notice of Award.