Overview

GridFTP is an extension of the standard File transfer Protocol (FTP) for high-speed, reliable, and secure data transfer. Because GridFTP provides a more reliable and high performance file transfer (compared to protocols such as SCP or rsync), it enables the transmission of very large files. GridFTP also addresses the problem of incompatibility between storage and access systems. (You can read more about the advantages of GridFTP here.

To use GridFTP, we recommend you use Globus. Globus uses endpoints to make transfers.

Contents

Accessing Globus

Globus can be used as a web application. To access it, go to https://www.globus.org/. Next, click Log In in the upper right-hand corner

On the next page, enter The University of Arizona in the search field and click the result.

This will take you through the standard university WebAuth login process. Once you successfully log in, you will be placed in a File Manager window. The various steps for setting up endpoints, initiating transfers, and viewing a transfer's progress can be found in the sections below.

Globus Connect Personal

To transfer files to/from your personal computer with Globus, you'll need to have a local endpoint set up. This can be achieved using Globus Connect Personal. Official documentation on how to install the relevant software and configure a local endpoint can be found in Globus' offical how-to documentation. An overview is shown for Mac, Linux, and Windows below.

To start, regardless of operating system, go to https://www.globus.org/, log in, navigate to the Collections tab, and select ⨁ Get Globus Connect Personal

From there, choose your operating system to proceed with the download and setup process

Mac

Once the installation is complete, open the .dmg file and drag/drop the Globus Connect Personal icon into your Applications folder. Then open Globus Connect Personal from Applications.

This will prompt you to log in via the university WebAuth process in a browser session. Once you've logged in, enter an identifying label for your local machine and grant Globus access

This will bring you back to your local Globus Connect Personal installation. You will fill out your local display name for your endpoint and click Save.

Windows

Once your installation is complete, open the .exe file to initiate the install. Click Yes to allow Globus Connect Personal to make changes to your device, select the install location, and click Install. Once the install is complete, make sure the Run Globus Connect Personal box is checked and click Finish.

Globus Connect Personal will then open and begin the configuration process. Click Log In to continue.

This will open a web browser where you will go through the typical UArizona WebAuth login process. Once you're logged in, give your local endpoint a descriptive name and click Allow.

This will bring you back to your local install. Enter a descriptive local name for your endpoint and click Save.

Linux

Once your installation is complete, open a terminal, navigate to your Downloads directory, and unpack the tar archive. Next, change into the unpacked directory and execute the globusconnectpersonal binary:

[user@ubuntu ~]$ cd Downloads
[user@ubuntu Downloads]$ tar xzvf globusconnectpersonal-latest.tgz
[user@ubuntu Downloads]$ cd globusconnectpersonal-3.2.0
[user@ubuntu globusconnectpersonal-3.2.0]$ ./globusconnectpersonal

This will bring up a graphical application. Click Log In to continue

This will open a web browser where you will need to go through the typical university WebAuth process. Once you're logged in, give your endpoint a name and click Allow.

This will bring you back to your local installation. Give your machine a descriptive name, then select Save.

Your setup should now be complete and your endpoint will now be usable to initiate transfers.

HPC-Managed Globus Endpoints

HPC Storage

HPC Storage Endpoint

The endpoint for HPC can be found by searching UA HPC Filesystems under the Collections tab.

Click the result to open information on this endpoint. Click Open in File Manager to access your HPC files.

The default location is your /home on HPC. You can navigate through by double-clicking directories, or by entering a full path in the Path search bar and hitting enter. This method can be used to access any /xdisk or /groups directories you have access to.

Rental Storage

Storage Rental Endpoint

The endpoint for rental storage (found on the filexfer nodes under /rental) can be found by searching UA Rental Storage Filesystem under the Collections tab.

This will open details on the endpoint. Click Open in File Manager to view the contents.

The root for this endpoint is /rental. Faculty members who have rented storage will have a directory with their NetID in this space. Find the one relevant to you and double-click to access.

Google Drive

Google Drive Endpoint

Warning: University Google Drive storage is going away in spring, 2023. See: Google Drive
The Globus endpoint is configured to have its root directory in your My Drive. This means you won't be able to access objects stored in your Shared With Me location on Google Drive. If you wish to access them, you can set up a shortcut in Google Drive to move objects from Shared With Me to My Drive.

Under the Endpoints tab (see: left-hand side menu) Search for UA Google Drive and select the result UA Google Drive with the subheading Managed Public Endpoint.
After clicking the result, navigate to the Credentials tab
This should bring you to a page to register your credentials. Click Continue.
Select your university account on the next page to proceed
Select Allow to give Globus access to your files
Once your credentials are set up, navigate back to the UA Google Drive endpoint, go to the Collections tab and select Add a Guest Collection
Under Directory, enter /My Drive to set the correct working directory for your collection. Next, give your collection a descriptive Display Name so you can identify it. When you're ready, click Create Collection
Once your collection is ready, you can find it under the Collections tab under Shareable By You. Clicking the name will open it in the File Manager window allowing you to initiate transfers and browse the contents.

Tier 2 AWS Storage

AWS S3 Endpoint (UITS Subsidized Tier 2 Storage)

Under the Collections tab, enter UA AWS S3 in the search bar. In the results, you should see the name UA AWS S3 show up with the description Managed Mapped Collection. Click the endpoint's name to proceed
Next, select the Credentials tab. If you are prompted for Authentication/Consent, click Continue
If requested, authenticate by selecting your Arizona email address, then Allow.
You will then be returned to the Credentials tab. From there, link to your AWS S3 Bucket by entering your public and private keys in the provided fields
Once you've added your keys, navigate back to the UA AWS S3 collection, go to the Collections tab, and click Add a Guest Collection on the right
Under Create New Guest Collection, click Browse next to the Directory field to find your group's AWS bucket. You will find it under /ua-rt-t2-faculty_netid/ where faculty_netid is the NetID of the faculty member who requested the bucket. Under Display Name, enter a descriptive name that you can use to identify your bucket. Once you've completed the process, click Create Collection.

If you encounter Authentication/Consent Required after clicking Browse, click Continue, select your university credentials, and click Allow. That should bring you back to the Browse window.
To find and use your new collection, navigate to the Collections tab, go to Shareable By You, and select the name. That will open your collection in the File Manager window allowing you to view the contents and initiate transfers.

Making Transfers

Transfers can be made between any two endpoints of your choosing using the File Manager window in the Globus web application. In this example, we'll make a transfer between a Globus Connect Personal endpoint and the primary HPC storage array.

Go to the File Manager tab in the Globus web application. Make sure you have the dual-panel mode enabled (upper right-hand corner shown with the red arrow below) to allow you to open two endpoints. Start with opening your first endpoint by clicking the Search bar on the left-hand side.
This will open a window where you can search for your first endpoint. In this example, we'll use UA HPC Filesystems. Click the result.
Now, on the left-hand side you should see the contents of your home directory on HPC. You can navigate through the various directories by double-clicking the folder icons, or can enter a full path in the Path search bar. To open a second connection, click the Search bar on the right-hand side.
You can search for your next endpoint in the same way as we searched for UA HPC Filesystems. You can also find recently used endpoints and your collections (e.g., a Google Drive collection or personal endpoint) under the Recent and Your Collections tabs. In this example, we'll go to Your Collections, find a personal endpoint, and click the result.
Now you should be back in the File Manager window with two endpoints open. A transfer can be made from one endpoint to another by selecting the item(s) you want to transfer, then clicking the Start button.

Monitoring Your Transfers

When you initiate a transfer following the instructions in the Making Transfers section above, a green box will pop up confirming the request.

You can get additional information the Activity panel on the left-hand side of the page. This will show you active and past transfers as well as their status. You can view additional details about your transfers by clicking the > shown on the right-hand side next to the target task. You can also cancel a transfer by clicking the × on the right.

Once your transfer has completed, you should receive an email with its status.