[Globus] Command-line Globus (in beluga cluster)

source:https://docs.computecanada.ca/wiki/Globus#Command_Line_Interface_.28CLI.29

             https://hpc.nih.gov/storage/globus.html

 

 

 

Command Line Interface (CLI)[edit]

Installing[edit]

The Globus command line interface is a python module which can be installed using pip. Below are the steps to install Globus CLI on one of our clusters.

1. Create a virtual environment to install the Globus CLI into (see creating and using a virtual environment).
   $ virtualenv $HOME/.globus-cli-virtualenv
2. Activate the virtual environment
   $ source $HOME/.globus-cli-virtualenv/bin/activate
3. Install Globus CLI into the virtual environment (see installing modules).
   $ pip install globus-cli

---

Note: in the example below, the command-line globus transfer is started on Helix, the interactive data transfer node. However, the transfer to the NIH HPC endpoint ('NIH HPC Data Transfer') will go via the 8 HPC data transfer nodes.

Sample session:

[user@helix ~$ globus login

Please login to Globus here:
---------------------------
https://auth.globus.org/long/globus/URL/
---------------------------

 

When you point a web browser to the URL that is provided, you will need to authenticate against the NIH domain, with your usual NIH login username and password. You will then see a page like the following

 

When you click Allow, you will get a page with a long authorization code. Cut-and-paste that code back into your terminal window:

Enter the resulting Authorization Code here: aabbccddeeffgg1122334455

You have successfully logged in to the Globus CLI as username@globusid.org
You can always check your current identity with
  globus whoami 
Logout of the Globus CLI with
  globus logout

[user@biowulf ~]$ globus whoami
user@globusid.org

 

Depending on how recently you signed up for Globus, the command 'globus whoami' may show you your Globus userid (user@globusid.org) or your NIH userid (user@nih.gov). Once you have logged in to Globus, you can set up transfers. You will need the ID of the endpoint, which you can find using the 'globus endpoint search' command. In the example below, the user obtains a list of NIH endpoints, and then gets a listing of the user's files on one of those endpoints.

[user@biowulf ~]$ globus endpoint search NIH

[user@biowulf ~]$ globus transfer --recursive --no-verify-checksum \ 
	d8eb36b6-6d04-11e5-ba46-22000b92c6ec:/data1/5GB-in-small-files/ \ 
    	e2620047-6d04-11e5-ba46-22000b92c6ec:/scratch/$USER/5GB-in-small-files/
Message: The transfer has been accepted and a task has been created and queued for execution
Task ID: 6924b21e-f54b-11e6-ba69-22000b9a448b

 

You can use the method above in your Biowulf batch scripts, for example, to transfer data at the end of a job. [Example]

Documentation about the Globus CLI