Rucio CLI: Examples¶
Rucio provides several commands for the end-user. See man pages.
Getting user information¶
The first thing you might try is to check who you are:
$ rucio whoami
status : ACTIVE
account : jdoe
account_type : SERVICE
created_at : 2014-01-17T07:52:18
updated_at : 2014-01-17T07:52:18
suspended_at : None
deleted_at : None
email : jdoe@blahblah.com
You can switch between different accounts by setting the RUCIO_ACCOUNT variable:
$ export RUCIO_ACCOUNT=root
$ rucio whoami
status : ACTIVE
account : jdoe
account_type : SERVICE
created_at : 2014-01-17T07:51:59
updated_at : 2014-01-17T07:51:59
suspended_at : None
deleted_at : None
email : root@blahblah.com
If you try to authenticate with an account that is not mapped with your credentials:
$ export RUCIO_ACCOUNT=janedoe
$ rucio whoami
cannot get auth_token
2018-01-30 16:50:08,554 ERROR Cannot authenticate.
Details: x509 authentication failed
2018-01-30 16:50:08,554 ERROR Please verify that your proxy is still valid and renew it if needed.
Querrying basic information about RSEs¶
You can query the list of available RSEs:
$ rucio list-rses
SITE1_DISK
SITE1_TAPE
SITE2_DISK
SITE2_SCRATCH
SITE3_TAPE
If the RSEs are tagged with attributes you can build RSE expressions and query the sites matching these expression:
$ rucio list-rses --expression "tier=1&disk=1"
SITE1_DISK
SITE2_DISK
Querying information about DIDs¶
To list all the possible scopes:
$ rucio list-scopes
mc
data
user.jdoe
user.janedoe
You can query the DIDs matching a certain pattern. It always requires to specify the scope in which you want to search:
$ rucio list-dids user.jdoe:*
+-------------------------------------------+--------------+
| SCOPE:NAME | [DID TYPE] |
|-------------------------------------------+--------------|
| user.jdoe:user.jdoe.test.container.1234.1 | CONTAINER |
| user.jdoe:user.jdoe.test.container.1234.2 | CONTAINER |
| user.jdoe:user.jdoe.test.dataset.1 | DATASET |
| user.jdoe:user.jdoe.test.dataset.2 | DATASET |
| user.jdoe:test.file.1 | FILE |
| user.jdoe:test.file.2 | FILE |
| user.jdoe:test.file.3 | FILE |
|-------------------------------------------+--------------|
You can filter by key/value, e.g.:
$ rucio list-dids --filter type=CONTAINER
+-------------------------------------------+--------------+
| SCOPE:NAME | [DID TYPE] |
|-------------------------------------------+--------------|
| user.jdoe:user.jdoe.test.container.1234.1 | CONTAINER |
| user.jdoe:user.jdoe.test.container.1234.2 | CONTAINER |
|-------------------------------------------+--------------|
If you want to resolve a collection (CONTAINER or DATASET) into the list of its constituents:
$ rucio list-content user.jdoe:user.jdoe.test.container.1234.1
+------------------------------------+--------------+
| SCOPE:NAME | [DID TYPE] |
|------------------------------------+--------------|
| user.jdoe:user.jdoe.test.dataset.1 | DATASET |
| user.jdoe:user.jdoe.test.dataset.2 | DATASET |
+------------------------------------+--------------+
You can resolve also the collections (CONTAINER or DATASET) into the list of files:
$ rucio list-content user.jdoe:user.jdoe.test.container.1234.1
+-----------------------+--------------------------------------+-------------+------------+----------+
| SCOPE:NAME | GUID | ADLER32 | FILESIZE | EVENTS |
|-----------------------+--------------------------------------+-------------+------------+----------|
| user.jdoe:test.file.1 | 9DF32550-D0D1-4482-9A26-0FBC46D6902A | ad:56fb0723 | 39.247 kB | |
| user.jdoe:test.file.2 | 67E8CF14-F953-45F3-B3F5-E6143F89915F | ad:e3e573b5 | 636.075 kB | |
| user.jdoe:test.file.3 | 32CD7F8E-944B-4EA4-83E3-BABE48DB5751 | ad:22849380 | 641.427 kB | |
+-----------------------+--------------------------------------+-------------+------------+----------+
Total files : 3
Total size : 1.316 MB:
Rules operations¶
You can create a new rule like this:
$ rucio add-rules --lifetime 1209600 user.jdoe:user.jdoe.test.container.1234.1 1 "tier=1&disk=1"
a12e5664555a4f12b3cc6991db5accf9
The command returns the rule_id of the rule.
You can list the rules for a particular DID:
$ rucio list-rules user.jdoe:user.jdoe.test.container.1234.1
ID ACCOUNT SCOPE:NAME STATE[OK/REPL/STUCK] RSE_EXPRESSION COPIES EXPIRES (UTC)
-------------------------------- --------- ----------------------------------------- ---------------------- ------------------ -------- -------------------
a12e5664555a4f12b3cc6991db5accf9 jdoe user.jdoe:user.jdoe.test.container.1234.1 OK[3/0/0] tier=1&disk=1 1 2018-02-09 03:57:46
b0fcde2acbdb489b874c3c4537595adc janedoe user.jdoe:user.jdoe.test.container.1234.1 REPLICATING[4/1/1] tier=1&tape=1 2
4a6bd85c13384bd6836fbc06e8b316d7 mc user.jdoe:user.jdoe.test.container.1234.1 OK[3/0/0] tier=1&tape=1 2
The state indicate how many locks (physical replicas of the files) are OK, Replicating or Stuck
Accessing files¶
The command to download DIDs locally is called rucio download. It supports various sets of option. You can invoke it like this:
# rucio download user.jdoe:user.jdoe.test.container.1234.1
2018-02-02 15:13:08,450 INFO Thread 1/3 : Starting the download of user.jdoe:test.file.2
2018-02-02 15:13:08,451 INFO Thread 2/3 : Starting the download of user.jdoe:test.file.3
2018-02-02 15:13:08,451 INFO Thread 3/3 : Starting the download of user.jdoe:test.file.1
2018-02-02 15:13:08,503 INFO Thread 1/3 : File user.jdoe:test.file.2 trying from SITE1_DISK
2018-02-02 15:13:08,549 INFO Thread 2/3 : File user.jdoe:test.file.3 trying from SITE2_DISK
2018-02-02 15:13:08,551 INFO Thread 3/3 : File user.jdoe:test.file.1 trying from SITE1_DISK
2018-02-02 15:13:10,399 INFO Thread 3/3 : File user.jdoe:test.file.1 successfully downloaded from SITE1_DISK
2018-02-02 15:13:10,415 INFO Thread 2/3 : File user.jdoe:test.file.3 successfully downloaded from SITE2_DISK
2018-02-02 15:13:10,420 INFO Thread 3/3 : File user.jdoe:test.file.1 successfully downloaded. 39.247 kB in 1.85 seconds = 0.02 MBps
2018-02-02 15:13:10,537 INFO Thread 2/3 : File user.jdoe:test.file.3 successfully downloaded. 641.427 kB in 1.87 seconds = 0.34 MBps
2018-02-02 15:13:10,614 INFO Thread 1/3 : File user.jdoe:test.file.2 successfully downloaded from SITE1_DISK
2018-02-02 15:13:10,633 INFO Thread 1/3 : File user.jdoe:test.file.2 successfully downloaded. 636.075 kB in 2.11 seconds = 0.3 MBps
----------------------------------
Download summary
----------------------------------------
DID user.jdoe:user.jdoe.test.container.1234.1
Total files : 3
Downloaded files : 3
Files already found locally : 0
Files that cannot be downloaded : 0