Deprecated

This page is deprecated and may contain information that is no longer up to date.

EOS-CTA Protocol¶

Transport Layer¶

The communication channel between EOS and CTA is the XRootD Scalable Service Interface v2 (SSIv2) protocol.

The XRootD SSI protocol allows maximum parallelisation of requests served to the CTA Frontend, i.e. without per-file serialization. The CTA Frontend internally packs requests together to improve database and object store bandwidth, to meet the performance requirements described in link.

Protocol Layer¶

All Requests and Responses sent between EOS to CTA are defined and serialized using Google Protocol Buffers v3.

A workflow event of type proto: is assigned by the administrator to directories where events should be propagated to CTA.
When a workflow event occurs, a notification message will be sent to the CTA Frontend. Notifications are synchronous: EOS will receive an acknowledgement or an error message.
The workflows do not set any file properties on the EOS side. These are set by CTA via the xattr map in the Response message or by one of the URL callbacks specified below.
The Protocol Buffer definition provides the CTA Frontend with everything EOS knows about the file. The CTA Frontend will select only the information it needs from the Protocol Buffer message.

The Protocol Buffers are defined in the xrootd-ssi-protobuf-interface project, which is shared between EOS and CTA. This project comprises:

A set of generic C++ headers to bind a set of Protocol Buffer definitions to the XRootD SSIv2 transport layer (The headers are not specific to EOS or CTA; in principle they can be used for any project which has a client and server communicating using protocol buffers over XRootD SSIv2).
The Protocol Buffer definitions which instantiate the EOS--CTA interface.

Protocol Buffer URL Fields¶

The event notification Protocol Buffer includes three URL fields which are constructed by EOS:

message Service {
  string name = 1;    //< name of the service
  string url  = 2;    //< access url of the service
}

message Workflow {
  ...
  cta.common.Service instance = 5;    //< instance information
  ...
}

message Transport {
  string dst_url    = 1;    //< transport destination URL
  string report_url = 2;    //< URL to report successful archiving
}

The access URL and report URL are sent with archive events.

The transport destination URL is sent with retrieve events.

egin{alertbox}[Retrieval Notification Message] Possibly we need another callback URL to provide notification of a successful retrieval\slash error message in case of a failed retrieval (under discussion).

Service Access URL¶

The Service Access URL Request.notification.wf.instance.url is used by the CTA Tape Server daemon cta-taped to read the disk file from the EOS MGM during an archive event.

Example URL:

root://ctapps-eossrv01.cern.ch//eos/ctapps/preprod/ed8f3136/0/test099999?eos.lfn=291198a

which is composed as follows:

root://<hostname>.cern.ch/<path>/<filename>?eos.lfn=<fid>

<hostname> is the hostname of the EOS MGM
<path>/<filename> is the absolute EOS path to the file
<fid> is the file ID, given in hex format as fxid:<fid-hex>

File Archived Report URL¶

The File Archived Report URL Request.notification.transport.report_url is used by the Tape Server to asynchronously report to the EOS MGM that a file has been safely archived to tape.

The URL is composed as follows:

eosQuery://<hostname>.cern.ch/<path>/<filename>?mgm.pcmd=event&mgm.fid=<fid-hex>& mgm.logid=cta&mgm.event=archived&mgm.workflow=default&mgm.path=<path>/<filename>& mgm.ruid=<ruid>&mgm.rgid=<rgid>

mgm.pcmd tells the MGM to execute a workflow event (event)
mgm.fid gives the file ID of the file that was archived, in hexadecimal digits
mgm.logid gives the Log ID (cta)
mgm.event sets which workflow event to execute (archived)
mgm.workflow sets which workflow to execute (default)
mgm.path gives the path to the EOS disk file
mgm.ruid sets the user ID of the user invoking the workflow
mgm.rgid sets the group ID of the user invoking the workflow

The MGM will respond by executing the sys.workflow.archived.default workflow against the specified file as user/group <ruid>:<rgid>.

Transport Destination URL¶

The Transport Destination URL Request.notification.transport.dst_url is used by the Tape Server to write a tape file to the EOS MGM during a retrieve event.

The URL is composed as follows:

root://<hostname>.cern.ch/<path>/<filename>?eos.lfn=<fid>&eos.ruid=<ruid>& eos.rgid=<rgid>&eos.injection=1&eos.workflow=none

eos.lfn specifies the file ID, given in hex format as fxid:<fid-hex>
eos.ruid sets the user ID of the user invoking the workflow
eos.rgid sets the group ID of the user invoking the workflow
eos.injection=1 informs EOS that we are sending it data to write to the file
eos.workflow sets which workflow event to execute (none)

Note: When EOS writes a file to disk, the closew.default workflow will be triggered, to archive the file on tape. This workflow should obviously not be triggered when the file being written is being retrieved from tape. The URL overrides the default workflow by explicitly setting it to none.

Protocol Buffer versions¶

If the Protocol Buffer definitions are updated:

First install the new version into CTA.
Next, upgrade EOS instances (one-at-a-time).

CTA must maintain compatibility with old protocol versions until the upgrades have been rolled out to all experiments.

EOS support of tape notions¶

EOS keeps track of tape-related information using extended attributes:

The CTA_StorageClass attribute indicates that one or more copies should be stored on tape under the specified Storage Class. The CTA database will specify the number of tape copies that should be created for that Storage Class.
EOS reserves filesystem ID 65535 for tape operations. This should be specified in all tape directories using the extended attribute CTA_TapeFsId="65535". EOS treats FS ID 65535 as a phantom filesystem in the EOS namespace, using it to record the number of copies stored on tape.