Scheduler

Scheduler is a central part of the CTA system which is responsible for deciding when a tape shall be mounted into a tape drive. As the time to mount a tape is significant — O(1 min) — tape systems need to accumulate transfer requests in multiple queues before it is worth mounting a tape. The Scheduler decision is important for efficient throughput to be achieved. The scheduling workflow involves most parts of the CTA infrastructure: the CTA Frontend, Disk Instance, Tape Server, Catalogue and SchedulerDB backend for metadata operations.

The initial part of the scheduling workflow runs on the side of the CTA Frontend which interacts with the Disk Instance. Once a file has been written to a Disk Instance or a tape read has been requested by the Disk Instance, the Scheduler makes sure to record the requested transfers in SchedulerDB with all necessary metadata and acknowledges it back.

The scheduling decision is then made by the Tape Servers, where the most of the scheduling work is being executed. Whenever a Tape Daemon has a Tape Drive free to mount a tape, a Tape Drive session is started, which polls the SchedulerDB for work and asks a Scheduler if it is worth to mount a tape. Each Tape Drive session looks at all the work which exists in the SchedulerDB and creates a summary of all the transfer requests. These summaries are categorized by several parameters (e.g.: archive/retrieve transfer type and status, Tape Pool, existing mounts (if any), Mount Policy, etc.). Since each queue summary contains either archive or retrieve requests, it means, that a tape will be always mounted either for read (retrieve) or write (archive) transfers to be served and never for the mix of the two transfer types.

The Scheduler then filters these queue summaries further based on the matches with the constraints of the Tape Drive session, which makes such a poll (i.e. Tape Pool it belongs to and similar), as well as with other Scheduler configuration parameters (e.g. the match of the minimal request age with the Mount Policy, min/max number of drives per Virtual Organization, etc.). Each match of all the criteria with a particular queue is called a Tape Mount (existing or potential future mounts). The result of the scheduling decision is a list of Tape Mounts ordered in terms of priority of the Mount Policies. If the Tape Drive session finds a compatible Tape Mount in this list, mounting of a tape is triggered. The execution of the transfer itself then continues to poll the transfers from the Scheduler DB for its particular Tape Mount until the end of the tape is reached, until there are no more transfers to fulfill or other criteria to unmount a tape are met.

For more information, please check the Scheduler Architecture in the development section of this documentation.