Skip to content

Scheduler Backend Evolution#

Starting point: 


stateDiagram-v2
    direction LR
    classDef whitesystem fill:white
    classDef needsreview fill:#ff8c1a
    classDef allok fill:#66b3ff
    classDef almostimplemented fill:yellow
    Scheduler<br>(cta/scheduler):::allok --> SchedulerDatabase<br>(cta/scheduler):::allok
    OStoreDB<br>(cta/scheduler/OStoreDB):::needsreview --> objectstore<br>(cta/objectstore):::needsreview
    SchedulerDatabase<br>(cta/scheduler) --> OStoreDB<br>(cta/scheduler/OStoreDB)

  • ensuring compatibility with Scheduler:
  • copy of OStoreDB methods put in cta/scheduler/rdbms making them throw exceptions
  • useful strategy to learn/review all the bits from the very start (i.e. neither taped not ctafrontend start up)
  • to start running the system, step by step approach: crash → implement → test

Midterm goal (end of 2025): - functional PostgresSched backend (Archive/Retrieve and Repack workflows)


stateDiagram-v2
    direction LR
    classDef whitesystem fill:white
    classDef needsreview fill:#ff8c1a
    classDef allok fill:#66b3ff
    classDef almostimplemented fill:yellow
    Scheduler<br>(cta/scheduler):::allok --> SchedulerDatabase<br>(cta/scheduler):::allok
    rdbms<br>(cta/scheduler/rdbms):::allok --> rdbms<br>(cta/rdbms):::allok
    SchedulerDatabase<br>(cta/scheduler) --> rdbms<br>(cta/scheduler/rdbms):::allok

Our longterm target: - re-think and re-implement part of the Scheduler and SchedulerDatabase (+ refactor the cta/scheduler/rdbms code) to exploit the advantages of relational DB (PostgreSQL) and remove biases in scheduling introduced by objectstore


stateDiagram-v2
    direction LR
    classDef allok fill:#5cd65c
    Scheduler<br>(cta/scheduler):::allok --> SchedulerDatabase<br>(cta/scheduler):::allok
    rdbms<br>(cta/scheduler/rdbms):::allok --> rbms<br>(cta/rdbms):::allok
    SchedulerDatabase<br>(cta/scheduler) --> rdbms<br>(cta/scheduler/rdbms):::allok

Current development: - Archive and Retrieve workflows are functional - WIP: Repack workflows - WIP: Scaling infrastructure deployment for tests - no changes/improvements to the Scheduler or SchedulerDatabase unless necessary (creating tickets with these for the future)