Skip to content

EOS space discussions

Reasons to make the FST garbage collector aware of EOS spaces

  1. During testing of the MGM GC with two EOS spaces it would be nice to be able to selectively turn FST garbage collection on and off per EOS space. This is a nice to have requirement, it is not mission critical.
  2. Julien has pointed out that he will be deploying CTA in configurations where: 2.1 A single host will contain the file systems of multiple EOS spaces. 2.2 A single disk storage device may contain the file systems of multiple EOS spaces. I understand that this is still under discussion with Andreas.

With such configurations it may again be useful to turn FST garbage collection on and off per EOS space. Is this a mission critical requirement? I don't know and we need to discuss.

Reason not to make FST garbage collector aware of EOS spaces

  1. The complexity of the "simple" python garbage collector is no longer simple. Testing and debugging has taught me that I should not be pushing the complexity envelope with python. Two major time consumers have emerged. Firstly the lack of static type checking has turnis a simply type changes into nighmares. Replacing a string with an array of strings just works when you wish it wouldn't. Iterating over a string gives you each invidual letter as opposed to teh list giving you string item. Secondly unit testsing has to try and execute EVERY path through the code because only then will all the syntax problems be found.
  2. The true partner of the FST garbage collector daemon is the EOS FST daemon. Unlike the FST garbage collector which is host driven, the EOS FST daemon is file system driven. A given EOS FST can be told to work on all of the file systems on a given host or it can be told to work on only one. Julien currently deploys more than one EOS FST daemon per host. One day the FST garbage collector logic may have to be ported to the "file scanner" logic within the EOS FST. The garbage collection logic would then be file system driven and not host or EOS space driven.
  3. Current deployments will be difficult to understand when there isn't a one to one mapping between FST garbage collectors and EOS FST daemons.

Discussion for 13:30 on Wednesday 18 MArch 2020

The discussion should center around answering the following question: * Should we(I) give up the idea of an FST garbage collector driven by host and EOS space and go for one driven by file system instead? What are the production deployment requirements of CTA?