Skip to content

C++ versus Python

The FST garbage collector is currently implemented in Python. This decision was made in order to make it easier for tape operators to modify and improve the garbage collection logic. This section describes the many reasons to move away from Python and use C++ instead and more specifically to add C++ code to the EOS FST daemon itself.

Over time more requests are being made to increase the complexity of the FST garbage collector. For example there is an ongoing dicussion to make the garbage collector aware of either EOS spaces, individual EOS file systems or even both. The modifications required to implement these requests are not simple and are therefore undermining the idea of a simple Python program that tape operators can modify with ease.

The static type checking of C++ helps developers make "safer" changes to a program. Python does not provide static type checking. I'm not pythonic in the way I think. I like to be able to change the type of an object in my code and have the compiler automatically take me around the code to the points where the new data type is causing a breakage. Python goes against this by trying to make different types work together.

Python is an interpretted language and therefore unit testing is not only essential it is also required to execute all paths through the code base in order to test for syntax errors.

The EOS FST daemon already contains "file scanner" code. The EOS team has suggested that I add the CTA garbage collection logic to this existing code.

Discussions between Julien and Andreas are pointing towards the possibility of running a RAIN organisation across the FSTs of the EOSCTA instances. We are currently using a single replica layout. The "stagerrm" actions of the FST garbage collector are orchestrated by the EOS MGM and therefore the removal of "old" disk replicas should be independent of the EOS disk layout. Even though this is true the mapping between local FST files and EOS logical file IDs and the way free space is determined may be different between different EOS disk layouts. Putting the FST garbage collector logic inside the EOS FST daemon will either remove all dependencies on the different EOS disk layouts or it will facilitate the handling of them.