EOS converters and file identifiers

An EOS converter changes the file ID of the file being converted.

When a converter works on a file it:

1. Creates a file with a new file ID in the /proc/converter directory.
2. Merges all of the metadata from the original file.
3. Renames the new file to be on top of the existing file.

EOS documentation about conversion

• Converter Engine
• Converter
• Automatic Layout Conversion hooks

According to this documentation:

Converter serves two purposes: For Group Balancer and GEO Balancer it is used to rewrite files with to a new location.

Regarding the group balancer: the files are written RR style between groups. If a group is full, there is not free space left on the underlying set of disks, and the bandwidth is limited to the groups on which some freespace is available.

To recover bandwidth, the group that is full must be rebalanced to groups with some freespace so that space is liberated and all the groups can again be written to.

This allows to liberate more bandwidth by enabling again write to all groups.

Auto Repair broken replicas

According to the Auto Repair EOS documentation:

You can configure the FSTs to regulary scan all files for file/block checksum errors. If you enable the auto repair function the FSTs will call to trigger an asynchronous rewrite of the corrupted file using the Converter.

This means that a broken replica triggers a conversion process on on the file on the same space to repair the broken layout.

Other use cases for conversion

Conversion is also needed to change the underlying disk replica layout: this was heavily used to convert Alice 2-replica layout to RAIN layout.

Conversion also allows to change move file content between 2 spaces.