[jira] [Created] (JAMES-3578) Post 3.6.x Cassandra data schema sanitizing

["Benoit Tellier (Jira)" <se...@james.apache.org>]

Benoit Tellier created JAMES-3578:
-------------------------------------

             Summary: Post 3.6.x Cassandra data schema sanitizing
                 Key: JAMES-3578
                 URL: https://issues.apache.org/jira/browse/JAMES-3578
             Project: James Server
          Issue Type: Improvement
          Components: cassandra, mailbox, RRT
            Reporter: Benoit Tellier
             Fix For: 3.7.0


For multiple reasons, our Cassandra schemas evolve over time and we end up adapting the tables structure over time.

To migrate data from the old to the new table organisation, the Apache James project:

 - Support backward compatibility and will lookup the old tables if needed, sometime performing on the fly data migration.
 - Provide migration tasks for moving the data from the old tables to the new.

Of course all this logic is complex and needs to be maintained:

 - More tests needs to be ran as we need to ensure correctness regardless of the schema version
 - More bugs resulting from interaction between this migration logic and other features like read-repairs
 - More code meaning heavier maintenance costs
 - More Cassandra tables being created, which have a detrimental impact on performance (marginal yes but still... https://thelastpickle.com/blog/2020/11/25/impacts-of-many-tables-on-cassandra.html )

Here are the migration we currently support:

{code:java}
allMigrationClazzBinder.addBinding(FROM_V5_TO_V6).to(MailboxPathV2Migration.class);
        allMigrationClazzBinder.addBinding(FROM_V6_TO_V7).to(MappingsSourcesMigration.class);
        allMigrationClazzBinder.addBinding(FROM_V7_TO_V8).to(MailboxPathV3Migration.class);
        allMigrationClazzBinder.addBinding(FROM_V8_TO_V9).to(MessageV3Migration.class);
        allMigrationClazzBinder.addBinding(FROM_V9_TO_V10).to(AclV2Migration.class);
{code}

I propose to drop MailboxPathV2Migration, MappingsSourcesMigration, MailboxPathV3Migration.

Doing so allows dropping 2 tables and remove 3.000 lines of code. Users will need to upgrade from 3.6.x to perform these migrations before moving on to 3.7.x...




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org