[jira] [Commented] (JAMES-3578) Post 3.6.x Cassandra data schema sanitizing

["Raphael Ouazana (Jira)" <se...@james.apache.org>]

    [ https://issues.apache.org/jira/browse/JAMES-3578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17340043#comment-17340043 ] 

Raphael Ouazana commented on JAMES-3578:
----------------------------------------

Could we at least have a documentation page explaining clearly what is the upgrade path for each version?

> Post 3.6.x Cassandra data schema sanitizing
> -------------------------------------------
>
>                 Key: JAMES-3578
>                 URL: https://issues.apache.org/jira/browse/JAMES-3578
>             Project: James Server
>          Issue Type: Improvement
>          Components: cassandra, mailbox, RRT
>            Reporter: Benoit Tellier
>            Priority: Major
>             Fix For: 3.7.0
>
>
> For multiple reasons, our Cassandra schemas evolve over time and we end up adapting the tables structure over time.
> To migrate data from the old to the new table organisation, the Apache James project:
>  - Support backward compatibility and will lookup the old tables if needed, sometime performing on the fly data migration.
>  - Provide migration tasks for moving the data from the old tables to the new.
> Of course all this logic is complex and needs to be maintained:
>  - More tests needs to be ran as we need to ensure correctness regardless of the schema version
>  - More bugs resulting from interaction between this migration logic and other features like read-repairs
>  - More code meaning heavier maintenance costs
>  - More Cassandra tables being created, which have a detrimental impact on performance (marginal yes but still... https://thelastpickle.com/blog/2020/11/25/impacts-of-many-tables-on-cassandra.html )
> Here are the migration we currently support:
> {code:java}
> allMigrationClazzBinder.addBinding(FROM_V5_TO_V6).to(MailboxPathV2Migration.class);
>         allMigrationClazzBinder.addBinding(FROM_V6_TO_V7).to(MappingsSourcesMigration.class);
>         allMigrationClazzBinder.addBinding(FROM_V7_TO_V8).to(MailboxPathV3Migration.class);
>         allMigrationClazzBinder.addBinding(FROM_V8_TO_V9).to(MessageV3Migration.class);
>         allMigrationClazzBinder.addBinding(FROM_V9_TO_V10).to(AclV2Migration.class);
> {code}
> I propose to drop MailboxPathV2Migration, MappingsSourcesMigration, MailboxPathV3Migration.
> Doing so allows dropping 2 tables and remove 3.000 lines of code. Users will need to upgrade from 3.6.x to perform these migrations before moving on to 3.7.x...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org