[jira] [Updated] (IGNITE-15404) Rework distributed configuration flow

["Andrey Mashenkov (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IGNITE-15404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andrey Mashenkov updated IGNITE-15404:
--------------------------------------
    Summary: Rework distributed configuration flow  (was: Rework disctributed configuration flow)

> Rework distributed configuration flow
> -------------------------------------
>
>                 Key: IGNITE-15404
>                 URL: https://issues.apache.org/jira/browse/IGNITE-15404
>             Project: Ignite
>          Issue Type: Improvement
>            Reporter: Alexander Lapin
>            Assignee: Alexander Lapin
>            Priority: Major
>              Labels: ignite-3
>             Fix For: 3.0.0-alpha3
>
>          Time Spent: 9h 10m
>  Remaining Estimate: 0h
>
> Based on prototyping carried out within IGNITE-15374 it's requited to rework distributed configuration flow processing:
>  # Use configuration registry instead of meta storage manager.
>  ## Use configuration manager for internal configuration processing. Blocked by IGNITE-15047
>  ## Use configuration manager in order to retrieve distributed up-to-date values.
>  ## Use configuration listeners instead of meta storage watches.
>  # Favor one-step table creation (calculate assignments and inner schema together with initial table configuration update).
> Please pay attention that during prototyping plenty of bugs were found that should be fixed during refactoring. To name some:
>  # Columns validation is skipped if column was added or removed during alterTable() See onConfigurationChanged() for more details.
> {code:java}
> if (!newTbl.columns().namedListKeys().equals(oldTbl.columns().namedListKeys()))
>     return true;
> {code}
>  # table identified ignores update counter
> {code:java}
> UUID tblId = new UUID(ctx.storageRevision(), 0L);
> {code}
> Of course in addition to fixing known issues we will introduce new bugs during refactoring.
> *Following items were implemented:*
> 1. Unnecessary phases of assignment calculation and schema preparation as a reaction on distributed event were simplified to in-place calculation within table creation:
>  
> {code:java}
> clusterCfgMgr
>     .configurationRegistry()
>     .getConfiguration(TablesConfiguration.KEY)
>     .tables()
>     .change(
>         change -> change.create(
>             name,
>             (ch) -> {
>                 tableInitChange.accept(ch);
>                 ((ExtendedTableChange)ch).
>                     // Table id specification.
>                     changeId(tblId.toString()).
>                     // Affinity assignments calculation.
>                     changeAssignments(
>                         ByteUtils.toBytes(
>                             AffinityService.calculateAssignments(
>                                 baselineMgr.nodes(),
>                                 ch.partitions(),
>                                 ch.replicas()
>                             )
>                         )
>                     ).
>                     // Table schema preparation.
>                     changeSchemas(
>                         schemasCh -> schemasCh.create(
>                             String.valueOf(INITIAL_SCHEMA_VERSION),
>                             schemaCh -> schemaCh.changeSchema(
>                                 ByteUtils.toBytes(
>                                     SchemaService.prepareSchemaDescriptor(
>                                         ((ExtendedTableView)ch).schemas().size(),
>                                         ch
>                                     )
>                                 )
>                             )
>                         )
>                     );
>             }
>         )
>     )
> {code}
>  
> Besides obvious that allowed to simplify AffinityManager and SchemaManager that were converted to simple stateless services.
> 2. Same is about table id: instead of using meta storage revision and update counter (this wasn't implemented though) explicit table id generation was used. This is the main reason for the large number of changed files.
> 3. Inner local events were substituted by common Futures within create/alter/dropTable.
> Few more tickets were created for further improvements:
> [IGNITE-15414 Schema validation refactoring with configuration validators.|https://issues.apache.org/jira/browse/IGNITE-15414]
> [IGNITE-15480 Refactor index column mapping based on column id automatically generated on column creation with the help of internal configuration extensions.|https://issues.apache.org/jira/browse/IGNITE-15480] 
> [IGNITE-15485 Support table rename operation.|https://issues.apache.org/jira/browse/IGNITE-15485] 
> Some tickets for further refactoring are still awaited:
> [IGNITE-15409 Listener with any placeholders|https://issues.apache.org/jira/browse/IGNITE-15409] 
> [IGNITE-15412 Configuration registry: add ability to retrieve distributed value|https://issues.apache.org/jira/browse/IGNITE-15412] That will allow to remove metastorage from table manager completely and get rid of current verison of TableManager#tableNamesConfigured
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)