You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@amoro.apache.org by "baiyangtx (via GitHub)" <gi...@apache.org> on 2024/11/27 11:43:12 UTC

[I] [Improvement]: Make Controller Depend on Stateless Services [amoro]

baiyangtx opened a new issue, #3346:
URL: https://github.com/apache/amoro/issues/3346

   ### Search before asking
   
   - [X] I have searched in the [issues](https://github.com/apache/amoro/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### What would you like to be improved?
   
   To support multi-node deployment of AMS REST services(as mentioned in issue #2810 ), we must refactor the Controller implementation and its dependent service layer to be stateless.
   
   ### How should we improve?
   
   Based on the previous discussion at #2997, the current services will be split into two types: stateless services and stateful services.
   
   Firstly, these two types of services will adhere to the following standards:
   
   1. All stateless service interfaces will have the suffix `Manager` such as `CatalogManager` or `OptimizerManager`.
   
   2. Stateless services can provide more complex functionalities than direct database access, such as CatalogCache, etc., but they must assume that state changes will occur on other nodes. Therefore, stateless services need to pay attention to maintaining the consistency between in-memory objects and the database state.
   
   3. Stateful service interfaces will have the suffix `Service` such as `TableService` or `OptimizingService`, and will share a common parent interface named `StateService`, which includes an `initialize` method and a `dispose` method. All methods of the stateful service interfaces must check the initialization state.
   
   4. Stateless services will be initialized before the leader election, and stateful services will be initialized after the election, and will only be initialized on the master node.
   
   
   
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Subtasks
   
   I will break down the task into several subtasks to advance the goal in parallel.
   
   ### Refactoring of the TableService Interface
   
   - [ ] **CatalogManager**: The TableService interface will no longer inherit from `CatalogService`, and `CatalogService` will be modified to become a stateless `CatalogManager`.
   - [ ] **InternalTableManager**: Interfaces related to `TableMetadata` management will be extracted into `InternalTableManager`, which will be used by `IcebergRestCatalogServer`.
   - [ ] **TableBlockerManager**: Interfaces related to `TableBlocker` will be extracted into `TableBlockerManager`, which will be used by `MixedFormat` and **Table Thrift API**.
   - [ ] **MaintainedTableManager**: Interfaces related to `ServerTableIdentifier` and `TableRuntimeMeta` will be extracted into `MaintainedTableManager`, which will be used by **Process and Optimizing-related controllers**.
   - [ ] **TableService**: The remaining functionality of `TableService` will be retained, including the maintenance of `TableRuntime` and the implementation of the `TableRuntimeHandler` listener chain invocation.
   
   ### Refactoring of the DefaultOptimizingService Interface
   
   - [ ] Extract the implementation of `OptimizerManager` from `DefaultOptimizingService` and reimplement it as a stateless service.
   - [ ] Refer to the design of **TableConfig refresh** and `TableRuntimeHandler` to implement the refresh and exploration of `OptimizingGroupConfig`(due to modifications may not take place on the master node).
   
   
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Improvement]: Make Controller Depend on Stateless Services [amoro]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #3346:
URL: https://github.com/apache/amoro/issues/3346#issuecomment-3304900861

   This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Improvement]: Make Controller Depend on Stateless Services [amoro]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #3346: [Improvement]: Make Controller Depend on Stateless Services
URL: https://github.com/apache/amoro/issues/3346


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Improvement]: Make Controller Depend on Stateless Services [amoro]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #3346:
URL: https://github.com/apache/amoro/issues/3346#issuecomment-3363664190

   This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@amoro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org