[GitHub] [hudi] xushiyan commented on pull request #4309: [HUDI-3016][RFC-43] Proposal to implement Table Management Service

[GitBox <gi...@apache.org>]

xushiyan commented on PR #4309:
URL: https://github.com/apache/hudi/pull/4309#issuecomment-1293182401

   > if we delegate the scheduling also to TMS, we can't guarantee that there won't be any other inflight operation while scheduling compaction
   
   @nsivabalan TMS is a downstream listener to metaserver (or any timeline server used if no metaserver) so TMS is aware of all inflight commits on the registered tables, and we should use that info to generate plans.
   
   > can regular writers take care of scheduling table services by themselves and only delegate execution to TMS?
   
   Let's clarify: TMS is responsible for plan generation and scheduling, while execution is send to a separate cluster, and TMS is monitoring the execution. You're proposing to basically delegating execution, it's possible but not really making use of TMS capabilities. I'd suggest full delegation to TMS; we're starting a server, so making full use of it will justify the design and the cost.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org