You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "stream2000 (via GitHub)" <gi...@apache.org> on 2023/06/14 03:21:24 UTC

[GitHub] [hudi] stream2000 opened a new issue, #8960: [SUPPORT] Do we need a multi table service tools for hudi？

stream2000 opened a new issue, #8960:
URL: https://github.com/apache/hudi/issues/8960

   **Describe the problem you faced**
   
   Now we have `HoodieMultiTableDeltaStreamer` using spark to ingest multi tables into hudi, and we can also ingest multi tables using a single flink in some platforms like alicloud vvp.  In such scenario we may want to run all tables services outside of the ingestion job to improve stability. 
   
   Currently, hudi provides a bunch of offline table service jobs but we still need to run them manually or through some schedule tools. What about introducing a stand-alone multi table service runner that will continuously run table services for all tables under one/some db paths?  
   
   I noticed that hudi has already introduce table management service and meta server. But I believe that a single spark job for all table services in a db will help improve the ease of use of hudi. 
   
   I can help to contribute the tool if hdi community needs this~ 
   
   **To Reproduce**
   
   Steps to reproduce the behavior:
   
   1.
   2.
   3.
   4.
   
   **Expected behavior**
   
   A clear and concise description of what you expected to happen.
   
   **Environment Description**
   
   * Hudi version :
   
   * Spark version :
   
   * Hive version :
   
   * Hadoop version :
   
   * Storage (HDFS/S3/GCS..) :
   
   * Running on Docker? (yes/no) :
   
   
   **Additional context**
   
   Add any other context about the problem here.
   
   **Stacktrace**
   
   ```Add the stacktrace of the error.```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] codope closed issue #8960: [SUPPORT] Do we need a multi table service tools for hudi？

Posted by "codope (via GitHub)" <gi...@apache.org>.
codope closed issue #8960: [SUPPORT] Do we need a multi table service tools for hudi？ 
URL: https://github.com/apache/hudi/issues/8960


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] ad1happy2go commented on issue #8960: [SUPPORT] Do we need a multi table service tools for hudi？

Posted by "ad1happy2go (via GitHub)" <gi...@apache.org>.
ad1happy2go commented on issue #8960:
URL: https://github.com/apache/hudi/issues/8960#issuecomment-1621226720

   Thanks @stream2000. This feature will be helpful and will improve usability. Feel free to contribute here.
   
   Created a JIRA to track - https://issues.apache.org/jira/browse/HUDI-6481
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org