You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "leosanqing (via GitHub)" <gi...@apache.org> on 2023/09/27 10:22:16 UTC

[GitHub] [hudi] leosanqing opened a new issue, #9793: [SUPPORT] When is the clustering service used?

leosanqing opened a new issue, #9793:
URL: https://github.com/apache/hudi/issues/9793

   **_Tips before filing an issue_**
   
   
   
   **Describe the problem you faced**
   
   I know that the clustering service is mainly to solve the problem of too many small files and split large files (such as consistent Hash bucket index). But when used in Flink, the bucket index does not have the problem of small files, and the Flink State Index writes data will be written to small files first, so there should not be a large number of small files.
   
   So when will the clustering service be used? Is it related to the index type?
   
   Thx
   
   
   **Expected behavior**
   
   A clear and concise description of what you expected to happen.
   
   **Environment Description**
   
   * Hudi version : 0.13
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [SUPPORT] When is the clustering service used? [hudi]

Posted by "ad1happy2go (via GitHub)" <gi...@apache.org>.

ad1happy2go commented on issue #9793:
URL: https://github.com/apache/hudi/issues/9793#issuecomment-1748292798

   @leosanqing Closing this issue as @danny0405 clarifies. Feel free to open in case of any more questions on this. Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] danny0405 commented on issue #9793: [SUPPORT] When is the clustering service used?

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.

danny0405 commented on issue #9793:
URL: https://github.com/apache/hudi/issues/9793#issuecomment-1740177039

    The clustering service is a more general optimization for small files, it can rearrrange the file layouts to merge the small file groups into larger ones. The normal write path like Flink can sove some small file issues but it only optimizes the current writing bucket, it can not do multi-filegroup merging.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [SUPPORT] When is the clustering service used? [hudi]

Posted by "codope (via GitHub)" <gi...@apache.org>.

codope closed issue #9793: [SUPPORT] When is the clustering service used?
URL: https://github.com/apache/hudi/issues/9793


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org