[GitHub] [hudi] HEPBO3AH commented on issue #3984: [SUPPORT] Upgrade from 0.8.0 to 0.9.0 removes functionality and decreases performance

["HEPBO3AH (via GitHub)" <gi...@apache.org>]

HEPBO3AH commented on issue #3984:
URL: https://github.com/apache/hudi/issues/3984#issuecomment-1724835210

   Hi @cb149 , we are seeing a very slow stage during clustering where the file creation takes from 1 to 5 minutes depending on the amount of data. Similar to what you've seen here:
   >21/12/21 09:03:01 INFO io.HoodieCreateHandle: New CreateHandle for partition :year=2021/month=12/day=21 with fileId 5ec7339d-b2a3-4cee-97bb-ea60b637e411-0
   21/12/21 09:04:02 INFO io.HoodieCreateHandle: Closing the file 5ec7339d-b2a3-4cee-97bb-ea60b637e411-0 as we are done with all the records 3402219
   21/12/21 09:04:03 INFO io.HoodieCreateHandle: CreateHandle for partitionPath year=2021/month=12/day=21 fileID 5ec7339d-b2a3-4cee-97bb-ea60b637e411-0, took 62292 ms.
   
   The CreateHandle takes 50 sec to cluster 3 files of 3.1mb size where BLOOM was 3mb out of those.
   
   Did you manage to get to the bottom of this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org