You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "njalan (via GitHub)" <gi...@apache.org> on 2023/09/14 14:10:10 UTC

[GitHub] [hudi] njalan commented on issue #9529: S3 storage cpu is high when spark streaming is upserting data into hudi

njalan commented on issue #9529:
URL: https://github.com/apache/hudi/issues/9529#issuecomment-1719531578

   @ad1happy2go I  have upgraded 40% hudi tables  from 0.9 to 0.13.1 with metadata enabled.  But based on the metrics from s3 server the list files operations is not reduced.  Below links clear mentioned that it metadata can eliminate the requirement for the "list files" operation.  
   https://hudi.apache.org/docs/metadata/
   I can see the /.hoodie/metadata/files under base table path.
   Did I miss anything? Why there is no list object reduction?
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org