You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/01/10 13:37:55 UTC

[GitHub] [hudi] cb149 edited a comment on issue #3984: [SUPPORT] Upgrade from 0.8.0 to 0.9.0 removes functionality and decreases performance

cb149 edited a comment on issue #3984:
URL: https://github.com/apache/hudi/issues/3984#issuecomment-1008878036


   > hi @cb149 @nsivabalan @xushiyan I have found this problem, just need to `set hoodie.file.index.enable=false` to work
   > 
   > ```
   > val tripsSnapshotDF = spark.read.format("hudi")
   >   .option("hoodie.file.index.enable", "false")
   >   .load(basePath) 
   > ```
   
   HI @XuQianJin-Stars that solves the problem but decreases the performance extremely, since it takes a very long time before the Stage in Spark is visible.
   
   E.g. as a workaround I am using `....where("_partition like 'year=2021/month=6/%'").count` , which takes like 5 seconds total, while using _hoodie.file.index.enable false_ takes multiple minutes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org