You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "ad1happy2go (via GitHub)" <gi...@apache.org> on 2023/03/20 18:31:44 UTC

[GitHub] [hudi] ad1happy2go commented on issue #8215: [SUPPORT] spark-shell cannot obtain the latest data

ad1happy2go commented on issue #8215:
URL: https://github.com/apache/hudi/issues/8215#issuecomment-1476740454

   @LiJie20190102 I was able to reproduce the issue. When I had spark shell opened, it was still giving me old data while when querying on hive it was giving the latest data. 
   
   In spark shell, it caches the old data frame and when we run again it reads from the cached data frame. That is why we see old data. When I tried terminating the shell and start again, spark shell was also able to fetch the latest data.
   
   <img width="1723" alt="image" src="https://user-images.githubusercontent.com/63430370/226430841-e3923efe-f475-4172-b3b7-27da224a328f.png">
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org