[GitHub] [spark] dongjoon-hyun edited a comment on issue #25460: [SPARK-25474][SQL][FOLLOW-UP] fallback to hdfs when relation table stats is not available

[GitBox <gi...@apache.org>]

dongjoon-hyun edited a comment on issue #25460: [SPARK-25474][SQL][FOLLOW-UP] fallback to hdfs when relation table stats is not available
URL: https://github.com/apache/spark/pull/25460#issuecomment-521892398
 
 
   To @shahidki31 . 
   
   @maropu and @cloud-fan meant the corner case when the table size is equal to the user configuration value (not 8.0EB). Let say we set the configuration to 1GB and we have a static table T1 whose size happens to be 1GB. In that case, every query on that tables might invoke this function. Although it's a very special case, but it's a regression.
   
   So, @cloud-fan and @maropu suggested to close this PR and proceed with #24715 .
   
   I'm +1 for that suggestion because that is the correct way.
   
   I know that you are worrying that #24715 doesn't resolve 8.0EB issue. However, that should be covered by your UTs in the previous PR. In the worst case, some of your code might be reverted. However, your test cases should survive there. It's your contribution. I believe @wangyum 's PR will pass your test cases in addition to his new test code. That's the way we make Apache Spark stronger.
   
   How do you think about this, @shahidki31 ? It's a way of collaboration.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org