You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by rdblue <gi...@git.apache.org> on 2018/06/06 20:29:01 UTC

[GitHub] spark issue #21319: [SPARK-24267][SQL] explicitly keep DataSourceReader in D...

Github user rdblue commented on the issue:

    https://github.com/apache/spark/pull/21319
  
    Here's the commit with my changes to support v2 stats in the visitor, sorry it took so long for me to find the time!
    https://github.com/apache/spark/pull/21503/commits/9d3a11e68bca6c5a56a2be47fb09395350362ac5
    
    The stats visitor now matches on PhysicalOperation to get accurate stats for v2, or any other data source that wants to report more accurate stats that aren't available because push-down happens when converting from logical plan to physical plan.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org