[GitHub] spark pull request #19868: [SPARK-22676] Avoid iterating all partition paths...

[cloud-fan <gi...@git.apache.org>]

Github user cloud-fan commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19868#discussion_r180713699
  
    --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
    @@ -176,12 +176,13 @@ class HadoopTableReader(
                   val matches = fs.globStatus(pathPattern)
                   matches.foreach(fileStatus => existPathSet += fileStatus.getPath.toString)
                 }
    -            // convert  /demo/data/year/month/day  to  /demo/data/*/*/*/
    +            // convert  /demo/data/year/month/day  to  /demo/data/year/month/*/
    --- End diff --
    
    This is a pretty old logic. Can you explain what's going on here and why your change works? It can help other people to understand your change quickly,


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org