You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ma...@apache.org on 2014/08/14 22:00:24 UTC

git commit: Revert [SPARK-3011][SQL] _temporary directory should be filtered out by sqlContext.parquetFile

Repository: spark
Updated Branches:
  refs/heads/master 962210675 -> a7f8a4f5e


Revert  [SPARK-3011][SQL] _temporary directory should be filtered out by sqlContext.parquetFile

Reverts #1924 due to build failures with hadoop 0.23.

Author: Michael Armbrust <mi...@databricks.com>

Closes #1949 from marmbrus/revert1924 and squashes the following commits:

6bff940 [Michael Armbrust] Revert "[SPARK-3011][SQL] _temporary directory should be filtered out by sqlContext.parquetFile"


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a7f8a4f5
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a7f8a4f5
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a7f8a4f5

Branch: refs/heads/master
Commit: a7f8a4f5ee757450ce8d4028021441435081cf53
Parents: 9622106
Author: Michael Armbrust <mi...@databricks.com>
Authored: Thu Aug 14 13:00:21 2014 -0700
Committer: Michael Armbrust <mi...@databricks.com>
Committed: Thu Aug 14 13:00:21 2014 -0700

----------------------------------------------------------------------
 .../main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala    | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/a7f8a4f5/sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala
----------------------------------------------------------------------
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala b/sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala
index b0579f7..c79a9ac 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/parquet/ParquetTypes.scala
@@ -378,8 +378,7 @@ private[parquet] object ParquetTypesConverter extends Logging {
 
     val children = fs.listStatus(path).filterNot { status =>
       val name = status.getPath.getName
-      name(0) == '.' || name == FileOutputCommitter.SUCCEEDED_FILE_NAME || 
-        name == FileOutputCommitter.TEMP_DIR_NAME
+      name(0) == '.' || name == FileOutputCommitter.SUCCEEDED_FILE_NAME
     }
 
     // NOTE (lian): Parquet "_metadata" file can be very slow if the file consists of lots of row


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org