You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by li...@apache.org on 2015/06/19 07:02:00 UTC
spark git commit: [SPARK-8458] [SQL] Don't strip scheme part of
output path when writing ORC files
Repository: spark
Updated Branches:
refs/heads/master 3eaed8769 -> a71cbbdea
[SPARK-8458] [SQL] Don't strip scheme part of output path when writing ORC files
`Path.toUri.getPath` strips scheme part of output path (from `file:///foo` to `/foo`), which causes ORC data source only writes to the file system configured in Hadoop configuration. Should use `Path.toString` instead.
Author: Cheng Lian <li...@databricks.com>
Closes #6892 from liancheng/spark-8458 and squashes the following commits:
87f8199 [Cheng Lian] Don't strip scheme of output path when writing ORC files
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a71cbbde
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a71cbbde
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a71cbbde
Branch: refs/heads/master
Commit: a71cbbdea581573192a59bf8472861c463c40fcb
Parents: 3eaed87
Author: Cheng Lian <li...@databricks.com>
Authored: Thu Jun 18 22:01:52 2015 -0700
Committer: Cheng Lian <li...@databricks.com>
Committed: Thu Jun 18 22:01:52 2015 -0700
----------------------------------------------------------------------
.../src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/a71cbbde/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala
----------------------------------------------------------------------
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala
index 77f1ca9..dbce39f 100644
--- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala
+++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala
@@ -111,7 +111,7 @@ private[orc] class OrcOutputWriter(
new OrcOutputFormat().getRecordWriter(
new Path(path, filename).getFileSystem(conf),
conf.asInstanceOf[JobConf],
- new Path(path, filename).toUri.getPath,
+ new Path(path, filename).toString,
Reporter.NULL
).asInstanceOf[RecordWriter[NullWritable, Writable]]
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org