You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "kevin yu (JIRA)" <ji...@apache.org> on 2016/06/03 16:25:59 UTC
[jira] [Commented] (SPARK-15731) orc writer directory permissions
[ https://issues.apache.org/jira/browse/SPARK-15731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314364#comment-15314364 ]
kevin yu commented on SPARK-15731:
----------------------------------
Hi Ran: I tried on my machine for orc file, the partition directories has the x permission. Did I do anything different than yours?
scala> spark.createDataFrame(data).toDF("a", "b").write.format("orc").mode("append").partitionBy("a").save("/Users/qianyangyu/sparkcp/spark-15731orc2")
Qianyangs-MBP:sparkcp qianyangyu$ ls -al spark-15731orc2/
total 8
drwxr-xr-x 10 qianyangyu staff 340 Jun 3 09:11 .
drwxr-xr-x 15 qianyangyu staff 510 Jun 3 09:11 ..
-rw-r--r-- 1 qianyangyu staff 8 Jun 3 09:11 ._SUCCESS.crc
-rw-r--r-- 1 qianyangyu staff 0 Jun 3 09:11 _SUCCESS
drwxr-xr-x 4 qianyangyu staff 136 Jun 3 09:11 a=1
drwxr-xr-x 4 qianyangyu staff 136 Jun 3 09:11 a=3
drwxr-xr-x 4 qianyangyu staff 136 Jun 3 09:11 a=5
drwxr-xr-x 4 qianyangyu staff 136 Jun 3 09:11 a=7
drwxr-xr-x 4 qianyangyu staff 136 Jun 3 09:11 a=9
drwxr-xr-x 12 qianyangyu staff 408 Jun 3 09:11 a=__HIVE_DEFAULT_PARTITION__
Qianyangs-MBP:sparkcp qianyangyu$
> orc writer directory permissions
> --------------------------------
>
> Key: SPARK-15731
> URL: https://issues.apache.org/jira/browse/SPARK-15731
> Project: Spark
> Issue Type: Bug
> Reporter: Ran Haim
>
> When saving orc files with partitions, the partition directories created do not have x permission (even tough umask is 002), then no other users can get inside those directories to read the orc file.
> When writing parquet files there is no such issue.
> code example:
> datafrmae.write.format("orc").mode("append").partitionBy("date").save("/path")
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org