You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Wei Zheng (JIRA)" <ji...@apache.org> on 2018/02/15 07:49:00 UTC
[jira] [Commented] (SPARK-21521) History service requires user is
in any group
[ https://issues.apache.org/jira/browse/SPARK-21521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16365208#comment-16365208 ]
Wei Zheng commented on SPARK-21521:
-----------------------------------
We came across the same problem recently - SHS UI only shows the jobs for the user who started SHS service. Although that user is a super user (both in local FS and HDFS), it cannot read other users' job log files (due to rwxrwx---).
Special logic to tell whether a user is a super user is nice, but I don't know if that's doable, because that logic may be vendor specific. For those using HDFS maybe we can read dfs.permissions.supergroup from hdfs-site.xml and tell, but other system like MapR doesn't use hdfs-site.xml at all but has different configs. I don't know if that's the case for other vendors.
We currently work around this issue by changing LOG_FILE_PERMISSIONS from 770 to 774. I'm not sure if that's a safe change though.
> History service requires user is in any group
> ---------------------------------------------
>
> Key: SPARK-21521
> URL: https://issues.apache.org/jira/browse/SPARK-21521
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.2.0
> Reporter: Adrian Bridgett
> Priority: Major
>
> (Regression cf. 2.0.2)
> We run spark as several users, these write to the history location where the files are saved as those users with permissions of 770 (this is hardcoded in EventLoggingListener.scala).
> The history service runs as root so that it has permissions on these files (see https://spark.apache.org/docs/latest/security.html).
> This worked fine in v2.0.2, however in v2.2.0 the events are being skipped unless I add the root user into each users group at which point they are seen.
> We currently have all acls configuration unset.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org