You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "Suma Shivaprasad (JIRA)" <ji...@apache.org> on 2018/11/28 15:58:00 UTC
[jira] [Comment Edited] (YARN-9039) App ACLs are not validated when
serving logs from LogWebService
[ https://issues.apache.org/jira/browse/YARN-9039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16702059#comment-16702059 ]
Suma Shivaprasad edited comment on YARN-9039 at 11/28/18 3:57 PM:
------------------------------------------------------------------
Thanks [~bibinchundatt] for your suggestions. Unfortunately the Cloud Storage ACLs are a bit more complicated than that.
User specific folder access control could work if only a single user needs access to the folder. But gets cumbersome when implementing application specific acls on the logs objects stored under that folder i.e granting access to other users to read the logs
- There are limits on the size of bucket/IAM policies - https://aws.amazon.com/blogs/security/iam-policies-and-bucket-policies-and-acls-oh-my-controlling-access-to-s3-resources/.
- Having a bucket per user also has limitations due to the 100 buckets per account limits in S3.
In the long term, there are couple of options to address this
- Logs CLI talks to Log Webservice instead of talking to storage directly and only yarn user has access to write/read from the log aggregation storage bucket. Can track this effort in a separate jira to fix the issue of YARN CLI imposing ACLs
- All FileSystem calls go through a FS proxy which can authorize storage ACLs via an Authorization framework like Apache Ranger
In the current jira, we could address the issue of ATSv2 LogWebservice not checking ACLs while serving logs through REST/UI and I could upload a patch for the same
Makes sense?
Did not get what is the fix needed for LogAggregationHtmlBlock#checkAcls ? It doesnt seem to be calling LogAggregationFileController.readAggregatedLogs
was (Author: suma.shivaprasad):
Thanks [~bibinchundatt] for your suggestions. Unfortunately the Cloud Storage ACLs are a bit more complicated than that.
User specific folder access control could work if only a single user needs access to the folder. But gets cumbersome when implementing application specific acls on the logs objects stored under that folder i.e granting access to other users to read the logs
- There are limits on the size of bucket/IAM policies - https://aws.amazon.com/blogs/security/iam-policies-and-bucket-policies-and-acls-oh-my-controlling-access-to-s3-resources/.
- Having a bucket per user also has limitations due to the 100 buckets per account limits in S3.
In the long term, there are couple of options to address this
- Logs CLI talks to Log Webservice instead of talking to storage directly and only yarn user has access to write/read from the log aggregation storage bucket. Can track this effort in a separate jira to fix the issue of YARN CLI imposing ACLs
- All FileSystem calls go through a FS proxy which can authorize storage ACLs via an Authorization framework like Apache Ranger
In the current jira, we could address the issue of ATSv2 LogWebservice not checking ACLs while serving logs through REST/UI and I could upload a patch for the same
Makes sense?
> App ACLs are not validated when serving logs from LogWebService
> ---------------------------------------------------------------
>
> Key: YARN-9039
> URL: https://issues.apache.org/jira/browse/YARN-9039
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Priority: Critical
> Attachments: YARN-9039.1.patch, YARN-9039.2.patch
>
>
> App Acls are not being validated while serving logs through REST and UI2 via Log Webservice
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org