You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tez.apache.org by Xiaoyong Zhu <xi...@microsoft.com> on 2015/05/25 05:46:12 UTC

tez timeline domain configuration?

We are trying to enable tez ATS integration for Hadoop 2.4 and found there is such a setting according to the Tez website:

When using Tez with Apache Hadoop 2.4.x or 2.5.x, given that these versions are not fully secure, the following property also needs to be enabled:
<property>
   <name>tez.allow.disabled.timeline-domains</name>
   <value>true</value>
</property>

Could someone help to clarify what exactly does this mean? What's the behavior for turn on/off this config?

Thanks!

Xiaoyong

Re: tez timeline domain configuration?

Posted by Chris K Wensel <ch...@wensel.net>.
fwiw, it will also continue to dump the classloader exception into your logs whether you set the flag or not.

15/05/26 22:57:38 WARN client.TezClient: Could not instantiate object for org.apache.tez.dag.history.ats.acls.ATSHistoryACLPolicyManager. ACLs cannot be enforced correctly for history data in Timeline
org.apache.tez.dag.api.TezUncheckedException: Unable to load class: org.apache.tez.dag.history.ats.acls.ATSHistoryACLPolicyManager
        at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:45)
        at org.apache.tez.common.ReflectionUtils.createClazzInstance(ReflectionUtils.java:88)
        at org.apache.tez.client.TezClient.start(TezClient.java:317)
        at cascading.flow.tez.planner.Hadoop2TezFlowStepJob.internalNonBlockingStart(Hadoop2TezFlowStepJob.java:137)
        at cascading.flow.planner.FlowStepJob.blockOnJob(FlowStepJob.java:248)
        at cascading.flow.planner.FlowStepJob.start(FlowStepJob.java:172)
        at cascading.flow.planner.FlowStepJob.call(FlowStepJob.java:134)
        at cascading.flow.planner.FlowStepJob.call(FlowStepJob.java:45)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.tez.dag.history.ats.acls.ATSHistoryACLPolicyManager
        at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:274)
        at org.apache.tez.common.ReflectionUtils.getClazz(ReflectionUtils.java:43)
        ... 11 more


> On May 26, 2015, at 11:11 AM, Hitesh Shah <hi...@apache.org> wrote:
> 
> In hadoop 2.4, YARN timeline does not really support proper security. It does not have any ACL support ( implemented using something called domains ). 
> 
> The property that you mentioned needs to be set as it is a form of a warning to the user that you are running Tez with YARN Timeline with ACLs enabled but given that Timeline does not support ACLs, the history data will be visible to all users ( ACL enforcement not possible ).
> 
> In other words: If you turn off the config, the Tez code will error out saying that ACLs are enabled but timeline does not support ACLs. 
> 
> thanks
> — Hitesh
> 
> 
> On May 24, 2015, at 8:46 PM, Xiaoyong Zhu <xi...@microsoft.com> wrote:
> 
>> We are trying to enable tez ATS integration for Hadoop 2.4 and found there is such a setting according to the Tez website:
>> 
>> When using Tez with Apache Hadoop 2.4.x or 2.5.x, given that these versions are not fully secure, the following property also needs to be enabled:
>> <property>
>>   <name>tez.allow.disabled.timeline-domains</name>
>>   <value>true</value>
>> </property>
>> 
>> Could someone help to clarify what exactly does this mean? What’s the behavior for turn on/off this config?
>> 
>> Thanks!
>> 
>> Xiaoyong
> 

—
Chris K Wensel
chris@wensel.net

Re: tez timeline domain configuration?

Posted by Hitesh Shah <hi...@apache.org>.
In hadoop 2.4, YARN timeline does not really support proper security. It does not have any ACL support ( implemented using something called domains ). 

The property that you mentioned needs to be set as it is a form of a warning to the user that you are running Tez with YARN Timeline with ACLs enabled but given that Timeline does not support ACLs, the history data will be visible to all users ( ACL enforcement not possible ).
 
In other words: If you turn off the config, the Tez code will error out saying that ACLs are enabled but timeline does not support ACLs. 

thanks
— Hitesh


On May 24, 2015, at 8:46 PM, Xiaoyong Zhu <xi...@microsoft.com> wrote:

> We are trying to enable tez ATS integration for Hadoop 2.4 and found there is such a setting according to the Tez website:
>  
> When using Tez with Apache Hadoop 2.4.x or 2.5.x, given that these versions are not fully secure, the following property also needs to be enabled:
> <property>
>    <name>tez.allow.disabled.timeline-domains</name>
>    <value>true</value>
> </property>
>  
> Could someone help to clarify what exactly does this mean? What’s the behavior for turn on/off this config?
>  
> Thanks!
>  
> Xiaoyong