You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tez.apache.org by "Fabio C." <an...@gmail.com> on 2014/10/14 21:59:14 UTC
Submit dags from different users
Hi guys, right now I am performing some experiments with Tez. While trying
to launch DAGs from different users (just using the tez wordcount example)
I got this error:
"java.io.IOException: The ownership on the staging directory
hdfs://master:9000/tmp/tez/staging is not as expected. It is owned by user.
The directory must be owned by the submitter user1 or by user1"
user is the owner, user1 is the new user I am testing. What I want is to
launch two dags in the cluster at the same time from different users, (such
as wordcount and the join example).
Thanks in advance
Re: Submit dags from different users
Posted by Fabio <an...@gmail.com>.
Yeah, that solved the problem, thanks!
On 10/14/2014 10:02 PM, Siddharth Seth wrote:
> Fabio,
> Setting tez.staging-dir to something like /tmp/${user.name
> <http://user.name>}/staging in tez-site.xml should allow this.
>
> <property>
> <name>tez.staging-dir</name>
> <value>/tmp/${user.name <http://user.name>}/staging</value>
> </property>
>
> On Tue, Oct 14, 2014 at 12:59 PM, Fabio C. <anytek88@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi guys, right now I am performing some experiments with Tez.
> While trying to launch DAGs from different users (just using the
> tez wordcount example) I got this error:
>
> "java.io.IOException: The ownership on the staging directory
> hdfs://master:9000/tmp/tez/staging is not as expected. It is owned
> by user. The directory must be owned by the submitter user1 or by
> user1"
>
> user is the owner, user1 is the new user I am testing. What I want
> is to launch two dags in the cluster at the same time from
> different users, (such as wordcount and the join example).
>
> Thanks in advance
>
>
Re: Submit dags from different users
Posted by Siddharth Seth <ss...@apache.org>.
Fabio,
Setting tez.staging-dir to something like /tmp/${user.name}/staging in
tez-site.xml should allow this.
<property>
<name>tez.staging-dir</name>
<value>/tmp/${user.name}/staging</value>
</property>
On Tue, Oct 14, 2014 at 12:59 PM, Fabio C. <an...@gmail.com> wrote:
> Hi guys, right now I am performing some experiments with Tez. While trying
> to launch DAGs from different users (just using the tez wordcount example)
> I got this error:
>
> "java.io.IOException: The ownership on the staging directory
> hdfs://master:9000/tmp/tez/staging is not as expected. It is owned by user.
> The directory must be owned by the submitter user1 or by user1"
>
> user is the owner, user1 is the new user I am testing. What I want is to
> launch two dags in the cluster at the same time from different users, (such
> as wordcount and the join example).
>
> Thanks in advance
>
RE: Submit dags from different users
Posted by Bikas Saha <bi...@hortonworks.com>.
Tez writes temp job artifacts to the staging dir. So the user1 needs write
access to the staging dir.
Bikas
*From:* Fabio C. [mailto:anytek88@gmail.com]
*Sent:* Tuesday, October 14, 2014 12:59 PM
*To:* user@tez.apache.org
*Subject:* Submit dags from different users
Hi guys, right now I am performing some experiments with Tez. While trying
to launch DAGs from different users (just using the tez wordcount example)
I got this error:
"java.io.IOException: The ownership on the staging directory
hdfs://master:9000/tmp/tez/staging is not as expected. It is owned by user.
The directory must be owned by the submitter user1 or by user1"
user is the owner, user1 is the new user I am testing. What I want is to
launch two dags in the cluster at the same time from different users, (such
as wordcount and the join example).
Thanks in advance
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.