You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@tez.apache.org by Madhusudan Ramanna <m....@ymail.com> on 2016/10/04 18:11:39 UTC
adding local resource to classpath and/or java.library.path
Hello Folks,
We have an archive local resource that is being expanded to
PWD/<archive given name>/*.*
How do we add PWD/<archive given name> to classpath and java.library.path ?
thanks,Madhu
Re: adding local resource to classpath and/or java.library.path
Posted by Madhusudan Ramanna <m....@ymail.com>.
Yes, right now we're using "${PWD}/archivename"
Madhu
On Wednesday, October 5, 2016 10:28 AM, Hitesh Shah <hi...@apache.org> wrote:
From the current dir perspective, the classpath could be set up to use “$PWD/archivename/*” where PWD in the container’s env would point to the container’s working directory. This is a bit more robust as compared to using “./“. Both should ideally work in most scenarios but using “./“ is dependent on YARN implementation behavior.
— Hitesh
> On Oct 5, 2016, at 10:06 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
>
> Seems like with this approach, there is no need to have information on current dir.
>
> thanks,
> Madhu
>
>
> On Tuesday, October 4, 2016 4:44 PM, Hitesh Shah <hi...@apache.org> wrote:
>
>
> The env is one approach for augmenting classpath. The other approach which modifies classpath for both the AM and the task containers is to use “tez.cluster.additional.classpath.prefix” by setting it to something like “./archive name/*”
>
> — Hitesh
>
> > On Oct 4, 2016, at 4:38 PM, Madhusudan Ramanna <m....@ymail.com> wrote:
> >
> > Actually, we solved it using
> >
> > tez.task.launch.env in tez-site.xml
> >
> > thanks,
> > Madhu
> >
> >
> > On Tuesday, October 4, 2016 11:13 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
> >
> >
> > Please note that we need this in the tez containers
> >
> > thanks,
> > Madhu
> >
> >
> > On Tuesday, October 4, 2016 11:11 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
> >
> >
> > Hello Folks,
> >
> > We have an archive local resource that is being expanded to
> >
> > PWD/<archive given name>/*.*
> >
> > How do we add PWD/<archive given name> to classpath and java.library.path ?
> >
> > thanks,
> > Madhu
> >
> >
> >
> >
>
>
Re: adding local resource to classpath and/or java.library.path
Posted by Hitesh Shah <hi...@apache.org>.
From the current dir perspective, the classpath could be set up to use “$PWD/archivename/*” where PWD in the container’s env would point to the container’s working directory. This is a bit more robust as compared to using “./“. Both should ideally work in most scenarios but using “./“ is dependent on YARN implementation behavior.
— Hitesh
> On Oct 5, 2016, at 10:06 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
>
> Seems like with this approach, there is no need to have information on current dir.
>
> thanks,
> Madhu
>
>
> On Tuesday, October 4, 2016 4:44 PM, Hitesh Shah <hi...@apache.org> wrote:
>
>
> The env is one approach for augmenting classpath. The other approach which modifies classpath for both the AM and the task containers is to use “tez.cluster.additional.classpath.prefix” by setting it to something like “./archive name/*”
>
> — Hitesh
>
> > On Oct 4, 2016, at 4:38 PM, Madhusudan Ramanna <m....@ymail.com> wrote:
> >
> > Actually, we solved it using
> >
> > tez.task.launch.env in tez-site.xml
> >
> > thanks,
> > Madhu
> >
> >
> > On Tuesday, October 4, 2016 11:13 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
> >
> >
> > Please note that we need this in the tez containers
> >
> > thanks,
> > Madhu
> >
> >
> > On Tuesday, October 4, 2016 11:11 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
> >
> >
> > Hello Folks,
> >
> > We have an archive local resource that is being expanded to
> >
> > PWD/<archive given name>/*.*
> >
> > How do we add PWD/<archive given name> to classpath and java.library.path ?
> >
> > thanks,
> > Madhu
> >
> >
> >
> >
>
>
Re: adding local resource to classpath and/or java.library.path
Posted by Madhusudan Ramanna <m....@ymail.com>.
Seems like with this approach, there is no need to have information on current dir.
thanks,Madhu
On Tuesday, October 4, 2016 4:44 PM, Hitesh Shah <hi...@apache.org> wrote:
The env is one approach for augmenting classpath. The other approach which modifies classpath for both the AM and the task containers is to use “tez.cluster.additional.classpath.prefix” by setting it to something like “./archive name/*”
— Hitesh
> On Oct 4, 2016, at 4:38 PM, Madhusudan Ramanna <m....@ymail.com> wrote:
>
> Actually, we solved it using
>
> tez.task.launch.env in tez-site.xml
>
> thanks,
> Madhu
>
>
> On Tuesday, October 4, 2016 11:13 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
>
>
> Please note that we need this in the tez containers
>
> thanks,
> Madhu
>
>
> On Tuesday, October 4, 2016 11:11 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
>
>
> Hello Folks,
>
> We have an archive local resource that is being expanded to
>
> PWD/<archive given name>/*.*
>
> How do we add PWD/<archive given name> to classpath and java.library.path ?
>
> thanks,
> Madhu
>
>
>
>
Re: adding local resource to classpath and/or java.library.path
Posted by Hitesh Shah <hi...@apache.org>.
The env is one approach for augmenting classpath. The other approach which modifies classpath for both the AM and the task containers is to use “tez.cluster.additional.classpath.prefix” by setting it to something like “./archive name/*”
— Hitesh
> On Oct 4, 2016, at 4:38 PM, Madhusudan Ramanna <m....@ymail.com> wrote:
>
> Actually, we solved it using
>
> tez.task.launch.env in tez-site.xml
>
> thanks,
> Madhu
>
>
> On Tuesday, October 4, 2016 11:13 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
>
>
> Please note that we need this in the tez containers
>
> thanks,
> Madhu
>
>
> On Tuesday, October 4, 2016 11:11 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
>
>
> Hello Folks,
>
> We have an archive local resource that is being expanded to
>
> PWD/<archive given name>/*.*
>
> How do we add PWD/<archive given name> to classpath and java.library.path ?
>
> thanks,
> Madhu
>
>
>
>
Re: adding local resource to classpath and/or java.library.path
Posted by Madhusudan Ramanna <m....@ymail.com>.
Actually, we solved it using
tez.task.launch.env in tez-site.xml
thanks,Madhu
On Tuesday, October 4, 2016 11:13 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
Please note that we need this in the tez containers
thanks,Madhu
On Tuesday, October 4, 2016 11:11 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
Hello Folks,
We have an archive local resource that is being expanded to
PWD/<archive given name>/*.*
How do we add PWD/<archive given name> to classpath and java.library.path ?
thanks,Madhu
Re: adding local resource to classpath and/or java.library.path
Posted by Madhusudan Ramanna <m....@ymail.com>.
Please note that we need this in the tez containers
thanks,Madhu
On Tuesday, October 4, 2016 11:11 AM, Madhusudan Ramanna <m....@ymail.com> wrote:
Hello Folks,
We have an archive local resource that is being expanded to
PWD/<archive given name>/*.*
How do we add PWD/<archive given name> to classpath and java.library.path ?
thanks,Madhu