You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airavata.apache.org by Saminda Wijeratne <sa...@gmail.com> on 2014/01/10 19:14:51 UTC
Predefined parent working directory for jobs
Hi Devs,
I want the parent directory of each working directory of the jobs to be a
predefined path defined for each remote host. What is the easiest way to do
this?
Thanks,
Saminda
Re: Predefined parent working directory for jobs
Posted by Lahiru Gunathilake <gl...@gmail.com>.
Hi Suresh,
Yes Suresh you are correct but I was under the impression that we need to
enforce this thing in server side to avoid application description creation
error etc.
Each server have multiple application descriptors which are highly coupled
with each host, so scratch working directories for each host has to
configure manually (we don't re use them, have to create one for each host).
Regards
Lahiru
On Mon, Jan 13, 2014 at 11:14 AM, Suresh Marru <sm...@apache.org> wrote:
> Saminda, Lahiru,
>
> Why not use the staticWorkingDirectory option within Application
> Deployment Description [1]? The intention of this option was, within the
> provider, if this is provided, it ignores the scratchWorkingDirectory and
> always uses a static working directory location.
>
> Also, if I re-read Saminda’s original request, is it not the normal GFac
> behavior? I mean, within the specified directory
> scratchWorkingDirectory=foo on host A and scratchWorkingDirectory=bar on
> hostB, all new sub-directories will be created. What is the precise
> difference from this behaviour?
>
> Suresh
>
> [1] -
> https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-schema/src/main/resources/schemas/ApplicationDeploymentDescription.xsd
>
> On Jan 13, 2014, at 9:53 AM, Lahiru Gunathilake <gl...@gmail.com> wrote:
>
> > Hi Saminda,
> >
> > You can do this easily by writing another Handler and make sure you call
> this handler after AppDescriptorCheckHandler. You can have properties in
> your handler configuration and read those properties and configure
> different hosts with different base directory locations. So your handler
> configuration could looks like this.
> >
> > <InHandlers>
> > <Handler class="org.apache.airavata.gfac.handler.
> AppDescriptorCheckHandler"/>
> > <Handler
> class="org.apache.airavata.gfac.handler.BaseDirectorySetupHandler">
> > <property name="trestles"
> value="/home/trestles/base-directory"/>
> > </Handler>
> >
> > </InHandlers>
> > With handler properties you do not have to hard-code the host specific
> directories or create another configuration file.
> >
> > Hope this helps !
> >
> > Regards
> > Lahiru
> >
> >
> > On Fri, Jan 10, 2014 at 3:35 PM, Saminda Wijeratne <sa...@gmail.com>
> wrote:
> > A specific gateway will have access to the directory location "foo" in a
> certain remote host A. Therefore the gateway would like to have all the
> working directories for the jobs running in remote host A to be located
> inside "foo". Similarly for jobs running in remote host B, all the working
> directories should be created inside the directory location "bar" of which
> the gateway has access.
> >
> > The reason why the working directories needs to be located in locations
> accessible to the gateways is because the gateway may perform manual data
> transfer.
> >
> > Thanks,
> > Saminda
> >
> > On Fri, Jan 10, 2014 at 2:04 PM, Raminder Singh <
> raminderjsingh@gmail.com> wrote:
> > Hi Saminda,
> >
> > Do you want all the jobs should use same working directory?
> AppDescriptorCheckHandler in GFAC handles working directory, input-output
> directory in GFAC. You can control these configuration by customizing this
> input handler. Please describe the use case and i will be able to provide
> better input.
> >
> > Thanks
> > Raminder
> >
> > On Jan 10, 2014, at 1:14 PM, Saminda Wijeratne <sa...@gmail.com>
> wrote:
> >
> >> Hi Devs,
> >>
> >> I want the parent directory of each working directory of the jobs to be
> a predefined path defined for each remote host. What is the easiest way to
> do this?
> >>
> >> Thanks,
> >> Saminda
> >
> >
> >
> >
> >
> > --
> > System Analyst Programmer
> > PTI Lab
> > Indiana University
>
>
--
System Analyst Programmer
PTI Lab
Indiana University
Re: Predefined parent working directory for jobs
Posted by Saminda Wijeratne <sa...@gmail.com>.
Hi All,
Thanks for all the responses.
Lahiru, I will look at implementing the class *BaseDirectorySetupHandler. *Is
there a way which the gateway id and user can be obtained inside the
handler?
Suresh, this was also my initial thought. to use the
applicationDeploymentDescriptor properties. While it is a convenient place
for administrators to specify since it's documented in schema (so to
speak), they have to do it for each application deployment and if the
location changes they have to manually update all of them. But since I'm
aiming for a demo at the moment this indeed is a good way to go as a short
term solution.
On Mon, Jan 13, 2014 at 11:14 AM, Suresh Marru <sm...@apache.org> wrote:
> Saminda, Lahiru,
>
> Why not use the staticWorkingDirectory option within Application
> Deployment Description [1]? The intention of this option was, within the
> provider, if this is provided, it ignores the scratchWorkingDirectory and
> always uses a static working directory location.
>
> Also, if I re-read Saminda’s original request, is it not the normal GFac
> behavior? I mean, within the specified directory
> scratchWorkingDirectory=foo on host A and scratchWorkingDirectory=bar on
> hostB, all new sub-directories will be created. What is the precise
> difference from this behaviour?
>
> Suresh
>
> [1] -
> https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-schema/src/main/resources/schemas/ApplicationDeploymentDescription.xsd
>
> On Jan 13, 2014, at 9:53 AM, Lahiru Gunathilake <gl...@gmail.com> wrote:
>
> > Hi Saminda,
> >
> > You can do this easily by writing another Handler and make sure you call
> this handler after AppDescriptorCheckHandler. You can have properties in
> your handler configuration and read those properties and configure
> different hosts with different base directory locations. So your handler
> configuration could looks like this.
> >
> > <InHandlers>
> > <Handler class="org.apache.airavata.gfac.handler.
> AppDescriptorCheckHandler"/>
> > <Handler
> class="org.apache.airavata.gfac.handler.BaseDirectorySetupHandler">
> > <property name="trestles"
> value="/home/trestles/base-directory"/>
> > </Handler>
> >
> > </InHandlers>
> > With handler properties you do not have to hard-code the host specific
> directories or create another configuration file.
> >
> > Hope this helps !
> >
> > Regards
> > Lahiru
> >
> >
> > On Fri, Jan 10, 2014 at 3:35 PM, Saminda Wijeratne <sa...@gmail.com>
> wrote:
> > A specific gateway will have access to the directory location "foo" in a
> certain remote host A. Therefore the gateway would like to have all the
> working directories for the jobs running in remote host A to be located
> inside "foo". Similarly for jobs running in remote host B, all the working
> directories should be created inside the directory location "bar" of which
> the gateway has access.
> >
> > The reason why the working directories needs to be located in locations
> accessible to the gateways is because the gateway may perform manual data
> transfer.
> >
> > Thanks,
> > Saminda
> >
> > On Fri, Jan 10, 2014 at 2:04 PM, Raminder Singh <
> raminderjsingh@gmail.com> wrote:
> > Hi Saminda,
> >
> > Do you want all the jobs should use same working directory?
> AppDescriptorCheckHandler in GFAC handles working directory, input-output
> directory in GFAC. You can control these configuration by customizing this
> input handler. Please describe the use case and i will be able to provide
> better input.
> >
> > Thanks
> > Raminder
> >
> > On Jan 10, 2014, at 1:14 PM, Saminda Wijeratne <sa...@gmail.com>
> wrote:
> >
> >> Hi Devs,
> >>
> >> I want the parent directory of each working directory of the jobs to be
> a predefined path defined for each remote host. What is the easiest way to
> do this?
> >>
> >> Thanks,
> >> Saminda
> >
> >
> >
> >
> >
> > --
> > System Analyst Programmer
> > PTI Lab
> > Indiana University
>
>
Re: Predefined parent working directory for jobs
Posted by Suresh Marru <sm...@apache.org>.
Saminda, Lahiru,
Why not use the staticWorkingDirectory option within Application Deployment Description [1]? The intention of this option was, within the provider, if this is provided, it ignores the scratchWorkingDirectory and always uses a static working directory location.
Also, if I re-read Saminda’s original request, is it not the normal GFac behavior? I mean, within the specified directory scratchWorkingDirectory=foo on host A and scratchWorkingDirectory=bar on hostB, all new sub-directories will be created. What is the precise difference from this behaviour?
Suresh
[1] - https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/gfac-schema/src/main/resources/schemas/ApplicationDeploymentDescription.xsd
On Jan 13, 2014, at 9:53 AM, Lahiru Gunathilake <gl...@gmail.com> wrote:
> Hi Saminda,
>
> You can do this easily by writing another Handler and make sure you call this handler after AppDescriptorCheckHandler. You can have properties in your handler configuration and read those properties and configure different hosts with different base directory locations. So your handler configuration could looks like this.
>
> <InHandlers>
> <Handler class="org.apache.airavata.gfac.handler. AppDescriptorCheckHandler"/>
> <Handler class="org.apache.airavata.gfac.handler.BaseDirectorySetupHandler">
> <property name="trestles" value="/home/trestles/base-directory"/>
> </Handler>
>
> </InHandlers>
> With handler properties you do not have to hard-code the host specific directories or create another configuration file.
>
> Hope this helps !
>
> Regards
> Lahiru
>
>
> On Fri, Jan 10, 2014 at 3:35 PM, Saminda Wijeratne <sa...@gmail.com> wrote:
> A specific gateway will have access to the directory location "foo" in a certain remote host A. Therefore the gateway would like to have all the working directories for the jobs running in remote host A to be located inside "foo". Similarly for jobs running in remote host B, all the working directories should be created inside the directory location "bar" of which the gateway has access.
>
> The reason why the working directories needs to be located in locations accessible to the gateways is because the gateway may perform manual data transfer.
>
> Thanks,
> Saminda
>
> On Fri, Jan 10, 2014 at 2:04 PM, Raminder Singh <ra...@gmail.com> wrote:
> Hi Saminda,
>
> Do you want all the jobs should use same working directory? AppDescriptorCheckHandler in GFAC handles working directory, input-output directory in GFAC. You can control these configuration by customizing this input handler. Please describe the use case and i will be able to provide better input.
>
> Thanks
> Raminder
>
> On Jan 10, 2014, at 1:14 PM, Saminda Wijeratne <sa...@gmail.com> wrote:
>
>> Hi Devs,
>>
>> I want the parent directory of each working directory of the jobs to be a predefined path defined for each remote host. What is the easiest way to do this?
>>
>> Thanks,
>> Saminda
>
>
>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University
Re: Predefined parent working directory for jobs
Posted by Lahiru Gunathilake <gl...@gmail.com>.
Hi Saminda,
You can do this easily by writing another Handler and make sure you call
this handler after AppDescriptorCheckHandler. You can have properties in
your handler configuration and read those properties and configure
different hosts with different base directory locations. So your handler
configuration could looks like this.
<InHandlers>
<Handler
class="org.apache.airavata.gfac.handler. AppDescriptorCheckHandler"/>
*<Handler
class="org.apache.airavata.gfac.handler.BaseDirectorySetupHandler">*
* <property name="trestles"
value="/home/trestles/base-directory"/>*
* </Handler>*
</InHandlers>
With handler properties you do not have to hard-code the host specific
directories or create another configuration file.
Hope this helps !
Regards
Lahiru
On Fri, Jan 10, 2014 at 3:35 PM, Saminda Wijeratne <sa...@gmail.com>wrote:
> A specific gateway will have access to the directory location "foo" in a
> certain remote host A. Therefore the gateway would like to have all the
> working directories for the jobs running in remote host A to be located
> inside "foo". Similarly for jobs running in remote host B, all the working
> directories should be created inside the directory location "bar" of which
> the gateway has access.
>
> The reason why the working directories needs to be located in locations
> accessible to the gateways is because the gateway may perform manual data
> transfer.
>
> Thanks,
> Saminda
>
> On Fri, Jan 10, 2014 at 2:04 PM, Raminder Singh <ra...@gmail.com>wrote:
>
>> Hi Saminda,
>>
>> Do you want all the jobs should use same working
>> directory? AppDescriptorCheckHandler in GFAC handles working directory,
>> input-output directory in GFAC. You can control these configuration
>> by customizing this input handler. Please describe the use case and i will
>> be able to provide better input.
>>
>> Thanks
>> Raminder
>>
>> On Jan 10, 2014, at 1:14 PM, Saminda Wijeratne <sa...@gmail.com>
>> wrote:
>>
>> Hi Devs,
>>
>> I want the parent directory of each working directory of the jobs to be a
>> predefined path defined for each remote host. What is the easiest way to do
>> this?
>>
>> Thanks,
>> Saminda
>>
>>
>>
>
--
System Analyst Programmer
PTI Lab
Indiana University
Re: Predefined parent working directory for jobs
Posted by Saminda Wijeratne <sa...@gmail.com>.
A specific gateway will have access to the directory location "foo" in a
certain remote host A. Therefore the gateway would like to have all the
working directories for the jobs running in remote host A to be located
inside "foo". Similarly for jobs running in remote host B, all the working
directories should be created inside the directory location "bar" of which
the gateway has access.
The reason why the working directories needs to be located in locations
accessible to the gateways is because the gateway may perform manual data
transfer.
Thanks,
Saminda
On Fri, Jan 10, 2014 at 2:04 PM, Raminder Singh <ra...@gmail.com>wrote:
> Hi Saminda,
>
> Do you want all the jobs should use same working
> directory? AppDescriptorCheckHandler in GFAC handles working directory,
> input-output directory in GFAC. You can control these configuration
> by customizing this input handler. Please describe the use case and i will
> be able to provide better input.
>
> Thanks
> Raminder
>
> On Jan 10, 2014, at 1:14 PM, Saminda Wijeratne <sa...@gmail.com> wrote:
>
> Hi Devs,
>
> I want the parent directory of each working directory of the jobs to be a
> predefined path defined for each remote host. What is the easiest way to do
> this?
>
> Thanks,
> Saminda
>
>
>
Re: Predefined parent working directory for jobs
Posted by Raminder Singh <ra...@gmail.com>.
Hi Saminda,
Do you want all the jobs should use same working directory? AppDescriptorCheckHandler in GFAC handles working directory, input-output directory in GFAC. You can control these configuration by customizing this input handler. Please describe the use case and i will be able to provide better input.
Thanks
Raminder
On Jan 10, 2014, at 1:14 PM, Saminda Wijeratne <sa...@gmail.com> wrote:
> Hi Devs,
>
> I want the parent directory of each working directory of the jobs to be a predefined path defined for each remote host. What is the easiest way to do this?
>
> Thanks,
> Saminda