airavata for parameteric sweep applications

["AMARNATH, Balachandar" <BA...@airbus.com>]

Hi,

Does airavata supports parametereric sweep tasks through foreach constructs in distributed nodes? From tutorial, what I learnt is forEach construct supports parameteric tasks but there was no mention about executing them on different nodes.

With thanks and regards
Balachandar




The information in this e-mail is confidential. The contents may not be disclosed or used by anyone other than the addressee. Access to this e-mail by anyone else is unauthorised.
If you are not the intended recipient, please notify Airbus immediately and delete this e-mail.
Airbus cannot accept any responsibility for the accuracy or completeness of this e-mail as it has been sent over public networks. If you have any concerns over the content of this message or its Accuracy or Integrity, please contact Airbus immediately.
All outgoing e-mails from Airbus are checked using regularly updated virus scanning software but you should take whatever measures you deem to be appropriate to ensure that this message and any attachments are virus free.

Re: airavata for parameteric sweep applications

[Suresh Marru <sm...@apache.org>]

Hi Balachandar,

The short answer is Yes you can execute different instances on different nodes but it will take some work. One solution is what Lahiru indicated but I assume you are asking for more first class way of scheduling nodes within parametric sweep. As with any Apache projects, Airavata features and roadmap is driven by community needs or interest and use cases by developers. So-far the use cases have been dominant by long running applications on Grid resources. Technically speaking the workflow execution schema was enhanced to support your use case [1] and it is supported [2]. 

Various airavata developers have explored and published use cases on parametric sweeps on grid and cloud resources and briefly touched upon the scheduling aspects you are asking. Attached is a example on how Chathura was looking at pruning parametric sweeps. You can modulate this feature to achieve the node level scheduling you are trying. Also, Heshan was looking at efficient scheduling on amazon ec2 cloud resources. You can easily override scheduling in that approach. Also, we had put out this topic as a GSoC project [3] last summer. Lastly, Pedro Da Silveira was developing a component to add to Airavata which enables advanced scheduling. 

Please describe your use case in detail and what resources you are trying to achieve this and we may be able to find overlap. If you have any development time and would like to submit patches, we will be happy to support you.

Cheers,
Suresh

[1] - https://issues.apache.org/jira/browse/AIRAVATA-152
[2] - https://issues.apache.org/jira/browse/AIRAVATA-403
[3] - https://issues.apache.org/jira/browse/AIRAVATA-344




On Nov 22, 2012, at 8:36 AM, Lahiru Gunathilake <gl...@gmail.com> wrote:

> Hi Amaranath,
> 
> If you want to execute them in different node you have to use ForEach for eacha node and for everynode you have to use EndForEach to collect the outputs of each node.
> 
> I have attached a sample workflow with multiple ForEachs.
> 
> 
> Lahiru
> 
> On Thu, Nov 22, 2012 at 4:16 AM, AMARNATH, Balachandar <BA...@airbus.com> wrote:
> Hi,
>  
> Does airavata supports parametereric sweep tasks through foreach constructs in distributed nodes? From tutorial, what I learnt is forEach construct supports parameteric tasks but there was no mention about executing them on different nodes.
>  
> With thanks and regards
> Balachandar
>  
>  
>  
>  The information in this e-mail is confidential. The contents may not be disclosed or used by anyone other than the addressee. Access to this e-mail by anyone else is unauthorised.
> If you are not the intended recipient, please notify Airbus immediately and delete this e-mail.
> Airbus cannot accept any responsibility for the accuracy or completeness of this e-mail as it has been sent over public networks. If you have any concerns over the content of this message or its Accuracy or Integrity, please contact Airbus immediately.
> All outgoing e-mails from Airbus are checked using regularly updated virus scanning software but you should take whatever measures you deem to be appropriate to ensure that this message and any attachments are virus free.
> 
> 
> 
> -- 
> System Analyst Programmer
> PTI Lab
> Indiana University
> 
> <forEachSample.xwf>

Re: airavata for parameteric sweep applications

[Lahiru Gunathilake <gl...@gmail.com>]

Hi Amaranath,

If you want to execute them in different node you have to use ForEach for
eacha node and for everynode you have to use EndForEach to collect the
outputs of each node.

I have attached a sample workflow with multiple ForEachs.


Lahiru

On Thu, Nov 22, 2012 at 4:16 AM, AMARNATH, Balachandar <
BALACHANDAR.AMARNATH@airbus.com> wrote:

>  Hi,
>
> Does airavata supports parametereric sweep tasks through foreach
> constructs in distributed nodes? From tutorial, what I learnt is forEach
> construct supports parameteric tasks but there was no mention about
> executing them on different nodes.
>
> With thanks and regards
> Balachandar
>
>
>
>
> The information in this e-mail is confidential. The contents may not be disclosed or used by anyone other than the addressee. Access to this e-mail by anyone else is unauthorised.
> If you are not the intended recipient, please notify Airbus immediately and delete this e-mail.
> Airbus cannot accept any responsibility for the accuracy or completeness of this e-mail as it has been sent over public networks. If you have any concerns over the content of this message or its Accuracy or Integrity, please contact Airbus immediately.
> All outgoing e-mails from Airbus are checked using regularly updated virus scanning software but you should take whatever measures you deem to be appropriate to ensure that this message and any attachments are virus free.
>
>


-- 
System Analyst Programmer
PTI Lab
Indiana University