You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airavata.apache.org by Isuru Ranawaka <ir...@gmail.com> on 2022/10/04 11:41:02 UTC
Airavata MetaSchedular
Hi all,
Airavata MetaSchedular is a managed job submission framework for
High-Performance Computing resources. Basically, It handles
-
User-based load balancing of Jobs between resource queues.
-
Dynamic Job scheduling across compute resources
-
Job queueing and managed submissions
and the following document [1] contains the captured use cases. Please have
a look and make suggestions, comments are highly appreciated.
I will update the thread as I proceed with the implementation.
thanks
Isuru
[1]
https://docs.google.com/document/d/1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I/edit?usp=sharing
--
Research Software Engineer
Indiana University, IN
Re: Airavata MetaSchedular
Posted by Eroma Abeysinghe <er...@gmail.com>.
Hi Emre,
Happy New Year !!!
We are carrying out our planning activities, so let me check our
schedules and send you a time.
Thanks,
Eroma
On Thu, Jan 5, 2023 at 10:58 AM Emre Brookes <em...@mso.umt.edu>
wrote:
> Hi Eroma,
>
> Hope you had good holidays and Happy 2023!
>
> When can we schedule our first meeting?
>
> Thanks,
> Emre
>
>
> Eroma Abeysinghe wrote:
> > Hi Emre,
> >
> > Yes, We'll meet with you in January to discuss the implementation and
> > testing with UltraScan.
> > Currently, the initial testing is being planned for the implementation.
> >
> > Thanks,
> > Eroma
> >
> >
> >
> > On Thu, Dec 8, 2022 at 9:07 AM Emre Brookes <emre.brookes@mso.umt.edu
> > <ma...@mso.umt.edu>> wrote:
> >
> > Hi Isuru,
> >
> > Thanks for the updates. Great to see progress is being made.
> > I plan to dive into the UltraScan side of this starting in January,
> > perhaps we can setup a meeting early January to see what is
> > implemented,
> > how the new UltraScan microservice can be integrated & begin
> > planning/running some tests?
> >
> > Best wishes,
> > Emre
> >
> >
> > Isuru Ranawaka wrote:
> > > *EXTERNAL EMAIL*
> > >
> > > Hi all,
> > >
> > > We have implemented the initial phase of the Airavata
> > Metascheduler.
> > > Following is the finalized architecture at this phase.
> > >
> > > image.png
> > > Airavata Experiments can be configured for auto-scheduling and for
> > > such Experiments Orchestrator invokes meta scheduler to check
> > whether
> > > the Experiment is schedulable. If it is schedulable Orchestrator
> > > submits the respective Process of the Experiment to Helix Framework
> > > for instant scheduling otherwise Experiment and Process are Queued.
> > >
> > >
> > > *Metascheduler Architecture*
> > >
> > > image.png
> > >
> > > Process scanner scans for already queued
> > Processes
> > > periodically and sends those Process information to the
> > MetaScheduler
> > > Engine. Metascheduler Engine evaluates the chance of successful
> > > submission based on configured scheduling policies. If it is
> > > submittable MetaScheduler engine invokes Orchestrator APIs to
> > initiate
> > > the Job Submission for those Processes through Helix Framework.
> > >
> > > * Compute Resource Monitoring Service
> > >
> > > Compute resource monitoring service uses Airavata Group Resource
> > > Profile information to periodically monitor configured Remote
> > Compute
> > > Resources and update the status of the Queues in the registry. So
> > > the information is consumed by Metacheduler Engine to schedule or
> > > reschedule Processes.
> > >
> > >
> > >
> > > thanks
> > > Isuru
> > >
> > >
> > >
> > >
> > > On Tue, Oct 4, 2022 at 7:41 AM Isuru Ranawaka
> > <irjanith@gmail.com <ma...@gmail.com>
> > > <mailto:irjanith@gmail.com <ma...@gmail.com>>> wrote:
> > >
> > > Hi all,
> > >
> > > Airavata MetaSchedular is a managed job submission framework
> for
> > > High-Performance Computing resources. Basically, It handles
> > >
> > > *
> > >
> > > User-based load balancing of Jobs between resource queues.
> > >
> > > *
> > >
> > > Dynamic Job scheduling across compute resources
> > >
> > > *
> > >
> > > Job queueing and managed submissions
> > >
> > > and the following document [1] contains the captured use cases.
> > > Please have a look and make suggestions, comments are highly
> > > appreciated.
> > > I will update the thread as I proceed with the implementation.
> > >
> > > thanks
> > > Isuru
> > >
> > >
> > >
> > >
> > [1]
> https://docs.google.com/document/d/1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I/edit?usp=sharing
> > <
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I%2Fedit%3Fusp%3Dsharing&data=05%7C01%7Cemre.brookes%40mso.umt.edu%7Cbe901cb022944847525b08dad9273cd3%7C68407ce503da49ffaf0a724be0d37c9d%7C0%7C0%7C638061059806579260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=nS4gzBpdmOWF04SLIpJeCwZEAjDsEPipQzUsaVz7L30%3D&reserved=0
> >
> > >
> > <
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I%2Fedit%3Fusp%3Dsharing&data=05%7C01%7Cemre.brookes%40mso.umt.edu%7Cef364bd1eacf40f9084708dad7c05f78%7C68407ce503da49ffaf0a724be0d37c9d%7C0%7C0%7C638059518517375866%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TakGZiJUBDCypMJMjsD0yNrjayREybgM8JBtQVu9hJM%3D&reserved=0
> > <
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I%2Fedit%3Fusp%3Dsharing&data=05%7C01%7Cemre.brookes%40mso.umt.edu%7Cbe901cb022944847525b08dad9273cd3%7C68407ce503da49ffaf0a724be0d37c9d%7C0%7C0%7C638061059806735502%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=sbHtACYIZgKqCyfBJDM0uuyNkg7xW8c5SgNPkGHcQv4%3D&reserved=0
> >>
> > >
> > >
> > > --
> > > Research Software Engineer
> > > Indiana University, IN
> > >
> > >
> > >
> > > --
> > > Research Software Engineer
> > > Indiana University, IN
> > >
> > >
> > > CAUTION: This message originated outside of UT Health San Antonio.
> > > Please exercise caution when clicking on links or opening
> > attachments.
> >
> >
> >
> > --
> > Thank You,
> > Best Regards,
> > Eroma
>
>
--
Thank You,
Best Regards,
Eroma
Re: Airavata MetaSchedular
Posted by Emre Brookes <em...@mso.umt.edu>.
Hi Eroma,
Hope you had good holidays and Happy 2023!
When can we schedule our first meeting?
Thanks,
Emre
Eroma Abeysinghe wrote:
> Hi Emre,
>
> Yes, We'll meet with you in January to discuss the implementation and
> testing with UltraScan.
> Currently, the initial testing is being planned for the implementation.
>
> Thanks,
> Eroma
>
>
>
> On Thu, Dec 8, 2022 at 9:07 AM Emre Brookes <emre.brookes@mso.umt.edu
> <ma...@mso.umt.edu>> wrote:
>
> Hi Isuru,
>
> Thanks for the updates. Great to see progress is being made.
> I plan to dive into the UltraScan side of this starting in January,
> perhaps we can setup a meeting early January to see what is
> implemented,
> how the new UltraScan microservice can be integrated & begin
> planning/running some tests?
>
> Best wishes,
> Emre
>
>
> Isuru Ranawaka wrote:
> > *EXTERNAL EMAIL*
> >
> > Hi all,
> >
> > We have implemented the initial phase of the Airavata
> Metascheduler.
> > Following is the finalized architecture at this phase.
> >
> > image.png
> > Airavata Experiments can be configured for auto-scheduling and for
> > such Experiments Orchestrator invokes meta scheduler to check
> whether
> > the Experiment is schedulable. If it is schedulable Orchestrator
> > submits the respective Process of the Experiment to Helix Framework
> > for instant scheduling otherwise Experiment and Process are Queued.
> >
> >
> > *Metascheduler Architecture*
> >
> > image.png
> >
> > Process scanner scans for already queued
> Processes
> > periodically and sends those Process information to the
> MetaScheduler
> > Engine. Metascheduler Engine evaluates the chance of successful
> > submission based on configured scheduling policies. If it is
> > submittable MetaScheduler engine invokes Orchestrator APIs to
> initiate
> > the Job Submission for those Processes through Helix Framework.
> >
> > * Compute Resource Monitoring Service
> >
> > Compute resource monitoring service uses Airavata Group Resource
> > Profile information to periodically monitor configured Remote
> Compute
> > Resources and update the status of the Queues in the registry. So
> > the information is consumed by Metacheduler Engine to schedule or
> > reschedule Processes.
> >
> >
> >
> > thanks
> > Isuru
> >
> >
> >
> >
> > On Tue, Oct 4, 2022 at 7:41 AM Isuru Ranawaka
> <irjanith@gmail.com <ma...@gmail.com>
> > <mailto:irjanith@gmail.com <ma...@gmail.com>>> wrote:
> >
> > Hi all,
> >
> > Airavata MetaSchedular is a managed job submission framework for
> > High-Performance Computing resources. Basically, It handles
> >
> > *
> >
> > User-based load balancing of Jobs between resource queues.
> >
> > *
> >
> > Dynamic Job scheduling across compute resources
> >
> > *
> >
> > Job queueing and managed submissions
> >
> > and the following document [1] contains the captured use cases.
> > Please have a look and make suggestions, comments are highly
> > appreciated.
> > I will update the thread as I proceed with the implementation.
> >
> > thanks
> > Isuru
> >
> >
> >
> >
> [1]https://docs.google.com/document/d/1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I/edit?usp=sharing
> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I%2Fedit%3Fusp%3Dsharing&data=05%7C01%7Cemre.brookes%40mso.umt.edu%7Cbe901cb022944847525b08dad9273cd3%7C68407ce503da49ffaf0a724be0d37c9d%7C0%7C0%7C638061059806579260%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=nS4gzBpdmOWF04SLIpJeCwZEAjDsEPipQzUsaVz7L30%3D&reserved=0>
> >
> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I%2Fedit%3Fusp%3Dsharing&data=05%7C01%7Cemre.brookes%40mso.umt.edu%7Cef364bd1eacf40f9084708dad7c05f78%7C68407ce503da49ffaf0a724be0d37c9d%7C0%7C0%7C638059518517375866%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TakGZiJUBDCypMJMjsD0yNrjayREybgM8JBtQVu9hJM%3D&reserved=0
> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I%2Fedit%3Fusp%3Dsharing&data=05%7C01%7Cemre.brookes%40mso.umt.edu%7Cbe901cb022944847525b08dad9273cd3%7C68407ce503da49ffaf0a724be0d37c9d%7C0%7C0%7C638061059806735502%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=sbHtACYIZgKqCyfBJDM0uuyNkg7xW8c5SgNPkGHcQv4%3D&reserved=0>>
> >
> >
> > --
> > Research Software Engineer
> > Indiana University, IN
> >
> >
> >
> > --
> > Research Software Engineer
> > Indiana University, IN
> >
> >
> > CAUTION: This message originated outside of UT Health San Antonio.
> > Please exercise caution when clicking on links or opening
> attachments.
>
>
>
> --
> Thank You,
> Best Regards,
> Eroma
Re: Airavata MetaSchedular
Posted by Eroma Abeysinghe <er...@gmail.com>.
Hi Emre,
Yes, We'll meet with you in January to discuss the implementation and
testing with UltraScan.
Currently, the initial testing is being planned for the implementation.
Thanks,
Eroma
On Thu, Dec 8, 2022 at 9:07 AM Emre Brookes <em...@mso.umt.edu>
wrote:
> Hi Isuru,
>
> Thanks for the updates. Great to see progress is being made.
> I plan to dive into the UltraScan side of this starting in January,
> perhaps we can setup a meeting early January to see what is implemented,
> how the new UltraScan microservice can be integrated & begin
> planning/running some tests?
>
> Best wishes,
> Emre
>
>
> Isuru Ranawaka wrote:
> > *EXTERNAL EMAIL*
> >
> > Hi all,
> >
> > We have implemented the initial phase of the Airavata Metascheduler.
> > Following is the finalized architecture at this phase.
> >
> > image.png
> > Airavata Experiments can be configured for auto-scheduling and for
> > such Experiments Orchestrator invokes meta scheduler to check whether
> > the Experiment is schedulable. If it is schedulable Orchestrator
> > submits the respective Process of the Experiment to Helix Framework
> > for instant scheduling otherwise Experiment and Process are Queued.
> >
> >
> > *Metascheduler Architecture*
> >
> > image.png
> >
> > Process scanner scans for already queued Processes
> > periodically and sends those Process information to the MetaScheduler
> > Engine. Metascheduler Engine evaluates the chance of successful
> > submission based on configured scheduling policies. If it is
> > submittable MetaScheduler engine invokes Orchestrator APIs to initiate
> > the Job Submission for those Processes through Helix Framework.
> >
> > * Compute Resource Monitoring Service
> >
> > Compute resource monitoring service uses Airavata Group Resource
> > Profile information to periodically monitor configured Remote Compute
> > Resources and update the status of the Queues in the registry. So
> > the information is consumed by Metacheduler Engine to schedule or
> > reschedule Processes.
> >
> >
> >
> > thanks
> > Isuru
> >
> >
> >
> >
> > On Tue, Oct 4, 2022 at 7:41 AM Isuru Ranawaka <irjanith@gmail.com
> > <ma...@gmail.com>> wrote:
> >
> > Hi all,
> >
> > Airavata MetaSchedular is a managed job submission framework for
> > High-Performance Computing resources. Basically, It handles
> >
> > *
> >
> > User-based load balancing of Jobs between resource queues.
> >
> > *
> >
> > Dynamic Job scheduling across compute resources
> >
> > *
> >
> > Job queueing and managed submissions
> >
> > and the following document [1] contains the captured use cases.
> > Please have a look and make suggestions, comments are highly
> > appreciated.
> > I will update the thread as I proceed with the implementation.
> >
> > thanks
> > Isuru
> >
> >
> >
> > [1]
> https://docs.google.com/document/d/1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I/edit?usp=sharing
> > <
> https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I%2Fedit%3Fusp%3Dsharing&data=05%7C01%7Cemre.brookes%40mso.umt.edu%7Cef364bd1eacf40f9084708dad7c05f78%7C68407ce503da49ffaf0a724be0d37c9d%7C0%7C0%7C638059518517375866%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TakGZiJUBDCypMJMjsD0yNrjayREybgM8JBtQVu9hJM%3D&reserved=0
> >
> >
> >
> > --
> > Research Software Engineer
> > Indiana University, IN
> >
> >
> >
> > --
> > Research Software Engineer
> > Indiana University, IN
> >
> >
> > CAUTION: This message originated outside of UT Health San Antonio.
> > Please exercise caution when clicking on links or opening attachments.
>
>
--
Thank You,
Best Regards,
Eroma
Re: Airavata MetaSchedular
Posted by Emre Brookes <em...@mso.umt.edu>.
Hi Isuru,
Thanks for the updates. Great to see progress is being made.
I plan to dive into the UltraScan side of this starting in January,
perhaps we can setup a meeting early January to see what is implemented,
how the new UltraScan microservice can be integrated & begin
planning/running some tests?
Best wishes,
Emre
Isuru Ranawaka wrote:
> *EXTERNAL EMAIL*
>
> Hi all,
>
> We have implemented the initial phase of the Airavata Metascheduler.
> Following is the finalized architecture at this phase.
>
> image.png
> Airavata Experiments can be configured for auto-scheduling and for
> such Experiments Orchestrator invokes meta scheduler to check whether
> the Experiment is schedulable. If it is schedulable Orchestrator
> submits the respective Process of the Experiment to Helix Framework
> for instant scheduling otherwise Experiment and Process are Queued.
>
>
> *Metascheduler Architecture*
>
> image.png
>
> Process scanner scans for already queued Processes
> periodically and sends those Process information to the MetaScheduler
> Engine. Metascheduler Engine evaluates the chance of successful
> submission based on configured scheduling policies. If it is
> submittable MetaScheduler engine invokes Orchestrator APIs to initiate
> the Job Submission for those Processes through Helix Framework.
>
> * Compute Resource Monitoring Service
>
> Compute resource monitoring service uses Airavata Group Resource
> Profile information to periodically monitor configured Remote Compute
> Resources and update the status of the Queues in the registry. So
> the information is consumed by Metacheduler Engine to schedule or
> reschedule Processes.
>
>
>
> thanks
> Isuru
>
>
>
>
> On Tue, Oct 4, 2022 at 7:41 AM Isuru Ranawaka <irjanith@gmail.com
> <ma...@gmail.com>> wrote:
>
> Hi all,
>
> Airavata MetaSchedular is a managed job submission framework for
> High-Performance Computing resources. Basically, It handles
>
> *
>
> User-based load balancing of Jobs between resource queues.
>
> *
>
> Dynamic Job scheduling across compute resources
>
> *
>
> Job queueing and managed submissions
>
> and the following document [1] contains the captured use cases.
> Please have a look and make suggestions, comments are highly
> appreciated.
> I will update the thread as I proceed with the implementation.
>
> thanks
> Isuru
>
>
>
> [1]https://docs.google.com/document/d/1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I/edit?usp=sharing
> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I%2Fedit%3Fusp%3Dsharing&data=05%7C01%7Cemre.brookes%40mso.umt.edu%7Cef364bd1eacf40f9084708dad7c05f78%7C68407ce503da49ffaf0a724be0d37c9d%7C0%7C0%7C638059518517375866%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=TakGZiJUBDCypMJMjsD0yNrjayREybgM8JBtQVu9hJM%3D&reserved=0>
>
>
> --
> Research Software Engineer
> Indiana University, IN
>
>
>
> --
> Research Software Engineer
> Indiana University, IN
>
>
> CAUTION: This message originated outside of UT Health San Antonio.
> Please exercise caution when clicking on links or opening attachments.
Re: Airavata MetaSchedular
Posted by Isuru Ranawaka <ir...@gmail.com>.
Hi all,
We have implemented the initial phase of the Airavata Metascheduler.
Following is the finalized architecture at this phase.
[image: image.png]
Airavata Experiments can be configured for auto-scheduling and for such
Experiments Orchestrator invokes meta scheduler to check whether
the Experiment is schedulable. If it is schedulable Orchestrator submits
the respective Process of the Experiment to Helix Framework for instant
scheduling otherwise Experiment and Process are Queued.
*Metascheduler Architecture*
[image: image.png]
Process scanner scans for already queued Processes
periodically and sends those Process information to the MetaScheduler
Engine. Metascheduler Engine evaluates the chance of successful submission
based on configured scheduling policies. If it is submittable MetaScheduler
engine invokes Orchestrator APIs to initiate the Job Submission for those
Processes through Helix Framework.
- Compute Resource Monitoring Service
Compute resource monitoring service uses Airavata Group Resource Profile
information to periodically monitor configured Remote Compute Resources and
update the status of the Queues in the registry. So the information is
consumed by Metacheduler Engine to schedule or reschedule Processes.
thanks
Isuru
On Tue, Oct 4, 2022 at 7:41 AM Isuru Ranawaka <ir...@gmail.com> wrote:
> Hi all,
>
> Airavata MetaSchedular is a managed job submission framework for
> High-Performance Computing resources. Basically, It handles
>
> -
>
> User-based load balancing of Jobs between resource queues.
> -
>
> Dynamic Job scheduling across compute resources
> -
>
> Job queueing and managed submissions
>
> and the following document [1] contains the captured use cases. Please
> have a look and make suggestions, comments are highly appreciated.
> I will update the thread as I proceed with the implementation.
>
> thanks
> Isuru
>
>
>
> [1]
> https://docs.google.com/document/d/1i_cAR7ulNpkEATt0VW4-xoYZyZTQbdnNAJVbnAlMI4I/edit?usp=sharing
>
>
> --
> Research Software Engineer
> Indiana University, IN
>
>
--
Research Software Engineer
Indiana University, IN