You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Aaron Carey <ac...@ilm.com> on 2015/05/13 13:06:18 UTC

Batch Scheduler with dependency support

Hi All,

I was just wondering if anyone out there knew of a good mesos batch scheduler which supports dependencies between tasks? (ie Task B cannot run until Task A is complete)

Thanks,
Aaron

Re: Batch Scheduler with dependency support

Posted by Tom Arnfeld <to...@duedil.com>.
@Tim We're running a combination of an in-house DAG scheduler we wrote a few years back (kind of like https://github.com/spotify/luigi) in combination with Jenkins. I'm not aware of a fully blown DAG scheduler that exists as a Mesos framework but Jenkins might be a way to go, it works well for us.




We also looked into https://github.com/Yelp/Tron many years back but didn't end up using it, perhaps that's a route you could take.




Maybe there's a gap in the market for a DAG scheduler that can run tasks on Mesos ;-)



--


Tom Arnfeld

Developer // DueDil






On Wednesday, May 13, 2015 at 9:15 pm, Tim St Clair <ts...@redhat.com>, wrote:

Hi Alex, 




Have you by chance integrated with any of the tradition batch DAG systems? 





http://pegasus.isi.edu/ , http://ccl.cse.nd.edu/software/makeflow/





I keep longing for folks with decades of experience in HTC&HPC to chime in "on-list". 




Subtle nudge ;-) 

Tim





From: "Alex Gaudio" <ad...@gmail.com>
To: user@mesos.apache.org
Sent: Wednesday, May 13, 2015 3:04:20 PM
Subject: Re: Batch Scheduler with dependency support



Hi Tim (and everyone else!),


I am the primary author of Stolos.  We use Stolos to run all of our batch jobs on Mesos.  The batch jobs are scripts we can run from the command-line.  Scripts range from bash scripts, Spark jobs and R scripts.


It's a great tool for us because, unlike Chronos, it lets us define a script as stage in a dependency chain, where the script can run with different parameters for different dependency contexts.  (The closest usage of this would be to have many Chronos servers, though this does not work in all cases).




The tool is a critical component of Sailthru's data science infrastructure, but I believe we are the only people who use the tool right now.




If you are interested in learning more, I'm happy to invest time to talk more about Stolos, what it does and how we use it!




Alex




On Wed, May 13, 2015 at 2:02 PM Tim Chen <ti...@mesosphere.io> wrote:

How are you running your batch jobs? Is the batch job script/executable an in-house app?





Tim





On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <an...@ishisystems.com> wrote:

You might want to have a look at stolos too:

 

https://github.com/sailthru/stolos

 

 Andras

 

 

From: Aaron Carey [mailto:acarey@ilm.com] 
Sent: Wednesday, May 13, 2015 11:54 AM
To: user@mesos.apache.org
Subject: RE: Batch Scheduler with dependency support




 


Thanks! I hadn't come across that one before :)




From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff Schroeder [jeffschroeder@computer.org]
Sent: 13 May 2015 16:39
To: user@mesos.apache.org
Subject: Re: Batch Scheduler with dependency support



Lookup Hubspot's Singularity





On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:


Thanks Jeff,





Any other options around as well?




From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff Schroeder [jeffschroeder@computer.org]
Sent: 13 May 2015 14:12
To: user@mesos.apache.org
Subject: Batch Scheduler with dependency support



It does both just as well, along with cron-like functionality. It is harder to install and takes a bit more understanding however. The official tutorial is a process that loops 100 times and then exits. 

 



http://aurora.apache.org/documentation/latest/tutorial/#the-script





Aurora is pretty much a superset of most other generic frameworks sans maybe hubspot's singularity.





On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:


I was under the impression Aurora was for long running services? Is it suitable for scheduling one of batch processes too?





thanks,
Aaron




From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff Schroeder [jeffschroeder@computer.org]
Sent: 13 May 2015 13:12
To: user@mesos.apache.org
Subject: Re: Batch Scheduler with dependency support



Apache Aurora does this and you can be explicit about the ordering 





On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:


Hi All,





I was just wondering if anyone out there knew of a good mesos batch scheduler which supports dependencies between tasks? (ie Task B cannot run until Task A is complete)





Thanks,
Aaron 










-- 
Text by Jeff, typos by iPhone
















-- 
Text by Jeff, typos by iPhone














-- 
Text by Jeff, typos by iPhone






























-- 



Cheers,
Timothy St. Clair
Red Hat Inc.

Re: [Junk released by User action] Re: Batch Scheduler with dependency support

Posted by Pablo Cingolani <pa...@gmail.com>.
Hi Everyone,
   I've just sent a mail to the list introducing BDS, my simple scripting
language that runs on Mesos:

   http://pcingola.github.io/BigDataScript/

@Aaron: It looks like BDS should fit what you are looking for. BDS is
designed for complex pipelines with many dependencies (at least that's
what most people are using it for).

@Dougals and @Sharma: BDS also has the same structure where
dependencies are resolved and then sent to the scheduler.

Please let me know what you think.
Yours

    Pablo





On Thu, May 14, 2015 at 12:45 PM, Sharma Podila <sp...@netflix.com> wrote:

> Hi Aaron, yeah, an in-house one.
>
>
> On Thu, May 14, 2015 at 1:09 AM, Aaron Carey <ac...@ilm.com> wrote:
>
>>  Hi Sharma,
>>
>> This sounds eerily familiar! Was this an in-house system you were working
>> on, or a commercial product?
>>
>> Thanks,
>> Aaron
>>
>>  ------------------------------
>> *From:* Sharma Podila [spodila@netflix.com]
>> *Sent:* 13 May 2015 23:49
>> *To:* user@mesos.apache.org
>> *Cc:* Douglas Thain; Brian Bockelman
>> *Subject:* [Junk released by User action] Re: Batch Scheduler with
>> dependency support
>>
>>    ​I keep longing for folks with decades of experience in HTC&HPC to
>>> chime in "on-list".
>>
>>
>>  FWIW, I come from that background, but, am not in that space at this
>> time. My prior life was in developing a (not open source) distributed job
>> scheduler and management system for batch and interactive jobs that handled
>> dependencies, deadlines, preemptions, advance reservation of resources,
>> etc. with multi-level priority and share tree hierarchy based allocation.
>> Typically, dependencies and deadlines are handled outside of schedulers and
>> fed into schedulers as task submission after dependencies have been met. We
>> found it more optimal to have the scheduler resolve dependencies and
>> deadlines inherently. This way, a high priority job dependent on another
>> low priority job can induce higher priority on that dependent job.
>> Similarly, a job with a deadline depending on another job's completion can
>> induce an earlier launch of the latter job in order to meet it's deadline.
>> Also, a dependent job can reserve its resources in advance, knowing the
>> expected completion time of its dependent jobs. This was important because
>> in that environment we always had more jobs to run than can run on
>> available resources. It wasn't unusual to have 10s of 1000s of jobs waiting
>> in queue to run during the day.
>>
>>  Not sure if this helps the original question in this thread in any way.
>> But, I am glad to share my learning, if that helps.
>>
>>  Sharma
>>
>>
>> On Wed, May 13, 2015 at 1:12 PM, Tim St Clair <ts...@redhat.com>
>> wrote:
>>
>>>  Hi Alex,
>>>
>>>  Have you by chance integrated with any of the tradition batch DAG
>>> systems?
>>>
>>>  http://pegasus.isi.edu/ , http://ccl.cse.nd.edu/software/makeflow/
>>>
>>>   ​​
>>> I keep longing for folks with decades of experience in HTC&HPC to chime
>>> in "on-list".
>>>
>>>  Subtle nudge ;-)
>>> Tim
>>>
>>>  ------------------------------
>>>
>>> *From: *"Alex Gaudio" <ad...@gmail.com>
>>> *To: *user@mesos.apache.org
>>> *Sent: *Wednesday, May 13, 2015 3:04:20 PM
>>>
>>> *Subject: *Re: Batch Scheduler with dependency support
>>>
>>>  Hi Tim (and everyone else!),
>>>
>>>  I am the primary author of Stolos.  We use Stolos to run all of our
>>> batch jobs on Mesos.  The batch jobs are scripts we can run from the
>>> command-line.  Scripts range from bash scripts, Spark jobs and R scripts.
>>>
>>>  It's a great tool for us because, unlike Chronos, it lets us define a
>>> script as stage in a dependency chain, where the script can run with
>>> different parameters for different dependency contexts.  (The closest usage
>>> of this would be to have many Chronos servers, though this does not work in
>>> all cases).
>>>
>>>  The tool is a critical component of Sailthru's data science
>>> infrastructure, but I believe we are the only people who use the tool right
>>> now.
>>>
>>>  If you are interested in learning more, I'm happy to invest time to
>>> talk more about Stolos, what it does and how we use it!
>>>
>>>  Alex
>>>
>>> On Wed, May 13, 2015 at 2:02 PM Tim Chen <ti...@mesosphere.io> wrote:
>>>
>>>> How are you running your batch jobs? Is the batch job script/executable
>>>> an in-house app?
>>>>
>>>>  Tim
>>>>
>>>> On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <
>>>> andras.kerekes@ishisystems.com> wrote:
>>>>
>>>>>  You might want to have a look at stolos too:
>>>>>
>>>>>
>>>>>
>>>>> https://github.com/sailthru/stolos
>>>>>
>>>>>
>>>>>
>>>>> Andras
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *From:* Aaron Carey [mailto:acarey@ilm.com]
>>>>> *Sent:* Wednesday, May 13, 2015 11:54 AM
>>>>> *To:* user@mesos.apache.org
>>>>> *Subject:* RE: Batch Scheduler with dependency support
>>>>>
>>>>>
>>>>>
>>>>> Thanks! I hadn't come across that one before :)
>>>>>  ------------------------------
>>>>>
>>>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of
>>>>> Jeff Schroeder [jeffschroeder@computer.org]
>>>>> *Sent:* 13 May 2015 16:39
>>>>> *To:* user@mesos.apache.org
>>>>> *Subject:* Re: Batch Scheduler with dependency support
>>>>>
>>>>> Lookup Hubspot's Singularity
>>>>>
>>>>>  On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>>>
>>>>> Thanks Jeff,
>>>>>
>>>>>  Any other options around as well?
>>>>>  ------------------------------
>>>>>
>>>>> *From:* jeffschroed@gmail.com <http://UrlBlockedError.aspx> [
>>>>> jeffschroed@gmail.com <http://UrlBlockedError.aspx>] on behalf of
>>>>> Jeff Schroeder [jeffschroeder@computer.org
>>>>> <http://UrlBlockedError.aspx>]
>>>>> *Sent:* 13 May 2015 14:12
>>>>> *To:* user@mesos.apache.org <http://UrlBlockedError.aspx>
>>>>> *Subject:* Batch Scheduler with dependency support
>>>>>
>>>>> It does both just as well, along with cron-like functionality. It is
>>>>> harder to install and takes a bit more understanding however. The official
>>>>> tutorial is a process that loops 100 times and then exits.
>>>>>
>>>>>
>>>>>
>>>>> http://aurora.apache.org/documentation/latest/tutorial/#the-script
>>>>>
>>>>>  Aurora is pretty much a superset of most other generic frameworks
>>>>> sans maybe hubspot's singularity.
>>>>>
>>>>>
>>>>> On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com
>>>>> <http://UrlBlockedError.aspx>> wrote:
>>>>>
>>>>> I was under the impression Aurora was for long running services? Is it
>>>>> suitable for scheduling one of batch processes too?
>>>>>
>>>>>  thanks,
>>>>> Aaron
>>>>>  ------------------------------
>>>>>
>>>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of
>>>>> Jeff Schroeder [jeffschroeder@computer.org]
>>>>> *Sent:* 13 May 2015 13:12
>>>>> *To:* user@mesos.apache.org
>>>>> *Subject:* Re: Batch Scheduler with dependency support
>>>>>
>>>>> Apache Aurora does this and you can be explicit about the ordering
>>>>>
>>>>>  On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>>>
>>>>> Hi All,
>>>>>
>>>>>  I was just wondering if anyone out there knew of a good mesos batch
>>>>> scheduler which supports dependencies between tasks? (ie Task B cannot run
>>>>> until Task A is complete)
>>>>>
>>>>>  Thanks,
>>>>> Aaron
>>>>>
>>>>>
>>>>>
>>>>>  --
>>>>> Text by Jeff, typos by iPhone
>>>>>
>>>>>
>>>>>
>>>>>  --
>>>>> Text by Jeff, typos by iPhone
>>>>>
>>>>>
>>>>>
>>>>>  --
>>>>> Text by Jeff, typos by iPhone
>>>>>
>>>>
>>>>
>>>
>>>
>>>  --
>>>  Cheers,
>>> Timothy St. Clair
>>> Red Hat Inc.
>>>
>>
>>
>

Re: [Junk released by User action] Re: Batch Scheduler with dependency support

Posted by Sharma Podila <sp...@netflix.com>.
Hi Aaron, yeah, an in-house one.


On Thu, May 14, 2015 at 1:09 AM, Aaron Carey <ac...@ilm.com> wrote:

>  Hi Sharma,
>
> This sounds eerily familiar! Was this an in-house system you were working
> on, or a commercial product?
>
> Thanks,
> Aaron
>
>  ------------------------------
> *From:* Sharma Podila [spodila@netflix.com]
> *Sent:* 13 May 2015 23:49
> *To:* user@mesos.apache.org
> *Cc:* Douglas Thain; Brian Bockelman
> *Subject:* [Junk released by User action] Re: Batch Scheduler with
> dependency support
>
>    ​I keep longing for folks with decades of experience in HTC&HPC to
>> chime in "on-list".
>
>
>  FWIW, I come from that background, but, am not in that space at this
> time. My prior life was in developing a (not open source) distributed job
> scheduler and management system for batch and interactive jobs that handled
> dependencies, deadlines, preemptions, advance reservation of resources,
> etc. with multi-level priority and share tree hierarchy based allocation.
> Typically, dependencies and deadlines are handled outside of schedulers and
> fed into schedulers as task submission after dependencies have been met. We
> found it more optimal to have the scheduler resolve dependencies and
> deadlines inherently. This way, a high priority job dependent on another
> low priority job can induce higher priority on that dependent job.
> Similarly, a job with a deadline depending on another job's completion can
> induce an earlier launch of the latter job in order to meet it's deadline.
> Also, a dependent job can reserve its resources in advance, knowing the
> expected completion time of its dependent jobs. This was important because
> in that environment we always had more jobs to run than can run on
> available resources. It wasn't unusual to have 10s of 1000s of jobs waiting
> in queue to run during the day.
>
>  Not sure if this helps the original question in this thread in any way.
> But, I am glad to share my learning, if that helps.
>
>  Sharma
>
>
> On Wed, May 13, 2015 at 1:12 PM, Tim St Clair <ts...@redhat.com> wrote:
>
>>  Hi Alex,
>>
>>  Have you by chance integrated with any of the tradition batch DAG
>> systems?
>>
>>  http://pegasus.isi.edu/ , http://ccl.cse.nd.edu/software/makeflow/
>>
>>   ​​
>> I keep longing for folks with decades of experience in HTC&HPC to chime
>> in "on-list".
>>
>>  Subtle nudge ;-)
>> Tim
>>
>>  ------------------------------
>>
>> *From: *"Alex Gaudio" <ad...@gmail.com>
>> *To: *user@mesos.apache.org
>> *Sent: *Wednesday, May 13, 2015 3:04:20 PM
>>
>> *Subject: *Re: Batch Scheduler with dependency support
>>
>>  Hi Tim (and everyone else!),
>>
>>  I am the primary author of Stolos.  We use Stolos to run all of our
>> batch jobs on Mesos.  The batch jobs are scripts we can run from the
>> command-line.  Scripts range from bash scripts, Spark jobs and R scripts.
>>
>>  It's a great tool for us because, unlike Chronos, it lets us define a
>> script as stage in a dependency chain, where the script can run with
>> different parameters for different dependency contexts.  (The closest usage
>> of this would be to have many Chronos servers, though this does not work in
>> all cases).
>>
>>  The tool is a critical component of Sailthru's data science
>> infrastructure, but I believe we are the only people who use the tool right
>> now.
>>
>>  If you are interested in learning more, I'm happy to invest time to
>> talk more about Stolos, what it does and how we use it!
>>
>>  Alex
>>
>> On Wed, May 13, 2015 at 2:02 PM Tim Chen <ti...@mesosphere.io> wrote:
>>
>>> How are you running your batch jobs? Is the batch job script/executable
>>> an in-house app?
>>>
>>>  Tim
>>>
>>> On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <
>>> andras.kerekes@ishisystems.com> wrote:
>>>
>>>>  You might want to have a look at stolos too:
>>>>
>>>>
>>>>
>>>> https://github.com/sailthru/stolos
>>>>
>>>>
>>>>
>>>> Andras
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *From:* Aaron Carey [mailto:acarey@ilm.com]
>>>> *Sent:* Wednesday, May 13, 2015 11:54 AM
>>>> *To:* user@mesos.apache.org
>>>> *Subject:* RE: Batch Scheduler with dependency support
>>>>
>>>>
>>>>
>>>> Thanks! I hadn't come across that one before :)
>>>>  ------------------------------
>>>>
>>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of
>>>> Jeff Schroeder [jeffschroeder@computer.org]
>>>> *Sent:* 13 May 2015 16:39
>>>> *To:* user@mesos.apache.org
>>>> *Subject:* Re: Batch Scheduler with dependency support
>>>>
>>>> Lookup Hubspot's Singularity
>>>>
>>>>  On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>>
>>>> Thanks Jeff,
>>>>
>>>>  Any other options around as well?
>>>>  ------------------------------
>>>>
>>>> *From:* jeffschroed@gmail.com <http://UrlBlockedError.aspx> [
>>>> jeffschroed@gmail.com <http://UrlBlockedError.aspx>] on behalf of Jeff
>>>> Schroeder [jeffschroeder@computer.org <http://UrlBlockedError.aspx>]
>>>> *Sent:* 13 May 2015 14:12
>>>> *To:* user@mesos.apache.org <http://UrlBlockedError.aspx>
>>>> *Subject:* Batch Scheduler with dependency support
>>>>
>>>> It does both just as well, along with cron-like functionality. It is
>>>> harder to install and takes a bit more understanding however. The official
>>>> tutorial is a process that loops 100 times and then exits.
>>>>
>>>>
>>>>
>>>> http://aurora.apache.org/documentation/latest/tutorial/#the-script
>>>>
>>>>  Aurora is pretty much a superset of most other generic frameworks
>>>> sans maybe hubspot's singularity.
>>>>
>>>>
>>>> On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com
>>>> <http://UrlBlockedError.aspx>> wrote:
>>>>
>>>> I was under the impression Aurora was for long running services? Is it
>>>> suitable for scheduling one of batch processes too?
>>>>
>>>>  thanks,
>>>> Aaron
>>>>  ------------------------------
>>>>
>>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of
>>>> Jeff Schroeder [jeffschroeder@computer.org]
>>>> *Sent:* 13 May 2015 13:12
>>>> *To:* user@mesos.apache.org
>>>> *Subject:* Re: Batch Scheduler with dependency support
>>>>
>>>> Apache Aurora does this and you can be explicit about the ordering
>>>>
>>>>  On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>>
>>>> Hi All,
>>>>
>>>>  I was just wondering if anyone out there knew of a good mesos batch
>>>> scheduler which supports dependencies between tasks? (ie Task B cannot run
>>>> until Task A is complete)
>>>>
>>>>  Thanks,
>>>> Aaron
>>>>
>>>>
>>>>
>>>>  --
>>>> Text by Jeff, typos by iPhone
>>>>
>>>>
>>>>
>>>>  --
>>>> Text by Jeff, typos by iPhone
>>>>
>>>>
>>>>
>>>>  --
>>>> Text by Jeff, typos by iPhone
>>>>
>>>
>>>
>>
>>
>>  --
>>  Cheers,
>> Timothy St. Clair
>> Red Hat Inc.
>>
>
>

RE: [Junk released by User action] Re: Batch Scheduler with dependency support

Posted by Aaron Carey <ac...@ilm.com>.
Hi Sharma,

This sounds eerily familiar! Was this an in-house system you were working on, or a commercial product?

Thanks,
Aaron

________________________________
From: Sharma Podila [spodila@netflix.com]
Sent: 13 May 2015 23:49
To: user@mesos.apache.org
Cc: Douglas Thain; Brian Bockelman
Subject: [Junk released by User action] Re: Batch Scheduler with dependency support

​I keep longing for folks with decades of experience in HTC&HPC to chime in "on-list".

FWIW, I come from that background, but, am not in that space at this time. My prior life was in developing a (not open source) distributed job scheduler and management system for batch and interactive jobs that handled dependencies, deadlines, preemptions, advance reservation of resources, etc. with multi-level priority and share tree hierarchy based allocation. Typically, dependencies and deadlines are handled outside of schedulers and fed into schedulers as task submission after dependencies have been met. We found it more optimal to have the scheduler resolve dependencies and deadlines inherently. This way, a high priority job dependent on another low priority job can induce higher priority on that dependent job. Similarly, a job with a deadline depending on another job's completion can induce an earlier launch of the latter job in order to meet it's deadline. Also, a dependent job can reserve its resources in advance, knowing the expected completion time of its dependent jobs. This was important because in that environment we always had more jobs to run than can run on available resources. It wasn't unusual to have 10s of 1000s of jobs waiting in queue to run during the day.

Not sure if this helps the original question in this thread in any way. But, I am glad to share my learning, if that helps.

Sharma


On Wed, May 13, 2015 at 1:12 PM, Tim St Clair <ts...@redhat.com>> wrote:
Hi Alex,

Have you by chance integrated with any of the tradition batch DAG systems?

http://pegasus.isi.edu/ , http://ccl.cse.nd.edu/software/makeflow/

​​
I keep longing for folks with decades of experience in HTC&HPC to chime in "on-list".

Subtle nudge ;-)
Tim

________________________________
From: "Alex Gaudio" <ad...@gmail.com>>
To: user@mesos.apache.org<ma...@mesos.apache.org>
Sent: Wednesday, May 13, 2015 3:04:20 PM

Subject: Re: Batch Scheduler with dependency support

Hi Tim (and everyone else!),

I am the primary author of Stolos.  We use Stolos to run all of our batch jobs on Mesos.  The batch jobs are scripts we can run from the command-line.  Scripts range from bash scripts, Spark jobs and R scripts.

It's a great tool for us because, unlike Chronos, it lets us define a script as stage in a dependency chain, where the script can run with different parameters for different dependency contexts.  (The closest usage of this would be to have many Chronos servers, though this does not work in all cases).

The tool is a critical component of Sailthru's data science infrastructure, but I believe we are the only people who use the tool right now.

If you are interested in learning more, I'm happy to invest time to talk more about Stolos, what it does and how we use it!

Alex

On Wed, May 13, 2015 at 2:02 PM Tim Chen <ti...@mesosphere.io>> wrote:
How are you running your batch jobs? Is the batch job script/executable an in-house app?

Tim

On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <an...@ishisystems.com>> wrote:
You might want to have a look at stolos too:

https://github.com/sailthru/stolos

Andras


From: Aaron Carey [mailto:acarey@ilm.com<ma...@ilm.com>]
Sent: Wednesday, May 13, 2015 11:54 AM
To: user@mesos.apache.org<ma...@mesos.apache.org>
Subject: RE: Batch Scheduler with dependency support

Thanks! I hadn't come across that one before :)
________________________________
From: jeffschroed@gmail.com<ma...@gmail.com> [jeffschroed@gmail.com<ma...@gmail.com>] on behalf of Jeff Schroeder [jeffschroeder@computer.org<ma...@computer.org>]
Sent: 13 May 2015 16:39
To: user@mesos.apache.org<ma...@mesos.apache.org>
Subject: Re: Batch Scheduler with dependency support
Lookup Hubspot's Singularity

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com>> wrote:
Thanks Jeff,

Any other options around as well?
________________________________
From: jeffschroed@gmail.com<http://UrlBlockedError.aspx> [jeffschroed@gmail.com<http://UrlBlockedError.aspx>] on behalf of Jeff Schroeder [jeffschroeder@computer.org<http://UrlBlockedError.aspx>]
Sent: 13 May 2015 14:12
To: user@mesos.apache.org<http://UrlBlockedError.aspx>
Subject: Batch Scheduler with dependency support
It does both just as well, along with cron-like functionality. It is harder to install and takes a bit more understanding however. The official tutorial is a process that loops 100 times and then exits.

http://aurora.apache.org/documentation/latest/tutorial/#the-script

Aurora is pretty much a superset of most other generic frameworks sans maybe hubspot's singularity.

On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com<http://UrlBlockedError.aspx>> wrote:
I was under the impression Aurora was for long running services? Is it suitable for scheduling one of batch processes too?

thanks,
Aaron
________________________________
From: jeffschroed@gmail.com<ma...@gmail.com> [jeffschroed@gmail.com<ma...@gmail.com>] on behalf of Jeff Schroeder [jeffschroeder@computer.org<ma...@computer.org>]
Sent: 13 May 2015 13:12
To: user@mesos.apache.org<ma...@mesos.apache.org>
Subject: Re: Batch Scheduler with dependency support
Apache Aurora does this and you can be explicit about the ordering

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com>> wrote:
Hi All,

I was just wondering if anyone out there knew of a good mesos batch scheduler which supports dependencies between tasks? (ie Task B cannot run until Task A is complete)

Thanks,
Aaron


--
Text by Jeff, typos by iPhone


--
Text by Jeff, typos by iPhone


--
Text by Jeff, typos by iPhone




--
Cheers,
Timothy St. Clair
Red Hat Inc.

Re: Batch Scheduler with dependency support

Posted by Douglas Thain <dt...@nd.edu>.
@Tim Thanks for the pingback on Makeflow.  At the moment, Makeflow
works with the most widely used batch schedulers (Condor, SGE, Torque,
etc) and we are looking into adding support for Mesos.  Makeflow
assumes that the underlying system has an queueing api of the form
(submit/wait/status/remove) so it may make more sense to submit tasks
to an intermediate service (e.g. Singularity?) that then interfaces
with Mesos to match resource offers with requests.  We are open to
suggestions here.

@Sharma +1 on your architectural observation.  Our experience is
similar in that you want to have one component responsible for
handling dependencies, and then pass off the ready tasks to a
scheduler.  If you mix scheduling of tasks with dependency management,
the implementation gets very complicated very quickly.  It also allows
for a wide variety of workflow languages to fit application needs:




On Wed, May 13, 2015 at 8:36 PM, Alex Gaudio <ad...@gmail.com> wrote:
> @Tim Chen
>
> Thank your for your comment!  We use Stolos run about 200k jobs a day, where
> some jobs are themselves Mesos frameworks (ie Spark jobs).  The tool seems
> so far to be scalable because it offloads all scaling problems to
> established tools.  Queue/job state is stored to a user-chosen queue
> backend, configuration is stored in a user-chosen configuration backend, and
> jobs themselves can be huge, parallelized jobs (Hadoop jobs or Spark jbos)
> or small one-off reports.
>
>
> @Tim St. Clair
>
> Pegasus and Makeflow look really neat!  Are there any plans to integrate
> them with Mesos?  I am very excited to check these out further, as Pegasus's
> documentation on job clustering looks particularly interesting!  There are
> some very interesting algorithms in this space.
>
> To your question about integrating with batch queuing systems, I'm not sure
> to what extent Stolos integrates with, competes with, or is compatible with
> batch queuing systems like Condor, Quincy, etc.  Stolos solves only one
> problem: As a wrapper around scripts, chooses whether to run the wrapped
> script or queue another wrapped script.  I wrote and still actively develop
> Stolos because I haven't found anything else that works with Mesos and meets
> our needs.  That said, it's totally worth looking at your suggestions more
> deeply!
>
>
> @Tom
>
> I would definitely agree with you that there is a "gap" in the Mesos world
> for a really good DAG scheduler.  Chronos sort of gets there, but when it
> comes down to it, I don't currently believe a tools should combine both job
> dependency management (which gets really complicated) with distributed
> crontab (which is a different set of problems).
>
>
> Alex
>
>
> On Wed, May 13, 2015 at 6:50 PM Sharma Podila <sp...@netflix.com> wrote:
>>>
>>> I keep longing for folks with decades of experience in HTC&HPC to chime
>>> in "on-list".
>>
>>
>> FWIW, I come from that background, but, am not in that space at this time.
>> My prior life was in developing a (not open source) distributed job
>> scheduler and management system for batch and interactive jobs that handled
>> dependencies, deadlines, preemptions, advance reservation of resources, etc.
>> with multi-level priority and share tree hierarchy based allocation.
>> Typically, dependencies and deadlines are handled outside of schedulers and
>> fed into schedulers as task submission after dependencies have been met. We
>> found it more optimal to have the scheduler resolve dependencies and
>> deadlines inherently. This way, a high priority job dependent on another low
>> priority job can induce higher priority on that dependent job. Similarly, a
>> job with a deadline depending on another job's completion can induce an
>> earlier launch of the latter job in order to meet it's deadline. Also, a
>> dependent job can reserve its resources in advance, knowing the expected
>> completion time of its dependent jobs. This was important because in that
>> environment we always had more jobs to run than can run on available
>> resources. It wasn't unusual to have 10s of 1000s of jobs waiting in queue
>> to run during the day.
>>
>> Not sure if this helps the original question in this thread in any way.
>> But, I am glad to share my learning, if that helps.
>>
>> Sharma
>>
>>
>> On Wed, May 13, 2015 at 1:12 PM, Tim St Clair <ts...@redhat.com> wrote:
>>>
>>> Hi Alex,
>>>
>>> Have you by chance integrated with any of the tradition batch DAG
>>> systems?
>>>
>>> http://pegasus.isi.edu/ , http://ccl.cse.nd.edu/software/makeflow/
>>>
>>> I keep longing for folks with decades of experience in HTC&HPC to chime
>>> in "on-list".
>>>
>>> Subtle nudge ;-)
>>> Tim
>>>
>>> ________________________________
>>>
>>> From: "Alex Gaudio" <ad...@gmail.com>
>>> To: user@mesos.apache.org
>>> Sent: Wednesday, May 13, 2015 3:04:20 PM
>>>
>>> Subject: Re: Batch Scheduler with dependency support
>>>
>>> Hi Tim (and everyone else!),
>>>
>>> I am the primary author of Stolos.  We use Stolos to run all of our batch
>>> jobs on Mesos.  The batch jobs are scripts we can run from the command-line.
>>> Scripts range from bash scripts, Spark jobs and R scripts.
>>>
>>> It's a great tool for us because, unlike Chronos, it lets us define a
>>> script as stage in a dependency chain, where the script can run with
>>> different parameters for different dependency contexts.  (The closest usage
>>> of this would be to have many Chronos servers, though this does not work in
>>> all cases).
>>>
>>> The tool is a critical component of Sailthru's data science
>>> infrastructure, but I believe we are the only people who use the tool right
>>> now.
>>>
>>> If you are interested in learning more, I'm happy to invest time to talk
>>> more about Stolos, what it does and how we use it!
>>>
>>> Alex
>>>
>>> On Wed, May 13, 2015 at 2:02 PM Tim Chen <ti...@mesosphere.io> wrote:
>>>>
>>>> How are you running your batch jobs? Is the batch job script/executable
>>>> an in-house app?
>>>>
>>>> Tim
>>>>
>>>> On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes
>>>> <an...@ishisystems.com> wrote:
>>>>>
>>>>> You might want to have a look at stolos too:
>>>>>
>>>>>
>>>>>
>>>>> https://github.com/sailthru/stolos
>>>>>
>>>>>
>>>>>
>>>>> Andras
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> From: Aaron Carey [mailto:acarey@ilm.com]
>>>>> Sent: Wednesday, May 13, 2015 11:54 AM
>>>>> To: user@mesos.apache.org
>>>>> Subject: RE: Batch Scheduler with dependency support
>>>>>
>>>>>
>>>>>
>>>>> Thanks! I hadn't come across that one before :)
>>>>>
>>>>> ________________________________
>>>>>
>>>>> From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
>>>>> Schroeder [jeffschroeder@computer.org]
>>>>> Sent: 13 May 2015 16:39
>>>>> To: user@mesos.apache.org
>>>>> Subject: Re: Batch Scheduler with dependency support
>>>>>
>>>>> Lookup Hubspot's Singularity
>>>>>
>>>>>
>>>>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>>>
>>>>> Thanks Jeff,
>>>>>
>>>>>
>>>>> Any other options around as well?
>>>>>
>>>>> ________________________________
>>>>>
>>>>> From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
>>>>> Schroeder [jeffschroeder@computer.org]
>>>>> Sent: 13 May 2015 14:12
>>>>> To: user@mesos.apache.org
>>>>> Subject: Batch Scheduler with dependency support
>>>>>
>>>>> It does both just as well, along with cron-like functionality. It is
>>>>> harder to install and takes a bit more understanding however. The official
>>>>> tutorial is a process that loops 100 times and then exits.
>>>>>
>>>>>
>>>>>
>>>>> http://aurora.apache.org/documentation/latest/tutorial/#the-script
>>>>>
>>>>>
>>>>> Aurora is pretty much a superset of most other generic frameworks sans
>>>>> maybe hubspot's singularity.
>>>>>
>>>>>
>>>>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>>>
>>>>> I was under the impression Aurora was for long running services? Is it
>>>>> suitable for scheduling one of batch processes too?
>>>>>
>>>>>
>>>>> thanks,
>>>>> Aaron
>>>>>
>>>>> ________________________________
>>>>>
>>>>> From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
>>>>> Schroeder [jeffschroeder@computer.org]
>>>>> Sent: 13 May 2015 13:12
>>>>> To: user@mesos.apache.org
>>>>> Subject: Re: Batch Scheduler with dependency support
>>>>>
>>>>> Apache Aurora does this and you can be explicit about the ordering
>>>>>
>>>>>
>>>>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>>>
>>>>> Hi All,
>>>>>
>>>>>
>>>>> I was just wondering if anyone out there knew of a good mesos batch
>>>>> scheduler which supports dependencies between tasks? (ie Task B cannot run
>>>>> until Task A is complete)
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Aaron
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Text by Jeff, typos by iPhone
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Text by Jeff, typos by iPhone
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Text by Jeff, typos by iPhone
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Cheers,
>>> Timothy St. Clair
>>> Red Hat Inc.
>>
>>
>

Re: Batch Scheduler with dependency support

Posted by Alex Gaudio <ad...@gmail.com>.
@Tim Chen

Thank your for your comment!  We use Stolos run about 200k jobs a day,
where some jobs are themselves Mesos frameworks (ie Spark jobs).  The tool
seems so far to be scalable because it offloads all scaling problems to
established tools.  Queue/job state is stored to a user-chosen queue
backend, configuration is stored in a user-chosen configuration backend,
and jobs themselves can be huge, parallelized jobs (Hadoop jobs or Spark
jbos) or small one-off reports.


@Tim St. Clair

Pegasus and Makeflow look really neat!  Are there any plans to integrate
them with Mesos?  I am very excited to check these out further, as
Pegasus's documentation on job clustering
<http://pegasus.isi.edu/wms/docs/latest/job_clustering.php> looks
particularly interesting!  There are some very interesting algorithms in
this space.

To your question about integrating with batch queuing systems, I'm not sure
to what extent Stolos integrates with, competes with, or is compatible with
batch queuing systems like Condor, Quincy, etc.  Stolos solves only one
problem: As a wrapper around scripts, chooses whether to run the wrapped
script or queue another wrapped script.  I wrote and still actively develop
Stolos because I haven't found anything else that works with Mesos and
meets our needs.  That said, it's totally worth looking at your suggestions
more deeply!


@Tom

I would definitely agree with you that there is a "gap" in the Mesos world
for a really good DAG scheduler.  Chronos sort of gets there, but when it
comes down to it, I don't currently believe a tools should combine both job
dependency management (which gets really complicated) with distributed
crontab (which is a different set of problems).


Alex


On Wed, May 13, 2015 at 6:50 PM Sharma Podila <sp...@netflix.com> wrote:

> ​I keep longing for folks with decades of experience in HTC&HPC to chime
>> in "on-list".
>
>
> FWIW, I come from that background, but, am not in that space at this time.
> My prior life was in developing a (not open source) distributed job
> scheduler and management system for batch and interactive jobs that handled
> dependencies, deadlines, preemptions, advance reservation of resources,
> etc. with multi-level priority and share tree hierarchy based allocation.
> Typically, dependencies and deadlines are handled outside of schedulers and
> fed into schedulers as task submission after dependencies have been met. We
> found it more optimal to have the scheduler resolve dependencies and
> deadlines inherently. This way, a high priority job dependent on another
> low priority job can induce higher priority on that dependent job.
> Similarly, a job with a deadline depending on another job's completion can
> induce an earlier launch of the latter job in order to meet it's deadline.
> Also, a dependent job can reserve its resources in advance, knowing the
> expected completion time of its dependent jobs. This was important because
> in that environment we always had more jobs to run than can run on
> available resources. It wasn't unusual to have 10s of 1000s of jobs waiting
> in queue to run during the day.
>
> Not sure if this helps the original question in this thread in any way.
> But, I am glad to share my learning, if that helps.
>
> Sharma
>
>
> On Wed, May 13, 2015 at 1:12 PM, Tim St Clair <ts...@redhat.com> wrote:
>
>> Hi Alex,
>>
>> Have you by chance integrated with any of the tradition batch DAG
>> systems?
>>
>> http://pegasus.isi.edu/ , http://ccl.cse.nd.edu/software/makeflow/
>>
>> ​​
>> I keep longing for folks with decades of experience in HTC&HPC to chime
>> in "on-list".
>>
>> Subtle nudge ;-)
>> Tim
>>
>> ------------------------------
>>
>> *From: *"Alex Gaudio" <ad...@gmail.com>
>> *To: *user@mesos.apache.org
>> *Sent: *Wednesday, May 13, 2015 3:04:20 PM
>>
>> *Subject: *Re: Batch Scheduler with dependency support
>>
>> Hi Tim (and everyone else!),
>>
>> I am the primary author of Stolos.  We use Stolos to run all of our batch
>> jobs on Mesos.  The batch jobs are scripts we can run from the
>> command-line.  Scripts range from bash scripts, Spark jobs and R scripts.
>>
>> It's a great tool for us because, unlike Chronos, it lets us define a
>> script as stage in a dependency chain, where the script can run with
>> different parameters for different dependency contexts.  (The closest usage
>> of this would be to have many Chronos servers, though this does not work in
>> all cases).
>>
>> The tool is a critical component of Sailthru's data science
>> infrastructure, but I believe we are the only people who use the tool right
>> now.
>>
>> If you are interested in learning more, I'm happy to invest time to talk
>> more about Stolos, what it does and how we use it!
>>
>> Alex
>>
>> On Wed, May 13, 2015 at 2:02 PM Tim Chen <ti...@mesosphere.io> wrote:
>>
>>> How are you running your batch jobs? Is the batch job script/executable
>>> an in-house app?
>>>
>>> Tim
>>>
>>> On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <
>>> andras.kerekes@ishisystems.com> wrote:
>>>
>>>> You might want to have a look at stolos too:
>>>>
>>>>
>>>>
>>>> https://github.com/sailthru/stolos
>>>>
>>>>
>>>>
>>>> Andras
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *From:* Aaron Carey [mailto:acarey@ilm.com]
>>>> *Sent:* Wednesday, May 13, 2015 11:54 AM
>>>> *To:* user@mesos.apache.org
>>>> *Subject:* RE: Batch Scheduler with dependency support
>>>>
>>>>
>>>>
>>>> Thanks! I hadn't come across that one before :)
>>>> ------------------------------
>>>>
>>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of
>>>> Jeff Schroeder [jeffschroeder@computer.org]
>>>> *Sent:* 13 May 2015 16:39
>>>> *To:* user@mesos.apache.org
>>>> *Subject:* Re: Batch Scheduler with dependency support
>>>>
>>>> Lookup Hubspot's Singularity
>>>>
>>>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>>
>>>> Thanks Jeff,
>>>>
>>>> Any other options around as well?
>>>> ------------------------------
>>>>
>>>> *From:* jeffschroed@gmail.com <http://UrlBlockedError.aspx> [
>>>> jeffschroed@gmail.com <http://UrlBlockedError.aspx>] on behalf of Jeff
>>>> Schroeder [jeffschroeder@computer.org <http://UrlBlockedError.aspx>]
>>>> *Sent:* 13 May 2015 14:12
>>>> *To:* user@mesos.apache.org <http://UrlBlockedError.aspx>
>>>> *Subject:* Batch Scheduler with dependency support
>>>>
>>>> It does both just as well, along with cron-like functionality. It is
>>>> harder to install and takes a bit more understanding however. The official
>>>> tutorial is a process that loops 100 times and then exits.
>>>>
>>>>
>>>>
>>>> http://aurora.apache.org/documentation/latest/tutorial/#the-script
>>>>
>>>> Aurora is pretty much a superset of most other generic frameworks sans
>>>> maybe hubspot's singularity.
>>>>
>>>>
>>>> On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com
>>>> <http://UrlBlockedError.aspx>> wrote:
>>>>
>>>> I was under the impression Aurora was for long running services? Is it
>>>> suitable for scheduling one of batch processes too?
>>>>
>>>> thanks,
>>>> Aaron
>>>> ------------------------------
>>>>
>>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of
>>>> Jeff Schroeder [jeffschroeder@computer.org]
>>>> *Sent:* 13 May 2015 13:12
>>>> *To:* user@mesos.apache.org
>>>> *Subject:* Re: Batch Scheduler with dependency support
>>>>
>>>> Apache Aurora does this and you can be explicit about the ordering
>>>>
>>>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>>
>>>> Hi All,
>>>>
>>>> I was just wondering if anyone out there knew of a good mesos batch
>>>> scheduler which supports dependencies between tasks? (ie Task B cannot run
>>>> until Task A is complete)
>>>>
>>>> Thanks,
>>>> Aaron
>>>>
>>>>
>>>>
>>>> --
>>>> Text by Jeff, typos by iPhone
>>>>
>>>>
>>>>
>>>> --
>>>> Text by Jeff, typos by iPhone
>>>>
>>>>
>>>>
>>>> --
>>>> Text by Jeff, typos by iPhone
>>>>
>>>
>>>
>>
>>
>> --
>> Cheers,
>> Timothy St. Clair
>> Red Hat Inc.
>>
>
>

Re: Batch Scheduler with dependency support

Posted by Sharma Podila <sp...@netflix.com>.
>
> ​I keep longing for folks with decades of experience in HTC&HPC to chime
> in "on-list".


FWIW, I come from that background, but, am not in that space at this time.
My prior life was in developing a (not open source) distributed job
scheduler and management system for batch and interactive jobs that handled
dependencies, deadlines, preemptions, advance reservation of resources,
etc. with multi-level priority and share tree hierarchy based allocation.
Typically, dependencies and deadlines are handled outside of schedulers and
fed into schedulers as task submission after dependencies have been met. We
found it more optimal to have the scheduler resolve dependencies and
deadlines inherently. This way, a high priority job dependent on another
low priority job can induce higher priority on that dependent job.
Similarly, a job with a deadline depending on another job's completion can
induce an earlier launch of the latter job in order to meet it's deadline.
Also, a dependent job can reserve its resources in advance, knowing the
expected completion time of its dependent jobs. This was important because
in that environment we always had more jobs to run than can run on
available resources. It wasn't unusual to have 10s of 1000s of jobs waiting
in queue to run during the day.

Not sure if this helps the original question in this thread in any way.
But, I am glad to share my learning, if that helps.

Sharma


On Wed, May 13, 2015 at 1:12 PM, Tim St Clair <ts...@redhat.com> wrote:

> Hi Alex,
>
> Have you by chance integrated with any of the tradition batch DAG systems?
>
> http://pegasus.isi.edu/ , http://ccl.cse.nd.edu/software/makeflow/
>
> ​​
> I keep longing for folks with decades of experience in HTC&HPC to chime in
> "on-list".
>
> Subtle nudge ;-)
> Tim
>
> ------------------------------
>
> *From: *"Alex Gaudio" <ad...@gmail.com>
> *To: *user@mesos.apache.org
> *Sent: *Wednesday, May 13, 2015 3:04:20 PM
>
> *Subject: *Re: Batch Scheduler with dependency support
>
> Hi Tim (and everyone else!),
>
> I am the primary author of Stolos.  We use Stolos to run all of our batch
> jobs on Mesos.  The batch jobs are scripts we can run from the
> command-line.  Scripts range from bash scripts, Spark jobs and R scripts.
>
> It's a great tool for us because, unlike Chronos, it lets us define a
> script as stage in a dependency chain, where the script can run with
> different parameters for different dependency contexts.  (The closest usage
> of this would be to have many Chronos servers, though this does not work in
> all cases).
>
> The tool is a critical component of Sailthru's data science
> infrastructure, but I believe we are the only people who use the tool right
> now.
>
> If you are interested in learning more, I'm happy to invest time to talk
> more about Stolos, what it does and how we use it!
>
> Alex
>
> On Wed, May 13, 2015 at 2:02 PM Tim Chen <ti...@mesosphere.io> wrote:
>
>> How are you running your batch jobs? Is the batch job script/executable
>> an in-house app?
>>
>> Tim
>>
>> On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <
>> andras.kerekes@ishisystems.com> wrote:
>>
>>> You might want to have a look at stolos too:
>>>
>>>
>>>
>>> https://github.com/sailthru/stolos
>>>
>>>
>>>
>>> Andras
>>>
>>>
>>>
>>>
>>>
>>> *From:* Aaron Carey [mailto:acarey@ilm.com]
>>> *Sent:* Wednesday, May 13, 2015 11:54 AM
>>> *To:* user@mesos.apache.org
>>> *Subject:* RE: Batch Scheduler with dependency support
>>>
>>>
>>>
>>> Thanks! I hadn't come across that one before :)
>>> ------------------------------
>>>
>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
>>> Schroeder [jeffschroeder@computer.org]
>>> *Sent:* 13 May 2015 16:39
>>> *To:* user@mesos.apache.org
>>> *Subject:* Re: Batch Scheduler with dependency support
>>>
>>> Lookup Hubspot's Singularity
>>>
>>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>
>>> Thanks Jeff,
>>>
>>> Any other options around as well?
>>> ------------------------------
>>>
>>> *From:* jeffschroed@gmail.com <http://UrlBlockedError.aspx> [
>>> jeffschroed@gmail.com <http://UrlBlockedError.aspx>] on behalf of Jeff
>>> Schroeder [jeffschroeder@computer.org <http://UrlBlockedError.aspx>]
>>> *Sent:* 13 May 2015 14:12
>>> *To:* user@mesos.apache.org <http://UrlBlockedError.aspx>
>>> *Subject:* Batch Scheduler with dependency support
>>>
>>> It does both just as well, along with cron-like functionality. It is
>>> harder to install and takes a bit more understanding however. The official
>>> tutorial is a process that loops 100 times and then exits.
>>>
>>>
>>>
>>> http://aurora.apache.org/documentation/latest/tutorial/#the-script
>>>
>>> Aurora is pretty much a superset of most other generic frameworks sans
>>> maybe hubspot's singularity.
>>>
>>>
>>> On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com
>>> <http://UrlBlockedError.aspx>> wrote:
>>>
>>> I was under the impression Aurora was for long running services? Is it
>>> suitable for scheduling one of batch processes too?
>>>
>>> thanks,
>>> Aaron
>>> ------------------------------
>>>
>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
>>> Schroeder [jeffschroeder@computer.org]
>>> *Sent:* 13 May 2015 13:12
>>> *To:* user@mesos.apache.org
>>> *Subject:* Re: Batch Scheduler with dependency support
>>>
>>> Apache Aurora does this and you can be explicit about the ordering
>>>
>>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>
>>> Hi All,
>>>
>>> I was just wondering if anyone out there knew of a good mesos batch
>>> scheduler which supports dependencies between tasks? (ie Task B cannot run
>>> until Task A is complete)
>>>
>>> Thanks,
>>> Aaron
>>>
>>>
>>>
>>> --
>>> Text by Jeff, typos by iPhone
>>>
>>>
>>>
>>> --
>>> Text by Jeff, typos by iPhone
>>>
>>>
>>>
>>> --
>>> Text by Jeff, typos by iPhone
>>>
>>
>>
>
>
> --
> Cheers,
> Timothy St. Clair
> Red Hat Inc.
>

Re: Batch Scheduler with dependency support

Posted by Tim St Clair <ts...@redhat.com>.
Hi Alex, 

Have you by chance integrated with any of the tradition batch DAG systems? 

http://pegasus.isi.edu/ , http://ccl.cse.nd.edu/software/makeflow/ 

I keep longing for folks with decades of experience in HTC&HPC to chime in "on-list". 

Subtle nudge ;-) 
Tim 

----- Original Message -----

> From: "Alex Gaudio" <ad...@gmail.com>
> To: user@mesos.apache.org
> Sent: Wednesday, May 13, 2015 3:04:20 PM
> Subject: Re: Batch Scheduler with dependency support

> Hi Tim (and everyone else!),

> I am the primary author of Stolos. We use Stolos to run all of our batch jobs
> on Mesos. The batch jobs are scripts we can run from the command-line.
> Scripts range from bash scripts, Spark jobs and R scripts.

> It's a great tool for us because, unlike Chronos, it lets us define a script
> as stage in a dependency chain, where the script can run with different
> parameters for different dependency contexts. (The closest usage of this
> would be to have many Chronos servers, though this does not work in all
> cases).

> The tool is a critical component of Sailthru's data science infrastructure,
> but I believe we are the only people who use the tool right now.

> If you are interested in learning more, I'm happy to invest time to talk more
> about Stolos, what it does and how we use it!

> Alex

> On Wed, May 13, 2015 at 2:02 PM Tim Chen < tim@mesosphere.io > wrote:

> > How are you running your batch jobs? Is the batch job script/executable an
> > in-house app?
> 

> > Tim
> 

> > On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <
> > andras.kerekes@ishisystems.com > wrote:
> 

> > > You might want to have a look at stolos too:
> > 
> 

> > > https://github.com/sailthru/stolos
> > 
> 

> > > Andras
> > 
> 

> > > From: Aaron Carey [mailto: acarey@ilm.com ]
> > 
> 
> > > Sent: Wednesday, May 13, 2015 11:54 AM
> > 
> 
> > > To: user@mesos.apache.org
> > 
> 
> > > Subject: RE: Batch Scheduler with dependency support
> > 
> 

> > > Thanks! I hadn't come across that one before :)
> > 
> 

> > > From: jeffschroed@gmail.com [ jeffschroed@gmail.com ] on behalf of Jeff
> > > Schroeder [ jeffschroeder@computer.org ]
> > 
> 
> > > Sent: 13 May 2015 16:39
> > 
> 
> > > To: user@mesos.apache.org
> > 
> 
> > > Subject: Re: Batch Scheduler with dependency support
> > 
> 

> > > Lookup Hubspot's Singularity
> > 
> 

> > > On Wednesday, May 13, 2015, Aaron Carey < acarey@ilm.com > wrote:
> > 
> 

> > > Thanks Jeff,
> > 
> 

> > > Any other options around as well?
> > 
> 

> > > From: jeffschroed@gmail.com [ jeffschroed@gmail.com ] on behalf of Jeff
> > > Schroeder [ jeffschroeder@computer.org ]
> > 
> 
> > > Sent: 13 May 2015 14:12
> > 
> 
> > > To: user@mesos.apache.org
> > 
> 
> > > Subject: Batch Scheduler with dependency support
> > 
> 

> > > It does both just as well, along with cron-like functionality. It is
> > > harder
> > > to install and takes a bit more understanding however. The official
> > > tutorial
> > > is a process that loops 100 times and then exits.
> > 
> 

> > > http://aurora.apache.org/documentation/latest/tutorial/#the-script
> > 
> 

> > > Aurora is pretty much a superset of most other generic frameworks sans
> > > maybe
> > > hubspot's singularity.
> > 
> 

> > > On Wednesday, May 13, 2015, Aaron Carey < acarey@ilm.com > wrote:
> > 
> 

> > > I was under the impression Aurora was for long running services? Is it
> > > suitable for scheduling one of batch processes too?
> > 
> 

> > > thanks,
> > 
> 
> > > Aaron
> > 
> 

> > > From: jeffschroed@gmail.com [ jeffschroed@gmail.com ] on behalf of Jeff
> > > Schroeder [ jeffschroeder@computer.org ]
> > 
> 
> > > Sent: 13 May 2015 13:12
> > 
> 
> > > To: user@mesos.apache.org
> > 
> 
> > > Subject: Re: Batch Scheduler with dependency support
> > 
> 

> > > Apache Aurora does this and you can be explicit about the ordering
> > 
> 

> > > On Wednesday, May 13, 2015, Aaron Carey < acarey@ilm.com > wrote:
> > 
> 

> > > Hi All,
> > 
> 

> > > I was just wondering if anyone out there knew of a good mesos batch
> > > scheduler
> > > which supports dependencies between tasks? (ie Task B cannot run until
> > > Task
> > > A is complete)
> > 
> 

> > > Thanks,
> > 
> 
> > > Aaron
> > 
> 

> > > --
> > 
> 
> > > Text by Jeff, typos by iPhone
> > 
> 

> > > --
> > 
> 
> > > Text by Jeff, typos by iPhone
> > 
> 

> > > --
> > 
> 
> > > Text by Jeff, typos by iPhone
> > 
> 

-- 
Cheers, 
Timothy St. Clair 
Red Hat Inc.

Re: Batch Scheduler with dependency support

Posted by Tim Chen <ti...@mesosphere.io>.
Hi Alex,

Thanks for replying and looks like a really interesting framework! I was
orginally aiming the question back to Aaron as he stated he's looking for a
batch job scheduler.

Do you have some rough stats how many jobs or data you guys are computing
with Stolos?

Tim



On Wed, May 13, 2015 at 1:04 PM, Alex Gaudio <ad...@gmail.com> wrote:

> Hi Tim (and everyone else!),
>
> I am the primary author of Stolos.  We use Stolos to run all of our batch
> jobs on Mesos.  The batch jobs are scripts we can run from the
> command-line.  Scripts range from bash scripts, Spark jobs and R scripts.
>
> It's a great tool for us because, unlike Chronos, it lets us define a
> script as stage in a dependency chain, where the script can run with
> different parameters for different dependency contexts.  (The closest usage
> of this would be to have many Chronos servers, though this does not work in
> all cases).
>
> The tool is a critical component of Sailthru's data science
> infrastructure, but I believe we are the only people who use the tool right
> now.
>
> If you are interested in learning more, I'm happy to invest time to talk
> more about Stolos, what it does and how we use it!
>
> Alex
>
> On Wed, May 13, 2015 at 2:02 PM Tim Chen <ti...@mesosphere.io> wrote:
>
>> How are you running your batch jobs? Is the batch job script/executable
>> an in-house app?
>>
>> Tim
>>
>> On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <
>> andras.kerekes@ishisystems.com> wrote:
>>
>>> You might want to have a look at stolos too:
>>>
>>>
>>>
>>> https://github.com/sailthru/stolos
>>>
>>>
>>>
>>> Andras
>>>
>>>
>>>
>>>
>>>
>>> *From:* Aaron Carey [mailto:acarey@ilm.com]
>>> *Sent:* Wednesday, May 13, 2015 11:54 AM
>>> *To:* user@mesos.apache.org
>>> *Subject:* RE: Batch Scheduler with dependency support
>>>
>>>
>>>
>>> Thanks! I hadn't come across that one before :)
>>> ------------------------------
>>>
>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
>>> Schroeder [jeffschroeder@computer.org]
>>> *Sent:* 13 May 2015 16:39
>>> *To:* user@mesos.apache.org
>>> *Subject:* Re: Batch Scheduler with dependency support
>>>
>>> Lookup Hubspot's Singularity
>>>
>>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>
>>> Thanks Jeff,
>>>
>>> Any other options around as well?
>>> ------------------------------
>>>
>>> *From:* jeffschroed@gmail.com <http://UrlBlockedError.aspx> [
>>> jeffschroed@gmail.com <http://UrlBlockedError.aspx>] on behalf of Jeff
>>> Schroeder [jeffschroeder@computer.org <http://UrlBlockedError.aspx>]
>>> *Sent:* 13 May 2015 14:12
>>> *To:* user@mesos.apache.org <http://UrlBlockedError.aspx>
>>> *Subject:* Batch Scheduler with dependency support
>>>
>>> It does both just as well, along with cron-like functionality. It is
>>> harder to install and takes a bit more understanding however. The official
>>> tutorial is a process that loops 100 times and then exits.
>>>
>>>
>>>
>>> http://aurora.apache.org/documentation/latest/tutorial/#the-script
>>>
>>> Aurora is pretty much a superset of most other generic frameworks sans
>>> maybe hubspot's singularity.
>>>
>>>
>>> On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com
>>> <http://UrlBlockedError.aspx>> wrote:
>>>
>>> I was under the impression Aurora was for long running services? Is it
>>> suitable for scheduling one of batch processes too?
>>>
>>> thanks,
>>> Aaron
>>> ------------------------------
>>>
>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
>>> Schroeder [jeffschroeder@computer.org]
>>> *Sent:* 13 May 2015 13:12
>>> *To:* user@mesos.apache.org
>>> *Subject:* Re: Batch Scheduler with dependency support
>>>
>>> Apache Aurora does this and you can be explicit about the ordering
>>>
>>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>
>>> Hi All,
>>>
>>> I was just wondering if anyone out there knew of a good mesos batch
>>> scheduler which supports dependencies between tasks? (ie Task B cannot run
>>> until Task A is complete)
>>>
>>> Thanks,
>>> Aaron
>>>
>>>
>>>
>>> --
>>> Text by Jeff, typos by iPhone
>>>
>>>
>>>
>>> --
>>> Text by Jeff, typos by iPhone
>>>
>>>
>>>
>>> --
>>> Text by Jeff, typos by iPhone
>>>
>>
>>

Re: Batch Scheduler with dependency support

Posted by Alex Gaudio <ad...@gmail.com>.
I should also mention that I run Stolos as a Mesos executor and another
tool I wrote, Relay.Mesos, as the Mesos Scheduler (that spins up Stolos
Executors).

The github links for these tools are:

https://github.com/sailthru/relay.mesos
https://github.com/sailthru/stolos

On Wed, May 13, 2015 at 4:04 PM Alex Gaudio <ad...@gmail.com> wrote:

> Hi Tim (and everyone else!),
>
> I am the primary author of Stolos.  We use Stolos to run all of our batch
> jobs on Mesos.  The batch jobs are scripts we can run from the
> command-line.  Scripts range from bash scripts, Spark jobs and R scripts.
>
> It's a great tool for us because, unlike Chronos, it lets us define a
> script as stage in a dependency chain, where the script can run with
> different parameters for different dependency contexts.  (The closest usage
> of this would be to have many Chronos servers, though this does not work in
> all cases).
>
> The tool is a critical component of Sailthru's data science
> infrastructure, but I believe we are the only people who use the tool right
> now.
>
> If you are interested in learning more, I'm happy to invest time to talk
> more about Stolos, what it does and how we use it!
>
> Alex
>
> On Wed, May 13, 2015 at 2:02 PM Tim Chen <ti...@mesosphere.io> wrote:
>
>> How are you running your batch jobs? Is the batch job script/executable
>> an in-house app?
>>
>> Tim
>>
>> On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <
>> andras.kerekes@ishisystems.com> wrote:
>>
>>> You might want to have a look at stolos too:
>>>
>>>
>>>
>>> https://github.com/sailthru/stolos
>>>
>>>
>>>
>>> Andras
>>>
>>>
>>>
>>>
>>>
>>> *From:* Aaron Carey [mailto:acarey@ilm.com]
>>> *Sent:* Wednesday, May 13, 2015 11:54 AM
>>> *To:* user@mesos.apache.org
>>> *Subject:* RE: Batch Scheduler with dependency support
>>>
>>>
>>>
>>> Thanks! I hadn't come across that one before :)
>>> ------------------------------
>>>
>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
>>> Schroeder [jeffschroeder@computer.org]
>>> *Sent:* 13 May 2015 16:39
>>> *To:* user@mesos.apache.org
>>> *Subject:* Re: Batch Scheduler with dependency support
>>>
>>> Lookup Hubspot's Singularity
>>>
>>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>
>>> Thanks Jeff,
>>>
>>> Any other options around as well?
>>> ------------------------------
>>>
>>> *From:* jeffschroed@gmail.com <http://UrlBlockedError.aspx> [
>>> jeffschroed@gmail.com <http://UrlBlockedError.aspx>] on behalf of Jeff
>>> Schroeder [jeffschroeder@computer.org <http://UrlBlockedError.aspx>]
>>> *Sent:* 13 May 2015 14:12
>>> *To:* user@mesos.apache.org <http://UrlBlockedError.aspx>
>>> *Subject:* Batch Scheduler with dependency support
>>>
>>> It does both just as well, along with cron-like functionality. It is
>>> harder to install and takes a bit more understanding however. The official
>>> tutorial is a process that loops 100 times and then exits.
>>>
>>>
>>>
>>> http://aurora.apache.org/documentation/latest/tutorial/#the-script
>>>
>>> Aurora is pretty much a superset of most other generic frameworks sans
>>> maybe hubspot's singularity.
>>>
>>>
>>> On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com
>>> <http://UrlBlockedError.aspx>> wrote:
>>>
>>> I was under the impression Aurora was for long running services? Is it
>>> suitable for scheduling one of batch processes too?
>>>
>>> thanks,
>>> Aaron
>>> ------------------------------
>>>
>>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
>>> Schroeder [jeffschroeder@computer.org]
>>> *Sent:* 13 May 2015 13:12
>>> *To:* user@mesos.apache.org
>>> *Subject:* Re: Batch Scheduler with dependency support
>>>
>>> Apache Aurora does this and you can be explicit about the ordering
>>>
>>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>>
>>> Hi All,
>>>
>>> I was just wondering if anyone out there knew of a good mesos batch
>>> scheduler which supports dependencies between tasks? (ie Task B cannot run
>>> until Task A is complete)
>>>
>>> Thanks,
>>> Aaron
>>>
>>>
>>>
>>> --
>>> Text by Jeff, typos by iPhone
>>>
>>>
>>>
>>> --
>>> Text by Jeff, typos by iPhone
>>>
>>>
>>>
>>> --
>>> Text by Jeff, typos by iPhone
>>>
>>
>>

Re: Batch Scheduler with dependency support

Posted by Alex Gaudio <ad...@gmail.com>.
Hi Tim (and everyone else!),

I am the primary author of Stolos.  We use Stolos to run all of our batch
jobs on Mesos.  The batch jobs are scripts we can run from the
command-line.  Scripts range from bash scripts, Spark jobs and R scripts.

It's a great tool for us because, unlike Chronos, it lets us define a
script as stage in a dependency chain, where the script can run with
different parameters for different dependency contexts.  (The closest usage
of this would be to have many Chronos servers, though this does not work in
all cases).

The tool is a critical component of Sailthru's data science infrastructure,
but I believe we are the only people who use the tool right now.

If you are interested in learning more, I'm happy to invest time to talk
more about Stolos, what it does and how we use it!

Alex

On Wed, May 13, 2015 at 2:02 PM Tim Chen <ti...@mesosphere.io> wrote:

> How are you running your batch jobs? Is the batch job script/executable an
> in-house app?
>
> Tim
>
> On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <
> andras.kerekes@ishisystems.com> wrote:
>
>> You might want to have a look at stolos too:
>>
>>
>>
>> https://github.com/sailthru/stolos
>>
>>
>>
>> Andras
>>
>>
>>
>>
>>
>> *From:* Aaron Carey [mailto:acarey@ilm.com]
>> *Sent:* Wednesday, May 13, 2015 11:54 AM
>> *To:* user@mesos.apache.org
>> *Subject:* RE: Batch Scheduler with dependency support
>>
>>
>>
>> Thanks! I hadn't come across that one before :)
>> ------------------------------
>>
>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
>> Schroeder [jeffschroeder@computer.org]
>> *Sent:* 13 May 2015 16:39
>> *To:* user@mesos.apache.org
>> *Subject:* Re: Batch Scheduler with dependency support
>>
>> Lookup Hubspot's Singularity
>>
>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>
>> Thanks Jeff,
>>
>> Any other options around as well?
>> ------------------------------
>>
>> *From:* jeffschroed@gmail.com <http://UrlBlockedError.aspx> [
>> jeffschroed@gmail.com <http://UrlBlockedError.aspx>] on behalf of Jeff
>> Schroeder [jeffschroeder@computer.org <http://UrlBlockedError.aspx>]
>> *Sent:* 13 May 2015 14:12
>> *To:* user@mesos.apache.org <http://UrlBlockedError.aspx>
>> *Subject:* Batch Scheduler with dependency support
>>
>> It does both just as well, along with cron-like functionality. It is
>> harder to install and takes a bit more understanding however. The official
>> tutorial is a process that loops 100 times and then exits.
>>
>>
>>
>> http://aurora.apache.org/documentation/latest/tutorial/#the-script
>>
>> Aurora is pretty much a superset of most other generic frameworks sans
>> maybe hubspot's singularity.
>>
>>
>> On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com
>> <http://UrlBlockedError.aspx>> wrote:
>>
>> I was under the impression Aurora was for long running services? Is it
>> suitable for scheduling one of batch processes too?
>>
>> thanks,
>> Aaron
>> ------------------------------
>>
>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
>> Schroeder [jeffschroeder@computer.org]
>> *Sent:* 13 May 2015 13:12
>> *To:* user@mesos.apache.org
>> *Subject:* Re: Batch Scheduler with dependency support
>>
>> Apache Aurora does this and you can be explicit about the ordering
>>
>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>
>> Hi All,
>>
>> I was just wondering if anyone out there knew of a good mesos batch
>> scheduler which supports dependencies between tasks? (ie Task B cannot run
>> until Task A is complete)
>>
>> Thanks,
>> Aaron
>>
>>
>>
>> --
>> Text by Jeff, typos by iPhone
>>
>>
>>
>> --
>> Text by Jeff, typos by iPhone
>>
>>
>>
>> --
>> Text by Jeff, typos by iPhone
>>
>
>

RE: Batch Scheduler with dependency support

Posted by Aaron Carey <ac...@ilm.com>.
It varies, some inhouse apps, some commerical software. Although we're wrapping everything in a docker container for consistency.

There are usually multiple steps for each task, and these steps can often be subdivided into multiple parallel processes. Different steps require different executables though.

Aaron

________________________________
From: Tim Chen [tim@mesosphere.io]
Sent: 13 May 2015 19:01
To: user@mesos.apache.org
Subject: Re: Batch Scheduler with dependency support

How are you running your batch jobs? Is the batch job script/executable an in-house app?

Tim

On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <an...@ishisystems.com>> wrote:
You might want to have a look at stolos too:

https://github.com/sailthru/stolos

Andras


From: Aaron Carey [mailto:acarey@ilm.com<ma...@ilm.com>]
Sent: Wednesday, May 13, 2015 11:54 AM
To: user@mesos.apache.org<ma...@mesos.apache.org>
Subject: RE: Batch Scheduler with dependency support

Thanks! I hadn't come across that one before :)
________________________________
From: jeffschroed@gmail.com<ma...@gmail.com> [jeffschroed@gmail.com<ma...@gmail.com>] on behalf of Jeff Schroeder [jeffschroeder@computer.org<ma...@computer.org>]
Sent: 13 May 2015 16:39
To: user@mesos.apache.org<ma...@mesos.apache.org>
Subject: Re: Batch Scheduler with dependency support
Lookup Hubspot's Singularity

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com>> wrote:
Thanks Jeff,

Any other options around as well?
________________________________
From: jeffschroed@gmail.com<http://UrlBlockedError.aspx> [jeffschroed@gmail.com<http://UrlBlockedError.aspx>] on behalf of Jeff Schroeder [jeffschroeder@computer.org<http://UrlBlockedError.aspx>]
Sent: 13 May 2015 14:12
To: user@mesos.apache.org<http://UrlBlockedError.aspx>
Subject: Batch Scheduler with dependency support
It does both just as well, along with cron-like functionality. It is harder to install and takes a bit more understanding however. The official tutorial is a process that loops 100 times and then exits.

http://aurora.apache.org/documentation/latest/tutorial/#the-script

Aurora is pretty much a superset of most other generic frameworks sans maybe hubspot's singularity.

On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com<http://UrlBlockedError.aspx>> wrote:
I was under the impression Aurora was for long running services? Is it suitable for scheduling one of batch processes too?

thanks,
Aaron
________________________________
From: jeffschroed@gmail.com<ma...@gmail.com> [jeffschroed@gmail.com<ma...@gmail.com>] on behalf of Jeff Schroeder [jeffschroeder@computer.org<ma...@computer.org>]
Sent: 13 May 2015 13:12
To: user@mesos.apache.org<ma...@mesos.apache.org>
Subject: Re: Batch Scheduler with dependency support
Apache Aurora does this and you can be explicit about the ordering

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com>> wrote:
Hi All,

I was just wondering if anyone out there knew of a good mesos batch scheduler which supports dependencies between tasks? (ie Task B cannot run until Task A is complete)

Thanks,
Aaron


--
Text by Jeff, typos by iPhone


--
Text by Jeff, typos by iPhone


--
Text by Jeff, typos by iPhone

Re: Batch Scheduler with dependency support

Posted by Tim Chen <ti...@mesosphere.io>.
How are you running your batch jobs? Is the batch job script/executable an
in-house app?

Tim

On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <
andras.kerekes@ishisystems.com> wrote:

> You might want to have a look at stolos too:
>
>
>
> https://github.com/sailthru/stolos
>
>
>
> Andras
>
>
>
>
>
> *From:* Aaron Carey [mailto:acarey@ilm.com]
> *Sent:* Wednesday, May 13, 2015 11:54 AM
> *To:* user@mesos.apache.org
> *Subject:* RE: Batch Scheduler with dependency support
>
>
>
> Thanks! I hadn't come across that one before :)
> ------------------------------
>
> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
> Schroeder [jeffschroeder@computer.org]
> *Sent:* 13 May 2015 16:39
> *To:* user@mesos.apache.org
> *Subject:* Re: Batch Scheduler with dependency support
>
> Lookup Hubspot's Singularity
>
> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>
> Thanks Jeff,
>
> Any other options around as well?
> ------------------------------
>
> *From:* jeffschroed@gmail.com <http://UrlBlockedError.aspx> [
> jeffschroed@gmail.com <http://UrlBlockedError.aspx>] on behalf of Jeff
> Schroeder [jeffschroeder@computer.org <http://UrlBlockedError.aspx>]
> *Sent:* 13 May 2015 14:12
> *To:* user@mesos.apache.org <http://UrlBlockedError.aspx>
> *Subject:* Batch Scheduler with dependency support
>
> It does both just as well, along with cron-like functionality. It is
> harder to install and takes a bit more understanding however. The official
> tutorial is a process that loops 100 times and then exits.
>
>
>
> http://aurora.apache.org/documentation/latest/tutorial/#the-script
>
> Aurora is pretty much a superset of most other generic frameworks sans
> maybe hubspot's singularity.
>
>
> On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com
> <http://UrlBlockedError.aspx>> wrote:
>
> I was under the impression Aurora was for long running services? Is it
> suitable for scheduling one of batch processes too?
>
> thanks,
> Aaron
> ------------------------------
>
> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
> Schroeder [jeffschroeder@computer.org]
> *Sent:* 13 May 2015 13:12
> *To:* user@mesos.apache.org
> *Subject:* Re: Batch Scheduler with dependency support
>
> Apache Aurora does this and you can be explicit about the ordering
>
> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>
> Hi All,
>
> I was just wondering if anyone out there knew of a good mesos batch
> scheduler which supports dependencies between tasks? (ie Task B cannot run
> until Task A is complete)
>
> Thanks,
> Aaron
>
>
>
> --
> Text by Jeff, typos by iPhone
>
>
>
> --
> Text by Jeff, typos by iPhone
>
>
>
> --
> Text by Jeff, typos by iPhone
>

RE: Batch Scheduler with dependency support

Posted by Aaron Carey <ac...@ilm.com>.
Thanks Andras!

This is very interesting, it comes quite close to what we're looking for,

Thanks,
Aaron

________________________________
From: Andras Kerekes [andras.kerekes@ishisystems.com]
Sent: 13 May 2015 17:46
To: user@mesos.apache.org
Subject: RE: Batch Scheduler with dependency support

You might want to have a look at stolos too:

https://github.com/sailthru/stolos

Andras


From: Aaron Carey [mailto:acarey@ilm.com]
Sent: Wednesday, May 13, 2015 11:54 AM
To: user@mesos.apache.org
Subject: RE: Batch Scheduler with dependency support

Thanks! I hadn't come across that one before :)
________________________________
From: jeffschroed@gmail.com<ma...@gmail.com> [jeffschroed@gmail.com] on behalf of Jeff Schroeder [jeffschroeder@computer.org]
Sent: 13 May 2015 16:39
To: user@mesos.apache.org<ma...@mesos.apache.org>
Subject: Re: Batch Scheduler with dependency support
Lookup Hubspot's Singularity

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com>> wrote:
Thanks Jeff,

Any other options around as well?
________________________________
From: jeffschroed@gmail.com<UrlBlockedError.aspx> [jeffschroed@gmail.com<UrlBlockedError.aspx>] on behalf of Jeff Schroeder [jeffschroeder@computer.org<UrlBlockedError.aspx>]
Sent: 13 May 2015 14:12
To: user@mesos.apache.org<UrlBlockedError.aspx>
Subject: Batch Scheduler with dependency support
It does both just as well, along with cron-like functionality. It is harder to install and takes a bit more understanding however. The official tutorial is a process that loops 100 times and then exits.

http://aurora.apache.org/documentation/latest/tutorial/#the-script

Aurora is pretty much a superset of most other generic frameworks sans maybe hubspot's singularity.

On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com<http://UrlBlockedError.aspx>> wrote:
I was under the impression Aurora was for long running services? Is it suitable for scheduling one of batch processes too?

thanks,
Aaron
________________________________
From: jeffschroed@gmail.com<ma...@gmail.com> [jeffschroed@gmail.com] on behalf of Jeff Schroeder [jeffschroeder@computer.org]
Sent: 13 May 2015 13:12
To: user@mesos.apache.org<ma...@mesos.apache.org>
Subject: Re: Batch Scheduler with dependency support
Apache Aurora does this and you can be explicit about the ordering

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com>> wrote:
Hi All,

I was just wondering if anyone out there knew of a good mesos batch scheduler which supports dependencies between tasks? (ie Task B cannot run until Task A is complete)

Thanks,
Aaron


--
Text by Jeff, typos by iPhone


--
Text by Jeff, typos by iPhone


--
Text by Jeff, typos by iPhone

RE: Batch Scheduler with dependency support

Posted by Andras Kerekes <an...@ishisystems.com>.
You might want to have a look at stolos too:

 

https://github.com/sailthru/stolos

 

Andras

 

 

From: Aaron Carey [mailto:acarey@ilm.com] 
Sent: Wednesday, May 13, 2015 11:54 AM
To: user@mesos.apache.org
Subject: RE: Batch Scheduler with dependency support

 

Thanks! I hadn't come across that one before :)

  _____  

From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
Schroeder [jeffschroeder@computer.org]
Sent: 13 May 2015 16:39
To: user@mesos.apache.org
Subject: Re: Batch Scheduler with dependency support

Lookup Hubspot's Singularity

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:

Thanks Jeff,

Any other options around as well?

  _____  

From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
Schroeder [jeffschroeder@computer.org]
Sent: 13 May 2015 14:12
To: user@mesos.apache.org
Subject: Batch Scheduler with dependency support

It does both just as well, along with cron-like functionality. It is harder
to install and takes a bit more understanding however. The official tutorial
is a process that loops 100 times and then exits. 

 

http://aurora.apache.org/documentation/latest/tutorial/#the-script

Aurora is pretty much a superset of most other generic frameworks sans maybe
hubspot's singularity.


On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com
<http://UrlBlockedError.aspx> > wrote:

I was under the impression Aurora was for long running services? Is it
suitable for scheduling one of batch processes too?

thanks,
Aaron

  _____  

From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
Schroeder [jeffschroeder@computer.org]
Sent: 13 May 2015 13:12
To: user@mesos.apache.org
Subject: Re: Batch Scheduler with dependency support

Apache Aurora does this and you can be explicit about the ordering 

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:

Hi All,

I was just wondering if anyone out there knew of a good mesos batch
scheduler which supports dependencies between tasks? (ie Task B cannot run
until Task A is complete)

Thanks,
Aaron 



-- 
Text by Jeff, typos by iPhone



-- 
Text by Jeff, typos by iPhone



-- 
Text by Jeff, typos by iPhone

RE: Batch Scheduler with dependency support

Posted by Aaron Carey <ac...@ilm.com>.
Thanks! I hadn't come across that one before :)

________________________________
From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff Schroeder [jeffschroeder@computer.org]
Sent: 13 May 2015 16:39
To: user@mesos.apache.org
Subject: Re: Batch Scheduler with dependency support

Lookup Hubspot's Singularity

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com>> wrote:
Thanks Jeff,

Any other options around as well?

________________________________
From: jeffschroed@gmail.com<UrlBlockedError.aspx> [jeffschroed@gmail.com<UrlBlockedError.aspx>] on behalf of Jeff Schroeder [jeffschroeder@computer.org<UrlBlockedError.aspx>]
Sent: 13 May 2015 14:12
To: user@mesos.apache.org<UrlBlockedError.aspx>
Subject: Batch Scheduler with dependency support

It does both just as well, along with cron-like functionality. It is harder to install and takes a bit more understanding however. The official tutorial is a process that loops 100 times and then exits.

http://aurora.apache.org/documentation/latest/tutorial/#the-script

Aurora is pretty much a superset of most other generic frameworks sans maybe hubspot's singularity.

On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com<http://UrlBlockedError.aspx>> wrote:
I was under the impression Aurora was for long running services? Is it suitable for scheduling one of batch processes too?

thanks,
Aaron

________________________________
From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff Schroeder [jeffschroeder@computer.org]
Sent: 13 May 2015 13:12
To: user@mesos.apache.org
Subject: Re: Batch Scheduler with dependency support

Apache Aurora does this and you can be explicit about the ordering

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
Hi All,

I was just wondering if anyone out there knew of a good mesos batch scheduler which supports dependencies between tasks? (ie Task B cannot run until Task A is complete)

Thanks,
Aaron


--
Text by Jeff, typos by iPhone


--
Text by Jeff, typos by iPhone


--
Text by Jeff, typos by iPhone

Re: Batch Scheduler with dependency support

Posted by Jeff Schroeder <je...@computer.org>.
Lookup Hubspot's Singularity

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:

>  Thanks Jeff,
>
> Any other options around as well?
>
>  ------------------------------
> *From:* jeffschroed@gmail.com
> <javascript:_e(%7B%7D,'cvml','jeffschroed@gmail.com');> [
> jeffschroed@gmail.com
> <javascript:_e(%7B%7D,'cvml','jeffschroed@gmail.com');>] on behalf of
> Jeff Schroeder [jeffschroeder@computer.org
> <javascript:_e(%7B%7D,'cvml','jeffschroeder@computer.org');>]
> *Sent:* 13 May 2015 14:12
> *To:* user@mesos.apache.org
> <javascript:_e(%7B%7D,'cvml','user@mesos.apache.org');>
> *Subject:* Batch Scheduler with dependency support
>
>  It does both just as well, along with cron-like functionality. It is
> harder to install and takes a bit more understanding however. The official
> tutorial is a process that loops 100 times and then exits.
>
>  http://aurora.apache.org/documentation/latest/tutorial/#the-script
>
> Aurora is pretty much a superset of most other generic frameworks sans
> maybe hubspot's singularity.
>
> On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com
> <http://UrlBlockedError.aspx>> wrote:
>
>>  I was under the impression Aurora was for long running services? Is it
>> suitable for scheduling one of batch processes too?
>>
>> thanks,
>> Aaron
>>
>>  ------------------------------
>> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
>> Schroeder [jeffschroeder@computer.org]
>> *Sent:* 13 May 2015 13:12
>> *To:* user@mesos.apache.org
>> *Subject:* Re: Batch Scheduler with dependency support
>>
>>  Apache Aurora does this and you can be explicit about the ordering
>>
>> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>>
>>>  Hi All,
>>>
>>> I was just wondering if anyone out there knew of a good mesos batch
>>> scheduler which supports dependencies between tasks? (ie Task B cannot run
>>> until Task A is complete)
>>>
>>> Thanks,
>>> Aaron
>>>
>>
>>
>> --
>> Text by Jeff, typos by iPhone
>>
>
>
> --
> Text by Jeff, typos by iPhone
>


-- 
Text by Jeff, typos by iPhone

RE: Batch Scheduler with dependency support

Posted by Aaron Carey <ac...@ilm.com>.
Thanks Jeff,

Any other options around as well?

________________________________
From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff Schroeder [jeffschroeder@computer.org]
Sent: 13 May 2015 14:12
To: user@mesos.apache.org
Subject: Batch Scheduler with dependency support

It does both just as well, along with cron-like functionality. It is harder to install and takes a bit more understanding however. The official tutorial is a process that loops 100 times and then exits.

http://aurora.apache.org/documentation/latest/tutorial/#the-script

Aurora is pretty much a superset of most other generic frameworks sans maybe hubspot's singularity.

On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com<UrlBlockedError.aspx>> wrote:
I was under the impression Aurora was for long running services? Is it suitable for scheduling one of batch processes too?

thanks,
Aaron

________________________________
From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff Schroeder [jeffschroeder@computer.org]
Sent: 13 May 2015 13:12
To: user@mesos.apache.org
Subject: Re: Batch Scheduler with dependency support

Apache Aurora does this and you can be explicit about the ordering

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
Hi All,

I was just wondering if anyone out there knew of a good mesos batch scheduler which supports dependencies between tasks? (ie Task B cannot run until Task A is complete)

Thanks,
Aaron


--
Text by Jeff, typos by iPhone


--
Text by Jeff, typos by iPhone

Batch Scheduler with dependency support

Posted by Jeff Schroeder <je...@computer.org>.
It does both just as well, along with cron-like functionality. It is harder
to install and takes a bit more understanding however. The official
tutorial is a process that loops 100 times and then exits.

http://aurora.apache.org/documentation/latest/tutorial/#the-script

Aurora is pretty much a superset of most other generic frameworks sans
maybe hubspot's singularity.

On Wednesday, May 13, 2015, Aaron Carey <acarey@ilm.com
<javascript:_e(%7B%7D,'cvml','acarey@ilm.com');>> wrote:

>  I was under the impression Aurora was for long running services? Is it
> suitable for scheduling one of batch processes too?
>
> thanks,
> Aaron
>
>  ------------------------------
> *From:* jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff
> Schroeder [jeffschroeder@computer.org]
> *Sent:* 13 May 2015 13:12
> *To:* user@mesos.apache.org
> *Subject:* Re: Batch Scheduler with dependency support
>
>  Apache Aurora does this and you can be explicit about the ordering
>
> On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:
>
>>  Hi All,
>>
>> I was just wondering if anyone out there knew of a good mesos batch
>> scheduler which supports dependencies between tasks? (ie Task B cannot run
>> until Task A is complete)
>>
>> Thanks,
>> Aaron
>>
>
>
> --
> Text by Jeff, typos by iPhone
>


-- 
Text by Jeff, typos by iPhone

RE: Batch Scheduler with dependency support

Posted by Aaron Carey <ac...@ilm.com>.
I was under the impression Aurora was for long running services? Is it suitable for scheduling one of batch processes too?

thanks,
Aaron

________________________________
From: jeffschroed@gmail.com [jeffschroed@gmail.com] on behalf of Jeff Schroeder [jeffschroeder@computer.org]
Sent: 13 May 2015 13:12
To: user@mesos.apache.org
Subject: Re: Batch Scheduler with dependency support

Apache Aurora does this and you can be explicit about the ordering

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com>> wrote:
Hi All,

I was just wondering if anyone out there knew of a good mesos batch scheduler which supports dependencies between tasks? (ie Task B cannot run until Task A is complete)

Thanks,
Aaron


--
Text by Jeff, typos by iPhone

Re: Batch Scheduler with dependency support

Posted by Jeff Schroeder <je...@computer.org>.
Apache Aurora does this and you can be explicit about the ordering

On Wednesday, May 13, 2015, Aaron Carey <ac...@ilm.com> wrote:

>  Hi All,
>
> I was just wondering if anyone out there knew of a good mesos batch
> scheduler which supports dependencies between tasks? (ie Task B cannot run
> until Task A is complete)
>
> Thanks,
> Aaron
>


-- 
Text by Jeff, typos by iPhone

Re: Batch Scheduler with dependency support

Posted by craig mcmillan <mc...@gmail.com>.
ah, sorry, no : i'm conflating the two ... we run all our batch jobs on 
a schedule

:c

On 13 May 2015, at 12:28, Aaron Carey wrote:

> Have you been using it for batch tasks? It seems great for time 
> dependent tasks but I wasn't aware it could do batch as well?
>
> Thanks!
> Aaron
> ________________________________________
> From: craig mcmillan [mccraigmccraig@gmail.com]
> Sent: 13 May 2015 12:09
> To: user@mesos.apache.org
> Subject: Re: Batch Scheduler with dependency support
>
> i've been using https://github.com/mesos/chronos
>
> :c
>
> On 13 May 2015, at 12:06, Aaron Carey wrote:
>
>> Hi All,
>>
>> I was just wondering if anyone out there knew of a good mesos batch
>> scheduler which supports dependencies between tasks? (ie Task B 
>> cannot
>> run until Task A is complete)
>>
>> Thanks,
>> Aaron

RE: Batch Scheduler with dependency support

Posted by Aaron Carey <ac...@ilm.com>.
Have you been using it for batch tasks? It seems great for time dependent tasks but I wasn't aware it could do batch as well?

Thanks!
Aaron
________________________________________
From: craig mcmillan [mccraigmccraig@gmail.com]
Sent: 13 May 2015 12:09
To: user@mesos.apache.org
Subject: Re: Batch Scheduler with dependency support

i've been using https://github.com/mesos/chronos

:c

On 13 May 2015, at 12:06, Aaron Carey wrote:

> Hi All,
>
> I was just wondering if anyone out there knew of a good mesos batch
> scheduler which supports dependencies between tasks? (ie Task B cannot
> run until Task A is complete)
>
> Thanks,
> Aaron

Re: Batch Scheduler with dependency support

Posted by craig mcmillan <mc...@gmail.com>.
i've been using https://github.com/mesos/chronos

:c

On 13 May 2015, at 12:06, Aaron Carey wrote:

> Hi All,
>
> I was just wondering if anyone out there knew of a good mesos batch 
> scheduler which supports dependencies between tasks? (ie Task B cannot 
> run until Task A is complete)
>
> Thanks,
> Aaron