You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by 萝卜丝炒饭 <14...@qq.com> on 2017/06/06 12:04:26 UTC
a stage can belong to more than one job please?
Hi all,
I read same code of spark about stage.
The constructor of stage keep the first job ID the stage was part of.
does that means a stage can belong to more than one job please? And I find the member jobIds is never used. It looks strange.
thanks adv
Re: a stage can belong to more than one job please?
Posted by 萝卜丝炒饭 <14...@qq.com>.
Hi Mark,
Thanks.
---Original---
From: "Mark Hamstra"<ma...@clearstorydata.com>
Date: 2017/6/6 23:27:43
To: "dev"<de...@spark.apache.org>;
Cc: "user"<us...@spark.apache.org>;
Subject: Re: a stage can belong to more than one job please?
Yes, a Stage can be part of more than one Job. The jobIds field of Stage is used repeatedly in the DAGScheduler.
On Tue, Jun 6, 2017 at 5:04 AM, 萝卜丝炒饭 <14...@qq.com> wrote:
Hi all,
I read same code of spark about stage.
The constructor of stage keep the first job ID the stage was part of.
does that means a stage can belong to more than one job please? And I find the member jobIds is never used. It looks strange.
thanks adv
Re: a stage can belong to more than one job please?
Posted by 萝卜丝炒饭 <14...@qq.com>.
Hi Mark,
Thanks.
---Original---
From: "Mark Hamstra"<ma...@clearstorydata.com>
Date: 2017/6/6 23:27:43
To: "dev"<de...@spark.apache.org>;
Cc: "user"<us...@spark.apache.org>;
Subject: Re: a stage can belong to more than one job please?
Yes, a Stage can be part of more than one Job. The jobIds field of Stage is used repeatedly in the DAGScheduler.
On Tue, Jun 6, 2017 at 5:04 AM, 萝卜丝炒饭 <14...@qq.com> wrote:
Hi all,
I read same code of spark about stage.
The constructor of stage keep the first job ID the stage was part of.
does that means a stage can belong to more than one job please? And I find the member jobIds is never used. It looks strange.
thanks adv
Re: a stage can belong to more than one job please?
Posted by Mark Hamstra <ma...@clearstorydata.com>.
Yes, a Stage can be part of more than one Job. The jobIds field of Stage is
used repeatedly in the DAGScheduler.
On Tue, Jun 6, 2017 at 5:04 AM, 萝卜丝炒饭 <14...@qq.com> wrote:
> Hi all,
>
> I read same code of spark about stage.
>
> The constructor of stage keep the first job ID the stage was part of.
> does that means a stage can belong to more than one job
> please? And I find the member jobIds is never used. It looks strange.
>
>
> thanks adv
>
Re: a stage can belong to more than one job please?
Posted by Mark Hamstra <ma...@clearstorydata.com>.
Yes, a Stage can be part of more than one Job. The jobIds field of Stage is
used repeatedly in the DAGScheduler.
On Tue, Jun 6, 2017 at 5:04 AM, 萝卜丝炒饭 <14...@qq.com> wrote:
> Hi all,
>
> I read same code of spark about stage.
>
> The constructor of stage keep the first job ID the stage was part of.
> does that means a stage can belong to more than one job
> please? And I find the member jobIds is never used. It looks strange.
>
>
> thanks adv
>