You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tez.apache.org by Jonathan Eagles <je...@gmail.com> on 2017/03/27 22:49:29 UTC
[DISCUSS] Merge TEZ-3334 to master
At this time, it may be advantageous to move the Tez Shuffle Handler
development and maintenance to the master line. This would allow the tez
community to make this feature available to a wider audience. I would like
to get community feedback on proposing a vote to merge the TEZ-3334 branch
to the master line. I can say that I have tested this extensively and for
auto-reduce parallelism jobs, this reduces the shuffle time dramatically on
highly reduced jobs.
A list of features can be found on the umbrella JIRA
https://issues.apache.org/jira/browse/TEZ-3334
In addition I have made a patch file available to see the 423KB of
differences so we can have a discussion over the changes.
Thoughts?
jeagles
Re: [DISCUSS] Merge TEZ-3334 to master
Posted by Jonathan Eagles <je...@gmail.com>.
Having address the critical feedback given on the TEZ-3334 jira, will start
a vote thread.
On Wed, Mar 29, 2017 at 2:33 PM, Siddharth Seth <ss...@apache.org> wrote:
> Makes sense to get the merge process started. Would be nice to have a
> little more time to look at the changes, and provide feedback. Targeting
> end of the week. Hopefully before that.
>
> On Wed, Mar 29, 2017 at 12:23 PM, Jason Lowe <jl...@yahoo-inc.com.invalid>
> wrote:
>
> > +1 for merging the new shuffle handler to master.
> > Jason
> >
> >
> > On Wednesday, March 29, 2017 9:41 AM, Jonathan Eagles <
> > jeagles@gmail.com> wrote:
> >
> >
> > Folks, let's have all feedback by EOB Thursday, and I'll start the VOTE
> on
> > Friday if that is the direction the community is heading.
> >
> > jeagles
> >
> > On Mon, Mar 27, 2017 at 5:49 PM, Jonathan Eagles <je...@gmail.com>
> > wrote:
> >
> > > At this time, it may be advantageous to move the Tez Shuffle Handler
> > > development and maintenance to the master line. This would allow the
> tez
> > > community to make this feature available to a wider audience. I would
> > like
> > > to get community feedback on proposing a vote to merge the TEZ-3334
> > branch
> > > to the master line. I can say that I have tested this extensively and
> for
> > > auto-reduce parallelism jobs, this reduces the shuffle time
> dramatically
> > on
> > > highly reduced jobs.
> > >
> > > A list of features can be found on the umbrella JIRA
> > > https://issues.apache.org/jira/browse/TEZ-3334
> > > In addition I have made a patch file available to see the 423KB of
> > > differences so we can have a discussion over the changes.
> > >
> > > Thoughts?
> > > jeagles
> > >
> >
> >
> >
> >
>
Re: [DISCUSS] Merge TEZ-3334 to master
Posted by Siddharth Seth <ss...@apache.org>.
Makes sense to get the merge process started. Would be nice to have a
little more time to look at the changes, and provide feedback. Targeting
end of the week. Hopefully before that.
On Wed, Mar 29, 2017 at 12:23 PM, Jason Lowe <jl...@yahoo-inc.com.invalid>
wrote:
> +1 for merging the new shuffle handler to master.
> Jason
>
>
> On Wednesday, March 29, 2017 9:41 AM, Jonathan Eagles <
> jeagles@gmail.com> wrote:
>
>
> Folks, let's have all feedback by EOB Thursday, and I'll start the VOTE on
> Friday if that is the direction the community is heading.
>
> jeagles
>
> On Mon, Mar 27, 2017 at 5:49 PM, Jonathan Eagles <je...@gmail.com>
> wrote:
>
> > At this time, it may be advantageous to move the Tez Shuffle Handler
> > development and maintenance to the master line. This would allow the tez
> > community to make this feature available to a wider audience. I would
> like
> > to get community feedback on proposing a vote to merge the TEZ-3334
> branch
> > to the master line. I can say that I have tested this extensively and for
> > auto-reduce parallelism jobs, this reduces the shuffle time dramatically
> on
> > highly reduced jobs.
> >
> > A list of features can be found on the umbrella JIRA
> > https://issues.apache.org/jira/browse/TEZ-3334
> > In addition I have made a patch file available to see the 423KB of
> > differences so we can have a discussion over the changes.
> >
> > Thoughts?
> > jeagles
> >
>
>
>
>
Re: [DISCUSS] Merge TEZ-3334 to master
Posted by Jason Lowe <jl...@yahoo-inc.com.INVALID>.
+1 for merging the new shuffle handler to master.
Jason
On Wednesday, March 29, 2017 9:41 AM, Jonathan Eagles <je...@gmail.com> wrote:
Folks, let's have all feedback by EOB Thursday, and I'll start the VOTE on
Friday if that is the direction the community is heading.
jeagles
On Mon, Mar 27, 2017 at 5:49 PM, Jonathan Eagles <je...@gmail.com> wrote:
> At this time, it may be advantageous to move the Tez Shuffle Handler
> development and maintenance to the master line. This would allow the tez
> community to make this feature available to a wider audience. I would like
> to get community feedback on proposing a vote to merge the TEZ-3334 branch
> to the master line. I can say that I have tested this extensively and for
> auto-reduce parallelism jobs, this reduces the shuffle time dramatically on
> highly reduced jobs.
>
> A list of features can be found on the umbrella JIRA
> https://issues.apache.org/jira/browse/TEZ-3334
> In addition I have made a patch file available to see the 423KB of
> differences so we can have a discussion over the changes.
>
> Thoughts?
> jeagles
>
Re: [DISCUSS] Merge TEZ-3334 to master
Posted by Jonathan Eagles <je...@gmail.com>.
Folks, let's have all feedback by EOB Thursday, and I'll start the VOTE on
Friday if that is the direction the community is heading.
jeagles
On Mon, Mar 27, 2017 at 5:49 PM, Jonathan Eagles <je...@gmail.com> wrote:
> At this time, it may be advantageous to move the Tez Shuffle Handler
> development and maintenance to the master line. This would allow the tez
> community to make this feature available to a wider audience. I would like
> to get community feedback on proposing a vote to merge the TEZ-3334 branch
> to the master line. I can say that I have tested this extensively and for
> auto-reduce parallelism jobs, this reduces the shuffle time dramatically on
> highly reduced jobs.
>
> A list of features can be found on the umbrella JIRA
> https://issues.apache.org/jira/browse/TEZ-3334
> In addition I have made a patch file available to see the 423KB of
> differences so we can have a discussion over the changes.
>
> Thoughts?
> jeagles
>