You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Jitendra Pandey <ji...@hortonworks.com> on 2013/09/06 22:44:11 UTC
Merge vectorization branch to trunk
Hi Folks,
Vectorized query execution work has been under development for around 6
months now. We have made good progress in making basic datatypes and
queries to work in vectorized mode and also we have laid sufficient
groundwork so that any future development can happen directly on trunk.
This is a community effort involving several developers including Eric
Hanson, Remus Rusanu, Sarvesh
Sakalanaga<https://issues.apache.org/jira/secure/ViewProfile.jspa?name=sarvesh.sn>,
Gopal V, Teddy Choi, Tony Murphy, Timothy Chen, Prashanth J and myself.
I believe vectorization branch is now ready to be merged to the trunk.
I propose to merge it immediately after the hive-0.12 branch is cut, so
that we get sufficient time before the next release.
thanks,
jitendra
--
<http://hortonworks.com/download/>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Merge vectorization branch to trunk
Posted by Carl Steinbach <cw...@gmail.com>.
On Mon, Sep 9, 2013 at 6:51 PM, Edward Capriolo <ed...@gmail.com>wrote:
> Can we vote to change the by-laws and then svn merge the branch? Seems
> better then making a big patch and losing the commit history.
Yes, we can do that, but amending the bylaws requires a 2/3 majority vote
from
active PMC members and a six day voting period. There are a couple other
feature
branches that people are working on right now, so it probably makes sense
to bite
the bullet and get this over with. I'll work on getting a proposal ready.
Thanks.
Carl
Re: Merge vectorization branch to trunk
Posted by Edward Capriolo <ed...@gmail.com>.
Can we vote to change the by-laws and then svn merge the branch? Seems
better then making a big patch and losing the commit history.
On Mon, Sep 9, 2013 at 9:44 PM, Carl Steinbach <cw...@gmail.com>wrote:
> > First of all I'd like to say thanks for your hard work! I also agree
> > that this should go in post 0.12 branch.
> >
>
> Ditto.
>
> As far as merging this into trunk goes, we're currently hamstrung
> by the fact that the Hive Bylaws don't include any provisions covering
> development work done on feature branches. Consequently,
> the only option currently available for getting this merged into trunk is
> to treat it like a regular patch, i.e. file a JIRA ticket, post a patch for
> review, get a +1 vote and then wait 24 hours before doing the
> actual commit.
>
> I think we should consider amending the bylaws to include the
> feature branch provisions that are currently found in the Hadoop
> project bylaws:
>
> http://hadoop.apache.org/bylaws.html
>
> Thanks.
>
> Carl
>
Re: Merge vectorization branch to trunk
Posted by Carl Steinbach <cw...@gmail.com>.
> First of all I'd like to say thanks for your hard work! I also agree
> that this should go in post 0.12 branch.
>
Ditto.
As far as merging this into trunk goes, we're currently hamstrung
by the fact that the Hive Bylaws don't include any provisions covering
development work done on feature branches. Consequently,
the only option currently available for getting this merged into trunk is
to treat it like a regular patch, i.e. file a JIRA ticket, post a patch for
review, get a +1 vote and then wait 24 hours before doing the
actual commit.
I think we should consider amending the bylaws to include the
feature branch provisions that are currently found in the Hadoop
project bylaws:
http://hadoop.apache.org/bylaws.html
Thanks.
Carl
Re: Merge vectorization branch to trunk
Posted by Jitendra Pandey <ji...@hortonworks.com>.
Hi Brock,
I will merge latest trunk to the branch and run all the tests again, and
provide you the run time difference. Trunk has moved ahead since the last
merge, therefore test results are a bit stale right now.
> What new modules or dependencies are added?
No new dependencies have been added.
> Can the feature be disabled if required?
Yes, there is boolean hive.vectorized.execution.enabled flag. If it is
false, vectorization optimization is completely ignored. By default, we
intend to keep it false for now, so that users who don't care about
vectorization don't have to change anything.
On Fri, Sep 6, 2013 at 2:24 PM, Brock Noland <br...@cloudera.com> wrote:
> Hi,
>
> First of all I'd like to say thanks for your hard work! I also agree
> that this should go in post 0.12 branch. I have a couple questions:
>
> Do all the tests pass?
> How long does ant test test take in comparison with trunk?
> What new modules or dependencies are added?
> Can the feature be disabled if required?
>
> Brock
>
> On Fri, Sep 6, 2013 at 3:44 PM, Jitendra Pandey
> <ji...@hortonworks.com> wrote:
> > Hi Folks,
> > Vectorized query execution work has been under development for around
> 6
> > months now. We have made good progress in making basic datatypes and
> > queries to work in vectorized mode and also we have laid sufficient
> > groundwork so that any future development can happen directly on trunk.
> > This is a community effort involving several developers including Eric
> > Hanson, Remus Rusanu, Sarvesh
> > Sakalanaga<
> https://issues.apache.org/jira/secure/ViewProfile.jspa?name=sarvesh.sn>,
> > Gopal V, Teddy Choi, Tony Murphy, Timothy Chen, Prashanth J and myself.
> > I believe vectorization branch is now ready to be merged to the trunk.
> > I propose to merge it immediately after the hive-0.12 branch is cut,
> so
> > that we get sufficient time before the next release.
> >
> > thanks,
> > jitendra
> > --
> > <http://hortonworks.com/download/>
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>
>
>
> --
> Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
>
--
<http://hortonworks.com/download/>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Merge vectorization branch to trunk
Posted by Brock Noland <br...@cloudera.com>.
Hi,
First of all I'd like to say thanks for your hard work! I also agree
that this should go in post 0.12 branch. I have a couple questions:
Do all the tests pass?
How long does ant test test take in comparison with trunk?
What new modules or dependencies are added?
Can the feature be disabled if required?
Brock
On Fri, Sep 6, 2013 at 3:44 PM, Jitendra Pandey
<ji...@hortonworks.com> wrote:
> Hi Folks,
> Vectorized query execution work has been under development for around 6
> months now. We have made good progress in making basic datatypes and
> queries to work in vectorized mode and also we have laid sufficient
> groundwork so that any future development can happen directly on trunk.
> This is a community effort involving several developers including Eric
> Hanson, Remus Rusanu, Sarvesh
> Sakalanaga<https://issues.apache.org/jira/secure/ViewProfile.jspa?name=sarvesh.sn>,
> Gopal V, Teddy Choi, Tony Murphy, Timothy Chen, Prashanth J and myself.
> I believe vectorization branch is now ready to be merged to the trunk.
> I propose to merge it immediately after the hive-0.12 branch is cut, so
> that we get sufficient time before the next release.
>
> thanks,
> jitendra
> --
> <http://hortonworks.com/download/>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org