You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Ashutosh Chauhan <ha...@apache.org> on 2013/02/01 19:11:32 UTC
branch for ptf and windowing fuction
Hi all,
Harish and Prajkta are doing some cool work over at
https://issues.apache.org/jira/browse/HIVE-896 IMO its a very useful
feature for the community and our user base. Harish and Prajkta are making
steady progress on this for much last year in their github repo
https://github.com/hbutani/hive and much of the feature is now functional.
However, its quite a bit of work and new code which will take some time
before being ready for trunk. I propose that we create a new branch so that
further development of this happens in apache repo instead of github repo.
This gets us few benefits:
a) It will avoid the situation we ended up with HiveServer2 where a useful
new functionality came but in one big patch which made its review and thus
inclusion in mainline harder than it should have been.
b) Obvious advantages of development getting done in apache as oppose to
github which are:
i) It will make it easier for apache hive community members interested
in this work (like me) to follow progress.
ii) It will make it easier for apache hive community members interested
in this work to contribute.
iii) It will make it easier for apache community members to review the
work and provide feedback.
I further propose that we follow Commit-than-review policy for this feature
branch which will enable contributors to make rapid progress without
waiting for lengthy review cycles. Hive committers interested in work can
either review branch any time they want to provide feedback or can wait
till contributors declare work is complete and make a proposal to merge in
trunk and than review it than. This anyway is a throwaway branch not
intended to make releases out of it.
Unless I hear any objections, I will create a branch over the weekend.
Thanks,
Ashutosh
Re: branch for ptf and windowing fuction
Posted by Brock Noland <br...@cloudera.com>.
+1
On Fri, Feb 1, 2013 at 5:07 PM, Shreepadma Venugopalan <
shreepadma@cloudera.com> wrote:
> +1. This looks very useful.
>
> Shreepadma
>
>
> On Fri, Feb 1, 2013 at 11:36 AM, Ashish Thusoo <at...@qubole.com> wrote:
>
> > +1.
> >
> > This is certainly very useful work.
> >
> > Ashish
> >
> >
> > On Fri, Feb 1, 2013 at 11:31 AM, Carl Steinbach <cwsteinbach@gmail.com
> > >wrote:
> >
> > > +1
> > >
> > > On Fri, Feb 1, 2013 at 10:11 AM, Ashutosh Chauhan <
> hashutosh@apache.org
> > > >wrote:
> > >
> > > > Hi all,
> > > >
> > > > Harish and Prajkta are doing some cool work over at
> > > > https://issues.apache.org/jira/browse/HIVE-896 IMO its a very useful
> > > > feature for the community and our user base. Harish and Prajkta are
> > > making
> > > > steady progress on this for much last year in their github repo
> > > > https://github.com/hbutani/hive and much of the feature is now
> > > functional.
> > > > However, its quite a bit of work and new code which will take some
> time
> > > > before being ready for trunk. I propose that we create a new branch
> so
> > > that
> > > > further development of this happens in apache repo instead of github
> > > repo.
> > > > This gets us few benefits:
> > > > a) It will avoid the situation we ended up with HiveServer2 where a
> > > useful
> > > > new functionality came but in one big patch which made its review and
> > > thus
> > > > inclusion in mainline harder than it should have been.
> > > > b) Obvious advantages of development getting done in apache as oppose
> > to
> > > > github which are:
> > > > i) It will make it easier for apache hive community members
> > > interested
> > > > in this work (like me) to follow progress.
> > > > ii) It will make it easier for apache hive community members
> > > interested
> > > > in this work to contribute.
> > > > iii) It will make it easier for apache community members to
> review
> > > the
> > > > work and provide feedback.
> > > >
> > > > I further propose that we follow Commit-than-review policy for this
> > > feature
> > > > branch which will enable contributors to make rapid progress without
> > > > waiting for lengthy review cycles. Hive committers interested in work
> > can
> > > > either review branch any time they want to provide feedback or can
> wait
> > > > till contributors declare work is complete and make a proposal to
> merge
> > > in
> > > > trunk and than review it than. This anyway is a throwaway branch not
> > > > intended to make releases out of it.
> > > >
> > > > Unless I hear any objections, I will create a branch over the
> weekend.
> > > >
> > > > Thanks,
> > > > Ashutosh
> > > >
> > >
> >
>
--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
Re: branch for ptf and windowing fuction
Posted by Shreepadma Venugopalan <sh...@cloudera.com>.
+1. This looks very useful.
Shreepadma
On Fri, Feb 1, 2013 at 11:36 AM, Ashish Thusoo <at...@qubole.com> wrote:
> +1.
>
> This is certainly very useful work.
>
> Ashish
>
>
> On Fri, Feb 1, 2013 at 11:31 AM, Carl Steinbach <cwsteinbach@gmail.com
> >wrote:
>
> > +1
> >
> > On Fri, Feb 1, 2013 at 10:11 AM, Ashutosh Chauhan <hashutosh@apache.org
> > >wrote:
> >
> > > Hi all,
> > >
> > > Harish and Prajkta are doing some cool work over at
> > > https://issues.apache.org/jira/browse/HIVE-896 IMO its a very useful
> > > feature for the community and our user base. Harish and Prajkta are
> > making
> > > steady progress on this for much last year in their github repo
> > > https://github.com/hbutani/hive and much of the feature is now
> > functional.
> > > However, its quite a bit of work and new code which will take some time
> > > before being ready for trunk. I propose that we create a new branch so
> > that
> > > further development of this happens in apache repo instead of github
> > repo.
> > > This gets us few benefits:
> > > a) It will avoid the situation we ended up with HiveServer2 where a
> > useful
> > > new functionality came but in one big patch which made its review and
> > thus
> > > inclusion in mainline harder than it should have been.
> > > b) Obvious advantages of development getting done in apache as oppose
> to
> > > github which are:
> > > i) It will make it easier for apache hive community members
> > interested
> > > in this work (like me) to follow progress.
> > > ii) It will make it easier for apache hive community members
> > interested
> > > in this work to contribute.
> > > iii) It will make it easier for apache community members to review
> > the
> > > work and provide feedback.
> > >
> > > I further propose that we follow Commit-than-review policy for this
> > feature
> > > branch which will enable contributors to make rapid progress without
> > > waiting for lengthy review cycles. Hive committers interested in work
> can
> > > either review branch any time they want to provide feedback or can wait
> > > till contributors declare work is complete and make a proposal to merge
> > in
> > > trunk and than review it than. This anyway is a throwaway branch not
> > > intended to make releases out of it.
> > >
> > > Unless I hear any objections, I will create a branch over the weekend.
> > >
> > > Thanks,
> > > Ashutosh
> > >
> >
>
Re: branch for ptf and windowing fuction
Posted by Ashish Thusoo <at...@qubole.com>.
+1.
This is certainly very useful work.
Ashish
On Fri, Feb 1, 2013 at 11:31 AM, Carl Steinbach <cw...@gmail.com>wrote:
> +1
>
> On Fri, Feb 1, 2013 at 10:11 AM, Ashutosh Chauhan <hashutosh@apache.org
> >wrote:
>
> > Hi all,
> >
> > Harish and Prajkta are doing some cool work over at
> > https://issues.apache.org/jira/browse/HIVE-896 IMO its a very useful
> > feature for the community and our user base. Harish and Prajkta are
> making
> > steady progress on this for much last year in their github repo
> > https://github.com/hbutani/hive and much of the feature is now
> functional.
> > However, its quite a bit of work and new code which will take some time
> > before being ready for trunk. I propose that we create a new branch so
> that
> > further development of this happens in apache repo instead of github
> repo.
> > This gets us few benefits:
> > a) It will avoid the situation we ended up with HiveServer2 where a
> useful
> > new functionality came but in one big patch which made its review and
> thus
> > inclusion in mainline harder than it should have been.
> > b) Obvious advantages of development getting done in apache as oppose to
> > github which are:
> > i) It will make it easier for apache hive community members
> interested
> > in this work (like me) to follow progress.
> > ii) It will make it easier for apache hive community members
> interested
> > in this work to contribute.
> > iii) It will make it easier for apache community members to review
> the
> > work and provide feedback.
> >
> > I further propose that we follow Commit-than-review policy for this
> feature
> > branch which will enable contributors to make rapid progress without
> > waiting for lengthy review cycles. Hive committers interested in work can
> > either review branch any time they want to provide feedback or can wait
> > till contributors declare work is complete and make a proposal to merge
> in
> > trunk and than review it than. This anyway is a throwaway branch not
> > intended to make releases out of it.
> >
> > Unless I hear any objections, I will create a branch over the weekend.
> >
> > Thanks,
> > Ashutosh
> >
>
Re: branch for ptf and windowing fuction
Posted by Carl Steinbach <cw...@gmail.com>.
+1
On Fri, Feb 1, 2013 at 10:11 AM, Ashutosh Chauhan <ha...@apache.org>wrote:
> Hi all,
>
> Harish and Prajkta are doing some cool work over at
> https://issues.apache.org/jira/browse/HIVE-896 IMO its a very useful
> feature for the community and our user base. Harish and Prajkta are making
> steady progress on this for much last year in their github repo
> https://github.com/hbutani/hive and much of the feature is now functional.
> However, its quite a bit of work and new code which will take some time
> before being ready for trunk. I propose that we create a new branch so that
> further development of this happens in apache repo instead of github repo.
> This gets us few benefits:
> a) It will avoid the situation we ended up with HiveServer2 where a useful
> new functionality came but in one big patch which made its review and thus
> inclusion in mainline harder than it should have been.
> b) Obvious advantages of development getting done in apache as oppose to
> github which are:
> i) It will make it easier for apache hive community members interested
> in this work (like me) to follow progress.
> ii) It will make it easier for apache hive community members interested
> in this work to contribute.
> iii) It will make it easier for apache community members to review the
> work and provide feedback.
>
> I further propose that we follow Commit-than-review policy for this feature
> branch which will enable contributors to make rapid progress without
> waiting for lengthy review cycles. Hive committers interested in work can
> either review branch any time they want to provide feedback or can wait
> till contributors declare work is complete and make a proposal to merge in
> trunk and than review it than. This anyway is a throwaway branch not
> intended to make releases out of it.
>
> Unless I hear any objections, I will create a branch over the weekend.
>
> Thanks,
> Ashutosh
>
Re: branch for ptf and windowing fuction
Posted by Ashutosh Chauhan <ha...@apache.org>.
Hi all,
Cool. Seems like everyone is on board. I have created a new branch [1]
based of current trunk and have committed latest patch attached on HIVE-896
to it. Check it out. Feel free to open jiras for this work and put up
patches. I have added a new component called "ptf-windowing" on jira which
you could use for issues related to this work.
https://svn.apache.org/repos/asf/hive/branches/ptf-windowing/
Thanks,
Ashutosh
On Mon, Feb 4, 2013 at 8:54 AM, Ashutosh Chauhan <ha...@apache.org>wrote:
> Hi Arvind,
>
> Yeah thats the idea to do periodic merges to keep the branch in sync with
> trunk, otherwise merging it with trunk later on will get unnecessarily
> complicated.
>
> Thanks,
> Ashutosh
>
>
> On Mon, Feb 4, 2013 at 12:56 AM, Arvind Prabhakar <ar...@apache.org>wrote:
>
>> Hi Ashutosh,
>>
>> My +1 for the proposal for creating a separate branch for feature
>> development.
>>
>> I do have one question in this regard: how do you plan on keeping this
>> branch in sync with the trunk? If the branch is allowed to diverge
>> indefinitely, it is likely that the build from it will lag in features and
>> fixes that are otherwise available on the trunk. It will be great if you
>> could get the branch to first synchronize with the trunk and then follow a
>> policy where there are periodic merges from the trunk into the development
>> branch.
>>
>> Regards,
>> Arvind Prabhakar
>>
>> On Fri, Feb 1, 2013 at 10:11 AM, Ashutosh Chauhan <hashutosh@apache.org
>> >wrote:
>>
>> > Hi all,
>> >
>> > Harish and Prajkta are doing some cool work over at
>> > https://issues.apache.org/jira/browse/HIVE-896 IMO its a very useful
>> > feature for the community and our user base. Harish and Prajkta are
>> making
>> > steady progress on this for much last year in their github repo
>> > https://github.com/hbutani/hive and much of the feature is now
>> functional.
>> > However, its quite a bit of work and new code which will take some time
>> > before being ready for trunk. I propose that we create a new branch so
>> that
>> > further development of this happens in apache repo instead of github
>> repo.
>> > This gets us few benefits:
>> > a) It will avoid the situation we ended up with HiveServer2 where a
>> useful
>> > new functionality came but in one big patch which made its review and
>> thus
>> > inclusion in mainline harder than it should have been.
>> > b) Obvious advantages of development getting done in apache as oppose to
>> > github which are:
>> > i) It will make it easier for apache hive community members
>> interested
>> > in this work (like me) to follow progress.
>> > ii) It will make it easier for apache hive community members
>> interested
>> > in this work to contribute.
>> > iii) It will make it easier for apache community members to review
>> the
>> > work and provide feedback.
>> >
>> > I further propose that we follow Commit-than-review policy for this
>> feature
>> > branch which will enable contributors to make rapid progress without
>> > waiting for lengthy review cycles. Hive committers interested in work
>> can
>> > either review branch any time they want to provide feedback or can wait
>> > till contributors declare work is complete and make a proposal to merge
>> in
>> > trunk and than review it than. This anyway is a throwaway branch not
>> > intended to make releases out of it.
>> >
>> > Unless I hear any objections, I will create a branch over the weekend.
>> >
>> > Thanks,
>> > Ashutosh
>> >
>>
>
>
Re: branch for ptf and windowing fuction
Posted by Ashutosh Chauhan <ha...@apache.org>.
Hi Arvind,
Yeah thats the idea to do periodic merges to keep the branch in sync with
trunk, otherwise merging it with trunk later on will get unnecessarily
complicated.
Thanks,
Ashutosh
On Mon, Feb 4, 2013 at 12:56 AM, Arvind Prabhakar <ar...@apache.org> wrote:
> Hi Ashutosh,
>
> My +1 for the proposal for creating a separate branch for feature
> development.
>
> I do have one question in this regard: how do you plan on keeping this
> branch in sync with the trunk? If the branch is allowed to diverge
> indefinitely, it is likely that the build from it will lag in features and
> fixes that are otherwise available on the trunk. It will be great if you
> could get the branch to first synchronize with the trunk and then follow a
> policy where there are periodic merges from the trunk into the development
> branch.
>
> Regards,
> Arvind Prabhakar
>
> On Fri, Feb 1, 2013 at 10:11 AM, Ashutosh Chauhan <hashutosh@apache.org
> >wrote:
>
> > Hi all,
> >
> > Harish and Prajkta are doing some cool work over at
> > https://issues.apache.org/jira/browse/HIVE-896 IMO its a very useful
> > feature for the community and our user base. Harish and Prajkta are
> making
> > steady progress on this for much last year in their github repo
> > https://github.com/hbutani/hive and much of the feature is now
> functional.
> > However, its quite a bit of work and new code which will take some time
> > before being ready for trunk. I propose that we create a new branch so
> that
> > further development of this happens in apache repo instead of github
> repo.
> > This gets us few benefits:
> > a) It will avoid the situation we ended up with HiveServer2 where a
> useful
> > new functionality came but in one big patch which made its review and
> thus
> > inclusion in mainline harder than it should have been.
> > b) Obvious advantages of development getting done in apache as oppose to
> > github which are:
> > i) It will make it easier for apache hive community members
> interested
> > in this work (like me) to follow progress.
> > ii) It will make it easier for apache hive community members
> interested
> > in this work to contribute.
> > iii) It will make it easier for apache community members to review
> the
> > work and provide feedback.
> >
> > I further propose that we follow Commit-than-review policy for this
> feature
> > branch which will enable contributors to make rapid progress without
> > waiting for lengthy review cycles. Hive committers interested in work can
> > either review branch any time they want to provide feedback or can wait
> > till contributors declare work is complete and make a proposal to merge
> in
> > trunk and than review it than. This anyway is a throwaway branch not
> > intended to make releases out of it.
> >
> > Unless I hear any objections, I will create a branch over the weekend.
> >
> > Thanks,
> > Ashutosh
> >
>
Re: branch for ptf and windowing fuction
Posted by Arvind Prabhakar <ar...@apache.org>.
Hi Ashutosh,
My +1 for the proposal for creating a separate branch for feature
development.
I do have one question in this regard: how do you plan on keeping this
branch in sync with the trunk? If the branch is allowed to diverge
indefinitely, it is likely that the build from it will lag in features and
fixes that are otherwise available on the trunk. It will be great if you
could get the branch to first synchronize with the trunk and then follow a
policy where there are periodic merges from the trunk into the development
branch.
Regards,
Arvind Prabhakar
On Fri, Feb 1, 2013 at 10:11 AM, Ashutosh Chauhan <ha...@apache.org>wrote:
> Hi all,
>
> Harish and Prajkta are doing some cool work over at
> https://issues.apache.org/jira/browse/HIVE-896 IMO its a very useful
> feature for the community and our user base. Harish and Prajkta are making
> steady progress on this for much last year in their github repo
> https://github.com/hbutani/hive and much of the feature is now functional.
> However, its quite a bit of work and new code which will take some time
> before being ready for trunk. I propose that we create a new branch so that
> further development of this happens in apache repo instead of github repo.
> This gets us few benefits:
> a) It will avoid the situation we ended up with HiveServer2 where a useful
> new functionality came but in one big patch which made its review and thus
> inclusion in mainline harder than it should have been.
> b) Obvious advantages of development getting done in apache as oppose to
> github which are:
> i) It will make it easier for apache hive community members interested
> in this work (like me) to follow progress.
> ii) It will make it easier for apache hive community members interested
> in this work to contribute.
> iii) It will make it easier for apache community members to review the
> work and provide feedback.
>
> I further propose that we follow Commit-than-review policy for this feature
> branch which will enable contributors to make rapid progress without
> waiting for lengthy review cycles. Hive committers interested in work can
> either review branch any time they want to provide feedback or can wait
> till contributors declare work is complete and make a proposal to merge in
> trunk and than review it than. This anyway is a throwaway branch not
> intended to make releases out of it.
>
> Unless I hear any objections, I will create a branch over the weekend.
>
> Thanks,
> Ashutosh
>
Re: branch for ptf and windowing fuction
Posted by Namit Jain <nj...@fb.com>.
+1
On 2/2/13 11:01 AM, "Clark Yang (杨卓荦)" <ya...@gmail.com> wrote:
>+1
>
>Cheers,
>Zhuoluo (Clark) Yang
>
>
>2013/2/2 Jarek Jarcec Cecho <ja...@apache.org>
>
>> +1.
>>
>> Jarcec
>>
>> On Fri, Feb 01, 2013 at 10:11:32AM -0800, Ashutosh Chauhan wrote:
>> > Hi all,
>> >
>> > Harish and Prajkta are doing some cool work over at
>> > https://issues.apache.org/jira/browse/HIVE-896 IMO its a very useful
>> > feature for the community and our user base. Harish and Prajkta are
>> making
>> > steady progress on this for much last year in their github repo
>> > https://github.com/hbutani/hive and much of the feature is now
>> functional.
>> > However, its quite a bit of work and new code which will take some
>>time
>> > before being ready for trunk. I propose that we create a new branch so
>> that
>> > further development of this happens in apache repo instead of github
>> repo.
>> > This gets us few benefits:
>> > a) It will avoid the situation we ended up with HiveServer2 where a
>> useful
>> > new functionality came but in one big patch which made its review and
>> thus
>> > inclusion in mainline harder than it should have been.
>> > b) Obvious advantages of development getting done in apache as oppose
>>to
>> > github which are:
>> > i) It will make it easier for apache hive community members
>> interested
>> > in this work (like me) to follow progress.
>> > ii) It will make it easier for apache hive community members
>> interested
>> > in this work to contribute.
>> > iii) It will make it easier for apache community members to review
>> the
>> > work and provide feedback.
>> >
>> > I further propose that we follow Commit-than-review policy for this
>> feature
>> > branch which will enable contributors to make rapid progress without
>> > waiting for lengthy review cycles. Hive committers interested in work
>>can
>> > either review branch any time they want to provide feedback or can
>>wait
>> > till contributors declare work is complete and make a proposal to
>>merge
>> in
>> > trunk and than review it than. This anyway is a throwaway branch not
>> > intended to make releases out of it.
>> >
>> > Unless I hear any objections, I will create a branch over the weekend.
>> >
>> > Thanks,
>> > Ashutosh
>>
Re: branch for ptf and windowing fuction
Posted by "Clark Yang (杨卓荦)" <ya...@gmail.com>.
+1
Cheers,
Zhuoluo (Clark) Yang
2013/2/2 Jarek Jarcec Cecho <ja...@apache.org>
> +1.
>
> Jarcec
>
> On Fri, Feb 01, 2013 at 10:11:32AM -0800, Ashutosh Chauhan wrote:
> > Hi all,
> >
> > Harish and Prajkta are doing some cool work over at
> > https://issues.apache.org/jira/browse/HIVE-896 IMO its a very useful
> > feature for the community and our user base. Harish and Prajkta are
> making
> > steady progress on this for much last year in their github repo
> > https://github.com/hbutani/hive and much of the feature is now
> functional.
> > However, its quite a bit of work and new code which will take some time
> > before being ready for trunk. I propose that we create a new branch so
> that
> > further development of this happens in apache repo instead of github
> repo.
> > This gets us few benefits:
> > a) It will avoid the situation we ended up with HiveServer2 where a
> useful
> > new functionality came but in one big patch which made its review and
> thus
> > inclusion in mainline harder than it should have been.
> > b) Obvious advantages of development getting done in apache as oppose to
> > github which are:
> > i) It will make it easier for apache hive community members
> interested
> > in this work (like me) to follow progress.
> > ii) It will make it easier for apache hive community members
> interested
> > in this work to contribute.
> > iii) It will make it easier for apache community members to review
> the
> > work and provide feedback.
> >
> > I further propose that we follow Commit-than-review policy for this
> feature
> > branch which will enable contributors to make rapid progress without
> > waiting for lengthy review cycles. Hive committers interested in work can
> > either review branch any time they want to provide feedback or can wait
> > till contributors declare work is complete and make a proposal to merge
> in
> > trunk and than review it than. This anyway is a throwaway branch not
> > intended to make releases out of it.
> >
> > Unless I hear any objections, I will create a branch over the weekend.
> >
> > Thanks,
> > Ashutosh
>
Re: branch for ptf and windowing fuction
Posted by Jarek Jarcec Cecho <ja...@apache.org>.
+1.
Jarcec
On Fri, Feb 01, 2013 at 10:11:32AM -0800, Ashutosh Chauhan wrote:
> Hi all,
>
> Harish and Prajkta are doing some cool work over at
> https://issues.apache.org/jira/browse/HIVE-896 IMO its a very useful
> feature for the community and our user base. Harish and Prajkta are making
> steady progress on this for much last year in their github repo
> https://github.com/hbutani/hive and much of the feature is now functional.
> However, its quite a bit of work and new code which will take some time
> before being ready for trunk. I propose that we create a new branch so that
> further development of this happens in apache repo instead of github repo.
> This gets us few benefits:
> a) It will avoid the situation we ended up with HiveServer2 where a useful
> new functionality came but in one big patch which made its review and thus
> inclusion in mainline harder than it should have been.
> b) Obvious advantages of development getting done in apache as oppose to
> github which are:
> i) It will make it easier for apache hive community members interested
> in this work (like me) to follow progress.
> ii) It will make it easier for apache hive community members interested
> in this work to contribute.
> iii) It will make it easier for apache community members to review the
> work and provide feedback.
>
> I further propose that we follow Commit-than-review policy for this feature
> branch which will enable contributors to make rapid progress without
> waiting for lengthy review cycles. Hive committers interested in work can
> either review branch any time they want to provide feedback or can wait
> till contributors declare work is complete and make a proposal to merge in
> trunk and than review it than. This anyway is a throwaway branch not
> intended to make releases out of it.
>
> Unless I hear any objections, I will create a branch over the weekend.
>
> Thanks,
> Ashutosh