You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by Josh Wills <jw...@cloudera.com> on 2012/10/30 22:51:45 UTC
Crunch 0.4.0 release preparation
Hey all,
I'd like to get our final preparations in place for the 0.4.0 Crunch
release. Among the remaining issues, the only things that seem required for
the release are CRUNCH-63 (including the CHANGES.txt file), although it
would be great to include CRUNCH-105 (DOT files for the planner) if at all
possible.
I cc'd Roman on here so we can be sure that we position the release to be
compatible w/Bigtop. Thanks to Matthias, we have both support for .tar.gz
release files and binary release artifacts. We're using Hadoop 1.0.3 as our
default build target, although it is possible to do the build against the
Hadoop 2.0.x APIs via the crunch.platform property in the top-level POM
file.
Question for everyone: is there anything else you would like to include in
0.4.0 that is close to completion?
Question for Roman: is there anything else we should double-check on for
Bigtop compatibility?
Thanks,
Josh
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: Crunch 0.4.0 release preparation
Posted by Matthias Friedrich <ma...@mafr.de>.
Hi,
On Friday, 2012-11-09, Matthias Friedrich wrote:
> On Friday, 2012-11-09, Roman Shaposhnik wrote:
[...]
>> I'd be glad to review.
> Great! CRUNCH-108 adds some assembly excludes which I *think* should
> be fine (the created source artifact looked OK), but I'd like to make
> sure I didn't miss anything that could cause us trouble in the future.
Never mind, Josh already reviewed it. Got his mail seconds after I
wrote mine :)
Regards,
Matthias
Re: Crunch 0.4.0 release preparation
Posted by Matthias Friedrich <ma...@mafr.de>.
Hi Roman,
On Friday, 2012-11-09, Roman Shaposhnik wrote:
> On Thu, Nov 8, 2012 at 11:03 PM, Matthias Friedrich <ma...@mafr.de> wrote:
>> thanks for your review! We've discovered a few minor issues that we
>> already fixed (only CRUNCH-108 is waiting for review), but I'm going
>> to build a new RC this evening (CET).
> I'd be glad to review.
Great! CRUNCH-108 adds some assembly excludes which I *think* should
be fine (the created source artifact looked OK), but I'd like to make
sure I didn't miss anything that could cause us trouble in the future.
After that's done I'll create RC1, either today or tomorrow moring.
>> What kind of fat jar would you like to see and what do you need it for?
>> We already have a job-jar for our examples and I'd like to avoid adding
>> another artifact and going through a patch review cycle before building
>> the new RC.
> I was talking about crunch-examples-0.4.0-incubating-job.jar but I see
> it now being included in the bin assembly so everything is good.
Ah, cool. Thanks!
Regards,
Matthias
Re: Crunch 0.4.0 release preparation
Posted by Roman Shaposhnik <rv...@apache.org>.
Hi!
On Thu, Nov 8, 2012 at 11:03 PM, Matthias Friedrich <ma...@mafr.de> wrote:
> Hi Roman,
>
> thanks for your review! We've discovered a few minor issues that we
> already fixed (only CRUNCH-108 is waiting for review), but I'm going
> to build a new RC this evening (CET).
I'd be glad to review.
> What kind of fat jar would you like to see and what do you need it for?
> We already have a job-jar for our examples and I'd like to avoid adding
> another artifact and going through a patch review cycle before building
> the new RC.
I was talking about crunch-examples-0.4.0-incubating-job.jar but I see
it now being included in the bin assembly so everything is good.
Sorry for the noise.
Thanks,
Roman.
Re: Crunch 0.4.0 release preparation
Posted by Matthias Friedrich <ma...@mafr.de>.
Hi Roman,
thanks for your review! We've discovered a few minor issues that we
already fixed (only CRUNCH-108 is waiting for review), but I'm going
to build a new RC this evening (CET).
What kind of fat jar would you like to see and what do you need it for?
We already have a job-jar for our examples and I'd like to avoid adding
another artifact and going through a patch review cycle before building
the new RC.
Regards,
Matthias
On Thursday, 2012-11-08, Roman Shaposhnik wrote:
> On Tue, Nov 6, 2012 at 10:14 PM, Josh Wills <jw...@cloudera.com> wrote:
> >> Do you guys have an RC going? What's the URL?
> >
> >
> > We do: http://people.apache.org/~mafr/apache-crunch-0.4.0-incubating-rc0/
>
> This looks pretty happy as far as Bigtop is concerned. Let me also
> run some more tests on fully distributed clusters, but I think we're ok.
>
> It would still be very useful to make a fat jar part of the assembly though.
> Not sure whether this is worth re-spinning the RC for. If you think it is --
> I can help with the implementation.
>
> Thanks,
> Roman.
Re: Crunch 0.4.0 release preparation
Posted by Roman Shaposhnik <rv...@apache.org>.
On Tue, Nov 6, 2012 at 10:14 PM, Josh Wills <jw...@cloudera.com> wrote:
>> Do you guys have an RC going? What's the URL?
>
>
> We do: http://people.apache.org/~mafr/apache-crunch-0.4.0-incubating-rc0/
This looks pretty happy as far as Bigtop is concerned. Let me also
run some more tests on fully distributed clusters, but I think we're ok.
It would still be very useful to make a fat jar part of the assembly though.
Not sure whether this is worth re-spinning the RC for. If you think it is --
I can help with the implementation.
Thanks,
Roman.
Re: Crunch 0.4.0 release preparation
Posted by Josh Wills <jw...@cloudera.com>.
On Tue, Nov 6, 2012 at 5:17 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> On Sat, Nov 3, 2012 at 1:50 AM, Matthias Friedrich <ma...@mafr.de> wrote:
> >> I'll leave that one up to Matthias.
> >
> > Heh. We definitely need a real Maven wizard on the team, this stuff is
> > surprisingly tricky. I needed a separate moduleSet to make it work.
>
> I can help to the extent I know Maven. The best way is to have a JIRA
> I can post patches to.
>
> > Roman, could you try again? Bigtop is the primary consumer of the
> > binary release, we'd have to scrap the RC and restart the review and
> > voting process in case it doesn't work for you.
>
> Do you guys have an RC going? What's the URL?
>
We do: http://people.apache.org/~mafr/apache-crunch-0.4.0-incubating-rc0/
>
> Thanks,
> Roman.
>
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: Crunch 0.4.0 release preparation
Posted by Roman Shaposhnik <rv...@apache.org>.
On Sat, Nov 3, 2012 at 1:50 AM, Matthias Friedrich <ma...@mafr.de> wrote:
>> I'll leave that one up to Matthias.
>
> Heh. We definitely need a real Maven wizard on the team, this stuff is
> surprisingly tricky. I needed a separate moduleSet to make it work.
I can help to the extent I know Maven. The best way is to have a JIRA
I can post patches to.
> Roman, could you try again? Bigtop is the primary consumer of the
> binary release, we'd have to scrap the RC and restart the review and
> voting process in case it doesn't work for you.
Do you guys have an RC going? What's the URL?
Thanks,
Roman.
Re: Crunch 0.4.0 release preparation
Posted by Matthias Friedrich <ma...@mafr.de>.
On Friday, 2012-11-02, Josh Wills wrote:
> On Fri, Nov 2, 2012 at 6:19 PM, Roman Shaposhnik <rv...@apache.org> wrote:
[...]
>> It looks like you're in a good shape now. I will definitely track your
>> RC and test it in Bigtop just to make sure.
>> That said, I did notice a couple of minor things:
>> 1. any reason not to include examples-job.jar into assembly? The
>> way it stands
>> right now one has to jump through some hooks to make examples run.
> I'll leave that one up to Matthias.
Heh. We definitely need a real Maven wizard on the team, this stuff is
surprisingly tricky. I needed a separate moduleSet to make it work.
>> 2. when I run AverageBytesByIP and TotalBytesByIP examples I get no
>> output even though the job is marked as success. Am I missing something?
> No, you were right. When we put fake IP addresses in the example file,
> the regex in the examples stopped matching. I just submitted a
> 2-character fix that corrects it. Thanks for finding that, and thanks
> for weighing in!
Roman, could you try again? Bigtop is the primary consumer of the
binary release, we'd have to scrap the RC and restart the review and
voting process in case it doesn't work for you.
Regards,
Matthias
Re: Crunch 0.4.0 release preparation
Posted by Josh Wills <jw...@cloudera.com>.
On Fri, Nov 2, 2012 at 6:19 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> Hi!
>
> On Tue, Oct 30, 2012 at 2:51 PM, Josh Wills <jw...@cloudera.com> wrote:
>> I cc'd Roman on here so we can be sure that we position the release to be
>> compatible w/Bigtop. Thanks to Matthias, we have both support for .tar.gz
>> release files and binary release artifacts. We're using Hadoop 1.0.3 as our
>> default build target, although it is possible to do the build against the
>> Hadoop 2.0.x APIs via the crunch.platform property in the top-level POM
>> file.
>>
>> Question for everyone: is there anything else you would like to include in
>> 0.4.0 that is close to completion?
>> Question for Roman: is there anything else we should double-check on for
>> Bigtop compatibility?
>
> It looks like you're in a good shape now. I will definitely track your
> RC and test
> it in Bigtop just to make sure.
>
> That said, I did notice a couple of minor things:
> 1. any reason not to include examples-job.jar into assembly? The
> way it stands
> right now one has to jump through some hooks to make examples run.
I'll leave that one up to Matthias.
>
> 2. when I run AverageBytesByIP and TotalBytesByIP examples I get no
> output even though the job is marked as success. Am I missing something?
No, you were right. When we put fake IP addresses in the example file,
the regex in the examples stopped matching. I just submitted a
2-character fix that corrects it. Thanks for finding that, and thanks
for weighing in!
>
> Thanks,
> Roman.
--
Director of Data Science
Cloudera
Twitter: @josh_wills
Re: Crunch 0.4.0 release preparation
Posted by Roman Shaposhnik <rv...@apache.org>.
Hi!
On Tue, Oct 30, 2012 at 2:51 PM, Josh Wills <jw...@cloudera.com> wrote:
> I cc'd Roman on here so we can be sure that we position the release to be
> compatible w/Bigtop. Thanks to Matthias, we have both support for .tar.gz
> release files and binary release artifacts. We're using Hadoop 1.0.3 as our
> default build target, although it is possible to do the build against the
> Hadoop 2.0.x APIs via the crunch.platform property in the top-level POM
> file.
>
> Question for everyone: is there anything else you would like to include in
> 0.4.0 that is close to completion?
> Question for Roman: is there anything else we should double-check on for
> Bigtop compatibility?
It looks like you're in a good shape now. I will definitely track your
RC and test
it in Bigtop just to make sure.
That said, I did notice a couple of minor things:
1. any reason not to include examples-job.jar into assembly? The
way it stands
right now one has to jump through some hooks to make examples run.
2. when I run AverageBytesByIP and TotalBytesByIP examples I get no
output even though the job is marked as success. Am I missing something?
Thanks,
Roman.
Re: Crunch 0.4.0 release preparation
Posted by Josh Wills <jw...@cloudera.com>.
On Thu, Nov 1, 2012 at 2:26 AM, Gabriel Reid <ga...@gmail.com> wrote:
>
>
> On 01 Nov 2012, at 10:20, Matthias Friedrich <ma...@mafr.de> wrote:
>
> > In general I think we should have at least three people who know how
> > to cut a release so we can take turns and progress isn't blocked when
> > someone is busy.
>
> +1. I definitely want to get in on this too, although I won't be able to
> participate in it this week(end). I'm not familiar with the whole release
> process, and I definitely want to change that.
>
It's actually pretty painless. The only trouble I had last time was when I
had to make some changes to the initial RC, but we added instructions about
how to do that to the release guide:
https://cwiki.apache.org/confluence/display/CRUNCH/Release+Guide
Also, Matthias, I checked with Pat last time: you don't need to be part of
the Apache web of trust in order to do an incubator release, so don't worry
about that yet. We'll need to get everybody in once we're a TLP.
> - Gabriel
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: Crunch 0.4.0 release preparation
Posted by Gabriel Reid <ga...@gmail.com>.
On 01 Nov 2012, at 10:20, Matthias Friedrich <ma...@mafr.de> wrote:
> In general I think we should have at least three people who know how
> to cut a release so we can take turns and progress isn't blocked when
> someone is busy.
+1. I definitely want to get in on this too, although I won't be able to participate in it this week(end). I'm not familiar with the whole release process, and I definitely want to change that.
- Gabriel
Re: Crunch 0.4.0 release preparation
Posted by Matthias Friedrich <ma...@mafr.de>.
Hi,
On Wednesday, 2012-10-31, Josh Wills wrote:
> Matthias, I'd like to nominate you for release manager, if you're up for
> it. :) I'd also like Roman to weigh in on this thread so we're sure we have
> everything we need for Bigtop integration.
Sure, I'll do it unless someone else volunteers, but I might need a
bit of help in case we have to roll back a release candidate. You know
I'm no git expert.
In general I think we should have at least three people who know how
to cut a release so we can take turns and progress isn't blocked when
someone is busy.
> @Rahul, I'll try to get CRUNCH-97 in this week, but I'm not betting on it
> being ready.
Hmm, since CRUNCH-97 has the potential of being extremely useful, I
think we should spend some extra time to make sure it's as generally
applicable as possible. I expect that this may take a couple of review
cycles, so maybe it's better to postpone it (or add it and mark it as
beta?).
What do you think about building a first release candidate on Sunday
(in the afternoon CET)? After we get the thumbs up from Roman, of
course.
Regards,
Matthias
Re: Crunch 0.4.0 release preparation
Posted by Josh Wills <jo...@gmail.com>.
Matthias, I'd like to nominate you for release manager, if you're up for
it. :) I'd also like Roman to weigh in on this thread so we're sure we have
everything we need for Bigtop integration.
@Rahul, I'll try to get CRUNCH-97 in this week, but I'm not betting on it
being ready.
On Wed, Oct 31, 2012 at 10:45 AM, Gabriel Reid <ga...@gmail.com>wrote:
> Ok, I'll commit CRUNCH-105 (very) shortly.
>
> On 31 Oct 2012, at 18:21, Matthias Friedrich <ma...@mafr.de> wrote:
>
> > Hi,
> >
> > like Gabriel and Rahul I'd really like to see CRUNCH-105 (which looks
> > good to commit, BTW) as part of the release. Other than that, I agree
> > that waiting for some of the other open issues would delay the release
> > for too long.
> >
> > We've added cool stuff since 0.3.0, so let's release it :)
> >
> > Regards,
> > Matthias
> >
> > On Wednesday, 2012-10-31, Rahul wrote:
> >> Hi Josh,
> >>
> >> CRUNCH-105 would be a great new feature to have. CRUNCH-97 can be a
> >> nice add-on if possible.
> >> Also I would like to see fixes for CRUNCH-51 and CRUNCH-60 but I
> >> think they are not ready yet and should not hold the release.
> >>
> >> regards,
> >> Rahul
> >>
> >> On 31-10-2012 03:58, Josh Wills wrote:
> >>> On Tue, Oct 30, 2012 at 3:08 PM, Gabriel Reid <gabriel.reid@gmail.com
> >wrote:
> >>>
> >>>> Hi Josh,
> >>>>
> >>>> Yep, I'd like to include CRUNCH-105 as well if possible.
> >>>>
> >>>> The only other potential issue that I see for inclusion is CRUNCH-51
> >>>> (fixing sorting to use multiple reducers). I actually think that this
> one
> >>>> is
> >>>> still too far from being ready, but I just want to put that out there
> to
> >>>> make
> >>>> sure it's clear that it's not (yet) in.
> >>>>
> >>> Understood. I would have liked to get MSCR fusion and the text parsing
> >>> stuff in too, but they're not there yet.
> >>>
> >>>
> >>>> - Gabriel
> >>>>
> >>>> On 30 Oct 2012, at 22:51, Josh Wills <jw...@cloudera.com> wrote:
> >>>>
> >>>>> Hey all,
> >>>>>
> >>>>> I'd like to get our final preparations in place for the 0.4.0 Crunch
> >>>>> release. Among the remaining issues, the only things that seem
> required
> >>>> for
> >>>>> the release are CRUNCH-63 (including the CHANGES.txt file), although
> it
> >>>>> would be great to include CRUNCH-105 (DOT files for the planner) if
> at
> >>>> all
> >>>>> possible.
> >>>>>
> >>>>> I cc'd Roman on here so we can be sure that we position the release
> to be
> >>>>> compatible w/Bigtop. Thanks to Matthias, we have both support for
> .tar.gz
> >>>>> release files and binary release artifacts. We're using Hadoop 1.0.3
> as
> >>>> our
> >>>>> default build target, although it is possible to do the build
> against the
> >>>>> Hadoop 2.0.x APIs via the crunch.platform property in the top-level
> POM
> >>>>> file.
> >>>>>
> >>>>> Question for everyone: is there anything else you would like to
> include
> >>>> in
> >>>>> 0.4.0 that is close to completion?
> >>>>> Question for Roman: is there anything else we should double-check on
> for
> >>>>> Bigtop compatibility?
> >>>>>
> >>>>> Thanks,
> >>>>> Josh
> >>>>>
> >>>>> --
> >>>>> Director of Data Science
> >>>>> Cloudera <http://www.cloudera.com>
> >>>>> Twitter: @josh_wills <http://twitter.com/josh_wills>
> >>>>
> >>>
> >>
>
Re: Crunch 0.4.0 release preparation
Posted by Gabriel Reid <ga...@gmail.com>.
Ok, I'll commit CRUNCH-105 (very) shortly.
On 31 Oct 2012, at 18:21, Matthias Friedrich <ma...@mafr.de> wrote:
> Hi,
>
> like Gabriel and Rahul I'd really like to see CRUNCH-105 (which looks
> good to commit, BTW) as part of the release. Other than that, I agree
> that waiting for some of the other open issues would delay the release
> for too long.
>
> We've added cool stuff since 0.3.0, so let's release it :)
>
> Regards,
> Matthias
>
> On Wednesday, 2012-10-31, Rahul wrote:
>> Hi Josh,
>>
>> CRUNCH-105 would be a great new feature to have. CRUNCH-97 can be a
>> nice add-on if possible.
>> Also I would like to see fixes for CRUNCH-51 and CRUNCH-60 but I
>> think they are not ready yet and should not hold the release.
>>
>> regards,
>> Rahul
>>
>> On 31-10-2012 03:58, Josh Wills wrote:
>>> On Tue, Oct 30, 2012 at 3:08 PM, Gabriel Reid <ga...@gmail.com>wrote:
>>>
>>>> Hi Josh,
>>>>
>>>> Yep, I'd like to include CRUNCH-105 as well if possible.
>>>>
>>>> The only other potential issue that I see for inclusion is CRUNCH-51
>>>> (fixing sorting to use multiple reducers). I actually think that this one
>>>> is
>>>> still too far from being ready, but I just want to put that out there to
>>>> make
>>>> sure it's clear that it's not (yet) in.
>>>>
>>> Understood. I would have liked to get MSCR fusion and the text parsing
>>> stuff in too, but they're not there yet.
>>>
>>>
>>>> - Gabriel
>>>>
>>>> On 30 Oct 2012, at 22:51, Josh Wills <jw...@cloudera.com> wrote:
>>>>
>>>>> Hey all,
>>>>>
>>>>> I'd like to get our final preparations in place for the 0.4.0 Crunch
>>>>> release. Among the remaining issues, the only things that seem required
>>>> for
>>>>> the release are CRUNCH-63 (including the CHANGES.txt file), although it
>>>>> would be great to include CRUNCH-105 (DOT files for the planner) if at
>>>> all
>>>>> possible.
>>>>>
>>>>> I cc'd Roman on here so we can be sure that we position the release to be
>>>>> compatible w/Bigtop. Thanks to Matthias, we have both support for .tar.gz
>>>>> release files and binary release artifacts. We're using Hadoop 1.0.3 as
>>>> our
>>>>> default build target, although it is possible to do the build against the
>>>>> Hadoop 2.0.x APIs via the crunch.platform property in the top-level POM
>>>>> file.
>>>>>
>>>>> Question for everyone: is there anything else you would like to include
>>>> in
>>>>> 0.4.0 that is close to completion?
>>>>> Question for Roman: is there anything else we should double-check on for
>>>>> Bigtop compatibility?
>>>>>
>>>>> Thanks,
>>>>> Josh
>>>>>
>>>>> --
>>>>> Director of Data Science
>>>>> Cloudera <http://www.cloudera.com>
>>>>> Twitter: @josh_wills <http://twitter.com/josh_wills>
>>>>
>>>
>>
Re: Crunch 0.4.0 release preparation
Posted by Matthias Friedrich <ma...@mafr.de>.
Hi,
like Gabriel and Rahul I'd really like to see CRUNCH-105 (which looks
good to commit, BTW) as part of the release. Other than that, I agree
that waiting for some of the other open issues would delay the release
for too long.
We've added cool stuff since 0.3.0, so let's release it :)
Regards,
Matthias
On Wednesday, 2012-10-31, Rahul wrote:
> Hi Josh,
>
> CRUNCH-105 would be a great new feature to have. CRUNCH-97 can be a
> nice add-on if possible.
> Also I would like to see fixes for CRUNCH-51 and CRUNCH-60 but I
> think they are not ready yet and should not hold the release.
>
> regards,
> Rahul
>
> On 31-10-2012 03:58, Josh Wills wrote:
> >On Tue, Oct 30, 2012 at 3:08 PM, Gabriel Reid <ga...@gmail.com>wrote:
> >
> >>Hi Josh,
> >>
> >>Yep, I'd like to include CRUNCH-105 as well if possible.
> >>
> >>The only other potential issue that I see for inclusion is CRUNCH-51
> >>(fixing sorting to use multiple reducers). I actually think that this one
> >>is
> >>still too far from being ready, but I just want to put that out there to
> >>make
> >>sure it's clear that it's not (yet) in.
> >>
> >Understood. I would have liked to get MSCR fusion and the text parsing
> >stuff in too, but they're not there yet.
> >
> >
> >>- Gabriel
> >>
> >>On 30 Oct 2012, at 22:51, Josh Wills <jw...@cloudera.com> wrote:
> >>
> >>>Hey all,
> >>>
> >>>I'd like to get our final preparations in place for the 0.4.0 Crunch
> >>>release. Among the remaining issues, the only things that seem required
> >>for
> >>>the release are CRUNCH-63 (including the CHANGES.txt file), although it
> >>>would be great to include CRUNCH-105 (DOT files for the planner) if at
> >>all
> >>>possible.
> >>>
> >>>I cc'd Roman on here so we can be sure that we position the release to be
> >>>compatible w/Bigtop. Thanks to Matthias, we have both support for .tar.gz
> >>>release files and binary release artifacts. We're using Hadoop 1.0.3 as
> >>our
> >>>default build target, although it is possible to do the build against the
> >>>Hadoop 2.0.x APIs via the crunch.platform property in the top-level POM
> >>>file.
> >>>
> >>>Question for everyone: is there anything else you would like to include
> >>in
> >>>0.4.0 that is close to completion?
> >>>Question for Roman: is there anything else we should double-check on for
> >>>Bigtop compatibility?
> >>>
> >>>Thanks,
> >>>Josh
> >>>
> >>>--
> >>>Director of Data Science
> >>>Cloudera <http://www.cloudera.com>
> >>>Twitter: @josh_wills <http://twitter.com/josh_wills>
> >>
> >
>
Re: Crunch 0.4.0 release preparation
Posted by Rahul <rs...@xebia.com>.
Hi Josh,
CRUNCH-105 would be a great new feature to have. CRUNCH-97 can be a nice
add-on if possible.
Also I would like to see fixes for CRUNCH-51 and CRUNCH-60 but I think
they are not ready yet and should not hold the release.
regards,
Rahul
On 31-10-2012 03:58, Josh Wills wrote:
> On Tue, Oct 30, 2012 at 3:08 PM, Gabriel Reid <ga...@gmail.com>wrote:
>
>> Hi Josh,
>>
>> Yep, I'd like to include CRUNCH-105 as well if possible.
>>
>> The only other potential issue that I see for inclusion is CRUNCH-51
>> (fixing sorting to use multiple reducers). I actually think that this one
>> is
>> still too far from being ready, but I just want to put that out there to
>> make
>> sure it's clear that it's not (yet) in.
>>
> Understood. I would have liked to get MSCR fusion and the text parsing
> stuff in too, but they're not there yet.
>
>
>> - Gabriel
>>
>> On 30 Oct 2012, at 22:51, Josh Wills <jw...@cloudera.com> wrote:
>>
>>> Hey all,
>>>
>>> I'd like to get our final preparations in place for the 0.4.0 Crunch
>>> release. Among the remaining issues, the only things that seem required
>> for
>>> the release are CRUNCH-63 (including the CHANGES.txt file), although it
>>> would be great to include CRUNCH-105 (DOT files for the planner) if at
>> all
>>> possible.
>>>
>>> I cc'd Roman on here so we can be sure that we position the release to be
>>> compatible w/Bigtop. Thanks to Matthias, we have both support for .tar.gz
>>> release files and binary release artifacts. We're using Hadoop 1.0.3 as
>> our
>>> default build target, although it is possible to do the build against the
>>> Hadoop 2.0.x APIs via the crunch.platform property in the top-level POM
>>> file.
>>>
>>> Question for everyone: is there anything else you would like to include
>> in
>>> 0.4.0 that is close to completion?
>>> Question for Roman: is there anything else we should double-check on for
>>> Bigtop compatibility?
>>>
>>> Thanks,
>>> Josh
>>>
>>> --
>>> Director of Data Science
>>> Cloudera <http://www.cloudera.com>
>>> Twitter: @josh_wills <http://twitter.com/josh_wills>
>>
>
Re: Crunch 0.4.0 release preparation
Posted by Josh Wills <jw...@cloudera.com>.
On Tue, Oct 30, 2012 at 3:08 PM, Gabriel Reid <ga...@gmail.com>wrote:
> Hi Josh,
>
> Yep, I'd like to include CRUNCH-105 as well if possible.
>
> The only other potential issue that I see for inclusion is CRUNCH-51
> (fixing sorting to use multiple reducers). I actually think that this one
> is
> still too far from being ready, but I just want to put that out there to
> make
> sure it's clear that it's not (yet) in.
>
Understood. I would have liked to get MSCR fusion and the text parsing
stuff in too, but they're not there yet.
>
> - Gabriel
>
> On 30 Oct 2012, at 22:51, Josh Wills <jw...@cloudera.com> wrote:
>
> > Hey all,
> >
> > I'd like to get our final preparations in place for the 0.4.0 Crunch
> > release. Among the remaining issues, the only things that seem required
> for
> > the release are CRUNCH-63 (including the CHANGES.txt file), although it
> > would be great to include CRUNCH-105 (DOT files for the planner) if at
> all
> > possible.
> >
> > I cc'd Roman on here so we can be sure that we position the release to be
> > compatible w/Bigtop. Thanks to Matthias, we have both support for .tar.gz
> > release files and binary release artifacts. We're using Hadoop 1.0.3 as
> our
> > default build target, although it is possible to do the build against the
> > Hadoop 2.0.x APIs via the crunch.platform property in the top-level POM
> > file.
> >
> > Question for everyone: is there anything else you would like to include
> in
> > 0.4.0 that is close to completion?
> > Question for Roman: is there anything else we should double-check on for
> > Bigtop compatibility?
> >
> > Thanks,
> > Josh
> >
> > --
> > Director of Data Science
> > Cloudera <http://www.cloudera.com>
> > Twitter: @josh_wills <http://twitter.com/josh_wills>
>
>
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: Crunch 0.4.0 release preparation
Posted by Gabriel Reid <ga...@gmail.com>.
Hi Josh,
Yep, I'd like to include CRUNCH-105 as well if possible.
The only other potential issue that I see for inclusion is CRUNCH-51
(fixing sorting to use multiple reducers). I actually think that this one is
still too far from being ready, but I just want to put that out there to make
sure it's clear that it's not (yet) in.
- Gabriel
On 30 Oct 2012, at 22:51, Josh Wills <jw...@cloudera.com> wrote:
> Hey all,
>
> I'd like to get our final preparations in place for the 0.4.0 Crunch
> release. Among the remaining issues, the only things that seem required for
> the release are CRUNCH-63 (including the CHANGES.txt file), although it
> would be great to include CRUNCH-105 (DOT files for the planner) if at all
> possible.
>
> I cc'd Roman on here so we can be sure that we position the release to be
> compatible w/Bigtop. Thanks to Matthias, we have both support for .tar.gz
> release files and binary release artifacts. We're using Hadoop 1.0.3 as our
> default build target, although it is possible to do the build against the
> Hadoop 2.0.x APIs via the crunch.platform property in the top-level POM
> file.
>
> Question for everyone: is there anything else you would like to include in
> 0.4.0 that is close to completion?
> Question for Roman: is there anything else we should double-check on for
> Bigtop compatibility?
>
> Thanks,
> Josh
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: Crunch 0.4.0 release preparation
Posted by Matthias Friedrich <ma...@mafr.de>.
Thanks. I'm no JIRA black belt, so I won't touch anything :)
We have experts at work though that I can ask, unless someone here
knows what to do. Guys?
Regards,
Matthias
On Saturday, 2012-11-03, Josh Wills wrote:
> 0.5.0 created, but I couldn't figure out how to configure the rich text
> renderer. I made you an admin so you can take a look.
>
>
> On Sat, Nov 3, 2012 at 1:55 AM, Matthias Friedrich <ma...@mafr.de> wrote:
>
> > Hi,
> >
> > I'll use this thread for any release related information :)
> >
> > Josh, could you create a 0.5.0 version in JIRA? Also, while you're at
> > it, is it possible to switch our description and comment fields to
> > rich text? It would be nice to have some support for code examples and
> > block quotes.
> >
> > Regards,
> > Matthias
> >
> > On Tuesday, 2012-10-30, Josh Wills wrote:
> > > Hey all,
> > >
> > > I'd like to get our final preparations in place for the 0.4.0 Crunch
> > > release. Among the remaining issues, the only things that seem required
> > for
> > > the release are CRUNCH-63 (including the CHANGES.txt file), although it
> > > would be great to include CRUNCH-105 (DOT files for the planner) if at
> > all
> > > possible.
> > >
> > > I cc'd Roman on here so we can be sure that we position the release to be
> > > compatible w/Bigtop. Thanks to Matthias, we have both support for .tar.gz
> > > release files and binary release artifacts. We're using Hadoop 1.0.3 as
> > our
> > > default build target, although it is possible to do the build against the
> > > Hadoop 2.0.x APIs via the crunch.platform property in the top-level POM
> > > file.
> > >
> > > Question for everyone: is there anything else you would like to include
> > in
> > > 0.4.0 that is close to completion?
> > > Question for Roman: is there anything else we should double-check on for
> > > Bigtop compatibility?
> > >
> > > Thanks,
> > > Josh
> > >
> > > --
> > > Director of Data Science
> > > Cloudera <http://www.cloudera.com>
> > > Twitter: @josh_wills <http://twitter.com/josh_wills>
> >
Re: Crunch 0.4.0 release preparation
Posted by Josh Wills <jo...@gmail.com>.
0.5.0 created, but I couldn't figure out how to configure the rich text
renderer. I made you an admin so you can take a look.
On Sat, Nov 3, 2012 at 1:55 AM, Matthias Friedrich <ma...@mafr.de> wrote:
> Hi,
>
> I'll use this thread for any release related information :)
>
> Josh, could you create a 0.5.0 version in JIRA? Also, while you're at
> it, is it possible to switch our description and comment fields to
> rich text? It would be nice to have some support for code examples and
> block quotes.
>
> Regards,
> Matthias
>
> On Tuesday, 2012-10-30, Josh Wills wrote:
> > Hey all,
> >
> > I'd like to get our final preparations in place for the 0.4.0 Crunch
> > release. Among the remaining issues, the only things that seem required
> for
> > the release are CRUNCH-63 (including the CHANGES.txt file), although it
> > would be great to include CRUNCH-105 (DOT files for the planner) if at
> all
> > possible.
> >
> > I cc'd Roman on here so we can be sure that we position the release to be
> > compatible w/Bigtop. Thanks to Matthias, we have both support for .tar.gz
> > release files and binary release artifacts. We're using Hadoop 1.0.3 as
> our
> > default build target, although it is possible to do the build against the
> > Hadoop 2.0.x APIs via the crunch.platform property in the top-level POM
> > file.
> >
> > Question for everyone: is there anything else you would like to include
> in
> > 0.4.0 that is close to completion?
> > Question for Roman: is there anything else we should double-check on for
> > Bigtop compatibility?
> >
> > Thanks,
> > Josh
> >
> > --
> > Director of Data Science
> > Cloudera <http://www.cloudera.com>
> > Twitter: @josh_wills <http://twitter.com/josh_wills>
>
Re: Crunch 0.4.0 release preparation
Posted by Matthias Friedrich <ma...@mafr.de>.
Hi,
I'll use this thread for any release related information :)
Josh, could you create a 0.5.0 version in JIRA? Also, while you're at
it, is it possible to switch our description and comment fields to
rich text? It would be nice to have some support for code examples and
block quotes.
Regards,
Matthias
On Tuesday, 2012-10-30, Josh Wills wrote:
> Hey all,
>
> I'd like to get our final preparations in place for the 0.4.0 Crunch
> release. Among the remaining issues, the only things that seem required for
> the release are CRUNCH-63 (including the CHANGES.txt file), although it
> would be great to include CRUNCH-105 (DOT files for the planner) if at all
> possible.
>
> I cc'd Roman on here so we can be sure that we position the release to be
> compatible w/Bigtop. Thanks to Matthias, we have both support for .tar.gz
> release files and binary release artifacts. We're using Hadoop 1.0.3 as our
> default build target, although it is possible to do the build against the
> Hadoop 2.0.x APIs via the crunch.platform property in the top-level POM
> file.
>
> Question for everyone: is there anything else you would like to include in
> 0.4.0 that is close to completion?
> Question for Roman: is there anything else we should double-check on for
> Bigtop compatibility?
>
> Thanks,
> Josh
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>