You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by Doug Cutting <cu...@apache.org> on 2010/07/19 22:05:55 UTC
1.4.0 release soon?
I'd like to make a 1.4.0 release of Avro in the next month.
The list of unresolved 1.4.0 issues is at:
http://tinyurl.com/avro140todo
Can developers please look at this list?
If there are issues that you're assigned that you don't expect to fix
soon, please unset their fix-version.
If there are issues that you feel absolutely must be fixed in 1.4.0,
please mark them as blockers.
Thanks,
Doug
Re: 1.4.0 release soon?
Posted by Scott Carey <sc...@richrelevance.com>.
>
> P.S. Anyone ever on the IRC chat? I've tried several times to find it and never seen the channel listed, let alone anyone in it. Is http://avro.apache.org/irc.html correct? I've managed to get this far in life as an IRC noob, so it could be user error :)
>
Well, I'm a noob. I guess the channel just isn't indexed, it does not show up on my client's list of channels but manually joining works just fine.
Re: 1.4.0 release soon?
Posted by Jeff Hammerbacher <ha...@cloudera.com>.
>
> Would you like me to try to knock off AVRO-573 this week?
>
Yes, please! With AVRO-573 and AVRO-544, the HBase gateway will be in very
good shape.
Re: 1.4.0 release soon?
Posted by Doug Cutting <cu...@apache.org>.
On 07/19/2010 03:26 PM, Jeff Hammerbacher wrote:
> AVRO-230 (shared directory with test schemas) would be great, but I'm not
> sure it's a blocker. Do we want to make AVRO-570 (Python implementation of
> MapReduce) a blocker? I can see the pros (suss out sharp edges in the
> implementation) and cons (it's a new feature that's going to take a bit of
> work to implement).
As bad as I want this, in general I don't like to hold off releases for
a feature if there are already some other useful new features that are
ready. However if waiting a week or two will make a difference then we
should consider it. Has someone committed to working on AVRO-570?
FYI, I've talked to Flip from InfoChimps, and he's hoping to start work
on a Ruby implementation of MapReduce sometime soon. It'd be wonderful
to have either of these in 1.4.0, but if they don't make that, we can
always make another release soon after that does include them.
> I've added AVRO-588 (Avro usable and testable on Python 2.4). I'll need to
> augment the Python RPC to handle system errors correctly after AVRO-573 goes
> in, so I've filed AVRO-599 as a blocker as well. Lastly, I've been meaning
> to get to AVRO-528 (Twisted support) and AVRO-283 (change Python
> implementation from dictionaries to objects) forever, so the release seems
> as good as ever.
Would you like me to try to knock off AVRO-573 this week?
Doug
Re: 1.4.0 release soon?
Posted by Jeff Hammerbacher <ha...@cloudera.com>.
Well, my detailed plan for now is to look at Doug's implementation on the
Java side and mimic it. Once I get it working, I'll remove the Java-isms and
make it more Pythonic.
Creating classes and dynamically adding properties seems like how things
will work; it's going to be a bit "magical". If you want to sketch out your
implementation direction further, or even get a little code down, I'll be
happy to spend a bit more time on my critique.
On Mon, Aug 2, 2010 at 11:11 AM, Esteve Fernandez <es...@apache.org> wrote:
> Hi
>
> On Tue, Jul 20, 2010 at 12:26 AM, Jeff Hammerbacher <ha...@cloudera.com>
> wrote:
> > I've added AVRO-588 (Avro usable and testable on Python 2.4). I'll need
> to
> > augment the Python RPC to handle system errors correctly after AVRO-573
> goes
> > in, so I've filed AVRO-599 as a blocker as well. Lastly, I've been
> meaning
> > to get to AVRO-528 (Twisted support) and AVRO-283 (change Python
> > implementation from dictionaries to objects) forever, so the release
> seems
> > as good as ever.
>
> I'd love to see AVRO-283, actually I started putting together a patch
> a while ago, but I thought it was too "magical" and wouldn't get as
> chance of getting accepted, as it created classes on the fly and so
> on.
>
> How did you plan to implement it? I'd be very happy to help if I can.
>
> Cheers.
>
Re: 1.4.0 release soon?
Posted by Esteve Fernandez <es...@apache.org>.
Hi
On Tue, Jul 20, 2010 at 12:26 AM, Jeff Hammerbacher <ha...@cloudera.com> wrote:
> I've added AVRO-588 (Avro usable and testable on Python 2.4). I'll need to
> augment the Python RPC to handle system errors correctly after AVRO-573 goes
> in, so I've filed AVRO-599 as a blocker as well. Lastly, I've been meaning
> to get to AVRO-528 (Twisted support) and AVRO-283 (change Python
> implementation from dictionaries to objects) forever, so the release seems
> as good as ever.
I'd love to see AVRO-283, actually I started putting together a patch
a while ago, but I thought it was too "magical" and wouldn't get as
chance of getting accepted, as it created classes on the fly and so
on.
How did you plan to implement it? I'd be very happy to help if I can.
Cheers.
Re: 1.4.0 release soon?
Posted by Jeff Hammerbacher <ha...@cloudera.com>.
AVRO-230 (shared directory with test schemas) would be great, but I'm not
sure it's a blocker. Do we want to make AVRO-570 (Python implementation of
MapReduce) a blocker? I can see the pros (suss out sharp edges in the
implementation) and cons (it's a new feature that's going to take a bit of
work to implement).
I've added AVRO-588 (Avro usable and testable on Python 2.4). I'll need to
augment the Python RPC to handle system errors correctly after AVRO-573 goes
in, so I've filed AVRO-599 as a blocker as well. Lastly, I've been meaning
to get to AVRO-528 (Twisted support) and AVRO-283 (change Python
implementation from dictionaries to objects) forever, so the release seems
as good as ever.
If anyone has Python chops and wants to help with the release, please
volunteer now! I'm also in IRC as "hammer" all day.
On Mon, Jul 19, 2010 at 1:42 PM, Scott Carey <sc...@richrelevance.com>wrote:
> I don't want to hold up 1.4.0 to get the Avro<>Pig stuff in there, that
> needs a lot of review and some refinement. It would be nice, but is not
> nearly as important as the other stuff in this release.
>
> I think we may have to move
> https://issues.apache.org/jira/browse/AVRO-545
> into 1.4. Our Java dependencies are a mess and we should be producing more
> refined jars -- 'core', 'mapred', 'devel', etc. This not only helps
> prevent circular dependancies, but also helps projects choose the feature
> subset they are interested in and not carry the baggage + dependancies of
> what they are not using.
>
> As a part of the pig work, I will look into producing multiple jar outputs
> and cleaning up our Java dependancies in the build.
>
> -Scott
>
> P.S. Anyone ever on the IRC chat? I've tried several times to find it and
> never seen the channel listed, let alone anyone in it. Is
> http://avro.apache.org/irc.html correct? I've managed to get this far in
> life as an IRC noob, so it could be user error :)
>
> On Jul 19, 2010, at 1:05 PM, Doug Cutting wrote:
>
> > I'd like to make a 1.4.0 release of Avro in the next month.
> >
> > The list of unresolved 1.4.0 issues is at:
> >
> > http://tinyurl.com/avro140todo
> >
> > Can developers please look at this list?
> >
> > If there are issues that you're assigned that you don't expect to fix
> > soon, please unset their fix-version.
> >
> > If there are issues that you feel absolutely must be fixed in 1.4.0,
> > please mark them as blockers.
> >
> > Thanks,
> >
> > Doug
>
>
Re: 1.4.0 release soon?
Posted by Scott Carey <sc...@richrelevance.com>.
I don't want to hold up 1.4.0 to get the Avro<>Pig stuff in there, that needs a lot of review and some refinement. It would be nice, but is not nearly as important as the other stuff in this release.
I think we may have to move
https://issues.apache.org/jira/browse/AVRO-545
into 1.4. Our Java dependencies are a mess and we should be producing more refined jars -- 'core', 'mapred', 'devel', etc. This not only helps prevent circular dependancies, but also helps projects choose the feature subset they are interested in and not carry the baggage + dependancies of what they are not using.
As a part of the pig work, I will look into producing multiple jar outputs and cleaning up our Java dependancies in the build.
-Scott
P.S. Anyone ever on the IRC chat? I've tried several times to find it and never seen the channel listed, let alone anyone in it. Is http://avro.apache.org/irc.html correct? I've managed to get this far in life as an IRC noob, so it could be user error :)
On Jul 19, 2010, at 1:05 PM, Doug Cutting wrote:
> I'd like to make a 1.4.0 release of Avro in the next month.
>
> The list of unresolved 1.4.0 issues is at:
>
> http://tinyurl.com/avro140todo
>
> Can developers please look at this list?
>
> If there are issues that you're assigned that you don't expect to fix
> soon, please unset their fix-version.
>
> If there are issues that you feel absolutely must be fixed in 1.4.0,
> please mark them as blockers.
>
> Thanks,
>
> Doug
Re: 1.4.0 release soon?
Posted by Doug Cutting <cu...@apache.org>.
I don't yet see a blocker with fix-version of 1.4.0 in Jira for this.
Can you please file one? Thanks!
Doug
On 08/24/2010 06:47 PM, Jeff Hodges wrote:
> We've got a new bug in the ruby side that needs a review. Just found today.
>
> On Tue, Aug 24, 2010 at 3:00 PM, Doug Cutting<cu...@apache.org> wrote:
>> On 08/24/2010 02:27 PM, Doug Cutting wrote:
>>>
>>> Unless I hear objections, tomorrow I will:
>>> - commit AVRO-86 and AVRO-598
>>> - revert lang/c to 1.3, re-opening C jiras fixed since 1.3
>>> - start trying to roll a 1.4.0 release candidate
>>>
>>> If there are any issues that you feel must be in 1.4.0, please mark them
>>> as blockers.
>>
>> One more thing I'll do tomorrow:
>>
>> - clear the fix-version of all non-blocker 1.4.0 issues
>>
>> Doug
>>
Re: 1.4.0 release soon?
Posted by Jeff Hodges <jh...@twitter.com>.
We've got a new bug in the ruby side that needs a review. Just found today.
On Tue, Aug 24, 2010 at 3:00 PM, Doug Cutting <cu...@apache.org> wrote:
> On 08/24/2010 02:27 PM, Doug Cutting wrote:
>>
>> Unless I hear objections, tomorrow I will:
>> - commit AVRO-86 and AVRO-598
>> - revert lang/c to 1.3, re-opening C jiras fixed since 1.3
>> - start trying to roll a 1.4.0 release candidate
>>
>> If there are any issues that you feel must be in 1.4.0, please mark them
>> as blockers.
>
> One more thing I'll do tomorrow:
>
> - clear the fix-version of all non-blocker 1.4.0 issues
>
> Doug
>
Re: 1.4.0 release soon?
Posted by Bruce Mitchener <br...@gmail.com>.
Sounds good to me.
- Bruce
On Wed, Aug 25, 2010 at 5:00 AM, Doug Cutting <cu...@apache.org> wrote:
> On 08/24/2010 02:27 PM, Doug Cutting wrote:
>
>> Unless I hear objections, tomorrow I will:
>> - commit AVRO-86 and AVRO-598
>> - revert lang/c to 1.3, re-opening C jiras fixed since 1.3
>> - start trying to roll a 1.4.0 release candidate
>>
>> If there are any issues that you feel must be in 1.4.0, please mark them
>> as blockers.
>>
>
> One more thing I'll do tomorrow:
>
> - clear the fix-version of all non-blocker 1.4.0 issues
>
> Doug
>
Re: 1.4.0 release soon?
Posted by Doug Cutting <cu...@apache.org>.
On 08/24/2010 02:27 PM, Doug Cutting wrote:
> Unless I hear objections, tomorrow I will:
> - commit AVRO-86 and AVRO-598
> - revert lang/c to 1.3, re-opening C jiras fixed since 1.3
> - start trying to roll a 1.4.0 release candidate
>
> If there are any issues that you feel must be in 1.4.0, please mark them
> as blockers.
One more thing I'll do tomorrow:
- clear the fix-version of all non-blocker 1.4.0 issues
Doug
Re: 1.4.0 release soon?
Posted by Doug Cutting <cu...@apache.org>.
Unless I hear objections, tomorrow I will:
- commit AVRO-86 and AVRO-598
- revert lang/c to 1.3, re-opening C jiras fixed since 1.3
- start trying to roll a 1.4.0 release candidate
If there are any issues that you feel must be in 1.4.0, please mark them
as blockers.
Thanks,
Doug
On 07/19/2010 01:05 PM, Doug Cutting wrote:
> I'd like to make a 1.4.0 release of Avro in the next month.
>
> The list of unresolved 1.4.0 issues is at:
>
> http://tinyurl.com/avro140todo
>
> Can developers please look at this list?
>
> If there are issues that you're assigned that you don't expect to fix
> soon, please unset their fix-version.
>
> If there are issues that you feel absolutely must be fixed in 1.4.0,
> please mark them as blockers.
>
> Thanks,
>
> Doug
Re: 1.4.0 release soon?
Posted by Doug Cutting <cu...@apache.org>.
On 07/19/2010 03:44 PM, Bruce Mitchener wrote:
> Avro C is in bad shape / a bad state. It was in the middle of some
> breaking API changes (and more are needed).
That's a shame. Generally we should try to keep trunk in a
ready-to-release state. Long-term changes that can't be done in a patch
should generally be done in a branch, not trunk.
> It is probably better to look at reverting some of the changes and
> just shipping what was in 1.3.x (even though the API there is not
> complete and one can't actually use it for much).
>
> I'm on the road for the next couple of days (in Macau), so I won't be
> able to do much in the short term.
Will you have a chance to work on this after that? I doubt I'd roll a
1.4.0 candidate for at least a week or so, and am willing to wait a bit
if improvements are imminent.
Doug
Re: 1.4.0 release soon?
Posted by Doug Cutting <cu...@apache.org>.
On 08/24/2010 01:29 PM, Matt Massie wrote:
> Is it possible to make the current trunk a devel branch before reverting for
> Bruce and I to finish up the work and merge into 1.4 later?
It's possible to do this anytime. All that someone would need to do is:
svn up -r '{2010-08-23}'
svn cp . https://svn.apache.org/repos/asf/avro/branches/c-dev
Doug
Re: 1.4.0 release soon?
Posted by Matt Massie <ma...@cloudera.com>.
+1 on prior to branching 1.4, revert trunk/lang/c to branch-1.3/lang/c.
Is it possible to make the current trunk a devel branch before reverting for
Bruce and I to finish up the work and merge into 1.4 later?
--
Matt
*Hadoop World 2010 -- October 12 | New York -- **Register
today!*<http://www.cloudera.com/company/press-center/hadoop-world-nyc/>
On Tue, Aug 24, 2010 at 12:07 PM, Doug Cutting <cu...@apache.org> wrote:
> On 07/19/2010 03:44 PM, Bruce Mitchener wrote:
>
>> Avro C is in bad shape / a bad state. It was in the middle of some
>> breaking API changes (and more are needed).
>>
>> It is probably better to look at reverting some of the changes and
>> just shipping what was in 1.3.x (even though the API there is not
>> complete and one can't actually use it for much).
>>
>
> The changes to lang/c since 1.3 was branched can be listed with:
>
> svn log -r 927600:HEAD lang/c
>
> There have been no lang/c changes merged to the 1.3 branch.
>
> The changes in trunk sound incomplete. I see two options:
> - prior to branching 1.4, revert trunk/lang/c to branch-1.3/lang/c; or
> - after branching 1.4, revert branch-1.4/lang/c to branch-1.3/lang/c
>
> My preference is to keep trunk releasable, so I prefer the former. We'd
> then re-open the Jira issues that were closed. If someone wants to continue
> the effort started, they can revive the reverted changes in a feature branch
> and merge them to trunk when they're releasable.
>
> Does that sound reasonable?
>
> Doug
>
Re: 1.4.0 release soon?
Posted by Doug Cutting <cu...@apache.org>.
On 07/19/2010 03:44 PM, Bruce Mitchener wrote:
> Avro C is in bad shape / a bad state. It was in the middle of some
> breaking API changes (and more are needed).
>
> It is probably better to look at reverting some of the changes and
> just shipping what was in 1.3.x (even though the API there is not
> complete and one can't actually use it for much).
The changes to lang/c since 1.3 was branched can be listed with:
svn log -r 927600:HEAD lang/c
There have been no lang/c changes merged to the 1.3 branch.
The changes in trunk sound incomplete. I see two options:
- prior to branching 1.4, revert trunk/lang/c to branch-1.3/lang/c; or
- after branching 1.4, revert branch-1.4/lang/c to branch-1.3/lang/c
My preference is to keep trunk releasable, so I prefer the former. We'd
then re-open the Jira issues that were closed. If someone wants to
continue the effort started, they can revive the reverted changes in a
feature branch and merge them to trunk when they're releasable.
Does that sound reasonable?
Doug
Re: 1.4.0 release soon?
Posted by Bruce Mitchener <br...@gmail.com>.
Avro C is in bad shape / a bad state. It was in the middle of some
breaking API changes (and more are needed).
It is probably better to look at reverting some of the changes and
just shipping what was in 1.3.x (even though the API there is not
complete and one can't actually use it for much).
I'm on the road for the next couple of days (in Macau), so I won't be
able to do much in the short term.
- Bruce
On Tue, Jul 20, 2010 at 3:05 AM, Doug Cutting <cu...@apache.org> wrote:
> I'd like to make a 1.4.0 release of Avro in the next month.
>
> The list of unresolved 1.4.0 issues is at:
>
> http://tinyurl.com/avro140todo
>
> Can developers please look at this list?
>
> If there are issues that you're assigned that you don't expect to fix soon,
> please unset their fix-version.
>
> If there are issues that you feel absolutely must be fixed in 1.4.0, please
> mark them as blockers.
>
> Thanks,
>
> Doug
>