You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by Ryan Blue <bl...@cloudera.com> on 2015/11/05 18:43:40 UTC
Re: [DISCUSS] Release and code management
Phil or Sam, any ideas about how to keep release management simple, but
be able to avoid blocking specific languages on under-maintained ones?
Also, looking at the release history we've had 3 releases in the last 2
years, and that's being generous to include 1.7.5 that was released in
August 2013. I don't think more release overhead would be that big of a
problem, and would be well worth keeping the languages that are well
maintained released and up-to-date.
rb
On 10/30/2015 09:37 AM, Ryan Blue wrote:
> I think Sean is right that we could continue to release several at once.
> We would almost certainly continue this practice for several languages
> that are mostly unmaintained (like perl and php). I also expect each
> language's release cadence to reflect the activity in that language,
> which I think is very important to maintain.
>
> I also don't want to underestimate the drawback of having a single
> version for multiple implementations. We can't use semantic verisoning
> for any of the implementations. If we bump the minor version (!) because
> of a breaking change in Java, but aren't making breaking changes to C,
> this is confusing to users.
>
> If we don't separate release vehicles, how can we improve version
> conventions?
>
> And how do we ensure timely releases that aren't blocked by other
> implementations? This affects how attractive this project is to new
> contributors. If the releases are seldom and contributions aren't
> available for months at a time, I think we have a problem.
>
> rb
>
> On 10/29/2015 04:51 PM, Philip Zeyliger wrote:
>> -0.
>>
>> If you divide the world into N releases, you'll end up having to do
>> release
>> management N times. I think this will make doing releases that much more
>> complicated, time-consuming, and error-prone.
>>
>> Note that you could separate release trains while remaining in a single
>> repo. I'd certainly prefer that than separating into many smaller repos.
>>
>> -- Philip
>>
>> On Thu, Oct 29, 2015 at 11:31 AM Ryan Blue <bl...@cloudera.com> wrote:
>>
>>> On 10/29/2015 11:28 AM, Sean Busbey wrote:
>>>> On Oct 29, 2015 1:19 PM, "Ryan Blue" <bl...@cloudera.com> wrote:
>>>>
>>>> Where would the language interop tests live if we don't break them out?
>>>>
>>>> (We already have interop tests, in case that was lost in my original
>>> email.)
>>>
>>> We could either keep them where they are or add a separate repo. Running
>>> them with a release candidate would have to be part of the release
>>> checks.
>>>
>>> rb
>>>
>>>
>>> --
>>> Ryan Blue
>>> Software Engineer
>>> Cloudera, Inc.
>>>
>>
>
>
--
Ryan Blue
Software Engineer
Cloudera, Inc.
Re: [DISCUSS] Release and code management
Posted by Ryan Blue <bl...@cloudera.com>.
On 11/16/2015 08:10 AM, Sam Groth wrote:
> So I have developed for a similar scenario where we had APIs in 3 different languages that needed to be kept in sync. The releases were split, and we only had some high level tests to verify compatibility and therefore had to be very careful what changes we made to avoid failures across many different use cases of our APIs. There were a few people who were required to review any changes to this API. I feel that in a project with as many implementations as Avro, it will be difficult to have this same level of review. If it's possible to write some linked tests for most of the implementations like Niels suggested, it should be ok to split the releases. We would still have to pay close attention to changes in these test cases.
>
> Sam
I agree with the need to validate that files are to spec, but we
currently have that problem regardless of whether we release components
as a group or individually. Right now we don't do much cross-language
testing at all so I think this should be a goal (an important one!), not
a requirement for changing our releases.
> On Saturday, November 14, 2015 7:54 AM, Niels Basjes <Ni...@basjes.nl> wrote:
>
>
> Hi,
>
> First of all a +1 from me to move to git.
Sounds like we have consensus around moving to git, so at least we can
get that started.
> Regarding the "how many parts and the release cycle";
>
> In general I'm in the "release often" camp.
> Yet I understand the needless confusion if a certain language has not
> changed.
>
> How about this idea:
>
> We create a separate project (avro-spec) with the specification/testcases
> and such that guard the language specification. All language specific
> implementations must reference this (perhaps by using the git feature of a
> "linked submodule") and run the tests during the language specific deploy.
> The version of this module therefor the version of the Avro format
> specification.
This is similar to what we do in Parquet: we have a parquet-format
project that has meaningful versions for file compatibility. The version
for this dependency doesn't necessarily determine the version of a file
because the API can choose what underlying version it uses.
> I think it would be good if these modules use a version that essentially is
> <avro-spec version>-<library version>
>
> This way the libraries can have a separate release cycle and still clearly
> indicate the supported Avro specifcation.
I think this makes sense. Right now, while we have only one version of
the file format, do we need to include the format version in all version
numbers? We could put off that step when we have more than one format
version to worry about.
rb
--
Ryan Blue
Software Engineer
Cloudera, Inc.
Re: [DISCUSS] Release and code management
Posted by Sam Groth <sg...@yahoo-inc.com.INVALID>.
So I have developed for a similar scenario where we had APIs in 3 different languages that needed to be kept in sync. The releases were split, and we only had some high level tests to verify compatibility and therefore had to be very careful what changes we made to avoid failures across many different use cases of our APIs. There were a few people who were required to review any changes to this API. I feel that in a project with as many implementations as Avro, it will be difficult to have this same level of review. If it's possible to write some linked tests for most of the implementations like Niels suggested, it should be ok to split the releases. We would still have to pay close attention to changes in these test cases.
Sam
On Saturday, November 14, 2015 7:54 AM, Niels Basjes <Ni...@basjes.nl> wrote:
Hi,
First of all a +1 from me to move to git.
Regarding the "how many parts and the release cycle";
In general I'm in the "release often" camp.
Yet I understand the needless confusion if a certain language has not
changed.
How about this idea:
We create a separate project (avro-spec) with the specification/testcases
and such that guard the language specification. All language specific
implementations must reference this (perhaps by using the git feature of a
"linked submodule") and run the tests during the language specific deploy.
The version of this module therefor the version of the Avro format
specification.
I think it would be good if these modules use a version that essentially is
<avro-spec version>-<library version>
This way the libraries can have a separate release cycle and still clearly
indicate the supported Avro specifcation.
My 2ct
Niels Basjes
On Thu, Nov 12, 2015 at 6:34 PM, Ryan Blue <bl...@cloudera.com> wrote:
> My responses are inline.
>
> rb
>
> On 11/06/2015 07:32 AM, Tom White wrote:
>
>> I'm not sure that moving to a model where there are releases of individual
>> components will increase the frequency of releases. There will still need
>> to be a release manager for each component, and then there's a danger that
>> the less maintained components will not get released at all.
>>
>
> My concern is that new components, like Miki's fastavro (python) and
> Matthieu's js implementation, already have significantly faster release
> cycles (see [1] and [2]). We want those to be successful in the Avro
> community and I think that blocking their fast release cycles on languages
> without maintainers is a community problem. Not delivering timely releases
> discourages participation.
>
> I think a release manager per release, even if it is a python-only
> release, is fine. As I said, we've had less than one release per year
> lately so it isn't going to be that much overhead. And to your point about
> making releases easier, I think it is much easier to learn how to do
> releases with a small project.
>
> We will need to discuss what to do about languages that aren't maintained.
> (Maybe in a separate thread?) I don't think we need to deprecate them as
> Phil suggested, but I also don't think it is right to continue releasing
> code with new version numbers that hasn't been updated in a year or more. I
> think separating the releases is the right thing to do here too: it signals
> to users that the component is still "current" but hasn't been released in
> a while. We're neither discouraging use or participation by deprecating it,
> nor are we making it appear more active than it is.
>
> [1]: https://github.com/tebeka/fastavro/releases
> [2]: https://github.com/mtth/avsc/releases
>
> I would rather continue to make the release process easier (Docker helps a
>> lot) so that any committer can do it. We should be able to use the Docker
>> work to run tests for all components with Jenkins to ensure that trunk is
>> always in a releasable state.
>>
>
> Docker does help quite a lot, but it doesn't help when the problem isn't
> build-related. If Ruby contributors want to get a feature into a release or
> perl has a blocking bug or the C# implementation has license issues, we end
> up blocking all components.
>
> Where are we with the licensing issues? If we can get those worked out then
>> I'd like to make a release (of all components).
>>
>
> The source tarball is nearly ready and a few implementations have updated
> license docs in the binary artifacts. We have a couple days work to go.
>
>
> I'm +1 on moving to git.
>>
>> Cheers,
>> Tom
>>
>> On Thu, Nov 5, 2015 at 12:45 PM, Ryan Blue <bl...@cloudera.com> wrote:
>>
>> It isn't just license problems, either. Releases that include all of the
>>> languages can be blocked by bugs that need to be fixed in those languages
>>> that are suggested during release planning.
>>>
>>> It is also necessary to make sure the older language implementations
>>> still
>>> build and pass tests, which can mean, for example, installing php and
>>> fixing any tests that currently break. Tom's recent work to port the
>>> build
>>> to docker really helps this situation, but that took patches to
>>> unmaintained implementations and will still require maintenance.
>>>
>>> I also disagree that it's always okay to re-release artifacts. Everything
>>> is moving toward semantic versioning and I think that Avro should as
>>> well.
>>> It is confusing to users to have an identical library released with a
>>> version number that indicates a breaking change (though it appears not to
>>> be by semver rules).
>>>
>>> Each language should adopt a release cadence that works for its
>>> contributors so that those contributors are able to use their work in
>>> timely releases. Otherwise, I'm afraid that we will see fewer
>>> contributions
>>> because of the long release cycle we currently have.
>>>
>>> rb
>>>
>>> On 11/05/2015 10:09 AM, Sean Busbey wrote:
>>>
>>> we are currently blocked on all releases because of licensing errors
>>>> in under-maintained libraries.
>>>>
>>>> https://issues.apache.org/jira/browse/AVRO-1722
>>>>
>>>> essentially Ryan and I slowly work our way through understanding each
>>>> code base enough to do an evaluation and update things.
>>>>
>>>> It's been over 2 months now and it's a crappy situation to put our
>>>> contributors in.
>>>>
>>>>
>>>> On Thu, Nov 5, 2015 at 11:59 AM, Philip Zeyliger <ph...@cloudera.com>
>>>> wrote:
>>>>
>>>> I think it's always ok to re-release artifacts where nothing's changed.
>>>>> So, how can you be blocked on another language's implementation if you
>>>>> simply change the version number and re-release?
>>>>>
>>>>> -- Philip
>>>>>
>>>>>
>>>>
>>> --
>>> Ryan Blue
>>> Software Engineer
>>> Cloudera, Inc.
>>>
>>>
>>
>
> --
> Ryan Blue
> Software Engineer
> Cloudera, Inc.
>
--
Best regards / Met vriendelijke groeten,
Niels Basjes
Re: [DISCUSS] Release and code management
Posted by Niels Basjes <Ni...@basjes.nl>.
Hi,
First of all a +1 from me to move to git.
Regarding the "how many parts and the release cycle";
In general I'm in the "release often" camp.
Yet I understand the needless confusion if a certain language has not
changed.
How about this idea:
We create a separate project (avro-spec) with the specification/testcases
and such that guard the language specification. All language specific
implementations must reference this (perhaps by using the git feature of a
"linked submodule") and run the tests during the language specific deploy.
The version of this module therefor the version of the Avro format
specification.
I think it would be good if these modules use a version that essentially is
<avro-spec version>-<library version>
This way the libraries can have a separate release cycle and still clearly
indicate the supported Avro specifcation.
My 2ct
Niels Basjes
On Thu, Nov 12, 2015 at 6:34 PM, Ryan Blue <bl...@cloudera.com> wrote:
> My responses are inline.
>
> rb
>
> On 11/06/2015 07:32 AM, Tom White wrote:
>
>> I'm not sure that moving to a model where there are releases of individual
>> components will increase the frequency of releases. There will still need
>> to be a release manager for each component, and then there's a danger that
>> the less maintained components will not get released at all.
>>
>
> My concern is that new components, like Miki's fastavro (python) and
> Matthieu's js implementation, already have significantly faster release
> cycles (see [1] and [2]). We want those to be successful in the Avro
> community and I think that blocking their fast release cycles on languages
> without maintainers is a community problem. Not delivering timely releases
> discourages participation.
>
> I think a release manager per release, even if it is a python-only
> release, is fine. As I said, we've had less than one release per year
> lately so it isn't going to be that much overhead. And to your point about
> making releases easier, I think it is much easier to learn how to do
> releases with a small project.
>
> We will need to discuss what to do about languages that aren't maintained.
> (Maybe in a separate thread?) I don't think we need to deprecate them as
> Phil suggested, but I also don't think it is right to continue releasing
> code with new version numbers that hasn't been updated in a year or more. I
> think separating the releases is the right thing to do here too: it signals
> to users that the component is still "current" but hasn't been released in
> a while. We're neither discouraging use or participation by deprecating it,
> nor are we making it appear more active than it is.
>
> [1]: https://github.com/tebeka/fastavro/releases
> [2]: https://github.com/mtth/avsc/releases
>
> I would rather continue to make the release process easier (Docker helps a
>> lot) so that any committer can do it. We should be able to use the Docker
>> work to run tests for all components with Jenkins to ensure that trunk is
>> always in a releasable state.
>>
>
> Docker does help quite a lot, but it doesn't help when the problem isn't
> build-related. If Ruby contributors want to get a feature into a release or
> perl has a blocking bug or the C# implementation has license issues, we end
> up blocking all components.
>
> Where are we with the licensing issues? If we can get those worked out then
>> I'd like to make a release (of all components).
>>
>
> The source tarball is nearly ready and a few implementations have updated
> license docs in the binary artifacts. We have a couple days work to go.
>
>
> I'm +1 on moving to git.
>>
>> Cheers,
>> Tom
>>
>> On Thu, Nov 5, 2015 at 12:45 PM, Ryan Blue <bl...@cloudera.com> wrote:
>>
>> It isn't just license problems, either. Releases that include all of the
>>> languages can be blocked by bugs that need to be fixed in those languages
>>> that are suggested during release planning.
>>>
>>> It is also necessary to make sure the older language implementations
>>> still
>>> build and pass tests, which can mean, for example, installing php and
>>> fixing any tests that currently break. Tom's recent work to port the
>>> build
>>> to docker really helps this situation, but that took patches to
>>> unmaintained implementations and will still require maintenance.
>>>
>>> I also disagree that it's always okay to re-release artifacts. Everything
>>> is moving toward semantic versioning and I think that Avro should as
>>> well.
>>> It is confusing to users to have an identical library released with a
>>> version number that indicates a breaking change (though it appears not to
>>> be by semver rules).
>>>
>>> Each language should adopt a release cadence that works for its
>>> contributors so that those contributors are able to use their work in
>>> timely releases. Otherwise, I'm afraid that we will see fewer
>>> contributions
>>> because of the long release cycle we currently have.
>>>
>>> rb
>>>
>>> On 11/05/2015 10:09 AM, Sean Busbey wrote:
>>>
>>> we are currently blocked on all releases because of licensing errors
>>>> in under-maintained libraries.
>>>>
>>>> https://issues.apache.org/jira/browse/AVRO-1722
>>>>
>>>> essentially Ryan and I slowly work our way through understanding each
>>>> code base enough to do an evaluation and update things.
>>>>
>>>> It's been over 2 months now and it's a crappy situation to put our
>>>> contributors in.
>>>>
>>>>
>>>> On Thu, Nov 5, 2015 at 11:59 AM, Philip Zeyliger <ph...@cloudera.com>
>>>> wrote:
>>>>
>>>> I think it's always ok to re-release artifacts where nothing's changed.
>>>>> So, how can you be blocked on another language's implementation if you
>>>>> simply change the version number and re-release?
>>>>>
>>>>> -- Philip
>>>>>
>>>>>
>>>>
>>> --
>>> Ryan Blue
>>> Software Engineer
>>> Cloudera, Inc.
>>>
>>>
>>
>
> --
> Ryan Blue
> Software Engineer
> Cloudera, Inc.
>
--
Best regards / Met vriendelijke groeten,
Niels Basjes
Re: [DISCUSS] Release and code management
Posted by Ryan Blue <bl...@cloudera.com>.
My responses are inline.
rb
On 11/06/2015 07:32 AM, Tom White wrote:
> I'm not sure that moving to a model where there are releases of individual
> components will increase the frequency of releases. There will still need
> to be a release manager for each component, and then there's a danger that
> the less maintained components will not get released at all.
My concern is that new components, like Miki's fastavro (python) and
Matthieu's js implementation, already have significantly faster release
cycles (see [1] and [2]). We want those to be successful in the Avro
community and I think that blocking their fast release cycles on
languages without maintainers is a community problem. Not delivering
timely releases discourages participation.
I think a release manager per release, even if it is a python-only
release, is fine. As I said, we've had less than one release per year
lately so it isn't going to be that much overhead. And to your point
about making releases easier, I think it is much easier to learn how to
do releases with a small project.
We will need to discuss what to do about languages that aren't
maintained. (Maybe in a separate thread?) I don't think we need to
deprecate them as Phil suggested, but I also don't think it is right to
continue releasing code with new version numbers that hasn't been
updated in a year or more. I think separating the releases is the right
thing to do here too: it signals to users that the component is still
"current" but hasn't been released in a while. We're neither
discouraging use or participation by deprecating it, nor are we making
it appear more active than it is.
[1]: https://github.com/tebeka/fastavro/releases
[2]: https://github.com/mtth/avsc/releases
> I would rather continue to make the release process easier (Docker helps a
> lot) so that any committer can do it. We should be able to use the Docker
> work to run tests for all components with Jenkins to ensure that trunk is
> always in a releasable state.
Docker does help quite a lot, but it doesn't help when the problem isn't
build-related. If Ruby contributors want to get a feature into a release
or perl has a blocking bug or the C# implementation has license issues,
we end up blocking all components.
> Where are we with the licensing issues? If we can get those worked out then
> I'd like to make a release (of all components).
The source tarball is nearly ready and a few implementations have
updated license docs in the binary artifacts. We have a couple days work
to go.
> I'm +1 on moving to git.
>
> Cheers,
> Tom
>
> On Thu, Nov 5, 2015 at 12:45 PM, Ryan Blue <bl...@cloudera.com> wrote:
>
>> It isn't just license problems, either. Releases that include all of the
>> languages can be blocked by bugs that need to be fixed in those languages
>> that are suggested during release planning.
>>
>> It is also necessary to make sure the older language implementations still
>> build and pass tests, which can mean, for example, installing php and
>> fixing any tests that currently break. Tom's recent work to port the build
>> to docker really helps this situation, but that took patches to
>> unmaintained implementations and will still require maintenance.
>>
>> I also disagree that it's always okay to re-release artifacts. Everything
>> is moving toward semantic versioning and I think that Avro should as well.
>> It is confusing to users to have an identical library released with a
>> version number that indicates a breaking change (though it appears not to
>> be by semver rules).
>>
>> Each language should adopt a release cadence that works for its
>> contributors so that those contributors are able to use their work in
>> timely releases. Otherwise, I'm afraid that we will see fewer contributions
>> because of the long release cycle we currently have.
>>
>> rb
>>
>> On 11/05/2015 10:09 AM, Sean Busbey wrote:
>>
>>> we are currently blocked on all releases because of licensing errors
>>> in under-maintained libraries.
>>>
>>> https://issues.apache.org/jira/browse/AVRO-1722
>>>
>>> essentially Ryan and I slowly work our way through understanding each
>>> code base enough to do an evaluation and update things.
>>>
>>> It's been over 2 months now and it's a crappy situation to put our
>>> contributors in.
>>>
>>>
>>> On Thu, Nov 5, 2015 at 11:59 AM, Philip Zeyliger <ph...@cloudera.com>
>>> wrote:
>>>
>>>> I think it's always ok to re-release artifacts where nothing's changed.
>>>> So, how can you be blocked on another language's implementation if you
>>>> simply change the version number and re-release?
>>>>
>>>> -- Philip
>>>>
>>>
>>
>> --
>> Ryan Blue
>> Software Engineer
>> Cloudera, Inc.
>>
>
--
Ryan Blue
Software Engineer
Cloudera, Inc.
Re: [DISCUSS] Release and code management
Posted by Philip Zeyliger <ph...@cloudera.com>.
The semantic version conversation is being conflated into here. I'm not a
true believer, but I'll simply point out that there is the library version
and the IPC protocol and file format version, and those need to be
discussed separately. I.e., it's very possible to change the Python API in
an incompatible way while retaining the same file formats, and so on.
If we don't have the people interested in maintaining a release for
language X, perhaps language X should imply be removed? It can be
re-retrieved when it's staffed.
I'm +1 on git.
I remain -0 on separating things.
On Fri, Nov 6, 2015 at 7:32 AM Tom White <to...@cloudera.com> wrote:
> I'm not sure that moving to a model where there are releases of individual
> components will increase the frequency of releases. There will still need
> to be a release manager for each component, and then there's a danger that
> the less maintained components will not get released at all.
>
> I would rather continue to make the release process easier (Docker helps a
> lot) so that any committer can do it. We should be able to use the Docker
> work to run tests for all components with Jenkins to ensure that trunk is
> always in a releasable state.
>
> Where are we with the licensing issues? If we can get those worked out then
> I'd like to make a release (of all components).
>
> I'm +1 on moving to git.
>
> Cheers,
> Tom
>
> On Thu, Nov 5, 2015 at 12:45 PM, Ryan Blue <bl...@cloudera.com> wrote:
>
> > It isn't just license problems, either. Releases that include all of the
> > languages can be blocked by bugs that need to be fixed in those languages
> > that are suggested during release planning.
> >
> > It is also necessary to make sure the older language implementations
> still
> > build and pass tests, which can mean, for example, installing php and
> > fixing any tests that currently break. Tom's recent work to port the
> build
> > to docker really helps this situation, but that took patches to
> > unmaintained implementations and will still require maintenance.
> >
> > I also disagree that it's always okay to re-release artifacts. Everything
> > is moving toward semantic versioning and I think that Avro should as
> well.
> > It is confusing to users to have an identical library released with a
> > version number that indicates a breaking change (though it appears not to
> > be by semver rules).
> >
> > Each language should adopt a release cadence that works for its
> > contributors so that those contributors are able to use their work in
> > timely releases. Otherwise, I'm afraid that we will see fewer
> contributions
> > because of the long release cycle we currently have.
> >
> > rb
> >
> > On 11/05/2015 10:09 AM, Sean Busbey wrote:
> >
> >> we are currently blocked on all releases because of licensing errors
> >> in under-maintained libraries.
> >>
> >> https://issues.apache.org/jira/browse/AVRO-1722
> >>
> >> essentially Ryan and I slowly work our way through understanding each
> >> code base enough to do an evaluation and update things.
> >>
> >> It's been over 2 months now and it's a crappy situation to put our
> >> contributors in.
> >>
> >>
> >> On Thu, Nov 5, 2015 at 11:59 AM, Philip Zeyliger <ph...@cloudera.com>
> >> wrote:
> >>
> >>> I think it's always ok to re-release artifacts where nothing's changed.
> >>> So, how can you be blocked on another language's implementation if you
> >>> simply change the version number and re-release?
> >>>
> >>> -- Philip
> >>>
> >>
> >
> > --
> > Ryan Blue
> > Software Engineer
> > Cloudera, Inc.
> >
>
Re: [DISCUSS] Release and code management
Posted by Tom White <to...@cloudera.com>.
I'm not sure that moving to a model where there are releases of individual
components will increase the frequency of releases. There will still need
to be a release manager for each component, and then there's a danger that
the less maintained components will not get released at all.
I would rather continue to make the release process easier (Docker helps a
lot) so that any committer can do it. We should be able to use the Docker
work to run tests for all components with Jenkins to ensure that trunk is
always in a releasable state.
Where are we with the licensing issues? If we can get those worked out then
I'd like to make a release (of all components).
I'm +1 on moving to git.
Cheers,
Tom
On Thu, Nov 5, 2015 at 12:45 PM, Ryan Blue <bl...@cloudera.com> wrote:
> It isn't just license problems, either. Releases that include all of the
> languages can be blocked by bugs that need to be fixed in those languages
> that are suggested during release planning.
>
> It is also necessary to make sure the older language implementations still
> build and pass tests, which can mean, for example, installing php and
> fixing any tests that currently break. Tom's recent work to port the build
> to docker really helps this situation, but that took patches to
> unmaintained implementations and will still require maintenance.
>
> I also disagree that it's always okay to re-release artifacts. Everything
> is moving toward semantic versioning and I think that Avro should as well.
> It is confusing to users to have an identical library released with a
> version number that indicates a breaking change (though it appears not to
> be by semver rules).
>
> Each language should adopt a release cadence that works for its
> contributors so that those contributors are able to use their work in
> timely releases. Otherwise, I'm afraid that we will see fewer contributions
> because of the long release cycle we currently have.
>
> rb
>
> On 11/05/2015 10:09 AM, Sean Busbey wrote:
>
>> we are currently blocked on all releases because of licensing errors
>> in under-maintained libraries.
>>
>> https://issues.apache.org/jira/browse/AVRO-1722
>>
>> essentially Ryan and I slowly work our way through understanding each
>> code base enough to do an evaluation and update things.
>>
>> It's been over 2 months now and it's a crappy situation to put our
>> contributors in.
>>
>>
>> On Thu, Nov 5, 2015 at 11:59 AM, Philip Zeyliger <ph...@cloudera.com>
>> wrote:
>>
>>> I think it's always ok to re-release artifacts where nothing's changed.
>>> So, how can you be blocked on another language's implementation if you
>>> simply change the version number and re-release?
>>>
>>> -- Philip
>>>
>>
>
> --
> Ryan Blue
> Software Engineer
> Cloudera, Inc.
>
Re: [DISCUSS] Release and code management
Posted by Ryan Blue <bl...@cloudera.com>.
It isn't just license problems, either. Releases that include all of the
languages can be blocked by bugs that need to be fixed in those
languages that are suggested during release planning.
It is also necessary to make sure the older language implementations
still build and pass tests, which can mean, for example, installing php
and fixing any tests that currently break. Tom's recent work to port the
build to docker really helps this situation, but that took patches to
unmaintained implementations and will still require maintenance.
I also disagree that it's always okay to re-release artifacts.
Everything is moving toward semantic versioning and I think that Avro
should as well. It is confusing to users to have an identical library
released with a version number that indicates a breaking change (though
it appears not to be by semver rules).
Each language should adopt a release cadence that works for its
contributors so that those contributors are able to use their work in
timely releases. Otherwise, I'm afraid that we will see fewer
contributions because of the long release cycle we currently have.
rb
On 11/05/2015 10:09 AM, Sean Busbey wrote:
> we are currently blocked on all releases because of licensing errors
> in under-maintained libraries.
>
> https://issues.apache.org/jira/browse/AVRO-1722
>
> essentially Ryan and I slowly work our way through understanding each
> code base enough to do an evaluation and update things.
>
> It's been over 2 months now and it's a crappy situation to put our
> contributors in.
>
>
> On Thu, Nov 5, 2015 at 11:59 AM, Philip Zeyliger <ph...@cloudera.com> wrote:
>> I think it's always ok to re-release artifacts where nothing's changed.
>> So, how can you be blocked on another language's implementation if you
>> simply change the version number and re-release?
>>
>> -- Philip
--
Ryan Blue
Software Engineer
Cloudera, Inc.
Re: [DISCUSS] Release and code management
Posted by Sean Busbey <bu...@cloudera.com>.
we are currently blocked on all releases because of licensing errors
in under-maintained libraries.
https://issues.apache.org/jira/browse/AVRO-1722
essentially Ryan and I slowly work our way through understanding each
code base enough to do an evaluation and update things.
It's been over 2 months now and it's a crappy situation to put our
contributors in.
On Thu, Nov 5, 2015 at 11:59 AM, Philip Zeyliger <ph...@cloudera.com> wrote:
> I think it's always ok to re-release artifacts where nothing's changed.
> So, how can you be blocked on another language's implementation if you
> simply change the version number and re-release?
>
> -- Philip
>
> On Thu, Nov 5, 2015 at 9:43 AM Ryan Blue <bl...@cloudera.com> wrote:
>
>> Phil or Sam, any ideas about how to keep release management simple, but
>> be able to avoid blocking specific languages on under-maintained ones?
>>
>> Also, looking at the release history we've had 3 releases in the last 2
>> years, and that's being generous to include 1.7.5 that was released in
>> August 2013. I don't think more release overhead would be that big of a
>> problem, and would be well worth keeping the languages that are well
>> maintained released and up-to-date.
>>
>> rb
>>
>> On 10/30/2015 09:37 AM, Ryan Blue wrote:
>> > I think Sean is right that we could continue to release several at once.
>> > We would almost certainly continue this practice for several languages
>> > that are mostly unmaintained (like perl and php). I also expect each
>> > language's release cadence to reflect the activity in that language,
>> > which I think is very important to maintain.
>> >
>> > I also don't want to underestimate the drawback of having a single
>> > version for multiple implementations. We can't use semantic verisoning
>> > for any of the implementations. If we bump the minor version (!) because
>> > of a breaking change in Java, but aren't making breaking changes to C,
>> > this is confusing to users.
>> >
>> > If we don't separate release vehicles, how can we improve version
>> > conventions?
>> >
>> > And how do we ensure timely releases that aren't blocked by other
>> > implementations? This affects how attractive this project is to new
>> > contributors. If the releases are seldom and contributions aren't
>> > available for months at a time, I think we have a problem.
>> >
>> > rb
>> >
>> > On 10/29/2015 04:51 PM, Philip Zeyliger wrote:
>> >> -0.
>> >>
>> >> If you divide the world into N releases, you'll end up having to do
>> >> release
>> >> management N times. I think this will make doing releases that much
>> more
>> >> complicated, time-consuming, and error-prone.
>> >>
>> >> Note that you could separate release trains while remaining in a single
>> >> repo. I'd certainly prefer that than separating into many smaller
>> repos.
>> >>
>> >> -- Philip
>> >>
>> >> On Thu, Oct 29, 2015 at 11:31 AM Ryan Blue <bl...@cloudera.com> wrote:
>> >>
>> >>> On 10/29/2015 11:28 AM, Sean Busbey wrote:
>> >>>> On Oct 29, 2015 1:19 PM, "Ryan Blue" <bl...@cloudera.com> wrote:
>> >>>>
>> >>>> Where would the language interop tests live if we don't break them
>> out?
>> >>>>
>> >>>> (We already have interop tests, in case that was lost in my original
>> >>> email.)
>> >>>
>> >>> We could either keep them where they are or add a separate repo.
>> Running
>> >>> them with a release candidate would have to be part of the release
>> >>> checks.
>> >>>
>> >>> rb
>> >>>
>> >>>
>> >>> --
>> >>> Ryan Blue
>> >>> Software Engineer
>> >>> Cloudera, Inc.
>> >>>
>> >>
>> >
>> >
>>
>>
>> --
>> Ryan Blue
>> Software Engineer
>> Cloudera, Inc.
>>
--
Sean
Re: [DISCUSS] Release and code management
Posted by Philip Zeyliger <ph...@cloudera.com>.
I think it's always ok to re-release artifacts where nothing's changed.
So, how can you be blocked on another language's implementation if you
simply change the version number and re-release?
-- Philip
On Thu, Nov 5, 2015 at 9:43 AM Ryan Blue <bl...@cloudera.com> wrote:
> Phil or Sam, any ideas about how to keep release management simple, but
> be able to avoid blocking specific languages on under-maintained ones?
>
> Also, looking at the release history we've had 3 releases in the last 2
> years, and that's being generous to include 1.7.5 that was released in
> August 2013. I don't think more release overhead would be that big of a
> problem, and would be well worth keeping the languages that are well
> maintained released and up-to-date.
>
> rb
>
> On 10/30/2015 09:37 AM, Ryan Blue wrote:
> > I think Sean is right that we could continue to release several at once.
> > We would almost certainly continue this practice for several languages
> > that are mostly unmaintained (like perl and php). I also expect each
> > language's release cadence to reflect the activity in that language,
> > which I think is very important to maintain.
> >
> > I also don't want to underestimate the drawback of having a single
> > version for multiple implementations. We can't use semantic verisoning
> > for any of the implementations. If we bump the minor version (!) because
> > of a breaking change in Java, but aren't making breaking changes to C,
> > this is confusing to users.
> >
> > If we don't separate release vehicles, how can we improve version
> > conventions?
> >
> > And how do we ensure timely releases that aren't blocked by other
> > implementations? This affects how attractive this project is to new
> > contributors. If the releases are seldom and contributions aren't
> > available for months at a time, I think we have a problem.
> >
> > rb
> >
> > On 10/29/2015 04:51 PM, Philip Zeyliger wrote:
> >> -0.
> >>
> >> If you divide the world into N releases, you'll end up having to do
> >> release
> >> management N times. I think this will make doing releases that much
> more
> >> complicated, time-consuming, and error-prone.
> >>
> >> Note that you could separate release trains while remaining in a single
> >> repo. I'd certainly prefer that than separating into many smaller
> repos.
> >>
> >> -- Philip
> >>
> >> On Thu, Oct 29, 2015 at 11:31 AM Ryan Blue <bl...@cloudera.com> wrote:
> >>
> >>> On 10/29/2015 11:28 AM, Sean Busbey wrote:
> >>>> On Oct 29, 2015 1:19 PM, "Ryan Blue" <bl...@cloudera.com> wrote:
> >>>>
> >>>> Where would the language interop tests live if we don't break them
> out?
> >>>>
> >>>> (We already have interop tests, in case that was lost in my original
> >>> email.)
> >>>
> >>> We could either keep them where they are or add a separate repo.
> Running
> >>> them with a release candidate would have to be part of the release
> >>> checks.
> >>>
> >>> rb
> >>>
> >>>
> >>> --
> >>> Ryan Blue
> >>> Software Engineer
> >>> Cloudera, Inc.
> >>>
> >>
> >
> >
>
>
> --
> Ryan Blue
> Software Engineer
> Cloudera, Inc.
>