You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Wes McKinney <we...@gmail.com> on 2019/06/07 15:42:20 UTC
[DISCUSS] Timing of release and making a 1.0.0 release marking Arrow
protocol stability
hi folks,
Our last release 0.13.0 occurred at the end of March. I think it would
be good to plot a course for the next release (0.14.0?) as soon as
possible. There are still a number of issues (such as the shared
library duplication issue in the Python wheels) that I think might
discourage us from releasing right now. Do you think that pushing for
a release candidate by the end of June is reasonable?
As a second matter (and this can be split off into a separate
discussion thread), the Arrow format and binary protocol has been
stable effectively since the 0.8.0 release which occurred in December
2017. While we have some details yet to iron out in compatibility
testing between implementations (for example, the Union question, see
mailing list discussion [1]) and new features (e.g. 64-bit offset
binary/string/list types), in theory these should not prevent us
necessarily from making a declaration of protocol stability. I think
this would have a lot of benefits for project onlookers to remove
various warnings around the codebase around stability and cautions
against persistence of protocol data. It's fair to say that if we _do_
make changes in the future, that there will be a transition path for
migrate persisted data, should it ever come to that.
I would suggest a "1.0.0" release either as our next release (instead
of 0.14.0) or the release right after that (if we need more time to
get affairs in order), with the guidance for users of:
PROTOCOL VERSION (1): Protocol version, so libraries bearing 1.x.y
will be forward and backwards compatible (though new metadata fields
introduced in newer versions will be dropped in older readers)
MAJOR VERSION (0): API changes possible (and indeed, likely) from
major release to major release
MINOR VERSION (0): No API changes, bug fix only release
Thoughts on the above?
Thanks
Wes
[1]: https://lists.apache.org/thread.html/e54e8ec096f665a8aef94155de3b6c567258c0d15209de4b966dd8da@%3Cdev.arrow.apache.org%3E
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Bryan Cutler <cu...@gmail.com>.
I'm a little late, but +1 on the plan for a 0.14.0 release followed by 1.0.0
On Sun, Jun 16, 2019 at 5:45 AM Wes McKinney <we...@gmail.com> wrote:
> I also agree that time-based releases are the only reasonable thing to
> do going forward. It also creates a sense of urgency to complete work
> by a certain date, in the sense of "the ship is leaving the port on X,
> all aboard!"
>
> The Blocker priority on JIRA can be used to communicate if some issue
> should prevent a release candidate from being cut, but contributors
> can also state whether there is some concerning issue here on the
> mailing list.
>
> With that being said, I think we should have the backlog closed and
> the project ready by June 24. A number of optional builds are still
> broken, and Flight is not set up in the Python packages, so those are
> the things that I would like to see sorted out by then, all the rest
> is nice-to-have.
>
> - Wes
>
> On Sun, Jun 16, 2019 at 3:38 AM Sutou Kouhei <ko...@clear-code.com> wrote:
> >
> > Hi,
> >
> > > If we want to stick to project-wide releases where all languages and
> > > components are released once, then time-based releases are the only
> > > reasonable scheme IMHO. Of course, there may still be release-blocking
> > > issues, but those should only be important regressions, not "this is a
> > > feature we'd definitely like to see in this release".
> >
> > I think so too.
> >
> > I'll make blocker tasks "nice-to-have" if these tasks aren't
> > critical for release. For example, I'll mark them
> > "nice-to-have" as a release manager if they doesn't fix not
> > buildable issues.
> >
> > In pre-release, the followings are helpful:
> >
> > * JIRA tidying
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-JIRAtidying
> >
> > * Testing API document generation
> > How? Documented?
> >
> > * Testing package builds and generated packages
> > * For package builds: Crossbow is used. Where is the document?
> > * For package test: How? Documented?
> >
> > * Testing the master on many platforms
> >
> > In release process, the followings are helpful:
> >
> > * Resolving issues reported by release manager
> >
> > * Testing RC
> >
> > In post-release, the followings are helpful. Some of them
> > requires PMC member or committer role but others can be worked
> > by everyone:
> >
> > * Updating the Arrow website
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingtheArrowwebsite
> > * Require committer role
> >
> > * Announcing release
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-Announcingrelease
> > * Require PMC member role?
> >
> > * Updating website with new API documentation
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingwebsitewithnewAPIdocumentation
> > * Require committer role
> >
> > * Updating C++ and Python packages
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingC++andPythonpackages
> > * Everyone can work on?
> >
> > * Updating Homebrew packages
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingHomebrewpackages
> > * Everyone can work on
> >
> > * Updating Ruby packages
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingRubypackages
> > * Require PMC member role?
> >
> > * Updating JavaScript packages
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingJavaScriptpackages
> > * Require PMC member role?
> >
> > * Updating .NET NuGet packages
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-Updating.NETNuGetpackages
> > * Require PMC member role?
> >
> > * Updating Rust packages
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingRustpackages
> > * Require PMC member role?
> >
> > * Updating CRAN packages
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingCRANpackages
> > * Require PMC member role?
> >
> >
> > Thanks,
> > --
> > kou
> >
> > In <20...@fsol>
> > "Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
> Arrow protocol stability" on Sat, 15 Jun 2019 11:48:06 +0200,
> > Antoine Pitrou <so...@pitrou.net> wrote:
> >
> > > On Fri, 14 Jun 2019 16:41:47 -0700
> > > Neal Richardson <ne...@gmail.com> wrote:
> > >>
> > >> I gather that this has all been done informally in the past, but
> would some
> > >> folks be interested in taking up this release-prep role for the
> various
> > >> languages and formalizing that responsibility a bit? By "formalize" I
> just
> > >> mean write down on the wiki (
> > >>
> https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.14.0+Release)
> who
> > >> is overseeing the release for each language and what the current
> status is.
> > >> That way, Kou and anyone else can see at a glance which subprojects
> are
> > >> ready for release and who to talk to about wrangling the others. For
> my
> > >> part, I've been curating the R backlog and 0.14 release scope and
> working
> > >> with Romain and others to get the essentials done, and I'd be happy to
> > >> record that I'm doing this and publicize when the key issues have been
> > >> addressed and we're ready to release. If different people were to do
> that
> > >> for the other 10 languages, we could eliminate some ambiguity as to
> what
> > >> the status is and who is taking responsibility for ensuring that the
> > >> necessary work gets done.
> > >
> > > The risk if we have that kind of process for 11 languages is that
> > > releases never get ready because there's always something left to
> > > improve, fix or clean up.
> > >
> > > If we want to stick to project-wide releases where all languages and
> > > components are released once, then time-based releases are the only
> > > reasonable scheme IMHO. Of course, there may still be release-blocking
> > > issues, but those should only be important regressions, not "this is a
> > > feature we'd definitely like to see in this release".
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> > >
>
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Wes McKinney <we...@gmail.com>.
I also agree that time-based releases are the only reasonable thing to
do going forward. It also creates a sense of urgency to complete work
by a certain date, in the sense of "the ship is leaving the port on X,
all aboard!"
The Blocker priority on JIRA can be used to communicate if some issue
should prevent a release candidate from being cut, but contributors
can also state whether there is some concerning issue here on the
mailing list.
With that being said, I think we should have the backlog closed and
the project ready by June 24. A number of optional builds are still
broken, and Flight is not set up in the Python packages, so those are
the things that I would like to see sorted out by then, all the rest
is nice-to-have.
- Wes
On Sun, Jun 16, 2019 at 3:38 AM Sutou Kouhei <ko...@clear-code.com> wrote:
>
> Hi,
>
> > If we want to stick to project-wide releases where all languages and
> > components are released once, then time-based releases are the only
> > reasonable scheme IMHO. Of course, there may still be release-blocking
> > issues, but those should only be important regressions, not "this is a
> > feature we'd definitely like to see in this release".
>
> I think so too.
>
> I'll make blocker tasks "nice-to-have" if these tasks aren't
> critical for release. For example, I'll mark them
> "nice-to-have" as a release manager if they doesn't fix not
> buildable issues.
>
> In pre-release, the followings are helpful:
>
> * JIRA tidying
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-JIRAtidying
>
> * Testing API document generation
> How? Documented?
>
> * Testing package builds and generated packages
> * For package builds: Crossbow is used. Where is the document?
> * For package test: How? Documented?
>
> * Testing the master on many platforms
>
> In release process, the followings are helpful:
>
> * Resolving issues reported by release manager
>
> * Testing RC
>
> In post-release, the followings are helpful. Some of them
> requires PMC member or committer role but others can be worked
> by everyone:
>
> * Updating the Arrow website
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingtheArrowwebsite
> * Require committer role
>
> * Announcing release
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-Announcingrelease
> * Require PMC member role?
>
> * Updating website with new API documentation
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingwebsitewithnewAPIdocumentation
> * Require committer role
>
> * Updating C++ and Python packages
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingC++andPythonpackages
> * Everyone can work on?
>
> * Updating Homebrew packages
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingHomebrewpackages
> * Everyone can work on
>
> * Updating Ruby packages
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingRubypackages
> * Require PMC member role?
>
> * Updating JavaScript packages
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingJavaScriptpackages
> * Require PMC member role?
>
> * Updating .NET NuGet packages
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-Updating.NETNuGetpackages
> * Require PMC member role?
>
> * Updating Rust packages
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingRustpackages
> * Require PMC member role?
>
> * Updating CRAN packages
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingCRANpackages
> * Require PMC member role?
>
>
> Thanks,
> --
> kou
>
> In <20...@fsol>
> "Re: [DISCUSS] Timing of release and making a 1.0.0 release marking Arrow protocol stability" on Sat, 15 Jun 2019 11:48:06 +0200,
> Antoine Pitrou <so...@pitrou.net> wrote:
>
> > On Fri, 14 Jun 2019 16:41:47 -0700
> > Neal Richardson <ne...@gmail.com> wrote:
> >>
> >> I gather that this has all been done informally in the past, but would some
> >> folks be interested in taking up this release-prep role for the various
> >> languages and formalizing that responsibility a bit? By "formalize" I just
> >> mean write down on the wiki (
> >> https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.14.0+Release) who
> >> is overseeing the release for each language and what the current status is.
> >> That way, Kou and anyone else can see at a glance which subprojects are
> >> ready for release and who to talk to about wrangling the others. For my
> >> part, I've been curating the R backlog and 0.14 release scope and working
> >> with Romain and others to get the essentials done, and I'd be happy to
> >> record that I'm doing this and publicize when the key issues have been
> >> addressed and we're ready to release. If different people were to do that
> >> for the other 10 languages, we could eliminate some ambiguity as to what
> >> the status is and who is taking responsibility for ensuring that the
> >> necessary work gets done.
> >
> > The risk if we have that kind of process for 11 languages is that
> > releases never get ready because there's always something left to
> > improve, fix or clean up.
> >
> > If we want to stick to project-wide releases where all languages and
> > components are released once, then time-based releases are the only
> > reasonable scheme IMHO. Of course, there may still be release-blocking
> > issues, but those should only be important regressions, not "this is a
> > feature we'd definitely like to see in this release".
> >
> > Regards
> >
> > Antoine.
> >
> >
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
> If we want to stick to project-wide releases where all languages and
> components are released once, then time-based releases are the only
> reasonable scheme IMHO. Of course, there may still be release-blocking
> issues, but those should only be important regressions, not "this is a
> feature we'd definitely like to see in this release".
I think so too.
I'll make blocker tasks "nice-to-have" if these tasks aren't
critical for release. For example, I'll mark them
"nice-to-have" as a release manager if they doesn't fix not
buildable issues.
In pre-release, the followings are helpful:
* JIRA tidying
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-JIRAtidying
* Testing API document generation
How? Documented?
* Testing package builds and generated packages
* For package builds: Crossbow is used. Where is the document?
* For package test: How? Documented?
* Testing the master on many platforms
In release process, the followings are helpful:
* Resolving issues reported by release manager
* Testing RC
In post-release, the followings are helpful. Some of them
requires PMC member or committer role but others can be worked
by everyone:
* Updating the Arrow website
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingtheArrowwebsite
* Require committer role
* Announcing release
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-Announcingrelease
* Require PMC member role?
* Updating website with new API documentation
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingwebsitewithnewAPIdocumentation
* Require committer role
* Updating C++ and Python packages
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingC++andPythonpackages
* Everyone can work on?
* Updating Homebrew packages
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingHomebrewpackages
* Everyone can work on
* Updating Ruby packages
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingRubypackages
* Require PMC member role?
* Updating JavaScript packages
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingJavaScriptpackages
* Require PMC member role?
* Updating .NET NuGet packages
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-Updating.NETNuGetpackages
* Require PMC member role?
* Updating Rust packages
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingRustpackages
* Require PMC member role?
* Updating CRAN packages
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-UpdatingCRANpackages
* Require PMC member role?
Thanks,
--
kou
In <20...@fsol>
"Re: [DISCUSS] Timing of release and making a 1.0.0 release marking Arrow protocol stability" on Sat, 15 Jun 2019 11:48:06 +0200,
Antoine Pitrou <so...@pitrou.net> wrote:
> On Fri, 14 Jun 2019 16:41:47 -0700
> Neal Richardson <ne...@gmail.com> wrote:
>>
>> I gather that this has all been done informally in the past, but would some
>> folks be interested in taking up this release-prep role for the various
>> languages and formalizing that responsibility a bit? By "formalize" I just
>> mean write down on the wiki (
>> https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.14.0+Release) who
>> is overseeing the release for each language and what the current status is.
>> That way, Kou and anyone else can see at a glance which subprojects are
>> ready for release and who to talk to about wrangling the others. For my
>> part, I've been curating the R backlog and 0.14 release scope and working
>> with Romain and others to get the essentials done, and I'd be happy to
>> record that I'm doing this and publicize when the key issues have been
>> addressed and we're ready to release. If different people were to do that
>> for the other 10 languages, we could eliminate some ambiguity as to what
>> the status is and who is taking responsibility for ensuring that the
>> necessary work gets done.
>
> The risk if we have that kind of process for 11 languages is that
> releases never get ready because there's always something left to
> improve, fix or clean up.
>
> If we want to stick to project-wide releases where all languages and
> components are released once, then time-based releases are the only
> reasonable scheme IMHO. Of course, there may still be release-blocking
> issues, but those should only be important regressions, not "this is a
> feature we'd definitely like to see in this release".
>
> Regards
>
> Antoine.
>
>
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Antoine Pitrou <so...@pitrou.net>.
On Fri, 14 Jun 2019 16:41:47 -0700
Neal Richardson <ne...@gmail.com> wrote:
>
> I gather that this has all been done informally in the past, but would some
> folks be interested in taking up this release-prep role for the various
> languages and formalizing that responsibility a bit? By "formalize" I just
> mean write down on the wiki (
> https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.14.0+Release) who
> is overseeing the release for each language and what the current status is.
> That way, Kou and anyone else can see at a glance which subprojects are
> ready for release and who to talk to about wrangling the others. For my
> part, I've been curating the R backlog and 0.14 release scope and working
> with Romain and others to get the essentials done, and I'd be happy to
> record that I'm doing this and publicize when the key issues have been
> addressed and we're ready to release. If different people were to do that
> for the other 10 languages, we could eliminate some ambiguity as to what
> the status is and who is taking responsibility for ensuring that the
> necessary work gets done.
The risk if we have that kind of process for 11 languages is that
releases never get ready because there's always something left to
improve, fix or clean up.
If we want to stick to project-wide releases where all languages and
components are released once, then time-based releases are the only
reasonable scheme IMHO. Of course, there may still be release-blocking
issues, but those should only be important regressions, not "this is a
feature we'd definitely like to see in this release".
Regards
Antoine.
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Neal Richardson <ne...@gmail.com>.
Thanks Kou for volunteering to be the release manager. I can't imagine
there are any objections :)
As this is my first release with the project, I'm eager to learn how things
work and to help however I can. One thing that occurs to me is the
management of the backlog for 0.14 and the release-readiness of each of the
languages and subprojects. Like Kou just did here for the release as a
whole, do individuals stand up and volunteer to oversee an individual
language/component? It seems unreasonable to expect one release manager to
know that all 11 languages are in good condition, have no critical issues
outstanding, and that all build artifacts (binaries, docs) build correctly;
and the project seems too large to assume that the work will just get done
on its own.
I gather that this has all been done informally in the past, but would some
folks be interested in taking up this release-prep role for the various
languages and formalizing that responsibility a bit? By "formalize" I just
mean write down on the wiki (
https://cwiki.apache.org/confluence/display/ARROW/Arrow+0.14.0+Release) who
is overseeing the release for each language and what the current status is.
That way, Kou and anyone else can see at a glance which subprojects are
ready for release and who to talk to about wrangling the others. For my
part, I've been curating the R backlog and 0.14 release scope and working
with Romain and others to get the essentials done, and I'd be happy to
record that I'm doing this and publicize when the key issues have been
addressed and we're ready to release. If different people were to do that
for the other 10 languages, we could eliminate some ambiguity as to what
the status is and who is taking responsibility for ensuring that the
necessary work gets done.
Thoughts?
Neal
On Wed, Jun 12, 2019 at 10:05 PM Sutou Kouhei <ko...@clear-code.com> wrote:
> Hi,
>
> I like the plan too.
>
> If nobody wants be a release manager for 0.14.0, I can be a
> release manager. I'm busy recently but I'll be able to make
> time from June 24.
>
>
> Thanks,
> --
> kou
>
> In <CA...@mail.gmail.com>
> "Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
> Arrow protocol stability" on Mon, 10 Jun 2019 14:19:51 -0700,
> Jacques Nadeau <ja...@apache.org> wrote:
>
> > Sounds good.
> >
> > On Mon, Jun 10, 2019 at 11:06 AM Wes McKinney <we...@gmail.com>
> wrote:
> >
> >> Hi all,
> >>
> >> OK, it sounds like there is reasonable consensus behind the plan:
> >>
> >> * Make a 0.14.0 release in the near future (later this month?)
> >> * Publicize that the next release will be 1.0.0, in a "speak now or
> >> hold your peace" fashion
> >> * Release 1.0.0 as following release. I would suggest not waiting too
> >> long, so late August / early September time frame
> >>
> >> I'm going to continue grooming the 0.14.0 backlog to help refine the
> >> scope of what still needs to be done for C++/Python to get the next
> >> release out. If the stakeholders in various project subcomponents
> >> could also groom the backlog and mark any blockers, that would be very
> >> helpful.
> >>
> >> I suggest shooting for a release candidate for 0.14.0 either the week
> >> of June 24 or July 1 (depending on where things stand)
> >>
> >> Thanks
> >> Wes
> >>
> >> On Mon, Jun 10, 2019 at 2:39 AM Sutou Kouhei <ko...@clear-code.com>
> wrote:
> >> >
> >> > Hi,
> >> >
> >> > I think that 0.14.0 is better for the next version.
> >> >
> >> > People who don't try Apache Arrow yet to wait 1.0.0 will use
> >> > Apache Arrow when we release 1.0.0. If 1.0.0 satisfies them,
> >> > we will get more users and contributors by 1.0.0. They may
> >> > not care protocol stability. They may just care "1.0.0".
> >> >
> >> > We'll be able to release less problem 1.0.0 by releasing
> >> > 0.14.0 as RC for 1.0.0. 0.14.0 will be used more people than
> >> > 1.0.0-RCX. 0.14.0 users will find critical problems.
> >> >
> >> >
> >> > Thanks,
> >> > --
> >> > kou
> >> >
> >> > In <CAK7Z5T885FnCrgZnTfJrm400kq_pz-=
> AzYVGwthEEfnxt1etjQ@mail.gmail.com>
> >> > "Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
> >> Arrow protocol stability" on Fri, 7 Jun 2019 22:28:22 -0700,
> >> > Micah Kornfield <em...@gmail.com> wrote:
> >> >
> >> > > A few thoughts:
> >> > > - I think we should iron out the remaining incompatibilities between
> >> java
> >> > > and C++ before going to 1.0.0 (at least Union and NullType), and I'm
> >> not
> >> > > sure I will have time to them before the next release, so I would
> >> prefer to
> >> > > try to aim for the subsequent release to make it 1.0.0
> >> > > - For 1.0.0 should we change the metadata format version to a new
> >> naming
> >> > > scheme [1] (seems like more of a hassle then it is worth)?
> >> > > - I'm a little concerned about the implications for
> >> forward-compatibility
> >> > > restrictions for format changes. For instance the large list types
> >> would
> >> > > not be forward compatible (at least by some definitions), similarly
> if
> >> we
> >> > > deal with compression [2] it would also seem to not be forward
> >> compatible.
> >> > > Would this mean we bump the format version number for each change
> even
> >> > > though they would be backwards compatible?
> >> > >
> >> > > Thanks,
> >> > > Micah
> >> > >
> >> > > [1]
> https://github.com/apache/arrow/blob/master/format/Schema.fbs#L22
> >> > > [2] https://issues.apache.org/jira/browse/ARROW-300
> >> > >
> >> > > On Fri, Jun 7, 2019 at 12:42 PM Wes McKinney <we...@gmail.com>
> >> wrote:
> >> > >
> >> > >> I agree re: marketing value of a 1.0.0 release.
> >> > >>
> >> > >> For the record, I think we should continue to allow the API of each
> >> > >> respective library component to evolve freely and allow the
> >> > >> individuals developing each to decide how to handle deprecations,
> API
> >> > >> changes, etc., as we have up until this point. The project is still
> >> > >> very much in "innovation mode" across the board, but some parts may
> >> > >> grow more conservative than others. Having roughly time-based
> releases
> >> > >> encourages everyone to be ready-to-release at any given time, and
> we
> >> > >> develop a steady cadence of getting new functionality and
> >> > >> improvements/fixes out the door.
> >> > >>
> >> > >> On Fri, Jun 7, 2019 at 1:25 PM Antoine Pitrou <an...@python.org>
> >> wrote:
> >> > >> >
> >> > >> >
> >> > >> > I think there's a marketing merit to issuing a 1.0.0 release.
> >> > >> >
> >> > >> > Regards
> >> > >> >
> >> > >> > Antoine.
> >> > >> >
> >> > >> >
> >> > >> > Le 07/06/2019 à 20:05, Wes McKinney a écrit :
> >> > >> > > So one idea is that we could call the next release 1.14.0. So
> the
> >> > >> > > second number is the API version number. This encodes a
> >> sequencing of
> >> > >> > > the evolution of the API. The library APIs are already
> decoupled
> >> from
> >> > >> > > the binary serialization protocol, so I think we merely have to
> >> state
> >> > >> > > that API changes and protocol changes are not related to each
> >> other.
> >> > >> > >
> >> > >> > > On Fri, Jun 7, 2019 at 12:58 PM Jacques Nadeau <
> >> jacques@apache.org>
> >> > >> wrote:
> >> > >> > >>
> >> > >> > >> It brings up an interesting point... do we couple the
> stability
> >> of
> >> > >> the apis
> >> > >> > >> with the stability of the protocol. If the protocol is
> stable, we
> >> > >> should
> >> > >> > >> start providing guarantees for it. How do we want to express
> >> these
> >> > >> > >> different velocities?
> >> > >> > >>
> >> > >> > >> On Fri, Jun 7, 2019 at 10:48 AM Antoine Pitrou <
> >> antoine@python.org>
> >> > >> wrote:
> >> > >> > >>
> >> > >> > >>>
> >> > >> > >>> Le 07/06/2019 à 19:44, Jacques Nadeau a écrit :
> >> > >> > >>>> On Fri, Jun 7, 2019 at 10:25 AM Antoine Pitrou <
> >> antoine@python.org>
> >> > >> > >>> wrote:
> >> > >> > >>>>
> >> > >> > >>>>> Hi Wes,
> >> > >> > >>>>>
> >> > >> > >>>>> Le 07/06/2019 à 17:42, Wes McKinney a écrit :
> >> > >> > >>>>>>
> >> > >> > >>>>>> I think
> >> > >> > >>>>>> this would have a lot of benefits for project onlookers to
> >> remove
> >> > >> > >>>>>> various warnings around the codebase around stability and
> >> cautions
> >> > >> > >>>>>> against persistence of protocol data. It's fair to say
> that
> >> if we
> >> > >> _do_
> >> > >> > >>>>>> make changes in the future, that there will be a
> transition
> >> path
> >> > >> for
> >> > >> > >>>>>> migrate persisted data, should it ever come to that.
> >> > >> > >>>>>
> >> > >> > >>>>> I think that's a good idea, but perhaps the stability
> promise
> >> > >> shouldn't
> >> > >> > >>>>> cover the Flight protocol yet?
> >> > >> > >>>>
> >> > >> > >>>> Agreed.
> >> > >> > >>>>
> >> > >> > >>>>>> I would suggest a "1.0.0" release either as our next
> release
> >> > >> (instead
> >> > >> > >>>>>> of 0.14.0) or the release right after that (if we need
> more
> >> time
> >> > >> to
> >> > >> > >>>>>> get affairs in order), with the guidance for users of:
> >> > >> > >>>>>
> >> > >> > >>>>> I think we should first do a regular 0.14.0 with all that's
> >> on our
> >> > >> plate
> >> > >> > >>>>> right now, then work towards a 1.0.0 as the release
> following
> >> that.
> >> > >> > >>>>
> >> > >> > >>>> What is different from your perspective? If the protocol
> hasn't
> >> > >> changed
> >> > >> > >>> in
> >> > >> > >>>> over a year, why not call it 1.0?
> >> > >> > >>>
> >> > >> > >>> I would say that perhaps some API cleanup is in order.
> Remove
> >> > >> > >>> deprecated ones, review experimental APIs, perhaps mark
> >> experimental
> >> > >> > >>> certain APIs that we forgot to...
> >> > >> > >>>
> >> > >> > >>> Regards
> >> > >> > >>>
> >> > >> > >>> Antoine.
> >> > >> > >>>
> >> > >>
> >>
>
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
I like the plan too.
If nobody wants be a release manager for 0.14.0, I can be a
release manager. I'm busy recently but I'll be able to make
time from June 24.
Thanks,
--
kou
In <CA...@mail.gmail.com>
"Re: [DISCUSS] Timing of release and making a 1.0.0 release marking Arrow protocol stability" on Mon, 10 Jun 2019 14:19:51 -0700,
Jacques Nadeau <ja...@apache.org> wrote:
> Sounds good.
>
> On Mon, Jun 10, 2019 at 11:06 AM Wes McKinney <we...@gmail.com> wrote:
>
>> Hi all,
>>
>> OK, it sounds like there is reasonable consensus behind the plan:
>>
>> * Make a 0.14.0 release in the near future (later this month?)
>> * Publicize that the next release will be 1.0.0, in a "speak now or
>> hold your peace" fashion
>> * Release 1.0.0 as following release. I would suggest not waiting too
>> long, so late August / early September time frame
>>
>> I'm going to continue grooming the 0.14.0 backlog to help refine the
>> scope of what still needs to be done for C++/Python to get the next
>> release out. If the stakeholders in various project subcomponents
>> could also groom the backlog and mark any blockers, that would be very
>> helpful.
>>
>> I suggest shooting for a release candidate for 0.14.0 either the week
>> of June 24 or July 1 (depending on where things stand)
>>
>> Thanks
>> Wes
>>
>> On Mon, Jun 10, 2019 at 2:39 AM Sutou Kouhei <ko...@clear-code.com> wrote:
>> >
>> > Hi,
>> >
>> > I think that 0.14.0 is better for the next version.
>> >
>> > People who don't try Apache Arrow yet to wait 1.0.0 will use
>> > Apache Arrow when we release 1.0.0. If 1.0.0 satisfies them,
>> > we will get more users and contributors by 1.0.0. They may
>> > not care protocol stability. They may just care "1.0.0".
>> >
>> > We'll be able to release less problem 1.0.0 by releasing
>> > 0.14.0 as RC for 1.0.0. 0.14.0 will be used more people than
>> > 1.0.0-RCX. 0.14.0 users will find critical problems.
>> >
>> >
>> > Thanks,
>> > --
>> > kou
>> >
>> > In <CA...@mail.gmail.com>
>> > "Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
>> Arrow protocol stability" on Fri, 7 Jun 2019 22:28:22 -0700,
>> > Micah Kornfield <em...@gmail.com> wrote:
>> >
>> > > A few thoughts:
>> > > - I think we should iron out the remaining incompatibilities between
>> java
>> > > and C++ before going to 1.0.0 (at least Union and NullType), and I'm
>> not
>> > > sure I will have time to them before the next release, so I would
>> prefer to
>> > > try to aim for the subsequent release to make it 1.0.0
>> > > - For 1.0.0 should we change the metadata format version to a new
>> naming
>> > > scheme [1] (seems like more of a hassle then it is worth)?
>> > > - I'm a little concerned about the implications for
>> forward-compatibility
>> > > restrictions for format changes. For instance the large list types
>> would
>> > > not be forward compatible (at least by some definitions), similarly if
>> we
>> > > deal with compression [2] it would also seem to not be forward
>> compatible.
>> > > Would this mean we bump the format version number for each change even
>> > > though they would be backwards compatible?
>> > >
>> > > Thanks,
>> > > Micah
>> > >
>> > > [1] https://github.com/apache/arrow/blob/master/format/Schema.fbs#L22
>> > > [2] https://issues.apache.org/jira/browse/ARROW-300
>> > >
>> > > On Fri, Jun 7, 2019 at 12:42 PM Wes McKinney <we...@gmail.com>
>> wrote:
>> > >
>> > >> I agree re: marketing value of a 1.0.0 release.
>> > >>
>> > >> For the record, I think we should continue to allow the API of each
>> > >> respective library component to evolve freely and allow the
>> > >> individuals developing each to decide how to handle deprecations, API
>> > >> changes, etc., as we have up until this point. The project is still
>> > >> very much in "innovation mode" across the board, but some parts may
>> > >> grow more conservative than others. Having roughly time-based releases
>> > >> encourages everyone to be ready-to-release at any given time, and we
>> > >> develop a steady cadence of getting new functionality and
>> > >> improvements/fixes out the door.
>> > >>
>> > >> On Fri, Jun 7, 2019 at 1:25 PM Antoine Pitrou <an...@python.org>
>> wrote:
>> > >> >
>> > >> >
>> > >> > I think there's a marketing merit to issuing a 1.0.0 release.
>> > >> >
>> > >> > Regards
>> > >> >
>> > >> > Antoine.
>> > >> >
>> > >> >
>> > >> > Le 07/06/2019 à 20:05, Wes McKinney a écrit :
>> > >> > > So one idea is that we could call the next release 1.14.0. So the
>> > >> > > second number is the API version number. This encodes a
>> sequencing of
>> > >> > > the evolution of the API. The library APIs are already decoupled
>> from
>> > >> > > the binary serialization protocol, so I think we merely have to
>> state
>> > >> > > that API changes and protocol changes are not related to each
>> other.
>> > >> > >
>> > >> > > On Fri, Jun 7, 2019 at 12:58 PM Jacques Nadeau <
>> jacques@apache.org>
>> > >> wrote:
>> > >> > >>
>> > >> > >> It brings up an interesting point... do we couple the stability
>> of
>> > >> the apis
>> > >> > >> with the stability of the protocol. If the protocol is stable, we
>> > >> should
>> > >> > >> start providing guarantees for it. How do we want to express
>> these
>> > >> > >> different velocities?
>> > >> > >>
>> > >> > >> On Fri, Jun 7, 2019 at 10:48 AM Antoine Pitrou <
>> antoine@python.org>
>> > >> wrote:
>> > >> > >>
>> > >> > >>>
>> > >> > >>> Le 07/06/2019 à 19:44, Jacques Nadeau a écrit :
>> > >> > >>>> On Fri, Jun 7, 2019 at 10:25 AM Antoine Pitrou <
>> antoine@python.org>
>> > >> > >>> wrote:
>> > >> > >>>>
>> > >> > >>>>> Hi Wes,
>> > >> > >>>>>
>> > >> > >>>>> Le 07/06/2019 à 17:42, Wes McKinney a écrit :
>> > >> > >>>>>>
>> > >> > >>>>>> I think
>> > >> > >>>>>> this would have a lot of benefits for project onlookers to
>> remove
>> > >> > >>>>>> various warnings around the codebase around stability and
>> cautions
>> > >> > >>>>>> against persistence of protocol data. It's fair to say that
>> if we
>> > >> _do_
>> > >> > >>>>>> make changes in the future, that there will be a transition
>> path
>> > >> for
>> > >> > >>>>>> migrate persisted data, should it ever come to that.
>> > >> > >>>>>
>> > >> > >>>>> I think that's a good idea, but perhaps the stability promise
>> > >> shouldn't
>> > >> > >>>>> cover the Flight protocol yet?
>> > >> > >>>>
>> > >> > >>>> Agreed.
>> > >> > >>>>
>> > >> > >>>>>> I would suggest a "1.0.0" release either as our next release
>> > >> (instead
>> > >> > >>>>>> of 0.14.0) or the release right after that (if we need more
>> time
>> > >> to
>> > >> > >>>>>> get affairs in order), with the guidance for users of:
>> > >> > >>>>>
>> > >> > >>>>> I think we should first do a regular 0.14.0 with all that's
>> on our
>> > >> plate
>> > >> > >>>>> right now, then work towards a 1.0.0 as the release following
>> that.
>> > >> > >>>>
>> > >> > >>>> What is different from your perspective? If the protocol hasn't
>> > >> changed
>> > >> > >>> in
>> > >> > >>>> over a year, why not call it 1.0?
>> > >> > >>>
>> > >> > >>> I would say that perhaps some API cleanup is in order. Remove
>> > >> > >>> deprecated ones, review experimental APIs, perhaps mark
>> experimental
>> > >> > >>> certain APIs that we forgot to...
>> > >> > >>>
>> > >> > >>> Regards
>> > >> > >>>
>> > >> > >>> Antoine.
>> > >> > >>>
>> > >>
>>
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Jacques Nadeau <ja...@apache.org>.
Sounds good.
On Mon, Jun 10, 2019 at 11:06 AM Wes McKinney <we...@gmail.com> wrote:
> Hi all,
>
> OK, it sounds like there is reasonable consensus behind the plan:
>
> * Make a 0.14.0 release in the near future (later this month?)
> * Publicize that the next release will be 1.0.0, in a "speak now or
> hold your peace" fashion
> * Release 1.0.0 as following release. I would suggest not waiting too
> long, so late August / early September time frame
>
> I'm going to continue grooming the 0.14.0 backlog to help refine the
> scope of what still needs to be done for C++/Python to get the next
> release out. If the stakeholders in various project subcomponents
> could also groom the backlog and mark any blockers, that would be very
> helpful.
>
> I suggest shooting for a release candidate for 0.14.0 either the week
> of June 24 or July 1 (depending on where things stand)
>
> Thanks
> Wes
>
> On Mon, Jun 10, 2019 at 2:39 AM Sutou Kouhei <ko...@clear-code.com> wrote:
> >
> > Hi,
> >
> > I think that 0.14.0 is better for the next version.
> >
> > People who don't try Apache Arrow yet to wait 1.0.0 will use
> > Apache Arrow when we release 1.0.0. If 1.0.0 satisfies them,
> > we will get more users and contributors by 1.0.0. They may
> > not care protocol stability. They may just care "1.0.0".
> >
> > We'll be able to release less problem 1.0.0 by releasing
> > 0.14.0 as RC for 1.0.0. 0.14.0 will be used more people than
> > 1.0.0-RCX. 0.14.0 users will find critical problems.
> >
> >
> > Thanks,
> > --
> > kou
> >
> > In <CA...@mail.gmail.com>
> > "Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
> Arrow protocol stability" on Fri, 7 Jun 2019 22:28:22 -0700,
> > Micah Kornfield <em...@gmail.com> wrote:
> >
> > > A few thoughts:
> > > - I think we should iron out the remaining incompatibilities between
> java
> > > and C++ before going to 1.0.0 (at least Union and NullType), and I'm
> not
> > > sure I will have time to them before the next release, so I would
> prefer to
> > > try to aim for the subsequent release to make it 1.0.0
> > > - For 1.0.0 should we change the metadata format version to a new
> naming
> > > scheme [1] (seems like more of a hassle then it is worth)?
> > > - I'm a little concerned about the implications for
> forward-compatibility
> > > restrictions for format changes. For instance the large list types
> would
> > > not be forward compatible (at least by some definitions), similarly if
> we
> > > deal with compression [2] it would also seem to not be forward
> compatible.
> > > Would this mean we bump the format version number for each change even
> > > though they would be backwards compatible?
> > >
> > > Thanks,
> > > Micah
> > >
> > > [1] https://github.com/apache/arrow/blob/master/format/Schema.fbs#L22
> > > [2] https://issues.apache.org/jira/browse/ARROW-300
> > >
> > > On Fri, Jun 7, 2019 at 12:42 PM Wes McKinney <we...@gmail.com>
> wrote:
> > >
> > >> I agree re: marketing value of a 1.0.0 release.
> > >>
> > >> For the record, I think we should continue to allow the API of each
> > >> respective library component to evolve freely and allow the
> > >> individuals developing each to decide how to handle deprecations, API
> > >> changes, etc., as we have up until this point. The project is still
> > >> very much in "innovation mode" across the board, but some parts may
> > >> grow more conservative than others. Having roughly time-based releases
> > >> encourages everyone to be ready-to-release at any given time, and we
> > >> develop a steady cadence of getting new functionality and
> > >> improvements/fixes out the door.
> > >>
> > >> On Fri, Jun 7, 2019 at 1:25 PM Antoine Pitrou <an...@python.org>
> wrote:
> > >> >
> > >> >
> > >> > I think there's a marketing merit to issuing a 1.0.0 release.
> > >> >
> > >> > Regards
> > >> >
> > >> > Antoine.
> > >> >
> > >> >
> > >> > Le 07/06/2019 à 20:05, Wes McKinney a écrit :
> > >> > > So one idea is that we could call the next release 1.14.0. So the
> > >> > > second number is the API version number. This encodes a
> sequencing of
> > >> > > the evolution of the API. The library APIs are already decoupled
> from
> > >> > > the binary serialization protocol, so I think we merely have to
> state
> > >> > > that API changes and protocol changes are not related to each
> other.
> > >> > >
> > >> > > On Fri, Jun 7, 2019 at 12:58 PM Jacques Nadeau <
> jacques@apache.org>
> > >> wrote:
> > >> > >>
> > >> > >> It brings up an interesting point... do we couple the stability
> of
> > >> the apis
> > >> > >> with the stability of the protocol. If the protocol is stable, we
> > >> should
> > >> > >> start providing guarantees for it. How do we want to express
> these
> > >> > >> different velocities?
> > >> > >>
> > >> > >> On Fri, Jun 7, 2019 at 10:48 AM Antoine Pitrou <
> antoine@python.org>
> > >> wrote:
> > >> > >>
> > >> > >>>
> > >> > >>> Le 07/06/2019 à 19:44, Jacques Nadeau a écrit :
> > >> > >>>> On Fri, Jun 7, 2019 at 10:25 AM Antoine Pitrou <
> antoine@python.org>
> > >> > >>> wrote:
> > >> > >>>>
> > >> > >>>>> Hi Wes,
> > >> > >>>>>
> > >> > >>>>> Le 07/06/2019 à 17:42, Wes McKinney a écrit :
> > >> > >>>>>>
> > >> > >>>>>> I think
> > >> > >>>>>> this would have a lot of benefits for project onlookers to
> remove
> > >> > >>>>>> various warnings around the codebase around stability and
> cautions
> > >> > >>>>>> against persistence of protocol data. It's fair to say that
> if we
> > >> _do_
> > >> > >>>>>> make changes in the future, that there will be a transition
> path
> > >> for
> > >> > >>>>>> migrate persisted data, should it ever come to that.
> > >> > >>>>>
> > >> > >>>>> I think that's a good idea, but perhaps the stability promise
> > >> shouldn't
> > >> > >>>>> cover the Flight protocol yet?
> > >> > >>>>
> > >> > >>>> Agreed.
> > >> > >>>>
> > >> > >>>>>> I would suggest a "1.0.0" release either as our next release
> > >> (instead
> > >> > >>>>>> of 0.14.0) or the release right after that (if we need more
> time
> > >> to
> > >> > >>>>>> get affairs in order), with the guidance for users of:
> > >> > >>>>>
> > >> > >>>>> I think we should first do a regular 0.14.0 with all that's
> on our
> > >> plate
> > >> > >>>>> right now, then work towards a 1.0.0 as the release following
> that.
> > >> > >>>>
> > >> > >>>> What is different from your perspective? If the protocol hasn't
> > >> changed
> > >> > >>> in
> > >> > >>>> over a year, why not call it 1.0?
> > >> > >>>
> > >> > >>> I would say that perhaps some API cleanup is in order. Remove
> > >> > >>> deprecated ones, review experimental APIs, perhaps mark
> experimental
> > >> > >>> certain APIs that we forgot to...
> > >> > >>>
> > >> > >>> Regards
> > >> > >>>
> > >> > >>> Antoine.
> > >> > >>>
> > >>
>
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Wes McKinney <we...@gmail.com>.
Hi all,
OK, it sounds like there is reasonable consensus behind the plan:
* Make a 0.14.0 release in the near future (later this month?)
* Publicize that the next release will be 1.0.0, in a "speak now or
hold your peace" fashion
* Release 1.0.0 as following release. I would suggest not waiting too
long, so late August / early September time frame
I'm going to continue grooming the 0.14.0 backlog to help refine the
scope of what still needs to be done for C++/Python to get the next
release out. If the stakeholders in various project subcomponents
could also groom the backlog and mark any blockers, that would be very
helpful.
I suggest shooting for a release candidate for 0.14.0 either the week
of June 24 or July 1 (depending on where things stand)
Thanks
Wes
On Mon, Jun 10, 2019 at 2:39 AM Sutou Kouhei <ko...@clear-code.com> wrote:
>
> Hi,
>
> I think that 0.14.0 is better for the next version.
>
> People who don't try Apache Arrow yet to wait 1.0.0 will use
> Apache Arrow when we release 1.0.0. If 1.0.0 satisfies them,
> we will get more users and contributors by 1.0.0. They may
> not care protocol stability. They may just care "1.0.0".
>
> We'll be able to release less problem 1.0.0 by releasing
> 0.14.0 as RC for 1.0.0. 0.14.0 will be used more people than
> 1.0.0-RCX. 0.14.0 users will find critical problems.
>
>
> Thanks,
> --
> kou
>
> In <CA...@mail.gmail.com>
> "Re: [DISCUSS] Timing of release and making a 1.0.0 release marking Arrow protocol stability" on Fri, 7 Jun 2019 22:28:22 -0700,
> Micah Kornfield <em...@gmail.com> wrote:
>
> > A few thoughts:
> > - I think we should iron out the remaining incompatibilities between java
> > and C++ before going to 1.0.0 (at least Union and NullType), and I'm not
> > sure I will have time to them before the next release, so I would prefer to
> > try to aim for the subsequent release to make it 1.0.0
> > - For 1.0.0 should we change the metadata format version to a new naming
> > scheme [1] (seems like more of a hassle then it is worth)?
> > - I'm a little concerned about the implications for forward-compatibility
> > restrictions for format changes. For instance the large list types would
> > not be forward compatible (at least by some definitions), similarly if we
> > deal with compression [2] it would also seem to not be forward compatible.
> > Would this mean we bump the format version number for each change even
> > though they would be backwards compatible?
> >
> > Thanks,
> > Micah
> >
> > [1] https://github.com/apache/arrow/blob/master/format/Schema.fbs#L22
> > [2] https://issues.apache.org/jira/browse/ARROW-300
> >
> > On Fri, Jun 7, 2019 at 12:42 PM Wes McKinney <we...@gmail.com> wrote:
> >
> >> I agree re: marketing value of a 1.0.0 release.
> >>
> >> For the record, I think we should continue to allow the API of each
> >> respective library component to evolve freely and allow the
> >> individuals developing each to decide how to handle deprecations, API
> >> changes, etc., as we have up until this point. The project is still
> >> very much in "innovation mode" across the board, but some parts may
> >> grow more conservative than others. Having roughly time-based releases
> >> encourages everyone to be ready-to-release at any given time, and we
> >> develop a steady cadence of getting new functionality and
> >> improvements/fixes out the door.
> >>
> >> On Fri, Jun 7, 2019 at 1:25 PM Antoine Pitrou <an...@python.org> wrote:
> >> >
> >> >
> >> > I think there's a marketing merit to issuing a 1.0.0 release.
> >> >
> >> > Regards
> >> >
> >> > Antoine.
> >> >
> >> >
> >> > Le 07/06/2019 à 20:05, Wes McKinney a écrit :
> >> > > So one idea is that we could call the next release 1.14.0. So the
> >> > > second number is the API version number. This encodes a sequencing of
> >> > > the evolution of the API. The library APIs are already decoupled from
> >> > > the binary serialization protocol, so I think we merely have to state
> >> > > that API changes and protocol changes are not related to each other.
> >> > >
> >> > > On Fri, Jun 7, 2019 at 12:58 PM Jacques Nadeau <ja...@apache.org>
> >> wrote:
> >> > >>
> >> > >> It brings up an interesting point... do we couple the stability of
> >> the apis
> >> > >> with the stability of the protocol. If the protocol is stable, we
> >> should
> >> > >> start providing guarantees for it. How do we want to express these
> >> > >> different velocities?
> >> > >>
> >> > >> On Fri, Jun 7, 2019 at 10:48 AM Antoine Pitrou <an...@python.org>
> >> wrote:
> >> > >>
> >> > >>>
> >> > >>> Le 07/06/2019 à 19:44, Jacques Nadeau a écrit :
> >> > >>>> On Fri, Jun 7, 2019 at 10:25 AM Antoine Pitrou <an...@python.org>
> >> > >>> wrote:
> >> > >>>>
> >> > >>>>> Hi Wes,
> >> > >>>>>
> >> > >>>>> Le 07/06/2019 à 17:42, Wes McKinney a écrit :
> >> > >>>>>>
> >> > >>>>>> I think
> >> > >>>>>> this would have a lot of benefits for project onlookers to remove
> >> > >>>>>> various warnings around the codebase around stability and cautions
> >> > >>>>>> against persistence of protocol data. It's fair to say that if we
> >> _do_
> >> > >>>>>> make changes in the future, that there will be a transition path
> >> for
> >> > >>>>>> migrate persisted data, should it ever come to that.
> >> > >>>>>
> >> > >>>>> I think that's a good idea, but perhaps the stability promise
> >> shouldn't
> >> > >>>>> cover the Flight protocol yet?
> >> > >>>>
> >> > >>>> Agreed.
> >> > >>>>
> >> > >>>>>> I would suggest a "1.0.0" release either as our next release
> >> (instead
> >> > >>>>>> of 0.14.0) or the release right after that (if we need more time
> >> to
> >> > >>>>>> get affairs in order), with the guidance for users of:
> >> > >>>>>
> >> > >>>>> I think we should first do a regular 0.14.0 with all that's on our
> >> plate
> >> > >>>>> right now, then work towards a 1.0.0 as the release following that.
> >> > >>>>
> >> > >>>> What is different from your perspective? If the protocol hasn't
> >> changed
> >> > >>> in
> >> > >>>> over a year, why not call it 1.0?
> >> > >>>
> >> > >>> I would say that perhaps some API cleanup is in order. Remove
> >> > >>> deprecated ones, review experimental APIs, perhaps mark experimental
> >> > >>> certain APIs that we forgot to...
> >> > >>>
> >> > >>> Regards
> >> > >>>
> >> > >>> Antoine.
> >> > >>>
> >>
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
I think that 0.14.0 is better for the next version.
People who don't try Apache Arrow yet to wait 1.0.0 will use
Apache Arrow when we release 1.0.0. If 1.0.0 satisfies them,
we will get more users and contributors by 1.0.0. They may
not care protocol stability. They may just care "1.0.0".
We'll be able to release less problem 1.0.0 by releasing
0.14.0 as RC for 1.0.0. 0.14.0 will be used more people than
1.0.0-RCX. 0.14.0 users will find critical problems.
Thanks,
--
kou
In <CA...@mail.gmail.com>
"Re: [DISCUSS] Timing of release and making a 1.0.0 release marking Arrow protocol stability" on Fri, 7 Jun 2019 22:28:22 -0700,
Micah Kornfield <em...@gmail.com> wrote:
> A few thoughts:
> - I think we should iron out the remaining incompatibilities between java
> and C++ before going to 1.0.0 (at least Union and NullType), and I'm not
> sure I will have time to them before the next release, so I would prefer to
> try to aim for the subsequent release to make it 1.0.0
> - For 1.0.0 should we change the metadata format version to a new naming
> scheme [1] (seems like more of a hassle then it is worth)?
> - I'm a little concerned about the implications for forward-compatibility
> restrictions for format changes. For instance the large list types would
> not be forward compatible (at least by some definitions), similarly if we
> deal with compression [2] it would also seem to not be forward compatible.
> Would this mean we bump the format version number for each change even
> though they would be backwards compatible?
>
> Thanks,
> Micah
>
> [1] https://github.com/apache/arrow/blob/master/format/Schema.fbs#L22
> [2] https://issues.apache.org/jira/browse/ARROW-300
>
> On Fri, Jun 7, 2019 at 12:42 PM Wes McKinney <we...@gmail.com> wrote:
>
>> I agree re: marketing value of a 1.0.0 release.
>>
>> For the record, I think we should continue to allow the API of each
>> respective library component to evolve freely and allow the
>> individuals developing each to decide how to handle deprecations, API
>> changes, etc., as we have up until this point. The project is still
>> very much in "innovation mode" across the board, but some parts may
>> grow more conservative than others. Having roughly time-based releases
>> encourages everyone to be ready-to-release at any given time, and we
>> develop a steady cadence of getting new functionality and
>> improvements/fixes out the door.
>>
>> On Fri, Jun 7, 2019 at 1:25 PM Antoine Pitrou <an...@python.org> wrote:
>> >
>> >
>> > I think there's a marketing merit to issuing a 1.0.0 release.
>> >
>> > Regards
>> >
>> > Antoine.
>> >
>> >
>> > Le 07/06/2019 à 20:05, Wes McKinney a écrit :
>> > > So one idea is that we could call the next release 1.14.0. So the
>> > > second number is the API version number. This encodes a sequencing of
>> > > the evolution of the API. The library APIs are already decoupled from
>> > > the binary serialization protocol, so I think we merely have to state
>> > > that API changes and protocol changes are not related to each other.
>> > >
>> > > On Fri, Jun 7, 2019 at 12:58 PM Jacques Nadeau <ja...@apache.org>
>> wrote:
>> > >>
>> > >> It brings up an interesting point... do we couple the stability of
>> the apis
>> > >> with the stability of the protocol. If the protocol is stable, we
>> should
>> > >> start providing guarantees for it. How do we want to express these
>> > >> different velocities?
>> > >>
>> > >> On Fri, Jun 7, 2019 at 10:48 AM Antoine Pitrou <an...@python.org>
>> wrote:
>> > >>
>> > >>>
>> > >>> Le 07/06/2019 à 19:44, Jacques Nadeau a écrit :
>> > >>>> On Fri, Jun 7, 2019 at 10:25 AM Antoine Pitrou <an...@python.org>
>> > >>> wrote:
>> > >>>>
>> > >>>>> Hi Wes,
>> > >>>>>
>> > >>>>> Le 07/06/2019 à 17:42, Wes McKinney a écrit :
>> > >>>>>>
>> > >>>>>> I think
>> > >>>>>> this would have a lot of benefits for project onlookers to remove
>> > >>>>>> various warnings around the codebase around stability and cautions
>> > >>>>>> against persistence of protocol data. It's fair to say that if we
>> _do_
>> > >>>>>> make changes in the future, that there will be a transition path
>> for
>> > >>>>>> migrate persisted data, should it ever come to that.
>> > >>>>>
>> > >>>>> I think that's a good idea, but perhaps the stability promise
>> shouldn't
>> > >>>>> cover the Flight protocol yet?
>> > >>>>
>> > >>>> Agreed.
>> > >>>>
>> > >>>>>> I would suggest a "1.0.0" release either as our next release
>> (instead
>> > >>>>>> of 0.14.0) or the release right after that (if we need more time
>> to
>> > >>>>>> get affairs in order), with the guidance for users of:
>> > >>>>>
>> > >>>>> I think we should first do a regular 0.14.0 with all that's on our
>> plate
>> > >>>>> right now, then work towards a 1.0.0 as the release following that.
>> > >>>>
>> > >>>> What is different from your perspective? If the protocol hasn't
>> changed
>> > >>> in
>> > >>>> over a year, why not call it 1.0?
>> > >>>
>> > >>> I would say that perhaps some API cleanup is in order. Remove
>> > >>> deprecated ones, review experimental APIs, perhaps mark experimental
>> > >>> certain APIs that we forgot to...
>> > >>>
>> > >>> Regards
>> > >>>
>> > >>> Antoine.
>> > >>>
>>
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Micah Kornfield <em...@gmail.com>.
A few thoughts:
- I think we should iron out the remaining incompatibilities between java
and C++ before going to 1.0.0 (at least Union and NullType), and I'm not
sure I will have time to them before the next release, so I would prefer to
try to aim for the subsequent release to make it 1.0.0
- For 1.0.0 should we change the metadata format version to a new naming
scheme [1] (seems like more of a hassle then it is worth)?
- I'm a little concerned about the implications for forward-compatibility
restrictions for format changes. For instance the large list types would
not be forward compatible (at least by some definitions), similarly if we
deal with compression [2] it would also seem to not be forward compatible.
Would this mean we bump the format version number for each change even
though they would be backwards compatible?
Thanks,
Micah
[1] https://github.com/apache/arrow/blob/master/format/Schema.fbs#L22
[2] https://issues.apache.org/jira/browse/ARROW-300
On Fri, Jun 7, 2019 at 12:42 PM Wes McKinney <we...@gmail.com> wrote:
> I agree re: marketing value of a 1.0.0 release.
>
> For the record, I think we should continue to allow the API of each
> respective library component to evolve freely and allow the
> individuals developing each to decide how to handle deprecations, API
> changes, etc., as we have up until this point. The project is still
> very much in "innovation mode" across the board, but some parts may
> grow more conservative than others. Having roughly time-based releases
> encourages everyone to be ready-to-release at any given time, and we
> develop a steady cadence of getting new functionality and
> improvements/fixes out the door.
>
> On Fri, Jun 7, 2019 at 1:25 PM Antoine Pitrou <an...@python.org> wrote:
> >
> >
> > I think there's a marketing merit to issuing a 1.0.0 release.
> >
> > Regards
> >
> > Antoine.
> >
> >
> > Le 07/06/2019 à 20:05, Wes McKinney a écrit :
> > > So one idea is that we could call the next release 1.14.0. So the
> > > second number is the API version number. This encodes a sequencing of
> > > the evolution of the API. The library APIs are already decoupled from
> > > the binary serialization protocol, so I think we merely have to state
> > > that API changes and protocol changes are not related to each other.
> > >
> > > On Fri, Jun 7, 2019 at 12:58 PM Jacques Nadeau <ja...@apache.org>
> wrote:
> > >>
> > >> It brings up an interesting point... do we couple the stability of
> the apis
> > >> with the stability of the protocol. If the protocol is stable, we
> should
> > >> start providing guarantees for it. How do we want to express these
> > >> different velocities?
> > >>
> > >> On Fri, Jun 7, 2019 at 10:48 AM Antoine Pitrou <an...@python.org>
> wrote:
> > >>
> > >>>
> > >>> Le 07/06/2019 à 19:44, Jacques Nadeau a écrit :
> > >>>> On Fri, Jun 7, 2019 at 10:25 AM Antoine Pitrou <an...@python.org>
> > >>> wrote:
> > >>>>
> > >>>>> Hi Wes,
> > >>>>>
> > >>>>> Le 07/06/2019 à 17:42, Wes McKinney a écrit :
> > >>>>>>
> > >>>>>> I think
> > >>>>>> this would have a lot of benefits for project onlookers to remove
> > >>>>>> various warnings around the codebase around stability and cautions
> > >>>>>> against persistence of protocol data. It's fair to say that if we
> _do_
> > >>>>>> make changes in the future, that there will be a transition path
> for
> > >>>>>> migrate persisted data, should it ever come to that.
> > >>>>>
> > >>>>> I think that's a good idea, but perhaps the stability promise
> shouldn't
> > >>>>> cover the Flight protocol yet?
> > >>>>
> > >>>> Agreed.
> > >>>>
> > >>>>>> I would suggest a "1.0.0" release either as our next release
> (instead
> > >>>>>> of 0.14.0) or the release right after that (if we need more time
> to
> > >>>>>> get affairs in order), with the guidance for users of:
> > >>>>>
> > >>>>> I think we should first do a regular 0.14.0 with all that's on our
> plate
> > >>>>> right now, then work towards a 1.0.0 as the release following that.
> > >>>>
> > >>>> What is different from your perspective? If the protocol hasn't
> changed
> > >>> in
> > >>>> over a year, why not call it 1.0?
> > >>>
> > >>> I would say that perhaps some API cleanup is in order. Remove
> > >>> deprecated ones, review experimental APIs, perhaps mark experimental
> > >>> certain APIs that we forgot to...
> > >>>
> > >>> Regards
> > >>>
> > >>> Antoine.
> > >>>
>
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Wes McKinney <we...@gmail.com>.
I agree re: marketing value of a 1.0.0 release.
For the record, I think we should continue to allow the API of each
respective library component to evolve freely and allow the
individuals developing each to decide how to handle deprecations, API
changes, etc., as we have up until this point. The project is still
very much in "innovation mode" across the board, but some parts may
grow more conservative than others. Having roughly time-based releases
encourages everyone to be ready-to-release at any given time, and we
develop a steady cadence of getting new functionality and
improvements/fixes out the door.
On Fri, Jun 7, 2019 at 1:25 PM Antoine Pitrou <an...@python.org> wrote:
>
>
> I think there's a marketing merit to issuing a 1.0.0 release.
>
> Regards
>
> Antoine.
>
>
> Le 07/06/2019 à 20:05, Wes McKinney a écrit :
> > So one idea is that we could call the next release 1.14.0. So the
> > second number is the API version number. This encodes a sequencing of
> > the evolution of the API. The library APIs are already decoupled from
> > the binary serialization protocol, so I think we merely have to state
> > that API changes and protocol changes are not related to each other.
> >
> > On Fri, Jun 7, 2019 at 12:58 PM Jacques Nadeau <ja...@apache.org> wrote:
> >>
> >> It brings up an interesting point... do we couple the stability of the apis
> >> with the stability of the protocol. If the protocol is stable, we should
> >> start providing guarantees for it. How do we want to express these
> >> different velocities?
> >>
> >> On Fri, Jun 7, 2019 at 10:48 AM Antoine Pitrou <an...@python.org> wrote:
> >>
> >>>
> >>> Le 07/06/2019 à 19:44, Jacques Nadeau a écrit :
> >>>> On Fri, Jun 7, 2019 at 10:25 AM Antoine Pitrou <an...@python.org>
> >>> wrote:
> >>>>
> >>>>> Hi Wes,
> >>>>>
> >>>>> Le 07/06/2019 à 17:42, Wes McKinney a écrit :
> >>>>>>
> >>>>>> I think
> >>>>>> this would have a lot of benefits for project onlookers to remove
> >>>>>> various warnings around the codebase around stability and cautions
> >>>>>> against persistence of protocol data. It's fair to say that if we _do_
> >>>>>> make changes in the future, that there will be a transition path for
> >>>>>> migrate persisted data, should it ever come to that.
> >>>>>
> >>>>> I think that's a good idea, but perhaps the stability promise shouldn't
> >>>>> cover the Flight protocol yet?
> >>>>
> >>>> Agreed.
> >>>>
> >>>>>> I would suggest a "1.0.0" release either as our next release (instead
> >>>>>> of 0.14.0) or the release right after that (if we need more time to
> >>>>>> get affairs in order), with the guidance for users of:
> >>>>>
> >>>>> I think we should first do a regular 0.14.0 with all that's on our plate
> >>>>> right now, then work towards a 1.0.0 as the release following that.
> >>>>
> >>>> What is different from your perspective? If the protocol hasn't changed
> >>> in
> >>>> over a year, why not call it 1.0?
> >>>
> >>> I would say that perhaps some API cleanup is in order. Remove
> >>> deprecated ones, review experimental APIs, perhaps mark experimental
> >>> certain APIs that we forgot to...
> >>>
> >>> Regards
> >>>
> >>> Antoine.
> >>>
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Antoine Pitrou <an...@python.org>.
I think there's a marketing merit to issuing a 1.0.0 release.
Regards
Antoine.
Le 07/06/2019 à 20:05, Wes McKinney a écrit :
> So one idea is that we could call the next release 1.14.0. So the
> second number is the API version number. This encodes a sequencing of
> the evolution of the API. The library APIs are already decoupled from
> the binary serialization protocol, so I think we merely have to state
> that API changes and protocol changes are not related to each other.
>
> On Fri, Jun 7, 2019 at 12:58 PM Jacques Nadeau <ja...@apache.org> wrote:
>>
>> It brings up an interesting point... do we couple the stability of the apis
>> with the stability of the protocol. If the protocol is stable, we should
>> start providing guarantees for it. How do we want to express these
>> different velocities?
>>
>> On Fri, Jun 7, 2019 at 10:48 AM Antoine Pitrou <an...@python.org> wrote:
>>
>>>
>>> Le 07/06/2019 à 19:44, Jacques Nadeau a écrit :
>>>> On Fri, Jun 7, 2019 at 10:25 AM Antoine Pitrou <an...@python.org>
>>> wrote:
>>>>
>>>>> Hi Wes,
>>>>>
>>>>> Le 07/06/2019 à 17:42, Wes McKinney a écrit :
>>>>>>
>>>>>> I think
>>>>>> this would have a lot of benefits for project onlookers to remove
>>>>>> various warnings around the codebase around stability and cautions
>>>>>> against persistence of protocol data. It's fair to say that if we _do_
>>>>>> make changes in the future, that there will be a transition path for
>>>>>> migrate persisted data, should it ever come to that.
>>>>>
>>>>> I think that's a good idea, but perhaps the stability promise shouldn't
>>>>> cover the Flight protocol yet?
>>>>
>>>> Agreed.
>>>>
>>>>>> I would suggest a "1.0.0" release either as our next release (instead
>>>>>> of 0.14.0) or the release right after that (if we need more time to
>>>>>> get affairs in order), with the guidance for users of:
>>>>>
>>>>> I think we should first do a regular 0.14.0 with all that's on our plate
>>>>> right now, then work towards a 1.0.0 as the release following that.
>>>>
>>>> What is different from your perspective? If the protocol hasn't changed
>>> in
>>>> over a year, why not call it 1.0?
>>>
>>> I would say that perhaps some API cleanup is in order. Remove
>>> deprecated ones, review experimental APIs, perhaps mark experimental
>>> certain APIs that we forgot to...
>>>
>>> Regards
>>>
>>> Antoine.
>>>
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Wes McKinney <we...@gmail.com>.
So one idea is that we could call the next release 1.14.0. So the
second number is the API version number. This encodes a sequencing of
the evolution of the API. The library APIs are already decoupled from
the binary serialization protocol, so I think we merely have to state
that API changes and protocol changes are not related to each other.
On Fri, Jun 7, 2019 at 12:58 PM Jacques Nadeau <ja...@apache.org> wrote:
>
> It brings up an interesting point... do we couple the stability of the apis
> with the stability of the protocol. If the protocol is stable, we should
> start providing guarantees for it. How do we want to express these
> different velocities?
>
> On Fri, Jun 7, 2019 at 10:48 AM Antoine Pitrou <an...@python.org> wrote:
>
> >
> > Le 07/06/2019 à 19:44, Jacques Nadeau a écrit :
> > > On Fri, Jun 7, 2019 at 10:25 AM Antoine Pitrou <an...@python.org>
> > wrote:
> > >
> > >> Hi Wes,
> > >>
> > >> Le 07/06/2019 à 17:42, Wes McKinney a écrit :
> > >>>
> > >>> I think
> > >>> this would have a lot of benefits for project onlookers to remove
> > >>> various warnings around the codebase around stability and cautions
> > >>> against persistence of protocol data. It's fair to say that if we _do_
> > >>> make changes in the future, that there will be a transition path for
> > >>> migrate persisted data, should it ever come to that.
> > >>
> > >> I think that's a good idea, but perhaps the stability promise shouldn't
> > >> cover the Flight protocol yet?
> > >
> > > Agreed.
> > >
> > >>> I would suggest a "1.0.0" release either as our next release (instead
> > >>> of 0.14.0) or the release right after that (if we need more time to
> > >>> get affairs in order), with the guidance for users of:
> > >>
> > >> I think we should first do a regular 0.14.0 with all that's on our plate
> > >> right now, then work towards a 1.0.0 as the release following that.
> > >
> > > What is different from your perspective? If the protocol hasn't changed
> > in
> > > over a year, why not call it 1.0?
> >
> > I would say that perhaps some API cleanup is in order. Remove
> > deprecated ones, review experimental APIs, perhaps mark experimental
> > certain APIs that we forgot to...
> >
> > Regards
> >
> > Antoine.
> >
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Jacques Nadeau <ja...@apache.org>.
It brings up an interesting point... do we couple the stability of the apis
with the stability of the protocol. If the protocol is stable, we should
start providing guarantees for it. How do we want to express these
different velocities?
On Fri, Jun 7, 2019 at 10:48 AM Antoine Pitrou <an...@python.org> wrote:
>
> Le 07/06/2019 à 19:44, Jacques Nadeau a écrit :
> > On Fri, Jun 7, 2019 at 10:25 AM Antoine Pitrou <an...@python.org>
> wrote:
> >
> >> Hi Wes,
> >>
> >> Le 07/06/2019 à 17:42, Wes McKinney a écrit :
> >>>
> >>> I think
> >>> this would have a lot of benefits for project onlookers to remove
> >>> various warnings around the codebase around stability and cautions
> >>> against persistence of protocol data. It's fair to say that if we _do_
> >>> make changes in the future, that there will be a transition path for
> >>> migrate persisted data, should it ever come to that.
> >>
> >> I think that's a good idea, but perhaps the stability promise shouldn't
> >> cover the Flight protocol yet?
> >
> > Agreed.
> >
> >>> I would suggest a "1.0.0" release either as our next release (instead
> >>> of 0.14.0) or the release right after that (if we need more time to
> >>> get affairs in order), with the guidance for users of:
> >>
> >> I think we should first do a regular 0.14.0 with all that's on our plate
> >> right now, then work towards a 1.0.0 as the release following that.
> >
> > What is different from your perspective? If the protocol hasn't changed
> in
> > over a year, why not call it 1.0?
>
> I would say that perhaps some API cleanup is in order. Remove
> deprecated ones, review experimental APIs, perhaps mark experimental
> certain APIs that we forgot to...
>
> Regards
>
> Antoine.
>
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Antoine Pitrou <an...@python.org>.
Le 07/06/2019 à 19:44, Jacques Nadeau a écrit :
> On Fri, Jun 7, 2019 at 10:25 AM Antoine Pitrou <an...@python.org> wrote:
>
>> Hi Wes,
>>
>> Le 07/06/2019 à 17:42, Wes McKinney a écrit :
>>>
>>> I think
>>> this would have a lot of benefits for project onlookers to remove
>>> various warnings around the codebase around stability and cautions
>>> against persistence of protocol data. It's fair to say that if we _do_
>>> make changes in the future, that there will be a transition path for
>>> migrate persisted data, should it ever come to that.
>>
>> I think that's a good idea, but perhaps the stability promise shouldn't
>> cover the Flight protocol yet?
>
> Agreed.
>
>>> I would suggest a "1.0.0" release either as our next release (instead
>>> of 0.14.0) or the release right after that (if we need more time to
>>> get affairs in order), with the guidance for users of:
>>
>> I think we should first do a regular 0.14.0 with all that's on our plate
>> right now, then work towards a 1.0.0 as the release following that.
>
> What is different from your perspective? If the protocol hasn't changed in
> over a year, why not call it 1.0?
I would say that perhaps some API cleanup is in order. Remove
deprecated ones, review experimental APIs, perhaps mark experimental
certain APIs that we forgot to...
Regards
Antoine.
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Jacques Nadeau <ja...@apache.org>.
On Fri, Jun 7, 2019 at 10:25 AM Antoine Pitrou <an...@python.org> wrote:
>
> Hi Wes,
>
> Le 07/06/2019 à 17:42, Wes McKinney a écrit :
> >
> > I think
> > this would have a lot of benefits for project onlookers to remove
> > various warnings around the codebase around stability and cautions
> > against persistence of protocol data. It's fair to say that if we _do_
> > make changes in the future, that there will be a transition path for
> > migrate persisted data, should it ever come to that.
>
> I think that's a good idea, but perhaps the stability promise shouldn't
> cover the Flight protocol yet?
>
Agreed.
>
> > I would suggest a "1.0.0" release either as our next release (instead
> > of 0.14.0) or the release right after that (if we need more time to
> > get affairs in order), with the guidance for users of:
>
> I think we should first do a regular 0.14.0 with all that's on our plate
> right now, then work towards a 1.0.0 as the release following that.
>
What is different from your perspective? If the protocol hasn't changed in
over a year, why not call it 1.0?
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Antoine Pitrou <an...@python.org>.
Hi Wes,
Le 07/06/2019 à 17:42, Wes McKinney a écrit :
>
> I think
> this would have a lot of benefits for project onlookers to remove
> various warnings around the codebase around stability and cautions
> against persistence of protocol data. It's fair to say that if we _do_
> make changes in the future, that there will be a transition path for
> migrate persisted data, should it ever come to that.
I think that's a good idea, but perhaps the stability promise shouldn't
cover the Flight protocol yet?
> I would suggest a "1.0.0" release either as our next release (instead
> of 0.14.0) or the release right after that (if we need more time to
> get affairs in order), with the guidance for users of:
I think we should first do a regular 0.14.0 with all that's on our plate
right now, then work towards a 1.0.0 as the release following that.
> PROTOCOL VERSION (1): Protocol version, so libraries bearing 1.x.y
> will be forward and backwards compatible (though new metadata fields
> introduced in newer versions will be dropped in older readers)
> MAJOR VERSION (0): API changes possible (and indeed, likely) from
> major release to major release
> MINOR VERSION (0): No API changes, bug fix only release
>
> Thoughts on the above?
That sounds reasonable to me.
Regards
Antoine.
Re: [DISCUSS] Timing of release and making a 1.0.0 release marking
Arrow protocol stability
Posted by Jacques Nadeau <ja...@apache.org>.
You didn't mention it specifically but one big thing I'd like to get into
the next release is prepackaged flight tools for c++, java and python.
As far as 1.0, I think its time and would vote for naming the next release
1.0. Thanks for bringing this up!
On Fri, Jun 7, 2019 at 8:43 AM Wes McKinney <we...@gmail.com> wrote:
> hi folks,
>
> Our last release 0.13.0 occurred at the end of March. I think it would
> be good to plot a course for the next release (0.14.0?) as soon as
> possible. There are still a number of issues (such as the shared
> library duplication issue in the Python wheels) that I think might
> discourage us from releasing right now. Do you think that pushing for
> a release candidate by the end of June is reasonable?
>
> As a second matter (and this can be split off into a separate
> discussion thread), the Arrow format and binary protocol has been
> stable effectively since the 0.8.0 release which occurred in December
> 2017. While we have some details yet to iron out in compatibility
> testing between implementations (for example, the Union question, see
> mailing list discussion [1]) and new features (e.g. 64-bit offset
> binary/string/list types), in theory these should not prevent us
> necessarily from making a declaration of protocol stability. I think
> this would have a lot of benefits for project onlookers to remove
> various warnings around the codebase around stability and cautions
> against persistence of protocol data. It's fair to say that if we _do_
> make changes in the future, that there will be a transition path for
> migrate persisted data, should it ever come to that.
>
> I would suggest a "1.0.0" release either as our next release (instead
> of 0.14.0) or the release right after that (if we need more time to
> get affairs in order), with the guidance for users of:
>
> PROTOCOL VERSION (1): Protocol version, so libraries bearing 1.x.y
> will be forward and backwards compatible (though new metadata fields
> introduced in newer versions will be dropped in older readers)
> MAJOR VERSION (0): API changes possible (and indeed, likely) from
> major release to major release
> MINOR VERSION (0): No API changes, bug fix only release
>
> Thoughts on the above?
>
> Thanks
> Wes
>
> [1]:
> https://lists.apache.org/thread.html/e54e8ec096f665a8aef94155de3b6c567258c0d15209de4b966dd8da@%3Cdev.arrow.apache.org%3E
>