You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Micah Kornfield <em...@gmail.com> on 2019/08/21 07:02:11 UTC
[Discuss][Java] Communicating module maturity
A recent issue with the JDBC adapter [1] made me realize we aren't doing
enough to communicate to consumers the maturity of various modules within
arrow. From the issue, it also seems like it is surprising that everything
is based off of off-heap data access.
To help with this I added a description to each module in a new PR [2] with
a preliminary annotation experimental/contrib [3] modules. The annotations
match my understanding of the current state of the world, but please
correct them if I got something wrong.
If anyone knows how tags on mvnrepository [4] are generated, I think it
would also be good to populate tags for experimental, contrib and
off-heap/nio code.
Is there anything else we could be doing?
Thanks,
Micah
[1] https://issues.apache.org/jira/browse/ARROW-6206
[2] https://github.com/apache/arrow/pull/5151
[3] My rough definitions for each annotation are:
1. Contrib - Limited users, not well tested in production
environments. Not necesserily optimized.
2. Experimental - Not finalized (very likely to be breaking API
changes).
Better definitions are welcome :)
[4] https://mvnrepository.com/artifact/org.apache.arrow/arrow-vector
Re: [Discuss][Java] Communicating module maturity
Posted by Micah Kornfield <em...@gmail.com>.
> The discussion in ARROW-6206 contains some mildly offensive language
> directly at the Arrow community, like "arrow is a team that picked up
> netty derived off-heap tools naively". Excuse me?
I'm trying my best to ignore language that isn't really productive to
solving technical problems :) If users become more engaged in the
community and want to understand the design philosophy, I think the
appropriate place for debate is the mailing list. I think I might try to
put together a blog post on my understanding of the intent behind the
current Java design, so we have something to point to when these types of
questions arise (this probably won't happen soon).
We're at a stage of project maturity where we are going to start to
> see more users with 0 contributions to the project complaining about
> various things.
Agreed, that is where I'm coming from
So we can do things to head off those complaints but
> we shouldn't allow people from outside the community decide what
> standards we need to hold ourselves to.
Also agreed, my motivation here is to really head off more format
complaints and least make some of the implementation details visible so
users can decided if they want to depend on a module.
Thanks,
Micah
On Wed, Aug 21, 2019 at 7:12 AM Wes McKinney <we...@gmail.com> wrote:
> hi Micah,
>
> I agree that documenting the maturity of components is a good idea.
>
> The discussion in ARROW-6206 contains some mildly offensive language
> directly at the Arrow community, like "arrow is a team that picked up
> netty derived off-heap tools naively". Excuse me? Documentation aside,
> I think such language runs afoul of our code of conduct
> (https://www.apache.org/foundation/policies/conduct).
>
> We're at a stage of project maturity where we are going to start to
> see more users with 0 contributions to the project complaining about
> various things. So we can do things to head off those complaints but
> we shouldn't allow people from outside the community decide what
> standards we need to hold ourselves to.
>
> - Wes
>
> On Wed, Aug 21, 2019 at 2:02 AM Micah Kornfield <em...@gmail.com>
> wrote:
> >
> > A recent issue with the JDBC adapter [1] made me realize we aren't doing
> > enough to communicate to consumers the maturity of various modules within
> > arrow. From the issue, it also seems like it is surprising that
> everything
> > is based off of off-heap data access.
> >
> > To help with this I added a description to each module in a new PR [2]
> with
> > a preliminary annotation experimental/contrib [3] modules. The
> annotations
> > match my understanding of the current state of the world, but please
> > correct them if I got something wrong.
> >
> > If anyone knows how tags on mvnrepository [4] are generated, I think it
> > would also be good to populate tags for experimental, contrib and
> > off-heap/nio code.
> >
> > Is there anything else we could be doing?
> >
> > Thanks,
> > Micah
> >
> > [1] https://issues.apache.org/jira/browse/ARROW-6206
> > [2] https://github.com/apache/arrow/pull/5151
> > [3] My rough definitions for each annotation are:
> > 1. Contrib - Limited users, not well tested in production
> > environments. Not necesserily optimized.
> > 2. Experimental - Not finalized (very likely to be breaking API
> > changes).
> > Better definitions are welcome :)
> > [4] https://mvnrepository.com/artifact/org.apache.arrow/arrow-vector
>
Re: [Discuss][Java] Communicating module maturity
Posted by Wes McKinney <we...@gmail.com>.
hi Micah,
I agree that documenting the maturity of components is a good idea.
The discussion in ARROW-6206 contains some mildly offensive language
directly at the Arrow community, like "arrow is a team that picked up
netty derived off-heap tools naively". Excuse me? Documentation aside,
I think such language runs afoul of our code of conduct
(https://www.apache.org/foundation/policies/conduct).
We're at a stage of project maturity where we are going to start to
see more users with 0 contributions to the project complaining about
various things. So we can do things to head off those complaints but
we shouldn't allow people from outside the community decide what
standards we need to hold ourselves to.
- Wes
On Wed, Aug 21, 2019 at 2:02 AM Micah Kornfield <em...@gmail.com> wrote:
>
> A recent issue with the JDBC adapter [1] made me realize we aren't doing
> enough to communicate to consumers the maturity of various modules within
> arrow. From the issue, it also seems like it is surprising that everything
> is based off of off-heap data access.
>
> To help with this I added a description to each module in a new PR [2] with
> a preliminary annotation experimental/contrib [3] modules. The annotations
> match my understanding of the current state of the world, but please
> correct them if I got something wrong.
>
> If anyone knows how tags on mvnrepository [4] are generated, I think it
> would also be good to populate tags for experimental, contrib and
> off-heap/nio code.
>
> Is there anything else we could be doing?
>
> Thanks,
> Micah
>
> [1] https://issues.apache.org/jira/browse/ARROW-6206
> [2] https://github.com/apache/arrow/pull/5151
> [3] My rough definitions for each annotation are:
> 1. Contrib - Limited users, not well tested in production
> environments. Not necesserily optimized.
> 2. Experimental - Not finalized (very likely to be breaking API
> changes).
> Better definitions are welcome :)
> [4] https://mvnrepository.com/artifact/org.apache.arrow/arrow-vector