You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Wes McKinney <we...@gmail.com> on 2019/07/26 17:48:51 UTC

Procedure for seeing through implementation of new Arrow format features

hi folks,

In https://github.com/apache/arrow/pull/4921 Antoine has implemented
binary and unicode type variants with 64-bit offsets (to permit
payloads exceeding 2GB). I believe Micah plans to follow on with a
Java implementation in a separate PR.

We previously have discussed that we should ensure that we have
multiple tested implementations of new Arrow columnar format features
-- we have not discussed how such changes are to be sequenced.

For the avoidance of doubt, my view on the procedure is that we should
not _release_ the project containing additions to the protocol files
without at least 2 independent implementations of those features (with
integration tests, where relevant). I think it's OK for one
implementation to land in master and then the other in a follow up PR.

We'd like to get https://github.com/apache/arrow/pull/4921 merged soon
(today or tomorrow if possible), so if anyone has a different opinion
about how to stage new format implementations in a non-awkward way,
could you please chime in?

Thank you,
Wes