[Format] Monotonic offsets for variable size (e.g. Utf8 and Binary) arrays

[Andrew Lamb <al...@influxdata.com>]

The C/C++ and Rust implementations of Arrow seem to assume (and validate)
that the offsets in a variable size array such as Utf8 and Binary are
monotonic, even for values that are null (when the validity is not set).

The existing format documentation does not seem to address this matter
explicitly, so I propose a change to make it explicit [1].

Since this is related  to the overall format and there may be previous
discussion on the topic I missed, I wanted to raise it on the mailing list
for broader input. I also searched the archives of the mailing list but was
not able to find anything previously on this topic

Thanks,
Andrew

[1] https://github.com/apache/arrow/pull/12019

Re: [Format] Monotonic offsets for variable size (e.g. Utf8 and Binary) arrays

[Wang Xudong <wx...@gmail.com>]

That sounds great, makes sense to me. Thanks!

--
xudong

Andrew Lamb <al...@influxdata.com> 于2021年12月22日周三 20:12写道：

> The C/C++ and Rust implementations of Arrow seem to assume (and validate)
> that the offsets in a variable size array such as Utf8 and Binary are
> monotonic, even for values that are null (when the validity is not set).
>
> The existing format documentation does not seem to address this matter
> explicitly, so I propose a change to make it explicit [1].
>
> Since this is related  to the overall format and there may be previous
> discussion on the topic I missed, I wanted to raise it on the mailing list
> for broader input. I also searched the archives of the mailing list but was
> not able to find anything previously on this topic
>
> Thanks,
> Andrew
>
> [1] https://github.com/apache/arrow/pull/12019
>