flatbuffers vectors and --gen-object-api

[John Muehlhausen <jg...@jgm.org>]

It seems as if Arrow expects for some vectors to be empty rather than null.
 (Examples: Footer.dictionaries, Field.children)

Anyone using --gen-object-api with flatc will get code that writes null
when (e.g.) _o->children.size() is zero in CreateField().

I may be missing something but I don't see a way to change this behavior in
flatc.

I understand that the object API is not as performant, but wanted to toss
out the question:

Do we want to tolerate null vectors as well as empty vectors so that other
writer implementations have this option?  E.g. if they choose to use
--gen-object-api ?

-John

Re: flatbuffers vectors and --gen-object-api

[Wes McKinney <we...@gmail.com>]

On Fri, Jul 5, 2019 at 10:53 AM John Muehlhausen <jg...@jgm.org> wrote:
>
> It seems as if Arrow expects for some vectors to be empty rather than null.
>  (Examples: Footer.dictionaries, Field.children)
>

When you say "Arrow expects" do you mean the C++ library, or
implementations in general?

> Anyone using --gen-object-api with flatc will get code that writes null
> when (e.g.) _o->children.size() is zero in CreateField().
>
> I may be missing something but I don't see a way to change this behavior in
> flatc.
>
> I understand that the object API is not as performant, but wanted to toss
> out the question:
>
> Do we want to tolerate null vectors as well as empty vectors so that other
> writer implementations have this option?  E.g. if they choose to use
> --gen-object-api ?
>

It seems like implementations ought to be robust to empty-as-null.
This issue would need to be probed somehow in the integration tests
(e.g. a C++ writer option that toggles empty vs. null) to be safe
about it.

> -John