[Go] Npyio / Arrow support

[Sebastien Binet <bi...@cern.ch>]

hi there,

Back in the days, for my particle physics work, I had to be able to read
and write numpy data files[1] (for interop with existing analysis
pipelines.)

I was wondering whether I couldn't integrate this Go package with Apache
Arrow:
- https://github.com/sbinet/npyio

and put it under github.com/apache/arrow/go as either
- github.com/apache/arrow/go/npyio, or
- github.com/apache/arrow/go/arrow/npyio (as a sibling of the already
existing csv package.)

that new npyio package would have builtin support for array.Interface and
tensor.Interface.

thoughts?

cheers,
-s

[1]: http://docs.scipy.org/doc/numpy/neps/npy-format.html

Re: [Go] Npyio / Arrow support

[Wes McKinney <we...@gmail.com>]

hi Sebastien,

Seems reasonable, and the integration between the libraries could be
useful (although npyio could be made to depend on the Arrow Go
libraries)

If there is the desire from the Arrow Go developers to merge in that
codebase, it seems perhaps old enough that we might have to do an IP
clearance -- we've done enough of these that it's become fairly
routine at this point. I've conducted so many IP clearances (7,
according to http://incubator.apache.org/ip-clearance/) that I would
really appreciate if another PMC member could assume ownership for
doing one.

- Wes
On Fri, Dec 7, 2018 at 2:50 AM Sebastien Binet <bi...@cern.ch> wrote:
>
> hi there,
>
> Back in the days, for my particle physics work, I had to be able to read
> and write numpy data files[1] (for interop with existing analysis
> pipelines.)
>
> I was wondering whether I couldn't integrate this Go package with Apache
> Arrow:
> - https://github.com/sbinet/npyio
>
> and put it under github.com/apache/arrow/go as either
> - github.com/apache/arrow/go/npyio, or
> - github.com/apache/arrow/go/arrow/npyio (as a sibling of the already
> existing csv package.)
>
> that new npyio package would have builtin support for array.Interface and
> tensor.Interface.
>
> thoughts?
>
> cheers,
> -s
>
> [1]: http://docs.scipy.org/doc/numpy/neps/npy-format.html