You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Joris Peeters <jo...@gmail.com> on 2018/02/13 13:41:54 UTC
Arrow for MATLAB?
Hello,
Is anyone aware of plans (or concrete projects) to add MATLAB bindings for
Arrow? I'm interested in exchanging data between Java, Python, ..., and
MATLAB - and Arrow sounds like a great solution.
I couldn't find any pre-existing effort, though, so curious if that is due
to a lack of interest or because there might be underlying reasons that
would make this very hard to achieve.
Best,
-Joris.
Re: Arrow for MATLAB?
Posted by Nick Haddad <Ni...@mathworks.com>.
Thanks Wes,
I will look these over!
-Nick
On 2/15/18, 12:45 PM, "Wes McKinney" <we...@gmail.com> wrote:
hi Nick,
I would strongly encourage you to focus your energies on supporting
the general Arrow streaming and file formats as described in
http://arrow.apache.org/docs/ipc.html. It's fine to support Feather,
so long as you restrict yourself to the API defined in
https://github.com/apache/arrow/blob/master/cpp/src/arrow/ipc/feather.h
As soon as we have reasonable R bindings for the Arrow C++ libraries,
I would like to replace the internal details of the Feather format
with the Arrow IPC format, deprecating the metadata defined in
https://github.com/apache/arrow/blob/master/cpp/src/arrow/ipc/feather.fbs
Please let us know how things go and if there's anything we can do to help.
Thanks,
Wes
On Thu, Feb 15, 2018 at 9:48 AM, Nick Haddad <Ni...@mathworks.com> wrote:
> Uwe and Joris,
>
> I work for MathWorks. We plan to contribute Feather read/write capability in the near future. Interesting idea about MATLAB and Arrow. We’ll take a look as well.
>
> Thanks
> -Nick
>
>
> On 2/14/18, 3:39 AM, "Joris Peeters" <jo...@gmail.com> wrote:
>
> Yeah, pinging Mathworks might be worth it. It feels like this is something
> that could be of great value to them. Aside from HDF5 & mat files (both on
> disk), it can be really tedious to efficiently share data from anywhere
> with MATLAB, so it's becoming increasingly isolated.
> I'm going to play around for a few days with Arrow & the Matlab C++ API,
> getting a bit more familiar with it and maybe hacking a small prototype
> together. Currently it's just out-of-hours, though, so don't expect major
> magic. :)
>
> Thanks,
> -J
>
> On Tue, Feb 13, 2018 at 2:33 PM, Phillip Cloud <cp...@gmail.com> wrote:
>
> > The MathWorks is in the process of starting to contribute. I spoke with
> > them a couple weeks ago about this and they were excited about it. I can
> > ping them to see if they are still interested.
> >
> > On Tue, Feb 13, 2018, 09:24 Uwe L. Korn <uw...@xhochy.com> wrote:
> >
> > > Hello Joris,
> > >
> > > this is only due to lack of someone doing it and probably due to lack of
> > > people that have the experience to do that. I had a short look at
> > Matlab's
> > > C++ API and the interfaces seem to be promising enough
> > > https://de.mathworks.com/help/matlab/matlab-data-array.html that once
> > > someone attempts it, it should not be hard to build.
> > >
> > > If you want to try to take a shot, we are happy to help if there are
> > > problems with the Arrow side of things.
> > >
> > > Uwe
> > >
> > > On Tue, Feb 13, 2018, at 2:41 PM, Joris Peeters wrote:
> > > > Hello,
> > > >
> > > > Is anyone aware of plans (or concrete projects) to add MATLAB bindings
> > > for
> > > > Arrow? I'm interested in exchanging data between Java, Python, ..., and
> > > > MATLAB - and Arrow sounds like a great solution.
> > > >
> > > > I couldn't find any pre-existing effort, though, so curious if that is
> > > due
> > > > to a lack of interest or because there might be underlying reasons that
> > > > would make this very hard to achieve.
> > > >
> > > > Best,
> > > > -Joris.
> > >
> >
>
>
Re: Arrow for MATLAB?
Posted by Wes McKinney <we...@gmail.com>.
hi Nick,
I would strongly encourage you to focus your energies on supporting
the general Arrow streaming and file formats as described in
http://arrow.apache.org/docs/ipc.html. It's fine to support Feather,
so long as you restrict yourself to the API defined in
https://github.com/apache/arrow/blob/master/cpp/src/arrow/ipc/feather.h
As soon as we have reasonable R bindings for the Arrow C++ libraries,
I would like to replace the internal details of the Feather format
with the Arrow IPC format, deprecating the metadata defined in
https://github.com/apache/arrow/blob/master/cpp/src/arrow/ipc/feather.fbs.
Please let us know how things go and if there's anything we can do to help.
Thanks,
Wes
On Thu, Feb 15, 2018 at 9:48 AM, Nick Haddad <Ni...@mathworks.com> wrote:
> Uwe and Joris,
>
> I work for MathWorks. We plan to contribute Feather read/write capability in the near future. Interesting idea about MATLAB and Arrow. We’ll take a look as well.
>
> Thanks
> -Nick
>
>
> On 2/14/18, 3:39 AM, "Joris Peeters" <jo...@gmail.com> wrote:
>
> Yeah, pinging Mathworks might be worth it. It feels like this is something
> that could be of great value to them. Aside from HDF5 & mat files (both on
> disk), it can be really tedious to efficiently share data from anywhere
> with MATLAB, so it's becoming increasingly isolated.
> I'm going to play around for a few days with Arrow & the Matlab C++ API,
> getting a bit more familiar with it and maybe hacking a small prototype
> together. Currently it's just out-of-hours, though, so don't expect major
> magic. :)
>
> Thanks,
> -J
>
> On Tue, Feb 13, 2018 at 2:33 PM, Phillip Cloud <cp...@gmail.com> wrote:
>
> > The MathWorks is in the process of starting to contribute. I spoke with
> > them a couple weeks ago about this and they were excited about it. I can
> > ping them to see if they are still interested.
> >
> > On Tue, Feb 13, 2018, 09:24 Uwe L. Korn <uw...@xhochy.com> wrote:
> >
> > > Hello Joris,
> > >
> > > this is only due to lack of someone doing it and probably due to lack of
> > > people that have the experience to do that. I had a short look at
> > Matlab's
> > > C++ API and the interfaces seem to be promising enough
> > > https://de.mathworks.com/help/matlab/matlab-data-array.html that once
> > > someone attempts it, it should not be hard to build.
> > >
> > > If you want to try to take a shot, we are happy to help if there are
> > > problems with the Arrow side of things.
> > >
> > > Uwe
> > >
> > > On Tue, Feb 13, 2018, at 2:41 PM, Joris Peeters wrote:
> > > > Hello,
> > > >
> > > > Is anyone aware of plans (or concrete projects) to add MATLAB bindings
> > > for
> > > > Arrow? I'm interested in exchanging data between Java, Python, ..., and
> > > > MATLAB - and Arrow sounds like a great solution.
> > > >
> > > > I couldn't find any pre-existing effort, though, so curious if that is
> > > due
> > > > to a lack of interest or because there might be underlying reasons that
> > > > would make this very hard to achieve.
> > > >
> > > > Best,
> > > > -Joris.
> > >
> >
>
>
Re: Arrow for MATLAB?
Posted by Nick Haddad <Ni...@mathworks.com>.
Uwe and Joris,
I work for MathWorks. We plan to contribute Feather read/write capability in the near future. Interesting idea about MATLAB and Arrow. We’ll take a look as well.
Thanks
-Nick
On 2/14/18, 3:39 AM, "Joris Peeters" <jo...@gmail.com> wrote:
Yeah, pinging Mathworks might be worth it. It feels like this is something
that could be of great value to them. Aside from HDF5 & mat files (both on
disk), it can be really tedious to efficiently share data from anywhere
with MATLAB, so it's becoming increasingly isolated.
I'm going to play around for a few days with Arrow & the Matlab C++ API,
getting a bit more familiar with it and maybe hacking a small prototype
together. Currently it's just out-of-hours, though, so don't expect major
magic. :)
Thanks,
-J
On Tue, Feb 13, 2018 at 2:33 PM, Phillip Cloud <cp...@gmail.com> wrote:
> The MathWorks is in the process of starting to contribute. I spoke with
> them a couple weeks ago about this and they were excited about it. I can
> ping them to see if they are still interested.
>
> On Tue, Feb 13, 2018, 09:24 Uwe L. Korn <uw...@xhochy.com> wrote:
>
> > Hello Joris,
> >
> > this is only due to lack of someone doing it and probably due to lack of
> > people that have the experience to do that. I had a short look at
> Matlab's
> > C++ API and the interfaces seem to be promising enough
> > https://de.mathworks.com/help/matlab/matlab-data-array.html that once
> > someone attempts it, it should not be hard to build.
> >
> > If you want to try to take a shot, we are happy to help if there are
> > problems with the Arrow side of things.
> >
> > Uwe
> >
> > On Tue, Feb 13, 2018, at 2:41 PM, Joris Peeters wrote:
> > > Hello,
> > >
> > > Is anyone aware of plans (or concrete projects) to add MATLAB bindings
> > for
> > > Arrow? I'm interested in exchanging data between Java, Python, ..., and
> > > MATLAB - and Arrow sounds like a great solution.
> > >
> > > I couldn't find any pre-existing effort, though, so curious if that is
> > due
> > > to a lack of interest or because there might be underlying reasons that
> > > would make this very hard to achieve.
> > >
> > > Best,
> > > -Joris.
> >
>
Re: Arrow for MATLAB?
Posted by Joris Peeters <jo...@gmail.com>.
Yeah, pinging Mathworks might be worth it. It feels like this is something
that could be of great value to them. Aside from HDF5 & mat files (both on
disk), it can be really tedious to efficiently share data from anywhere
with MATLAB, so it's becoming increasingly isolated.
I'm going to play around for a few days with Arrow & the Matlab C++ API,
getting a bit more familiar with it and maybe hacking a small prototype
together. Currently it's just out-of-hours, though, so don't expect major
magic. :)
Thanks,
-J
On Tue, Feb 13, 2018 at 2:33 PM, Phillip Cloud <cp...@gmail.com> wrote:
> The MathWorks is in the process of starting to contribute. I spoke with
> them a couple weeks ago about this and they were excited about it. I can
> ping them to see if they are still interested.
>
> On Tue, Feb 13, 2018, 09:24 Uwe L. Korn <uw...@xhochy.com> wrote:
>
> > Hello Joris,
> >
> > this is only due to lack of someone doing it and probably due to lack of
> > people that have the experience to do that. I had a short look at
> Matlab's
> > C++ API and the interfaces seem to be promising enough
> > https://de.mathworks.com/help/matlab/matlab-data-array.html that once
> > someone attempts it, it should not be hard to build.
> >
> > If you want to try to take a shot, we are happy to help if there are
> > problems with the Arrow side of things.
> >
> > Uwe
> >
> > On Tue, Feb 13, 2018, at 2:41 PM, Joris Peeters wrote:
> > > Hello,
> > >
> > > Is anyone aware of plans (or concrete projects) to add MATLAB bindings
> > for
> > > Arrow? I'm interested in exchanging data between Java, Python, ..., and
> > > MATLAB - and Arrow sounds like a great solution.
> > >
> > > I couldn't find any pre-existing effort, though, so curious if that is
> > due
> > > to a lack of interest or because there might be underlying reasons that
> > > would make this very hard to achieve.
> > >
> > > Best,
> > > -Joris.
> >
>
Re: Arrow for MATLAB?
Posted by Phillip Cloud <cp...@gmail.com>.
The MathWorks is in the process of starting to contribute. I spoke with
them a couple weeks ago about this and they were excited about it. I can
ping them to see if they are still interested.
On Tue, Feb 13, 2018, 09:24 Uwe L. Korn <uw...@xhochy.com> wrote:
> Hello Joris,
>
> this is only due to lack of someone doing it and probably due to lack of
> people that have the experience to do that. I had a short look at Matlab's
> C++ API and the interfaces seem to be promising enough
> https://de.mathworks.com/help/matlab/matlab-data-array.html that once
> someone attempts it, it should not be hard to build.
>
> If you want to try to take a shot, we are happy to help if there are
> problems with the Arrow side of things.
>
> Uwe
>
> On Tue, Feb 13, 2018, at 2:41 PM, Joris Peeters wrote:
> > Hello,
> >
> > Is anyone aware of plans (or concrete projects) to add MATLAB bindings
> for
> > Arrow? I'm interested in exchanging data between Java, Python, ..., and
> > MATLAB - and Arrow sounds like a great solution.
> >
> > I couldn't find any pre-existing effort, though, so curious if that is
> due
> > to a lack of interest or because there might be underlying reasons that
> > would make this very hard to achieve.
> >
> > Best,
> > -Joris.
>
Re: Arrow for MATLAB?
Posted by "Uwe L. Korn" <uw...@xhochy.com>.
Hello Joris,
this is only due to lack of someone doing it and probably due to lack of people that have the experience to do that. I had a short look at Matlab's C++ API and the interfaces seem to be promising enough https://de.mathworks.com/help/matlab/matlab-data-array.html that once someone attempts it, it should not be hard to build.
If you want to try to take a shot, we are happy to help if there are problems with the Arrow side of things.
Uwe
On Tue, Feb 13, 2018, at 2:41 PM, Joris Peeters wrote:
> Hello,
>
> Is anyone aware of plans (or concrete projects) to add MATLAB bindings for
> Arrow? I'm interested in exchanging data between Java, Python, ..., and
> MATLAB - and Arrow sounds like a great solution.
>
> I couldn't find any pre-existing effort, though, so curious if that is due
> to a lack of interest or because there might be underlying reasons that
> would make this very hard to achieve.
>
> Best,
> -Joris.