You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Mitar <mm...@gmail.com> on 2019/03/30 22:14:24 UTC
Support for numpy matrix
Hi!
It seems numpy's matrix is not supported in recent versions of pyarrow:
https://issues.apache.org/jira/browse/ARROW-3399
Any ideas why this would be happening?
Mitar
--
http://mitar.tnode.com/
https://twitter.com/mitar_m
Re: Support for numpy matrix
Posted by Mitar <mm...@gmail.com>.
Hi!
I agree. This is in fact all information which is already there. :-)
Mitar
On Sat, Mar 30, 2019 at 8:40 PM Wes McKinney <we...@gmail.com> wrote:
>
> hi Mitar,
>
> Let's discuss further on JIRA? It's best to keep all the information
> about the issue in one place.
>
> Thanks
>
> On Sat, Mar 30, 2019 at 7:42 PM Mitar <mm...@gmail.com> wrote:
> >
> > Hi!
> >
> > I added:
> >
> > serialization_context.register_type(
> > np.matrix, 'np.matrix',
> > custom_serializer=_serialize_numpy_array_list,
> > custom_deserializer=_deserialize_numpy_array_list)
> >
> > But it did not help. Probably also because np.matrix is a subclas of
> > np.ndarray anyway. So no change here.
> >
> > An interesting fact is that this worked in older versions of numpy,
> > but stopped in numpy 1.15.2. It works with numpy 1.14.3. So it is them
> > changing something.
> >
> >
> > Mitar
> >
> > On Sat, Mar 30, 2019 at 3:34 PM Philipp Moritz <pc...@gmail.com> wrote:
> > >
> > > Hey Mitar,
> > >
> > > It might be as simple as adding a handler here:
> > > https://github.com/apache/arrow/blob/master/python/pyarrow/serialization.py#L300
> > >
> > > Do you want to try that?
> > >
> > > -- Philipp.
> > >
> > > On Sat, Mar 30, 2019 at 3:22 PM Mitar <mm...@gmail.com> wrote:
> > >
> > > > Hi!
> > > >
> > > > I do not know where to start looking into this? Not sure if I have
> > > > enough knowledge about arrow to be able to make a PR.
> > > >
> > > >
> > > > Miar
> > > >
> > > > On Sat, Mar 30, 2019 at 3:17 PM Wes McKinney <we...@gmail.com> wrote:
> > > > >
> > > > > hi Mitar,
> > > > >
> > > > > I see you reported the issue on October 2 and no one has volunteered
> > > > > to fix it yet. Are you up to submit a PR?
> > > > >
> > > > > Thanks
> > > > > Wes
> > > > >
> > > > > On Sat, Mar 30, 2019 at 5:14 PM Mitar <mm...@gmail.com> wrote:
> > > > > >
> > > > > > Hi!
> > > > > >
> > > > > > It seems numpy's matrix is not supported in recent versions of pyarrow:
> > > > > >
> > > > > > https://issues.apache.org/jira/browse/ARROW-3399
> > > > > >
> > > > > > Any ideas why this would be happening?
> > > > > >
> > > > > >
> > > > > > Mitar
> > > > > >
> > > > > > --
> > > > > > http://mitar.tnode.com/
> > > > > > https://twitter.com/mitar_m
> > > >
> > > >
> > > >
> > > > --
> > > > http://mitar.tnode.com/
> > > > https://twitter.com/mitar_m
> > > >
> >
> >
> >
> > --
> > http://mitar.tnode.com/
> > https://twitter.com/mitar_m
--
http://mitar.tnode.com/
https://twitter.com/mitar_m
Re: Support for numpy matrix
Posted by Wes McKinney <we...@gmail.com>.
hi Mitar,
Let's discuss further on JIRA? It's best to keep all the information
about the issue in one place.
Thanks
On Sat, Mar 30, 2019 at 7:42 PM Mitar <mm...@gmail.com> wrote:
>
> Hi!
>
> I added:
>
> serialization_context.register_type(
> np.matrix, 'np.matrix',
> custom_serializer=_serialize_numpy_array_list,
> custom_deserializer=_deserialize_numpy_array_list)
>
> But it did not help. Probably also because np.matrix is a subclas of
> np.ndarray anyway. So no change here.
>
> An interesting fact is that this worked in older versions of numpy,
> but stopped in numpy 1.15.2. It works with numpy 1.14.3. So it is them
> changing something.
>
>
> Mitar
>
> On Sat, Mar 30, 2019 at 3:34 PM Philipp Moritz <pc...@gmail.com> wrote:
> >
> > Hey Mitar,
> >
> > It might be as simple as adding a handler here:
> > https://github.com/apache/arrow/blob/master/python/pyarrow/serialization.py#L300
> >
> > Do you want to try that?
> >
> > -- Philipp.
> >
> > On Sat, Mar 30, 2019 at 3:22 PM Mitar <mm...@gmail.com> wrote:
> >
> > > Hi!
> > >
> > > I do not know where to start looking into this? Not sure if I have
> > > enough knowledge about arrow to be able to make a PR.
> > >
> > >
> > > Miar
> > >
> > > On Sat, Mar 30, 2019 at 3:17 PM Wes McKinney <we...@gmail.com> wrote:
> > > >
> > > > hi Mitar,
> > > >
> > > > I see you reported the issue on October 2 and no one has volunteered
> > > > to fix it yet. Are you up to submit a PR?
> > > >
> > > > Thanks
> > > > Wes
> > > >
> > > > On Sat, Mar 30, 2019 at 5:14 PM Mitar <mm...@gmail.com> wrote:
> > > > >
> > > > > Hi!
> > > > >
> > > > > It seems numpy's matrix is not supported in recent versions of pyarrow:
> > > > >
> > > > > https://issues.apache.org/jira/browse/ARROW-3399
> > > > >
> > > > > Any ideas why this would be happening?
> > > > >
> > > > >
> > > > > Mitar
> > > > >
> > > > > --
> > > > > http://mitar.tnode.com/
> > > > > https://twitter.com/mitar_m
> > >
> > >
> > >
> > > --
> > > http://mitar.tnode.com/
> > > https://twitter.com/mitar_m
> > >
>
>
>
> --
> http://mitar.tnode.com/
> https://twitter.com/mitar_m
Re: Support for numpy matrix
Posted by Mitar <mm...@gmail.com>.
Hi!
I added:
serialization_context.register_type(
np.matrix, 'np.matrix',
custom_serializer=_serialize_numpy_array_list,
custom_deserializer=_deserialize_numpy_array_list)
But it did not help. Probably also because np.matrix is a subclas of
np.ndarray anyway. So no change here.
An interesting fact is that this worked in older versions of numpy,
but stopped in numpy 1.15.2. It works with numpy 1.14.3. So it is them
changing something.
Mitar
On Sat, Mar 30, 2019 at 3:34 PM Philipp Moritz <pc...@gmail.com> wrote:
>
> Hey Mitar,
>
> It might be as simple as adding a handler here:
> https://github.com/apache/arrow/blob/master/python/pyarrow/serialization.py#L300
>
> Do you want to try that?
>
> -- Philipp.
>
> On Sat, Mar 30, 2019 at 3:22 PM Mitar <mm...@gmail.com> wrote:
>
> > Hi!
> >
> > I do not know where to start looking into this? Not sure if I have
> > enough knowledge about arrow to be able to make a PR.
> >
> >
> > Miar
> >
> > On Sat, Mar 30, 2019 at 3:17 PM Wes McKinney <we...@gmail.com> wrote:
> > >
> > > hi Mitar,
> > >
> > > I see you reported the issue on October 2 and no one has volunteered
> > > to fix it yet. Are you up to submit a PR?
> > >
> > > Thanks
> > > Wes
> > >
> > > On Sat, Mar 30, 2019 at 5:14 PM Mitar <mm...@gmail.com> wrote:
> > > >
> > > > Hi!
> > > >
> > > > It seems numpy's matrix is not supported in recent versions of pyarrow:
> > > >
> > > > https://issues.apache.org/jira/browse/ARROW-3399
> > > >
> > > > Any ideas why this would be happening?
> > > >
> > > >
> > > > Mitar
> > > >
> > > > --
> > > > http://mitar.tnode.com/
> > > > https://twitter.com/mitar_m
> >
> >
> >
> > --
> > http://mitar.tnode.com/
> > https://twitter.com/mitar_m
> >
--
http://mitar.tnode.com/
https://twitter.com/mitar_m
Re: Support for numpy matrix
Posted by Philipp Moritz <pc...@gmail.com>.
Hey Mitar,
It might be as simple as adding a handler here:
https://github.com/apache/arrow/blob/master/python/pyarrow/serialization.py#L300
Do you want to try that?
-- Philipp.
On Sat, Mar 30, 2019 at 3:22 PM Mitar <mm...@gmail.com> wrote:
> Hi!
>
> I do not know where to start looking into this? Not sure if I have
> enough knowledge about arrow to be able to make a PR.
>
>
> Miar
>
> On Sat, Mar 30, 2019 at 3:17 PM Wes McKinney <we...@gmail.com> wrote:
> >
> > hi Mitar,
> >
> > I see you reported the issue on October 2 and no one has volunteered
> > to fix it yet. Are you up to submit a PR?
> >
> > Thanks
> > Wes
> >
> > On Sat, Mar 30, 2019 at 5:14 PM Mitar <mm...@gmail.com> wrote:
> > >
> > > Hi!
> > >
> > > It seems numpy's matrix is not supported in recent versions of pyarrow:
> > >
> > > https://issues.apache.org/jira/browse/ARROW-3399
> > >
> > > Any ideas why this would be happening?
> > >
> > >
> > > Mitar
> > >
> > > --
> > > http://mitar.tnode.com/
> > > https://twitter.com/mitar_m
>
>
>
> --
> http://mitar.tnode.com/
> https://twitter.com/mitar_m
>
Re: Support for numpy matrix
Posted by Wes McKinney <we...@gmail.com>.
Let's discuss on the JIRA issue. AFAIK we have no unit tests relating
to numpy.matrix so most likely this was an accidental "feature". If
there is no other volunteer to help on the issue it's hard to predict
when it will get done. numpy.matrix isn't used by very many people
On Sat, Mar 30, 2019 at 5:22 PM Mitar <mm...@gmail.com> wrote:
>
> Hi!
>
> I do not know where to start looking into this? Not sure if I have
> enough knowledge about arrow to be able to make a PR.
>
>
> Miar
>
> On Sat, Mar 30, 2019 at 3:17 PM Wes McKinney <we...@gmail.com> wrote:
> >
> > hi Mitar,
> >
> > I see you reported the issue on October 2 and no one has volunteered
> > to fix it yet. Are you up to submit a PR?
> >
> > Thanks
> > Wes
> >
> > On Sat, Mar 30, 2019 at 5:14 PM Mitar <mm...@gmail.com> wrote:
> > >
> > > Hi!
> > >
> > > It seems numpy's matrix is not supported in recent versions of pyarrow:
> > >
> > > https://issues.apache.org/jira/browse/ARROW-3399
> > >
> > > Any ideas why this would be happening?
> > >
> > >
> > > Mitar
> > >
> > > --
> > > http://mitar.tnode.com/
> > > https://twitter.com/mitar_m
>
>
>
> --
> http://mitar.tnode.com/
> https://twitter.com/mitar_m
Re: Support for numpy matrix
Posted by Mitar <mm...@gmail.com>.
Hi!
I do not know where to start looking into this? Not sure if I have
enough knowledge about arrow to be able to make a PR.
Miar
On Sat, Mar 30, 2019 at 3:17 PM Wes McKinney <we...@gmail.com> wrote:
>
> hi Mitar,
>
> I see you reported the issue on October 2 and no one has volunteered
> to fix it yet. Are you up to submit a PR?
>
> Thanks
> Wes
>
> On Sat, Mar 30, 2019 at 5:14 PM Mitar <mm...@gmail.com> wrote:
> >
> > Hi!
> >
> > It seems numpy's matrix is not supported in recent versions of pyarrow:
> >
> > https://issues.apache.org/jira/browse/ARROW-3399
> >
> > Any ideas why this would be happening?
> >
> >
> > Mitar
> >
> > --
> > http://mitar.tnode.com/
> > https://twitter.com/mitar_m
--
http://mitar.tnode.com/
https://twitter.com/mitar_m
Re: Support for numpy matrix
Posted by Wes McKinney <we...@gmail.com>.
hi Mitar,
I see you reported the issue on October 2 and no one has volunteered
to fix it yet. Are you up to submit a PR?
Thanks
Wes
On Sat, Mar 30, 2019 at 5:14 PM Mitar <mm...@gmail.com> wrote:
>
> Hi!
>
> It seems numpy's matrix is not supported in recent versions of pyarrow:
>
> https://issues.apache.org/jira/browse/ARROW-3399
>
> Any ideas why this would be happening?
>
>
> Mitar
>
> --
> http://mitar.tnode.com/
> https://twitter.com/mitar_m