You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by mariusvniekerk <ma...@gmail.com> on 2016/11/01 03:30:46 UTC
Re: Python Spark Improvements (forked from Spark Improvement
Proposals)
So i've been working on some very very early stage apache arrow integration.
My current plan it to emulate some of how the R function execution works.
If there are any other people working on similar efforts it would be good
idea to combine efforts.
I can see how much effort is involved in converting that PR to a spark
package so that people can try to use it. I think this is something that we
want some more community iteration on maybe?
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Python-Spark-Improvements-forked-from-Spark-Improvement-Proposals-tp19422p19670.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: Python Spark Improvements (forked from Spark Improvement Proposals)
Posted by Holden Karau <ho...@pigscanfly.ca>.
On that note there is some discussion on the Jira -
https://issues.apache.org/jira/browse/SPARK-13534 :)
On Mon, Oct 31, 2016 at 8:32 PM, Holden Karau <ho...@pigscanfly.ca> wrote:
> I believe Bryan is also working on this a little - and I'm a little busy
> with the other stuff but would love to stay in the loop on Arrow progress :)
>
>
> On Monday, October 31, 2016, mariusvniekerk <ma...@gmail.com>
> wrote:
>
>> So i've been working on some very very early stage apache arrow
>> integration.
>> My current plan it to emulate some of how the R function execution works.
>> If there are any other people working on similar efforts it would be good
>> idea to combine efforts.
>>
>> I can see how much effort is involved in converting that PR to a spark
>> package so that people can try to use it. I think this is something that
>> we
>> want some more community iteration on maybe?
>>
>>
>>
>>
>>
>> --
>> View this message in context: http://apache-spark-developers
>> -list.1001551.n3.nabble.com/Python-Spark-Improvements-
>> forked-from-Spark-Improvement-Proposals-tp19422p19670.html
>> Sent from the Apache Spark Developers List mailing list archive at
>> Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>
>
> --
> Cell : 425-233-8271
> Twitter: https://twitter.com/holdenkarau
>
>
--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau
Re: Python Spark Improvements (forked from Spark Improvement Proposals)
Posted by Holden Karau <ho...@pigscanfly.ca>.
I believe Bryan is also working on this a little - and I'm a little busy
with the other stuff but would love to stay in the loop on Arrow progress :)
On Monday, October 31, 2016, mariusvniekerk <ma...@gmail.com>
wrote:
> So i've been working on some very very early stage apache arrow
> integration.
> My current plan it to emulate some of how the R function execution works.
> If there are any other people working on similar efforts it would be good
> idea to combine efforts.
>
> I can see how much effort is involved in converting that PR to a spark
> package so that people can try to use it. I think this is something that
> we
> want some more community iteration on maybe?
>
>
>
>
>
> --
> View this message in context: http://apache-spark-
> developers-list.1001551.n3.nabble.com/Python-Spark-
> Improvements-forked-from-Spark-Improvement-Proposals-tp19422p19670.html
> Sent from the Apache Spark Developers List mailing list archive at
> Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org <javascript:;>
>
>
--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau