Re: Apache Arrow + Spark examples?

[Nirav Patel <np...@xactlycorp.com>]

bump.

I have same question at Petr. SPARK-13534 seem to only solve
de(serialization) issue involved between rdd and python objects. However,
isn't Arrow can be standard  for in-memory columnar representation? may be
alternative to spark current in-memory store (k-v blocks or tungsten)

Thanks
Nir

On Wed, Feb 24, 2016 at 3:56 AM, Petr Novak <os...@gmail.com> wrote:

>
> How Arrows collide with Tungsten and its binary in-memory format. It will
> still has to convert between them. I assume they use similar
> concepts/layout hence it is likely the conversion can be quite efficient.
> Or is there a change that the current Tungsten in memory format would be
> replaced by Arrows in the future. The same applies for Impala, Drill and
> all others. Is the goal to unify internal in-memory representation for all
> of them or the benefit is going to be in conversions faster by e.g. order
> of magnitude?
>
> Many thanks for any explanation,
> Petr
>

-- 


[image: What's New with Xactly] <http://www.xactlycorp.com/email-click/>

<https://www.nyse.com/quote/XNYS:XTLY>  [image: LinkedIn] 
<https://www.linkedin.com/company/xactly-corporation>  [image: Twitter] 
<https://twitter.com/Xactly>  [image: Facebook] 
<https://www.facebook.com/XactlyCorp>  [image: YouTube] 
<http://www.youtube.com/xactlycorporation>