Re: More about the use case

[moon soo Lee <mo...@apache.org>]

Hi Hao Ren,

Thanks for sharing your use case.

The simplest way i know is persist DataFrame from your spark application
and then read the data from Zeppelin. OFF_HEAP with Tachyon may help.

Best,
moon




On Tue, Sep 22, 2015 at 6:36 PM Hao Ren <in...@gmail.com> wrote:

> Hi, I am new to zeppelin, come from spark background.
>
> I would like to know whether zeppelin supports the following use case:
>
>
> Given a spark application (*not written in zeppelin*, *just a jar, etc*.),
> I would like to sent a DateFrame to a zeppelin server while the spark job
> is running, and the zeppelin displays the data in any kind of chart once it
> receives data from spark.
>
> In this case, zeppelin is just like a chart server which waits for some
> inputs from a spark job apart. The inputs can be a DataFrame, scala
> collection, etc.
>
> If any implicit APIs implemented for spark RDD or DataFrame exists, it
> would be like the following:
>
> // In my spark job
> dataFrame.sendToZeppelin(zplMasterURL)
>
>
> I need this feature because we have created a spark application for
> machine learning, and want to visualize the result. It is not possible to
> copy all code in zeppelin note book. We don't want load our application jar
> in zeppelin, neither. We prefer some zeppelin APIs allows us to send data
> to a zeppelin server from our spark job apart.
>
> Zeppelin chart is very clear and reactive, we really want to take
> advantage of this and avoid reinventing the wheel.
>
> Any help is highly appreciated.
>
> Thank you.
>
> --
> Hao Ren
>
> Data Engineer @ leboncoin
>
> Paris, France
>