You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@zeppelin.apache.org by Dave O'Flanagan <da...@boxever.com> on 2015/07/03 15:35:29 UTC
SparkR Interpreter
Hi,
Is there a SparkR interpreter in Zeppelin in the roadmap? It would be great
to have similar capability to the REPL provided by sparkR.sh from the Spark
distro.
Thanks,
Dave.
----
Dave O'Flanagan
CEO, Boxever
e: dave@boxever.com
t: +353 86 8518476
w: www.boxever.com
Re: SparkR Interpreter
Posted by fe...@hotmail.com.
By itself, you should be able to add just these calls to R to initialize SparkR (assuming Spark/SparkR 1.4.0+):
library(SparkR)
sc <- sparkR.init()
sqlContext <- sparkRSQL.init(sc)
On Sat, Jul 4, 2015 at 5:03 AM -0700, "Eric Charles" <er...@apache.org> wrote:
Thx for the kind words Felix. Really appreciated!
I will come back asap with:
1. ZEPPELIN-154
2. SparkR with our zeppelin-R interpreter.
On 2015-07-04 13:54, felixcheung_m@hotmail.com wrote:
>
> Very nice Eric. Interesting approach to use Rserve and knitr, which
> solve the ploting problem (Interesting projects at datalayer too)
>
> I've been working on the SparkR interpreter. There the challenges are
> getting it to work with Spark interpreter within Zeppelin. Would be
> interested in your thoughts.
>
>> From: Eric Charles
>> Sent: Saturday, July 4, 1:11 AM
>> Subject: Re: SparkR Interpreter
>> To: users@zeppelin.incubator.apache.org
>>
>> https://github.com/datalayer/zeppelin-R uses Rserve (GPL-2 License).
>>
>> Rserve for variable sharing across paragraph and knitr for html
>> rendering (with image plot) was our best bet. SparkR should be ok with
>> this (will check).
>>
>> Are you investing other alternatives to implement SparkR?
>>
>> Side note: I have opened ZEPPELIN-154 and will propose a PR to allow 3rd
>> party interpreters to be used in Zeppelin without the need to have them
>> in the Apache source tree.
>>
>> On 2015-07-04 00:10, moon soo Lee wrote:
>> > Hi,
>> >
>> > Yes, there is SparkR in roadmap. and work in progress.
>> > (
>> https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap)
>> >
>> > Thanks,
>> > moon
>> >
>> > On Fri, Jul 3, 2015 at 6:35 AM Dave O'Flanagan
>> > <david.oflanagan@boxever.com < mailto:david.oflanagan@boxever.com>>
>> wrote:
>> >
>> > Hi,
>> >
>> > Is there a SparkR interpreter in Zeppelin in the roadmap? It would
>> > be great to have similar capability to the REPL provided by
>> > sparkR.sh from the Spark distro.
>> >
>> > Thanks,
>> > Dave.
>> >
>> > ----
>> > Dave O'Flanagan
>> > CEO, Boxever
>> > e: dave@boxever.com < mailto:dave@boxever.com>
>> > t: +353 86 8518476
>> > w: www.boxever.com <http://www.boxever.com> <
>> http://www.boxever.com/>
>> >
>>
Re: SparkR Interpreter
Posted by Eric Charles <er...@apache.org>.
Thx for the kind words Felix. Really appreciated!
I will come back asap with:
1. ZEPPELIN-154
2. SparkR with our zeppelin-R interpreter.
On 2015-07-04 13:54, felixcheung_m@hotmail.com wrote:
>
> Very nice Eric. Interesting approach to use Rserve and knitr, which
> solve the ploting problem (Interesting projects at datalayer too)
>
> I've been working on the SparkR interpreter. There the challenges are
> getting it to work with Spark interpreter within Zeppelin. Would be
> interested in your thoughts.
>
>> From: Eric Charles
>> Sent: Saturday, July 4, 1:11 AM
>> Subject: Re: SparkR Interpreter
>> To: users@zeppelin.incubator.apache.org
>>
>> https://github.com/datalayer/zeppelin-R uses Rserve (GPL-2 License).
>>
>> Rserve for variable sharing across paragraph and knitr for html
>> rendering (with image plot) was our best bet. SparkR should be ok with
>> this (will check).
>>
>> Are you investing other alternatives to implement SparkR?
>>
>> Side note: I have opened ZEPPELIN-154 and will propose a PR to allow 3rd
>> party interpreters to be used in Zeppelin without the need to have them
>> in the Apache source tree.
>>
>> On 2015-07-04 00:10, moon soo Lee wrote:
>> > Hi,
>> >
>> > Yes, there is SparkR in roadmap. and work in progress.
>> > (
>> https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap)
>> >
>> > Thanks,
>> > moon
>> >
>> > On Fri, Jul 3, 2015 at 6:35 AM Dave O'Flanagan
>> > <david.oflanagan@boxever.com < mailto:david.oflanagan@boxever.com>>
>> wrote:
>> >
>> > Hi,
>> >
>> > Is there a SparkR interpreter in Zeppelin in the roadmap? It would
>> > be great to have similar capability to the REPL provided by
>> > sparkR.sh from the Spark distro.
>> >
>> > Thanks,
>> > Dave.
>> >
>> > ----
>> > Dave O'Flanagan
>> > CEO, Boxever
>> > e: dave@boxever.com < mailto:dave@boxever.com>
>> > t: +353 86 8518476
>> > w: www.boxever.com <http://www.boxever.com> <
>> http://www.boxever.com/>
>> >
>>
Re: SparkR Interpreter
Posted by fe...@hotmail.com.
Very nice Eric. Interesting approach to use Rserve and knitr, which solve the ploting problem (Interesting projects at datalayer too)
I've been working on the SparkR interpreter. There the challenges are getting it to work with Spark interpreter within Zeppelin. Would be interested in your thoughts.
From: Eric Charles
Sent: Saturday, July 4, 1:11 AM
Subject: Re: SparkR Interpreter
To: users@zeppelin.incubator.apache.org
https://github.com/datalayer/zeppelin-R uses Rserve (GPL-2 License).
Rserve for variable sharing across paragraph and knitr for html
rendering (with image plot) was our best bet. SparkR should be ok with
this (will check).
Are you investing other alternatives to implement SparkR?
Side note: I have opened ZEPPELIN-154 and will propose a PR to allow 3rd
party interpreters to be used in Zeppelin without the need to have them
in the Apache source tree.
On 2015-07-04 00:10, moon soo Lee wrote:
> Hi,
>
> Yes, there is SparkR in roadmap. and work in progress.
> ( https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap)
>
> Thanks,
> moon
>
> On Fri, Jul 3, 2015 at 6:35 AM Dave O'Flanagan
> <david.oflanagan@boxever.com < mailto:david.oflanagan@boxever.com>> wrote:
>
> Hi,
>
> Is there a SparkR interpreter in Zeppelin in the roadmap? It would
> be great to have similar capability to the REPL provided by
> sparkR.sh from the Spark distro.
>
> Thanks,
> Dave.
>
> ----
> Dave O'Flanagan
> CEO, Boxever
> e: dave@boxever.com < mailto:dave@boxever.com>
> t: +353 86 8518476
> w: www.boxever.com < http://www.boxever.com/>
>
Re: SparkR Interpreter
Posted by Eric Charles <er...@apache.org>.
https://github.com/datalayer/zeppelin-R uses Rserve (GPL-2 License).
Rserve for variable sharing across paragraph and knitr for html
rendering (with image plot) was our best bet. SparkR should be ok with
this (will check).
Are you investing other alternatives to implement SparkR?
Side note: I have opened ZEPPELIN-154 and will propose a PR to allow 3rd
party interpreters to be used in Zeppelin without the need to have them
in the Apache source tree.
On 2015-07-04 00:10, moon soo Lee wrote:
> Hi,
>
> Yes, there is SparkR in roadmap. and work in progress.
> (https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap)
>
> Thanks,
> moon
>
> On Fri, Jul 3, 2015 at 6:35 AM Dave O'Flanagan
> <david.oflanagan@boxever.com <ma...@boxever.com>> wrote:
>
> Hi,
>
> Is there a SparkR interpreter in Zeppelin in the roadmap? It would
> be great to have similar capability to the REPL provided by
> sparkR.sh from the Spark distro.
>
> Thanks,
> Dave.
>
> ----
> Dave O'Flanagan
> CEO, Boxever
> e: dave@boxever.com <ma...@boxever.com>
> t: +353 86 8518476
> w: www.boxever.com <http://www.boxever.com/>
>
Re: SparkR Interpreter
Posted by moon soo Lee <mo...@apache.org>.
Hi,
Yes, there is SparkR in roadmap. and work in progress.
(https://cwiki.apache.org/confluence/display/ZEPPELIN/Zeppelin+Roadmap)
Thanks,
moon
On Fri, Jul 3, 2015 at 6:35 AM Dave O'Flanagan <da...@boxever.com>
wrote:
> Hi,
>
> Is there a SparkR interpreter in Zeppelin in the roadmap? It would be
> great to have similar capability to the REPL provided by sparkR.sh from the
> Spark distro.
>
> Thanks,
> Dave.
>
> ----
> Dave O'Flanagan
> CEO, Boxever
> e: dave@boxever.com
> t: +353 86 8518476
> w: www.boxever.com
>