Integrate spark-shell into officially supported web ui/api plug-in? What do you think?

[Peng Cheng <pc...@uow.edu.au>]

This will be handy for demo and quick prototyping as the command-line REPL
doesn't support a lot of editor features, also, you don't need to ssh into
your worker/master if your client is behind an NAT wall. Since Spark
codebase has a minimalistic design philosophy I don't think this component
can make into the main repository. However it can be an independent project
that is also supported by the community (like Solr/ElasticSearch to Lucene)

I've reviewed and tested a few REPL web ui including:
- Scala-notebook: https://github.com/Bridgewater/scala-notebook
- Tinsmiths: https://github.com/kouphax/tinsmith
- IScala: https://github.com/mattpap/IScala
- Codebrew: https://codebrew.io/

however they are either too heavyweight, or their ILoop is buried very deep
(sometimes even in another library). I'm interested in working on this part,
has anyone experimented on similar solution before?

Yours Peng



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Integrate-spark-shell-into-officially-supported-web-ui-api-plug-in-What-do-you-think-tp8447.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Integrate spark-shell into officially supported web ui/api plug-in? What do you think?

[Peng Cheng <pc...@uow.edu.au>]

That would be really cool with IPython, But I' still wondering if all
language features are supported, namely I need these 2 in particular:
1. importing class and ILoop from external jars (so I can point it to
SparkILoop or Sparkbinding ILoop of Apache Mahout instead of Scala's default
ILoop)
2. implicit typecast/wrapper and implicit variable (widely used in
SparkContext.scala)
I'll be able to start experimentation immediately if someone can confirm
these features.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Integrate-spark-shell-into-officially-supported-web-ui-api-plug-in-What-do-you-think-tp8447p8469.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Re: Integrate spark-shell into officially supported web ui/api plug-in? What do you think?

[Chester Chen <ch...@alpinenow.com>]

I am more interested to use Scala.js as the front-end and just use Spark as
the back-end.

I thought that would be more than simply a spark-shell on browser. Similar
to ipython that would allows user to interact with other plotting libraries
in the browser such as D3 etc.

Chester


On Fri, Jun 27, 2014 at 1:24 PM, Peng Cheng <pc...@uow.edu.au> wrote:

> This will be handy for demo and quick prototyping as the command-line REPL
> doesn't support a lot of editor features, also, you don't need to ssh into
> your worker/master if your client is behind an NAT wall. Since Spark
> codebase has a minimalistic design philosophy I don't think this component
> can make into the main repository. However it can be an independent project
> that is also supported by the community (like Solr/ElasticSearch to Lucene)
>
> I've reviewed and tested a few REPL web ui including:
> - Scala-notebook: https://github.com/Bridgewater/scala-notebook
> - Tinsmiths: https://github.com/kouphax/tinsmith
> - IScala: https://github.com/mattpap/IScala
> - Codebrew: https://codebrew.io/
>
> however they are either too heavyweight, or their ILoop is buried very deep
> (sometimes even in another library). I'm interested in working on this
> part,
> has anyone experimented on similar solution before?
>
> Yours Peng
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Integrate-spark-shell-into-officially-supported-web-ui-api-plug-in-What-do-you-think-tp8447.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>