[jira] [Updated] (MAHOUT-1489) Interactive Scala & Spark Bindings Shell & Script processor

["Dmitriy Lyubimov (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MAHOUT-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmitriy Lyubimov updated MAHOUT-1489:
-------------------------------------

    Attachment: MAHOUT-1489.patch

First working patch. Also tracked by "shell" branch in my github/mahout-commits. 

seems to be working with single node cluster in STANDALONE mode. Also tested on-the-fly closures with `mapBlock()`.

to start 

(1) compile Mahout 
(2) install and compile spark 0.9.1 
(3) make sure MAHOUT_HOME, SPARK_HOME pointing to mahout and spark 0.9.1 respectively

Start Spark standalone cluster per instructions in Spark

to start shell, use (for example)

{code}
 MASTER=spark://BigHP:7077 bin/mahout spark-shell
{code}
Outstanding issues: 

(1) log level of course is not adjustable by Spark settings any more, since we use Mahout to start the jvm process. How do we set log4j.properties for Mahout??

(2) After exiting from shell, the terminal driver on Linux ubuntu is screwed for some reason. (use "stty sane" to restore sanity to terminal control driver). Not sure why, this does not happen to either spark or scala shell. 

> Interactive Scala & Spark Bindings Shell & Script processor
> -----------------------------------------------------------
>
>                 Key: MAHOUT-1489
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1489
>             Project: Mahout
>          Issue Type: New Feature
>    Affects Versions: 1.0
>            Reporter: Saikat Kanjilal
>            Assignee: Dmitriy Lyubimov
>             Fix For: 1.0
>
>         Attachments: MAHOUT-1489.patch
>
>
> Build an interactive shell /scripting (just like spark shell). Something very similar in R interactive/script runner mode.



--
This message was sent by Atlassian JIRA
(v6.2#6252)