[jira] [Commented] (MAHOUT-1685) Move Mahout shell to Spark 1.3+

["Andrew Palumbo (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAHOUT-1685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14612758#comment-14612758 ] 

Andrew Palumbo commented on MAHOUT-1685:
----------------------------------------

The Developer API supports most everything we need with the exception of the ability to override :

{code}
   private def sparkCleanUp(){
    echo("Stopping spark context.")
    intp.beQuietDuring {
      command("sc.stop()")
    }
  }
{code} 

which is a private method called when shutting down the shell.

we can easily fix this by naming the {{MahoutDistributedContext}} to {{sc}} (from {{sdc}})

Other than that it is pretty straightforward to upgrade.  Not sure about backwards compatibility.

> Move Mahout shell to Spark 1.3+
> -------------------------------
>
>                 Key: MAHOUT-1685
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1685
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Mahout spark shell
>            Reporter: Pat Ferrel
>            Assignee: Dmitriy Lyubimov
>            Priority: Critical
>             Fix For: 0.11.0
>
>         Attachments: mahout-shell-spark-1.3-errors.txt
>
>
> Building for Spark 1.3 we found several important APIS used by the shell are now marked package private in Spark, making them inaccessible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)