You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by Trevor Grant <tr...@gmail.com> on 2017/06/18 14:04:54 UTC
[DISCUSS] remove getMahoutHome from sparkbindings
getMahoutHome()
https://github.com/apache/mahout/blob/08e02602e947ff945b9bd73ab5f0b45863df3e53/spark/src/main/scala/org/apache/mahout/sparkbindings/package.scala#L208
seems to not be used anywhere other than the Flink bindings.
I personally don't like the idea of REQUIRING mahout to be fully installed-
and at the moment, at least for Spark- we don't need to. One can simply use
maven coordinates in POM or add them when launching spark
job/shell/zeppelin.
I'm assuming this method existed to support the old spark-shell, but I'd
like to drop it to prevent anyone from introducing new code that relies on
it.
However since I didn't put it there and don't know all use cases, I want to
open up for discussion before I start a JIRA ticket.
thoughts?
tg
Re: [DISCUSS] remove getMahoutHome from sparkbindings
Posted by Dmitriy Lyubimov <dl...@gmail.com>.
i think the idea was that mahout has control over application jars and
guarantees that the minimum amount of mahout jars is added to the
application. In order to do that, it needs to be able to figure the home
(even if it is an uber jar, as some devs have been pushing for for some
time now, so it might actually happen)
On Sun, Jun 18, 2017 at 7:04 AM, Trevor Grant <tr...@gmail.com>
wrote:
> getMahoutHome()
>
> https://github.com/apache/mahout/blob/08e02602e947ff945b9bd73ab5f0b4
> 5863df3e53/spark/src/main/scala/org/apache/mahout/
> sparkbindings/package.scala#L208
>
> seems to not be used anywhere other than the Flink bindings.
>
> I personally don't like the idea of REQUIRING mahout to be fully installed-
> and at the moment, at least for Spark- we don't need to. One can simply use
> maven coordinates in POM or add them when launching spark
> job/shell/zeppelin.
>
> I'm assuming this method existed to support the old spark-shell, but I'd
> like to drop it to prevent anyone from introducing new code that relies on
> it.
>
> However since I didn't put it there and don't know all use cases, I want to
> open up for discussion before I start a JIRA ticket.
>
> thoughts?
>
> tg
>