ItemSimilarityDriver java example - NoSuchMethodError/SparkConf

[Jeff Isenhart <je...@yahoo.com.INVALID>]

I am trying to run the ItemSimilarityDriver in java (7) and getting a NoSuchMethod ItemSimilarityDriver$.sparkConf()Lorg/apache/spark/SparkConf;
I must not have the correct version of spark. Which one works with mahout 1.0-SNAPSHOT?
Code:
   ItemSimilarityDriver.main(new String[]{"-i","/transactions.csv",     "-o", "/out",     "-fc",     "1",     "-ic",      "2"});
Relevant pom entries:
  <dependency>   <groupId>org.apache.mahout</groupId>   <artifactId>mahout-core</artifactId>   <version>1.0-SNAPSHOT</version>  </dependency>
  <dependency>   <groupId>org.apache.mahout</groupId>   <artifactId>mahout-math-scala_2.10</artifactId>   <version>1.0-SNAPSHOT</version>  </dependency>
  <dependency>   <groupId>org.apache.mahout</groupId>   <artifactId>mahout-math</artifactId>   <version>1.0-SNAPSHOT</version>  </dependency>  
  <dependency>   <groupId>org.apache.mahout</groupId>   <artifactId>mahout-spark</artifactId>   <version>1.0-SNAPSHOT</version>  </dependency>
    <dependency><groupId>org.apache.spark</groupId><artifactId>spark-core_2.10</artifactId><version>1.2.0</version>  </dependency>

Re: ItemSimilarityDriver java example - NoSuchMethodError/SparkConf

[Pat Ferrel <pa...@occamsmachete.com>]

Look in the parent pom.xml in your mahout directory. 

It should be Spark 1.1.0. but today we’ll update to 1.1.1 We believe but can’t guarantee that using older versions is possible.

This doesn’t look like a bad Spark version error. How are you calling ItemSimilarityDriver.main ? This looks  like something from the test suite where a test-only method for creating the Spark conf and context is used. Make sure you are not calling ItemSimilarityDriver.useContext before calling main.

It should be fine to use as below but you can also use the CLI with:

mahout spark-itemsimilarity -i /transactions.csv -o, /out -fc 1 -ic 2

This will use a master = “local” and the native filesystem's /transactions.csv for input. That doesn’t look correct.


On Mar 3, 2015, at 7:57 PM, Jeff Isenhart <je...@yahoo.com.INVALID> wrote:

I am trying to run the ItemSimilarityDriver in java (7) and getting a NoSuchMethod ItemSimilarityDriver$.sparkConf()Lorg/apache/spark/SparkConf;
I must not have the correct version of spark. Which one works with mahout 1.0-SNAPSHOT?
Code:
  ItemSimilarityDriver.main(new String[]{"-i","/transactions.csv",     "-o", "/out",     "-fc",     "1",     "-ic",      "2"});
Relevant pom entries:
 <dependency>   <groupId>org.apache.mahout</groupId>   <artifactId>mahout-core</artifactId>   <version>1.0-SNAPSHOT</version>  </dependency>
 <dependency>   <groupId>org.apache.mahout</groupId>   <artifactId>mahout-math-scala_2.10</artifactId>   <version>1.0-SNAPSHOT</version>  </dependency>
 <dependency>   <groupId>org.apache.mahout</groupId>   <artifactId>mahout-math</artifactId>   <version>1.0-SNAPSHOT</version>  </dependency>  
 <dependency>   <groupId>org.apache.mahout</groupId>   <artifactId>mahout-spark</artifactId>   <version>1.0-SNAPSHOT</version>  </dependency>
   <dependency><groupId>org.apache.spark</groupId><artifactId>spark-core_2.10</artifactId><version>1.2.0</version>  </dependency>