You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Hui Wen Han (JIRA)" <ji...@apache.org> on 2010/07/27 18:31:16 UTC
[jira] Created: (MAHOUT-450) RowSimilarityJob error
RowSimilarityJob error
-----------------------
Key: MAHOUT-450
URL: https://issues.apache.org/jira/browse/MAHOUT-450
Project: Mahout
Issue Type: Test
Components: Collaborative Filtering
Affects Versions: 0.4
Reporter: Hui Wen Han
Fix For: 0.4
when ran RowSimilarityJob job as following:
hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
got following error:
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
It seemed that here error:
public static class SimilarityReducer
extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
private DistributedVectorSimilarity similarity;
private int numberOfColumns;
@Override
protected void setup(Context ctx) throws IOException, InterruptedException {
super.setup(ctx);
similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME)); //this line,seems can not load the class.
numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
if (numberOfColumns < 1) {
throw new IllegalStateException("Number of columns was not correctly set!");
}
}
I guess that maybe be wrong here:
in SimilarityType:
SIMILARITY_COOCCURRENCE(DistributedCooccurrenceVectorSimilarity.class), //just the class name is needed,has no the ".class"
SIMILARITY_EUCLIDEAN_DISTANCE(DistributedEuclideanDistanceVectorSimilarity.class),
SIMILARITY_LOGLIKELIHOOD(DistributedLoglikelihoodVectorSimilarity.class),
SIMILARITY_PEARSON_CORRELATION(DistributedPearsonCorrelationVectorSimilarity.class),
SIMILARITY_TANIMOTO_COEFFICIENT(DistributedTanimotoCoefficientVectorSimilarity.class),
SIMILARITY_UNCENTERED_COSINE(DistributedUncenteredCosineVectorSimilarity.class),
SIMILARITY_UNCENTERED_ZERO_ASSUMING_COSINE(DistributedUncenteredZeroAssumingCosineVectorSimilarity.class);
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (MAHOUT-450) RowSimilarityJob error
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893120#action_12893120 ]
Sebastian Schelter edited comment on MAHOUT-450 at 7/28/10 6:22 AM:
--------------------------------------------------------------------
I see that you use a customly built jar file, can you check whether it contains the class that cannot be loaded (org.apache.commons.cli2.OptionException)?
You can unzip jars just like standard zip files. If you do this with core/target/mahout-core-0.4-SNAPSHOT.job you will see that the class we're talking about is contained in lib/commons-cli-2.0-mahout.jar under the path org/apache/commons/cli2/OptionException.class. Please check if you can find it in your jar too.
You could also try to run the RowSimilarityJob with the .job file and see if that works.
was (Author: ssc):
I see that you use a customly built jar file, can you check whether it contains the class that cannot be loaded (org.apache.commons.cli2.OptionException)?
You can unzip jars as just like standard zip files. If you do this with core/target/mahout-core-0.4-SNAPSHOT.job you will see that the class we're talking about is contained in lib/commons-cli-2.0-mahout.jar under the path org/apache/commons/cli2/OptionException.class. Please check if you can find it in your jar too.
You could also try to run the RowSimilarityJob with the .job file and see if that works.
> RowSimilarityJob error
> -----------------------
>
> Key: MAHOUT-450
> URL: https://issues.apache.org/jira/browse/MAHOUT-450
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Fix For: 0.4
>
>
> when ran RowSimilarityJob job as following:
> hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
> extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
> private DistributedVectorSimilarity similarity;
> private int numberOfColumns;
> @Override
> protected void setup(Context ctx) throws IOException, InterruptedException {
> super.setup(ctx);
> similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME)); //this line,seems can not load the class.
> numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
> if (numberOfColumns < 1) {
> throw new IllegalStateException("Number of columns was not correctly set!");
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-450) RowSimilarityJob error
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893116#action_12893116 ]
Hui Wen Han commented on MAHOUT-450:
------------------------------------
public static class SimilarityReducer
extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
private DistributedVectorSimilarity similarity;
private int numberOfColumns;
@Override
protected void setup(Context ctx) throws IOException, InterruptedException {
super.setup(ctx);
String className=ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME); //error here ,but we cah get DISTRIBUTED_SIMILARITY_CLASSNAME in RowWeightMapper
similarity = instantiateSimilarity(className); // I separate this line to two lines
numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
if (numberOfColumns < 1) {
throw new IllegalStateException("Number of columns was not correctly set!");
}
}
> RowSimilarityJob error
> -----------------------
>
> Key: MAHOUT-450
> URL: https://issues.apache.org/jira/browse/MAHOUT-450
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Fix For: 0.4
>
>
> when ran RowSimilarityJob job as following:
> hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
> extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
> private DistributedVectorSimilarity similarity;
> private int numberOfColumns;
> @Override
> protected void setup(Context ctx) throws IOException, InterruptedException {
> super.setup(ctx);
> similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME)); //this line,seems can not load the class.
> numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
> if (numberOfColumns < 1) {
> throw new IllegalStateException("Number of columns was not correctly set!");
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-450) RowSimilarityJob error
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893197#action_12893197 ]
Hui Wen Han commented on MAHOUT-450:
------------------------------------
I ran using mahout-core-0.4-SNAPSHOT.job,it worked correctly.
> RowSimilarityJob error
> -----------------------
>
> Key: MAHOUT-450
> URL: https://issues.apache.org/jira/browse/MAHOUT-450
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Fix For: 0.4
>
>
> when ran RowSimilarityJob job as following:
> hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
> extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
> private DistributedVectorSimilarity similarity;
> private int numberOfColumns;
> @Override
> protected void setup(Context ctx) throws IOException, InterruptedException {
> super.setup(ctx);
> similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME)); //this line,seems can not load the class.
> numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
> if (numberOfColumns < 1) {
> throw new IllegalStateException("Number of columns was not correctly set!");
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-450) RowSimilarityJob error
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892855#action_12892855 ]
Sebastian Schelter commented on MAHOUT-450:
-------------------------------------------
from a first look I'd say that org.apache.commons.cli2.OptionException from commons cli cannot be loaded, are you sure that your jar-file is correctly built?
> RowSimilarityJob error
> -----------------------
>
> Key: MAHOUT-450
> URL: https://issues.apache.org/jira/browse/MAHOUT-450
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Fix For: 0.4
>
>
> when ran RowSimilarityJob job as following:
> hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
> extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
> private DistributedVectorSimilarity similarity;
> private int numberOfColumns;
> @Override
> protected void setup(Context ctx) throws IOException, InterruptedException {
> super.setup(ctx);
> similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME)); //this line,seems can not load the class.
> numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
> if (numberOfColumns < 1) {
> throw new IllegalStateException("Number of columns was not correctly set!");
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-450) RowSimilarityJob error
Posted by "Sebastian Schelter (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893120#action_12893120 ]
Sebastian Schelter commented on MAHOUT-450:
-------------------------------------------
I see that you use a customly built jar file, can you check whether it contains the class that cannot be loaded (org.apache.commons.cli2.OptionException)?
You can unzip jars as just like standard zip files. If you do this with core/target/mahout-core-0.4-SNAPSHOT.job you will see that the class we're talking about is contained in lib/commons-cli-2.0-mahout.jar under the path org/apache/commons/cli2/OptionException.class. Please check if you can find it in your jar too.
You could also try to run the RowSimilarityJob with the .job file and see if that works.
> RowSimilarityJob error
> -----------------------
>
> Key: MAHOUT-450
> URL: https://issues.apache.org/jira/browse/MAHOUT-450
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Fix For: 0.4
>
>
> when ran RowSimilarityJob job as following:
> hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
> extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
> private DistributedVectorSimilarity similarity;
> private int numberOfColumns;
> @Override
> protected void setup(Context ctx) throws IOException, InterruptedException {
> super.setup(ctx);
> similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME)); //this line,seems can not load the class.
> numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
> if (numberOfColumns < 1) {
> throw new IllegalStateException("Number of columns was not correctly set!");
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAHOUT-450) RowSimilarityJob error
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved MAHOUT-450.
------------------------------
Fix Version/s: (was: 0.4)
Resolution: Not A Problem
Yes, you were missing dependencies, which are in the .job file. THis is what must be used.
> RowSimilarityJob error
> -----------------------
>
> Key: MAHOUT-450
> URL: https://issues.apache.org/jira/browse/MAHOUT-450
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
>
> when ran RowSimilarityJob job as following:
> hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
> extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
> private DistributedVectorSimilarity similarity;
> private int numberOfColumns;
> @Override
> protected void setup(Context ctx) throws IOException, InterruptedException {
> super.setup(ctx);
> similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME)); //this line,seems can not load the class.
> numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
> if (numberOfColumns < 1) {
> throw new IllegalStateException("Number of columns was not correctly set!");
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-450) RowSimilarityJob error
Posted by "Hui Wen Han (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hui Wen Han updated MAHOUT-450:
-------------------------------
Description:
when ran RowSimilarityJob job as following:
hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
got following error:
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
It seemed that here error:
public static class SimilarityReducer
extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
private DistributedVectorSimilarity similarity;
private int numberOfColumns;
@Override
protected void setup(Context ctx) throws IOException, InterruptedException {
super.setup(ctx);
similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME)); //this line,seems can not load the class.
numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
if (numberOfColumns < 1) {
throw new IllegalStateException("Number of columns was not correctly set!");
}
}
was:
when ran RowSimilarityJob job as following:
hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
got following error:
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
It seemed that here error:
public static class SimilarityReducer
extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
private DistributedVectorSimilarity similarity;
private int numberOfColumns;
@Override
protected void setup(Context ctx) throws IOException, InterruptedException {
super.setup(ctx);
similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME)); //this line,seems can not load the class.
numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
if (numberOfColumns < 1) {
throw new IllegalStateException("Number of columns was not correctly set!");
}
}
I guess that maybe be wrong here:
in SimilarityType:
SIMILARITY_COOCCURRENCE(DistributedCooccurrenceVectorSimilarity.class), //just the class name is needed,has no the ".class"
SIMILARITY_EUCLIDEAN_DISTANCE(DistributedEuclideanDistanceVectorSimilarity.class),
SIMILARITY_LOGLIKELIHOOD(DistributedLoglikelihoodVectorSimilarity.class),
SIMILARITY_PEARSON_CORRELATION(DistributedPearsonCorrelationVectorSimilarity.class),
SIMILARITY_TANIMOTO_COEFFICIENT(DistributedTanimotoCoefficientVectorSimilarity.class),
SIMILARITY_UNCENTERED_COSINE(DistributedUncenteredCosineVectorSimilarity.class),
SIMILARITY_UNCENTERED_ZERO_ASSUMING_COSINE(DistributedUncenteredZeroAssumingCosineVectorSimilarity.class);
> RowSimilarityJob error
> -----------------------
>
> Key: MAHOUT-450
> URL: https://issues.apache.org/jira/browse/MAHOUT-450
> Project: Mahout
> Issue Type: Test
> Components: Collaborative Filtering
> Affects Versions: 0.4
> Reporter: Hui Wen Han
> Fix For: 0.4
>
>
> when ran RowSimilarityJob job as following:
> hadoop jar ../../singlejar/prefetch-hadoop-1.0-full.jar org.apache.mahout.math.hadoop.similarity.RowSimilarityJob --input /steer/item/tap/temp/maybePruneItemUserMatrixPath --output /steer/item/tap/temp/similarityMatrix -s SIMILARITY_COOCCURRENCE -r 10000 -m 100
> got following error:
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:31 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_1, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> 10/07/27 12:14:44 INFO mapred.JobClient: Task Id : attempt_201007261131_0026_r_000000_2, Status : FAILED
> Error: java.lang.ClassNotFoundException: org.apache.commons.cli2.OptionException
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> at org.apache.mahout.math.hadoop.similarity.RowSimilarityJob$SimilarityReducer.setup(RowSimilarityJob.java:284)
> at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
> at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:566)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> It seemed that here error:
> public static class SimilarityReducer
> extends Reducer<WeightedRowPair,Cooccurrence,SimilarityMatrixEntryKey, MatrixEntryWritable> {
> private DistributedVectorSimilarity similarity;
> private int numberOfColumns;
> @Override
> protected void setup(Context ctx) throws IOException, InterruptedException {
> super.setup(ctx);
> similarity = instantiateSimilarity(ctx.getConfiguration().get(DISTRIBUTED_SIMILARITY_CLASSNAME)); //this line,seems can not load the class.
> numberOfColumns = ctx.getConfiguration().getInt(NUMBER_OF_COLUMNS, -1);
> if (numberOfColumns < 1) {
> throw new IllegalStateException("Number of columns was not correctly set!");
> }
> }
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.