You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by James Parker <ev...@gmail.com> on 2015/03/19 15:11:52 UTC
Mahout ALS RecommenderJob error
Hi,
I try ALS recommender using Java code. This is the steps i follow:
1. I create DatasetSplitter Job
2. I create ParallelALSFactorizationJob Job
3. I create FactorizationEvaluator Job
4. I create RecommenderJob Job
Everything is ok for 1 to 3. But when i run the RecommenderJob , i have
the following error
-------------------------------------------------------------------------------------------------------------------------------
java.lang.Exception: java.lang.RuntimeException:
java.lang.ClassCastException:
org.apache.mahout.math.map.OpenIntObjectHashMap cannot be cast to
org.apache.mahout.common.Pair
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:403)
.
.
----------------------------------------------------------------------------------------------------------------------------
I'm suprised to see that if i run steps 1 to 3 using my java code, and the
last step using the command line "mahout recommendfactorized ........"
with the files generates with my java code steps 1 to 3), recommendations
are correctly generated.
Thanks you very much
Evans
Re: Mahout ALS RecommenderJob error
Posted by Pat Ferrel <pa...@occamsmachete.com>.
The best I can say is that if it works as a command line process it will also work called from Java . Double check that what you are calling is the equivalent of the CLI and your arguments are the same. This is quite mature code.
On Mar 19, 2015, at 3:29 PM, James Parker <ev...@gmail.com> wrote:
Pointing recommenderJobInput to ratings.csv generate error: ratings.csv not
a SequenceFile
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1851).......
2015-03-19 21:55 GMT+01:00 Pat Ferrel <pa...@occamsmachete.com>:
> recommenderJobInput should point to the ratings.csv looks like it pointing
> to als output.
>
> On Mar 19, 2015, at 10:15 AM, James Parker <ev...@gmail.com>
> wrote:
>
> The step 3 is just to evaluate my recommender.
>
> I had follow the same steps using CLI and everything works well : (mahout
> splitDataset , mahout parallelALS, mahout recommendfactorized). I think
> that is something wrong with the RecommederJob class. Here is my java
> code:
>
>
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> import org.apache.hadoop.conf.Configuration;
> import org.apache.mahout.cf.taste.hadoop.als.DatasetSplitter;
> import org.apache.mahout.cf.taste.hadoop.als.ParallelALSFactorizationJob;
> import org.apache.mahout.cf.taste.hadoop.als.RecommenderJob;
>
> public class Test {
>
> public static String baseDirectory =
> MainAlsRecommender.class.getResource("/reco").getPath();
> public static String inputSplitDirectory = baseDirectory
> +"/ratings.csv";
> public static String outputSplitDirectory = baseDirectory ;
> public static String inputAlsDirectory = baseDirectory
> +"/trainingSet/";
> public static String outputAlsDirectory = baseDirectory +"/als/out";
> public static String recommenderJobUserFeatures = baseDirectory
> +"/als/out/U/";
> public static String recommenderJobItemFeatures = baseDirectory
> +"/als/out/M/";
> public static String recommenderJobInput = baseDirectory
> +"/als/out/userRatings/";
> public static String recommenderJobOutput = baseDirectory
> +"/recommendations/";
> public static String similarityMatrixDirectory = baseDirectory
> +"/similarityMatrix";
> public static String tmpDirectory = baseDirectory +"/tmp";
> public static String similarityClassname = "SIMILARITY_COSINE";
>
> public static void main(String[] args) throws Exception {
>
>
> Configuration config = new Configuration();
>
>
> DatasetSplitter datasetSplitter = new DatasetSplitter();
> String[] datasetSplitterArgs = {"--input",
> inputSplitDirectory, "--output", outputSplitDirectory,
> "--trainingPercentage", "0.9",
> "--probePercentage","0.1"};
> datasetSplitter.setConf(config);
> datasetSplitter.run(datasetSplitterArgs);
>
> //
>
> ParallelALSFactorizationJob parallelALSFactorizationJob = new
> ParallelALSFactorizationJob();
> String[] parallelAlsFactorisationArgs = {"--input",
> inputAlsDirectory, "--output", outputAlsDirectory, "--numFeatures", "20",
> "--numIterations","10","--lambda","0.065"};
> parallelALSFactorizationJob.setConf(config);
> parallelALSFactorizationJob.run(parallelAlsFactorisationArgs);
>
>
>
> // Mahout’s recommendfactorized function will compute
> recommendations based on ratings matrix
>
> RecommenderJob recommenderJob = new RecommenderJob();
> String[] recommenderJobArgs = {"--input", recommenderJobInput,
> "--output",recommenderJobOutput,"--userFeatures",
> recommenderJobUserFeatures ,
> "--itemFeatures",recommenderJobItemFeatures,
> "--numRecommendations","6","--maxRating","5"};
> recommenderJob.setConf(config);
> recommenderJob.run(recommenderJobArgs);
>
>
>
>
>
> }
> }
>
>
>
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Thanks you very much
>
>
>
> 2015-03-19 15:53 GMT+01:00 Pat Ferrel <pa...@occamsmachete.com>:
>
>> I think that means you are using the wrong input to #4. It looks like you
>> are trying to evaluate the ALS recs then create a recommender? To use ALS
>> for recs try the CLI first see “mahout recommendfactorized” with input =
>> (userid, itemid, rating). If #4 is to create all recs for all known users
>> then it is independent of all other jobs since it takes interactions or
>> ratings as input. not the output f a previous job.
>>
>>
>> On Mar 19, 2015, at 7:11 AM, James Parker <ev...@gmail.com>
> wrote:
>>
>> Hi,
>> I try ALS recommender using Java code. This is the steps i follow:
>> 1. I create DatasetSplitter Job
>>
>> 2. I create ParallelALSFactorizationJob Job
>>
>> 3. I create FactorizationEvaluator Job
>>
>> 4. I create RecommenderJob Job
>>
>> Everything is ok for 1 to 3. But when i run the RecommenderJob , i have
>> the following error
>>
>>
> -------------------------------------------------------------------------------------------------------------------------------
>> java.lang.Exception: java.lang.RuntimeException:
>> java.lang.ClassCastException:
>> org.apache.mahout.math.map.OpenIntObjectHashMap cannot be cast to
>> org.apache.mahout.common.Pair
>> at
>> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:403)
>> .
>> .
>>
>>
> ----------------------------------------------------------------------------------------------------------------------------
>>
>>
>> I'm suprised to see that if i run steps 1 to 3 using my java code, and
> the
>> last step using the command line "mahout recommendfactorized ........"
>> with the files generates with my java code steps 1 to 3),
> recommendations
>> are correctly generated.
>>
>> Thanks you very much
>>
>> Evans
>>
>>
>
>
Re: Mahout ALS RecommenderJob error
Posted by James Parker <ev...@gmail.com>.
Pointing recommenderJobInput to ratings.csv generate error: ratings.csv not
a SequenceFile
at
org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1851).......
2015-03-19 21:55 GMT+01:00 Pat Ferrel <pa...@occamsmachete.com>:
> recommenderJobInput should point to the ratings.csv looks like it pointing
> to als output.
>
> On Mar 19, 2015, at 10:15 AM, James Parker <ev...@gmail.com>
> wrote:
>
> The step 3 is just to evaluate my recommender.
>
> I had follow the same steps using CLI and everything works well : (mahout
> splitDataset , mahout parallelALS, mahout recommendfactorized). I think
> that is something wrong with the RecommederJob class. Here is my java
> code:
>
>
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> import org.apache.hadoop.conf.Configuration;
> import org.apache.mahout.cf.taste.hadoop.als.DatasetSplitter;
> import org.apache.mahout.cf.taste.hadoop.als.ParallelALSFactorizationJob;
> import org.apache.mahout.cf.taste.hadoop.als.RecommenderJob;
>
> public class Test {
>
> public static String baseDirectory =
> MainAlsRecommender.class.getResource("/reco").getPath();
> public static String inputSplitDirectory = baseDirectory
> +"/ratings.csv";
> public static String outputSplitDirectory = baseDirectory ;
> public static String inputAlsDirectory = baseDirectory
> +"/trainingSet/";
> public static String outputAlsDirectory = baseDirectory +"/als/out";
> public static String recommenderJobUserFeatures = baseDirectory
> +"/als/out/U/";
> public static String recommenderJobItemFeatures = baseDirectory
> +"/als/out/M/";
> public static String recommenderJobInput = baseDirectory
> +"/als/out/userRatings/";
> public static String recommenderJobOutput = baseDirectory
> +"/recommendations/";
> public static String similarityMatrixDirectory = baseDirectory
> +"/similarityMatrix";
> public static String tmpDirectory = baseDirectory +"/tmp";
> public static String similarityClassname = "SIMILARITY_COSINE";
>
> public static void main(String[] args) throws Exception {
>
>
> Configuration config = new Configuration();
>
>
> DatasetSplitter datasetSplitter = new DatasetSplitter();
> String[] datasetSplitterArgs = {"--input",
> inputSplitDirectory, "--output", outputSplitDirectory,
> "--trainingPercentage", "0.9",
> "--probePercentage","0.1"};
> datasetSplitter.setConf(config);
> datasetSplitter.run(datasetSplitterArgs);
>
> //
>
> ParallelALSFactorizationJob parallelALSFactorizationJob = new
> ParallelALSFactorizationJob();
> String[] parallelAlsFactorisationArgs = {"--input",
> inputAlsDirectory, "--output", outputAlsDirectory, "--numFeatures", "20",
> "--numIterations","10","--lambda","0.065"};
> parallelALSFactorizationJob.setConf(config);
> parallelALSFactorizationJob.run(parallelAlsFactorisationArgs);
>
>
>
> // Mahout’s recommendfactorized function will compute
> recommendations based on ratings matrix
>
> RecommenderJob recommenderJob = new RecommenderJob();
> String[] recommenderJobArgs = {"--input", recommenderJobInput,
> "--output",recommenderJobOutput,"--userFeatures",
> recommenderJobUserFeatures ,
> "--itemFeatures",recommenderJobItemFeatures,
> "--numRecommendations","6","--maxRating","5"};
> recommenderJob.setConf(config);
> recommenderJob.run(recommenderJobArgs);
>
>
>
>
>
> }
> }
>
>
>
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Thanks you very much
>
>
>
> 2015-03-19 15:53 GMT+01:00 Pat Ferrel <pa...@occamsmachete.com>:
>
> > I think that means you are using the wrong input to #4. It looks like you
> > are trying to evaluate the ALS recs then create a recommender? To use ALS
> > for recs try the CLI first see “mahout recommendfactorized” with input =
> > (userid, itemid, rating). If #4 is to create all recs for all known users
> > then it is independent of all other jobs since it takes interactions or
> > ratings as input. not the output f a previous job.
> >
> >
> > On Mar 19, 2015, at 7:11 AM, James Parker <ev...@gmail.com>
> wrote:
> >
> > Hi,
> > I try ALS recommender using Java code. This is the steps i follow:
> > 1. I create DatasetSplitter Job
> >
> > 2. I create ParallelALSFactorizationJob Job
> >
> > 3. I create FactorizationEvaluator Job
> >
> > 4. I create RecommenderJob Job
> >
> > Everything is ok for 1 to 3. But when i run the RecommenderJob , i have
> > the following error
> >
> >
> -------------------------------------------------------------------------------------------------------------------------------
> > java.lang.Exception: java.lang.RuntimeException:
> > java.lang.ClassCastException:
> > org.apache.mahout.math.map.OpenIntObjectHashMap cannot be cast to
> > org.apache.mahout.common.Pair
> > at
> > org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:403)
> > .
> > .
> >
> >
> ----------------------------------------------------------------------------------------------------------------------------
> >
> >
> > I'm suprised to see that if i run steps 1 to 3 using my java code, and
> the
> > last step using the command line "mahout recommendfactorized ........"
> > with the files generates with my java code steps 1 to 3),
> recommendations
> > are correctly generated.
> >
> > Thanks you very much
> >
> > Evans
> >
> >
>
>
Re: Mahout ALS RecommenderJob error
Posted by Pat Ferrel <pa...@occamsmachete.com>.
recommenderJobInput should point to the ratings.csv looks like it pointing to als output.
On Mar 19, 2015, at 10:15 AM, James Parker <ev...@gmail.com> wrote:
The step 3 is just to evaluate my recommender.
I had follow the same steps using CLI and everything works well : (mahout
splitDataset , mahout parallelALS, mahout recommendfactorized). I think
that is something wrong with the RecommederJob class. Here is my java
code:
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
import org.apache.hadoop.conf.Configuration;
import org.apache.mahout.cf.taste.hadoop.als.DatasetSplitter;
import org.apache.mahout.cf.taste.hadoop.als.ParallelALSFactorizationJob;
import org.apache.mahout.cf.taste.hadoop.als.RecommenderJob;
public class Test {
public static String baseDirectory =
MainAlsRecommender.class.getResource("/reco").getPath();
public static String inputSplitDirectory = baseDirectory
+"/ratings.csv";
public static String outputSplitDirectory = baseDirectory ;
public static String inputAlsDirectory = baseDirectory
+"/trainingSet/";
public static String outputAlsDirectory = baseDirectory +"/als/out";
public static String recommenderJobUserFeatures = baseDirectory
+"/als/out/U/";
public static String recommenderJobItemFeatures = baseDirectory
+"/als/out/M/";
public static String recommenderJobInput = baseDirectory
+"/als/out/userRatings/";
public static String recommenderJobOutput = baseDirectory
+"/recommendations/";
public static String similarityMatrixDirectory = baseDirectory
+"/similarityMatrix";
public static String tmpDirectory = baseDirectory +"/tmp";
public static String similarityClassname = "SIMILARITY_COSINE";
public static void main(String[] args) throws Exception {
Configuration config = new Configuration();
DatasetSplitter datasetSplitter = new DatasetSplitter();
String[] datasetSplitterArgs = {"--input",
inputSplitDirectory, "--output", outputSplitDirectory,
"--trainingPercentage", "0.9",
"--probePercentage","0.1"};
datasetSplitter.setConf(config);
datasetSplitter.run(datasetSplitterArgs);
//
ParallelALSFactorizationJob parallelALSFactorizationJob = new
ParallelALSFactorizationJob();
String[] parallelAlsFactorisationArgs = {"--input",
inputAlsDirectory, "--output", outputAlsDirectory, "--numFeatures", "20",
"--numIterations","10","--lambda","0.065"};
parallelALSFactorizationJob.setConf(config);
parallelALSFactorizationJob.run(parallelAlsFactorisationArgs);
// Mahout’s recommendfactorized function will compute
recommendations based on ratings matrix
RecommenderJob recommenderJob = new RecommenderJob();
String[] recommenderJobArgs = {"--input", recommenderJobInput,
"--output",recommenderJobOutput,"--userFeatures",
recommenderJobUserFeatures ,
"--itemFeatures",recommenderJobItemFeatures,
"--numRecommendations","6","--maxRating","5"};
recommenderJob.setConf(config);
recommenderJob.run(recommenderJobArgs);
}
}
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Thanks you very much
2015-03-19 15:53 GMT+01:00 Pat Ferrel <pa...@occamsmachete.com>:
> I think that means you are using the wrong input to #4. It looks like you
> are trying to evaluate the ALS recs then create a recommender? To use ALS
> for recs try the CLI first see “mahout recommendfactorized” with input =
> (userid, itemid, rating). If #4 is to create all recs for all known users
> then it is independent of all other jobs since it takes interactions or
> ratings as input. not the output f a previous job.
>
>
> On Mar 19, 2015, at 7:11 AM, James Parker <ev...@gmail.com> wrote:
>
> Hi,
> I try ALS recommender using Java code. This is the steps i follow:
> 1. I create DatasetSplitter Job
>
> 2. I create ParallelALSFactorizationJob Job
>
> 3. I create FactorizationEvaluator Job
>
> 4. I create RecommenderJob Job
>
> Everything is ok for 1 to 3. But when i run the RecommenderJob , i have
> the following error
>
> -------------------------------------------------------------------------------------------------------------------------------
> java.lang.Exception: java.lang.RuntimeException:
> java.lang.ClassCastException:
> org.apache.mahout.math.map.OpenIntObjectHashMap cannot be cast to
> org.apache.mahout.common.Pair
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:403)
> .
> .
>
> ----------------------------------------------------------------------------------------------------------------------------
>
>
> I'm suprised to see that if i run steps 1 to 3 using my java code, and the
> last step using the command line "mahout recommendfactorized ........"
> with the files generates with my java code steps 1 to 3), recommendations
> are correctly generated.
>
> Thanks you very much
>
> Evans
>
>
Re: Mahout ALS RecommenderJob error
Posted by James Parker <ev...@gmail.com>.
The step 3 is just to evaluate my recommender.
I had follow the same steps using CLI and everything works well : (mahout
splitDataset , mahout parallelALS, mahout recommendfactorized). I think
that is something wrong with the RecommederJob class. Here is my java
code:
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
import org.apache.hadoop.conf.Configuration;
import org.apache.mahout.cf.taste.hadoop.als.DatasetSplitter;
import org.apache.mahout.cf.taste.hadoop.als.ParallelALSFactorizationJob;
import org.apache.mahout.cf.taste.hadoop.als.RecommenderJob;
public class Test {
public static String baseDirectory =
MainAlsRecommender.class.getResource("/reco").getPath();
public static String inputSplitDirectory = baseDirectory
+"/ratings.csv";
public static String outputSplitDirectory = baseDirectory ;
public static String inputAlsDirectory = baseDirectory
+"/trainingSet/";
public static String outputAlsDirectory = baseDirectory +"/als/out";
public static String recommenderJobUserFeatures = baseDirectory
+"/als/out/U/";
public static String recommenderJobItemFeatures = baseDirectory
+"/als/out/M/";
public static String recommenderJobInput = baseDirectory
+"/als/out/userRatings/";
public static String recommenderJobOutput = baseDirectory
+"/recommendations/";
public static String similarityMatrixDirectory = baseDirectory
+"/similarityMatrix";
public static String tmpDirectory = baseDirectory +"/tmp";
public static String similarityClassname = "SIMILARITY_COSINE";
public static void main(String[] args) throws Exception {
Configuration config = new Configuration();
DatasetSplitter datasetSplitter = new DatasetSplitter();
String[] datasetSplitterArgs = {"--input",
inputSplitDirectory, "--output", outputSplitDirectory,
"--trainingPercentage", "0.9",
"--probePercentage","0.1"};
datasetSplitter.setConf(config);
datasetSplitter.run(datasetSplitterArgs);
//
ParallelALSFactorizationJob parallelALSFactorizationJob = new
ParallelALSFactorizationJob();
String[] parallelAlsFactorisationArgs = {"--input",
inputAlsDirectory, "--output", outputAlsDirectory, "--numFeatures", "20",
"--numIterations","10","--lambda","0.065"};
parallelALSFactorizationJob.setConf(config);
parallelALSFactorizationJob.run(parallelAlsFactorisationArgs);
// Mahout’s recommendfactorized function will compute
recommendations based on ratings matrix
RecommenderJob recommenderJob = new RecommenderJob();
String[] recommenderJobArgs = {"--input", recommenderJobInput,
"--output",recommenderJobOutput,"--userFeatures",
recommenderJobUserFeatures ,
"--itemFeatures",recommenderJobItemFeatures,
"--numRecommendations","6","--maxRating","5"};
recommenderJob.setConf(config);
recommenderJob.run(recommenderJobArgs);
}
}
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Thanks you very much
2015-03-19 15:53 GMT+01:00 Pat Ferrel <pa...@occamsmachete.com>:
> I think that means you are using the wrong input to #4. It looks like you
> are trying to evaluate the ALS recs then create a recommender? To use ALS
> for recs try the CLI first see “mahout recommendfactorized” with input =
> (userid, itemid, rating). If #4 is to create all recs for all known users
> then it is independent of all other jobs since it takes interactions or
> ratings as input. not the output f a previous job.
>
>
> On Mar 19, 2015, at 7:11 AM, James Parker <ev...@gmail.com> wrote:
>
> Hi,
> I try ALS recommender using Java code. This is the steps i follow:
> 1. I create DatasetSplitter Job
>
> 2. I create ParallelALSFactorizationJob Job
>
> 3. I create FactorizationEvaluator Job
>
> 4. I create RecommenderJob Job
>
> Everything is ok for 1 to 3. But when i run the RecommenderJob , i have
> the following error
>
> -------------------------------------------------------------------------------------------------------------------------------
> java.lang.Exception: java.lang.RuntimeException:
> java.lang.ClassCastException:
> org.apache.mahout.math.map.OpenIntObjectHashMap cannot be cast to
> org.apache.mahout.common.Pair
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:403)
> .
> .
>
> ----------------------------------------------------------------------------------------------------------------------------
>
>
> I'm suprised to see that if i run steps 1 to 3 using my java code, and the
> last step using the command line "mahout recommendfactorized ........"
> with the files generates with my java code steps 1 to 3), recommendations
> are correctly generated.
>
> Thanks you very much
>
> Evans
>
>
Re: Mahout ALS RecommenderJob error
Posted by Pat Ferrel <pa...@occamsmachete.com>.
I think that means you are using the wrong input to #4. It looks like you are trying to evaluate the ALS recs then create a recommender? To use ALS for recs try the CLI first see “mahout recommendfactorized” with input = (userid, itemid, rating). If #4 is to create all recs for all known users then it is independent of all other jobs since it takes interactions or ratings as input. not the output f a previous job.
On Mar 19, 2015, at 7:11 AM, James Parker <ev...@gmail.com> wrote:
Hi,
I try ALS recommender using Java code. This is the steps i follow:
1. I create DatasetSplitter Job
2. I create ParallelALSFactorizationJob Job
3. I create FactorizationEvaluator Job
4. I create RecommenderJob Job
Everything is ok for 1 to 3. But when i run the RecommenderJob , i have
the following error
-------------------------------------------------------------------------------------------------------------------------------
java.lang.Exception: java.lang.RuntimeException:
java.lang.ClassCastException:
org.apache.mahout.math.map.OpenIntObjectHashMap cannot be cast to
org.apache.mahout.common.Pair
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:403)
.
.
----------------------------------------------------------------------------------------------------------------------------
I'm suprised to see that if i run steps 1 to 3 using my java code, and the
last step using the command line "mahout recommendfactorized ........"
with the files generates with my java code steps 1 to 3), recommendations
are correctly generated.
Thanks you very much
Evans