Problem with mahout and AWS

[Iñigo Llamosas <in...@gmail.com>]

Hi,

I am trying to run a simple recommender on AWS, but I'm getting errors when
reducing. These are the jar-parameters lines:

s3://inigobucket/jars/mahout-core-0.8-SNAPSHOT-job.jar

org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob
-Dmapred.input.dir=s3://inigobucket/data/grouplens10m/ratings.dat
-Dmapred.output.dir=s3://inigobucket/output/
--recommenderClassName
org.apache.mahout.cf.taste.impl.recommender.slopeone.SlopeOneRecommender

Starts OK, but when reducing it gives 2 kind of problems.

-Heap space error. This confuses me because I had that error with a 2
m.small slave cluster but also with a 5 c1.medium slave cluster
-org.apache.hadoop.io.SecureIOUtils$AlreadyExistsException: EEXIST: File
exists error.

Any suggestion?

Many thanks,

Inigo

Re: Problem with mahout and AWS

[Sean Owen <sr...@gmail.com>]

You should give more detail about the errors. You are running out of
memory on the child workers. This is not surprising since the default
memory they allocate is fairly small, and you're running a complete
recommender system inside each mapper. It has not much to do with the
size of the instane you use.

I am not sure what the second thing is, you should give more detail.

On Fri, Jan 18, 2013 at 2:02 PM, Iñigo Llamosas <in...@gmail.com> wrote:
> Hi,
>
> I am trying to run a simple recommender on AWS, but I'm getting errors when
> reducing. These are the jar-parameters lines:
>
> s3://inigobucket/jars/mahout-core-0.8-SNAPSHOT-job.jar
>
> org.apache.mahout.cf.taste.hadoop.pseudo.RecommenderJob
> -Dmapred.input.dir=s3://inigobucket/data/grouplens10m/ratings.dat
> -Dmapred.output.dir=s3://inigobucket/output/
> --recommenderClassName
> org.apache.mahout.cf.taste.impl.recommender.slopeone.SlopeOneRecommender
>
> Starts OK, but when reducing it gives 2 kind of problems.
>
> -Heap space error. This confuses me because I had that error with a 2
> m.small slave cluster but also with a 5 c1.medium slave cluster
> -org.apache.hadoop.io.SecureIOUtils$AlreadyExistsException: EEXIST: File
> exists error.
>
> Any suggestion?
>
> Many thanks,
>
> Inigo