You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by Alejandro Bellogin Kouki <al...@uam.es> on 2011/12/15 09:30:22 UTC
Comparison of execution times for recommenders
Hi all,
some months ago I performed some efficiency comparisons between the
execution times of one implementation of mine and user- and item-based
CF recommenders in Mahout. By that time, I was using Mahout-0.3 and I
obtained some decent values, taking into account that I was measuring
the average recommendation time per user (that is, how long it took to
execute the method 'recommend(u, N)').
Yesterday, I decided to make the same tests with the latest stable
version (Mahout-0.5), and, to my suprise, the execution times in the
latest implementation have been multiplied by a factor of 2.
Do you have any idea of why is that happening? Is it documented
anywhere? I have specific numbers and code in case you want to check
them out.
I want to emphasize that I am measuring how much time the recommender
spends on generating a ranking, and, probably, if you have changed how
the plausible items to be recommended for a user are obtained, then this
change may have caused this situation.
Thank you in advance.
Best regards,
Alejandro
--
Alejandro Bellogin Kouki
http://rincon.uam.es/dir?cw=435275268554687
Re: Comparison of execution times for recommenders
Posted by Alejandro Bellogin Kouki <al...@uam.es>.
Thanks, I will try to provide more info in the next days, either using
the snapshot repository or the latest code from SVN.
Regards,
Alejandro
Isabel Drost escribió:
> On 15.12.2011 Sean Owen wrote:
>
>> Also I suggest you use the latest code in Subversion.
>>
>
> Or in the Apache Maven Snapshot repository, if that is easier for you. Any input
> on performance of the current Snapshot is highly welcome - in particluar when
> people are working towards a new release.
>
> Isabel
>
--
Alejandro Bellogin Kouki
http://rincon.uam.es/dir?cw=435275268554687
Re: Comparison of execution times for recommenders
Posted by Isabel Drost <is...@apache.org>.
On 15.12.2011 Sean Owen wrote:
> Also I suggest you use the latest code in Subversion.
Or in the Apache Maven Snapshot repository, if that is easier for you. Any input
on performance of the current Snapshot is highly welcome - in particluar when
people are working towards a new release.
Isabel
Re: Comparison of execution times for recommenders
Posted by Sean Owen <sr...@gmail.com>.
My guess is that some defaults changed somewhere... but I can't think of
anything relevant to these implementations.
Alejandro do you have the ability to run a quick profiling of both, and
point out where the new bottleneck is? then we would have better ideas.
Also I suggest you use the latest code in Subversion.
On Thu, Dec 15, 2011 at 9:58 AM, Alejandro Bellogin Kouki <
alejandro.bellogin@uam.es> wrote:
> Thanks Sebastian.
>
> This is the code I used for Mahout-0.3 and Mahout-0.5:
>
> int N = 1000;
> long totalTime = 0L;
> int n = 0;
> Recommender rec = null;
> final DataModel train = new FileDataModel(new File(trainFile));
> ItemSimilarity sim = new PearsonCorrelationSimilarity(**train);
> sim = new CachingItemSimilarity(sim, train);
> rec = new GenericItemBasedRecommender(**train, sim);
>
> LongPrimitiveIterator users = train.getUserIDs();
> while (users.hasNext()) {
> long u = users.nextLong();
> n++;
> long time = System.currentTimeMillis();
> rec.recommend(u, N);
> time = System.currentTimeMillis() - time;
> totalTime += time;
> }
> System.out.println("Average time - using cache " + strategy + "
> (ms): " + (1.0 * totalTime / n));
>
> In Mahout-0.3, the average time per user was 310.99 ms, and for 0.5 it was
> 649.55 ms.
>
> As I said in my previous mail, here I was only concerned on measuring the
> time performance of recommenders.
>
> Thanks,
> Alejandro
>
> Sebastian Schelter escribió:
>
> Hi Alejandro,
>>
>> you have to provide a detailed description of your benchmark.
>>
>> There is no such thing is "the efficiency" of a recommender. Mahout
>> offers a wide variety of implementations and components that can be
>> glued together to form a recommender. There are a lot of knobs to adjust
>> that offer trade-offs between execution time, scalability, quality and
>> recency.
>>
>> --sebastian
>>
>>
>> On 15.12.2011 09:30, Alejandro Bellogin Kouki wrote:
>>
>>
>>> Hi all,
>>>
>>> some months ago I performed some efficiency comparisons between the
>>> execution times of one implementation of mine and user- and item-based
>>> CF recommenders in Mahout. By that time, I was using Mahout-0.3 and I
>>> obtained some decent values, taking into account that I was measuring
>>> the average recommendation time per user (that is, how long it took to
>>> execute the method 'recommend(u, N)').
>>>
>>> Yesterday, I decided to make the same tests with the latest stable
>>> version (Mahout-0.5), and, to my suprise, the execution times in the
>>> latest implementation have been multiplied by a factor of 2.
>>>
>>> Do you have any idea of why is that happening? Is it documented
>>> anywhere? I have specific numbers and code in case you want to check
>>> them out.
>>>
>>> I want to emphasize that I am measuring how much time the recommender
>>> spends on generating a ranking, and, probably, if you have changed how
>>> the plausible items to be recommended for a user are obtained, then this
>>> change may have caused this situation.
>>>
>>> Thank you in advance.
>>>
>>> Best regards,
>>> Alejandro
>>>
>>>
>>>
>>
>>
>>
>>
>
> --
> Alejandro Bellogin Kouki
> http://rincon.uam.es/dir?cw=**435275268554687<http://rincon.uam.es/dir?cw=435275268554687>
>
>
Re: Comparison of execution times for recommenders
Posted by Alejandro Bellogin Kouki <al...@uam.es>.
Thanks Sebastian.
This is the code I used for Mahout-0.3 and Mahout-0.5:
int N = 1000;
long totalTime = 0L;
int n = 0;
Recommender rec = null;
final DataModel train = new FileDataModel(new File(trainFile));
ItemSimilarity sim = new PearsonCorrelationSimilarity(train);
sim = new CachingItemSimilarity(sim, train);
rec = new GenericItemBasedRecommender(train, sim);
LongPrimitiveIterator users = train.getUserIDs();
while (users.hasNext()) {
long u = users.nextLong();
n++;
long time = System.currentTimeMillis();
rec.recommend(u, N);
time = System.currentTimeMillis() - time;
totalTime += time;
}
System.out.println("Average time - using cache " + strategy + "
(ms): " + (1.0 * totalTime / n));
In Mahout-0.3, the average time per user was 310.99 ms, and for 0.5 it
was 649.55 ms.
As I said in my previous mail, here I was only concerned on measuring
the time performance of recommenders.
Thanks,
Alejandro
Sebastian Schelter escribió:
> Hi Alejandro,
>
> you have to provide a detailed description of your benchmark.
>
> There is no such thing is "the efficiency" of a recommender. Mahout
> offers a wide variety of implementations and components that can be
> glued together to form a recommender. There are a lot of knobs to adjust
> that offer trade-offs between execution time, scalability, quality and
> recency.
>
> --sebastian
>
>
> On 15.12.2011 09:30, Alejandro Bellogin Kouki wrote:
>
>> Hi all,
>>
>> some months ago I performed some efficiency comparisons between the
>> execution times of one implementation of mine and user- and item-based
>> CF recommenders in Mahout. By that time, I was using Mahout-0.3 and I
>> obtained some decent values, taking into account that I was measuring
>> the average recommendation time per user (that is, how long it took to
>> execute the method 'recommend(u, N)').
>>
>> Yesterday, I decided to make the same tests with the latest stable
>> version (Mahout-0.5), and, to my suprise, the execution times in the
>> latest implementation have been multiplied by a factor of 2.
>>
>> Do you have any idea of why is that happening? Is it documented
>> anywhere? I have specific numbers and code in case you want to check
>> them out.
>>
>> I want to emphasize that I am measuring how much time the recommender
>> spends on generating a ranking, and, probably, if you have changed how
>> the plausible items to be recommended for a user are obtained, then this
>> change may have caused this situation.
>>
>> Thank you in advance.
>>
>> Best regards,
>> Alejandro
>>
>>
>
>
>
--
Alejandro Bellogin Kouki
http://rincon.uam.es/dir?cw=435275268554687
Re: Comparison of execution times for recommenders
Posted by Sebastian Schelter <ss...@apache.org>.
Hi Alejandro,
you have to provide a detailed description of your benchmark.
There is no such thing is "the efficiency" of a recommender. Mahout
offers a wide variety of implementations and components that can be
glued together to form a recommender. There are a lot of knobs to adjust
that offer trade-offs between execution time, scalability, quality and
recency.
--sebastian
On 15.12.2011 09:30, Alejandro Bellogin Kouki wrote:
> Hi all,
>
> some months ago I performed some efficiency comparisons between the
> execution times of one implementation of mine and user- and item-based
> CF recommenders in Mahout. By that time, I was using Mahout-0.3 and I
> obtained some decent values, taking into account that I was measuring
> the average recommendation time per user (that is, how long it took to
> execute the method 'recommend(u, N)').
>
> Yesterday, I decided to make the same tests with the latest stable
> version (Mahout-0.5), and, to my suprise, the execution times in the
> latest implementation have been multiplied by a factor of 2.
>
> Do you have any idea of why is that happening? Is it documented
> anywhere? I have specific numbers and code in case you want to check
> them out.
>
> I want to emphasize that I am measuring how much time the recommender
> spends on generating a ranking, and, probably, if you have changed how
> the plausible items to be recommended for a user are obtained, then this
> change may have caused this situation.
>
> Thank you in advance.
>
> Best regards,
> Alejandro
>