You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by Lance Norskog <go...@gmail.com> on 2011/09/29 08:15:15 UTC
Static state in RecommenderEvaluator?
If you run this version of GroupLensRecommenderEvaluatorRunner, an odd thing
happens. The first evaluation will be number X, the second will be Y, the
third will be Y, ..., the Nth will be Y. The same Y as the second and the
third and the fourth. The RecommenderEvaluator implementation is stochastic,
but this should be a different number each time. Anyone know how this could
happen? (You can load the DataModel outside the loop, the numbers are the
same.)
public static void main(String... args) throws IOException, TasteException,
OptionException {
File ratingsFile = TasteOptionParser.getRatings(args);
long first = System.currentTimeMillis();
long last = first;
for(int i = 0; i < 30; i++) {
DataModel model = ratingsFile == null ? new GroupLensDataModel() : new
GroupLensDataModel(ratingsFile);
RecommenderEvaluator evaluator = new
AverageAbsoluteDifferenceRecommenderEvaluator();
double evaluation = evaluator.evaluate(new
GroupLensRecommenderBuilder(),
null,
model,
0.9,
0.3);
long now = System.currentTimeMillis();
log.info("Result #{}: {}", i, String.valueOf(evaluation));
log.info("\t{} ms", now - last);
last = System.currentTimeMillis();
}
log.info("Total time: {} ms", last - first);
}
--
Lance Norskog
goksron@gmail.com
Re: Static state in RecommenderEvaluator?
Posted by Sean Owen <sr...@gmail.com>.
I am talking about results from one run, from this loop. I saw 6 different
results.
I can't think of any likely reason that would cause it to return the same
results. The only thing I can imagine is that something is resetting the RNG
each time, since our tests are able to do that, but I don't see anything
like that triggere dhere.
On Thu, Sep 29, 2011 at 10:56 AM, Lance Norskog <go...@gmail.com> wrote:
> Yes, separate runs from the command line or Eclipse vary by +/- a bit. It
> is
> only when I run this loop inside one program run that I see the behavior.
>
> On Thu, Sep 29, 2011 at 12:53 AM, Sean Owen <sr...@gmail.com> wrote:
>
> > I don't see this behavior. Each result is slightly different by about
> +/-2%
> > when I run it, as expected.
> >
> > On Thu, Sep 29, 2011 at 7:15 AM, Lance Norskog <go...@gmail.com>
> wrote:
> >
> > > If you run this version of GroupLensRecommenderEvaluatorRunner, an odd
> > > thing
> > > happens. The first evaluation will be number X, the second will be Y,
> the
> > > third will be Y, ..., the Nth will be Y. The same Y as the second and
> the
> > > third and the fourth. The RecommenderEvaluator implementation is
> > > stochastic,
> > > but this should be a different number each time. Anyone know how this
> > could
> > > happen? (You can load the DataModel outside the loop, the numbers are
> the
> > > same.)
> > >
> > > public static void main(String... args) throws IOException,
> > > TasteException,
> > > OptionException {
> > > File ratingsFile = TasteOptionParser.getRatings(args);
> > > long first = System.currentTimeMillis();
> > > long last = first;
> > > for(int i = 0; i < 30; i++) {
> > > DataModel model = ratingsFile == null ? new GroupLensDataModel() :
> > new
> > > GroupLensDataModel(ratingsFile);
> > > RecommenderEvaluator evaluator = new
> > > AverageAbsoluteDifferenceRecommenderEvaluator();
> > > double evaluation = evaluator.evaluate(new
> > > GroupLensRecommenderBuilder(),
> > > null,
> > > model,
> > > 0.9,
> > > 0.3);
> > > long now = System.currentTimeMillis();
> > > log.info("Result #{}: {}", i, String.valueOf(evaluation));
> > > log.info("\t{} ms", now - last);
> > > last = System.currentTimeMillis();
> > > }
> > > log.info("Total time: {} ms", last - first);
> > > }
> > >
> > >
> > > --
> > > Lance Norskog
> > > goksron@gmail.com
> > >
> >
>
>
>
> --
> Lance Norskog
> goksron@gmail.com
>
Re: Static state in RecommenderEvaluator?
Posted by Lance Norskog <go...@gmail.com>.
Yes, separate runs from the command line or Eclipse vary by +/- a bit. It is
only when I run this loop inside one program run that I see the behavior.
On Thu, Sep 29, 2011 at 12:53 AM, Sean Owen <sr...@gmail.com> wrote:
> I don't see this behavior. Each result is slightly different by about +/-2%
> when I run it, as expected.
>
> On Thu, Sep 29, 2011 at 7:15 AM, Lance Norskog <go...@gmail.com> wrote:
>
> > If you run this version of GroupLensRecommenderEvaluatorRunner, an odd
> > thing
> > happens. The first evaluation will be number X, the second will be Y, the
> > third will be Y, ..., the Nth will be Y. The same Y as the second and the
> > third and the fourth. The RecommenderEvaluator implementation is
> > stochastic,
> > but this should be a different number each time. Anyone know how this
> could
> > happen? (You can load the DataModel outside the loop, the numbers are the
> > same.)
> >
> > public static void main(String... args) throws IOException,
> > TasteException,
> > OptionException {
> > File ratingsFile = TasteOptionParser.getRatings(args);
> > long first = System.currentTimeMillis();
> > long last = first;
> > for(int i = 0; i < 30; i++) {
> > DataModel model = ratingsFile == null ? new GroupLensDataModel() :
> new
> > GroupLensDataModel(ratingsFile);
> > RecommenderEvaluator evaluator = new
> > AverageAbsoluteDifferenceRecommenderEvaluator();
> > double evaluation = evaluator.evaluate(new
> > GroupLensRecommenderBuilder(),
> > null,
> > model,
> > 0.9,
> > 0.3);
> > long now = System.currentTimeMillis();
> > log.info("Result #{}: {}", i, String.valueOf(evaluation));
> > log.info("\t{} ms", now - last);
> > last = System.currentTimeMillis();
> > }
> > log.info("Total time: {} ms", last - first);
> > }
> >
> >
> > --
> > Lance Norskog
> > goksron@gmail.com
> >
>
--
Lance Norskog
goksron@gmail.com
Re: Static state in RecommenderEvaluator?
Posted by Sean Owen <sr...@gmail.com>.
I don't see this behavior. Each result is slightly different by about +/-2%
when I run it, as expected.
On Thu, Sep 29, 2011 at 7:15 AM, Lance Norskog <go...@gmail.com> wrote:
> If you run this version of GroupLensRecommenderEvaluatorRunner, an odd
> thing
> happens. The first evaluation will be number X, the second will be Y, the
> third will be Y, ..., the Nth will be Y. The same Y as the second and the
> third and the fourth. The RecommenderEvaluator implementation is
> stochastic,
> but this should be a different number each time. Anyone know how this could
> happen? (You can load the DataModel outside the loop, the numbers are the
> same.)
>
> public static void main(String... args) throws IOException,
> TasteException,
> OptionException {
> File ratingsFile = TasteOptionParser.getRatings(args);
> long first = System.currentTimeMillis();
> long last = first;
> for(int i = 0; i < 30; i++) {
> DataModel model = ratingsFile == null ? new GroupLensDataModel() : new
> GroupLensDataModel(ratingsFile);
> RecommenderEvaluator evaluator = new
> AverageAbsoluteDifferenceRecommenderEvaluator();
> double evaluation = evaluator.evaluate(new
> GroupLensRecommenderBuilder(),
> null,
> model,
> 0.9,
> 0.3);
> long now = System.currentTimeMillis();
> log.info("Result #{}: {}", i, String.valueOf(evaluation));
> log.info("\t{} ms", now - last);
> last = System.currentTimeMillis();
> }
> log.info("Total time: {} ms", last - first);
> }
>
>
> --
> Lance Norskog
> goksron@gmail.com
>