Setting Similarity classes in Benchmark .alg scripts

[Michael O'Leary <mi...@seomoz.org>]

I'd like to compare the relevance scores that are returned when using the
Similarity classes that are available in Lucene 4.x, and it seems like
using the Benchmark component would be a good way to do that. It looks like
the isn't currently a way to specify a Similarity class to use in the
config settings used in .alg files. I can see how the analyzer that is
specified in an .alg file is created in NewAnalyzerTask.java, and then used
in CreateIndexTask.java and ReadTokenTask. It seems like the Similarity
classes could be supported in a similar way. Are there other considerations
that also need to be taken account of? NewAnalyzerTask apparently only uses
Analyzers' default constructors (with a Version arg or with no args), and
some of the Similarity classes don't have default constructors, so it would
also be necessary to define ways to specify the arguments that the
Similarity class constructors take. Are there any classes that are
instantiated in the Benchmark component using non-default constructors that
I could look at as examples? On a scale of 1 to 10, how hard it is to add
features of this kind to the Benchmark component?
Thanks,
Mike

Re: Setting Similarity classes in Benchmark .alg scripts

[Robert Muir <rc...@gmail.com>]

Just to be sure what you are trying to do:

A) compare the relevance of different similarities? this is something
the benchmark.quality package (actually pretty much unrelated from the
rest of the benchmark package!) does, if you have some e.g. TREC
collection or whatever to test with.

B) compare the performance (speed) ?

In either case... patches welcome!

On Mon, Feb 4, 2013 at 6:01 PM, Michael O'Leary <mi...@seomoz.org> wrote:
> I'd like to compare the relevance scores that are returned when using the
> Similarity classes that are available in Lucene 4.x, and it seems like
> using the Benchmark component would be a good way to do that. It looks like
> the isn't currently a way to specify a Similarity class to use in the
> config settings used in .alg files. I can see how the analyzer that is
> specified in an .alg file is created in NewAnalyzerTask.java, and then used
> in CreateIndexTask.java and ReadTokenTask. It seems like the Similarity
> classes could be supported in a similar way. Are there other considerations
> that also need to be taken account of? NewAnalyzerTask apparently only uses
> Analyzers' default constructors (with a Version arg or with no args), and
> some of the Similarity classes don't have default constructors, so it would
> also be necessary to define ways to specify the arguments that the
> Similarity class constructors take. Are there any classes that are
> instantiated in the Benchmark component using non-default constructors that
> I could look at as examples? On a scale of 1 to 10, how hard it is to add
> features of this kind to the Benchmark component?
> Thanks,
> Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org