You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Mark Miller <ma...@gmail.com> on 2009/08/26 04:11:48 UTC
Lucene website - benchmarks page
This are very old and not very useful anymore. Should we pull this page?
Its kind of an embarrassment if we don't actually maintain it to be
remotely current. These all with Lucene 1.2, 1.3 ...
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene website - benchmarks page
Posted by Michael McCandless <lu...@mikemccandless.com>.
+1
Mike
On Wed, Aug 26, 2009 at 8:20 AM, Grant Ingersoll<gs...@apache.org> wrote:
> +1
>
> On Aug 25, 2009, at 10:11 PM, Mark Miller wrote:
>
>> This are very old and not very useful anymore. Should we pull this page?
>> Its kind of an embarrassment if we don't actually maintain it to be
>> remotely current. These all with Lucene 1.2, 1.3 ...
>>
>> --
>> - Mark
>>
>> http://www.lucidimagination.com
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene website - benchmarks page
Posted by Grant Ingersoll <gs...@apache.org>.
+1
On Aug 25, 2009, at 10:11 PM, Mark Miller wrote:
> This are very old and not very useful anymore. Should we pull this
> page?
> Its kind of an embarrassment if we don't actually maintain it to be
> remotely current. These all with Lucene 1.2, 1.3 ...
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene website - benchmarks page
Posted by Mark Miller <ma...@gmail.com>.
Chris Hostetter wrote:
> : Prob want to run it on decent hardware as well (eg mabye I shouldn't do
> : it with my 5200 rpm laptop drives).
>
> as long as both are run on the same hardware, and the page lists the
> hardware, it's the relative numbers that matter the most.
>
>
>
> -Hoss
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
Agreed to a degree - people like and read off raw numbers though. I like
to show Lucene in the best light possible (or least avg light - trust me
- these drives are so poky - basically I was just trying to get someone
else with normal hardware to do it).
And in the guessing my butt off space - perhaps 2.9 takes advantage of
faster hardware better than 2.4 did.
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene website - benchmarks page
Posted by Mark Miller <ma...@gmail.com>.
Noone had touched that page for many years :) Thats why I removed it.
If we add it back though, thats a great thought - we should mention it
and how to do a standard run.
Though if we use it to compare 2.4 to 2.9, the mentioning part goes
without saying.
- Mark
Matthew Hall wrote:
> That's great... perhaps it should be listed on the Benchmark page
> though ^^
>
> Matt
>
> Mark Miller wrote:
>> We have it - its the benchmark contrib - just run the mirco or standard
>> benchmark alg.
>>
>> Matthew Hall wrote:
>>
>>> Sorry to interject, but I think that the most useful thing that could
>>> be done to this page would be the creation of a standard benchmark
>>> test suite that someone could simply download and run on own systems.
>>>
>>> That way the benchmarking numbers for all users could be easily
>>> compared between boxes etc... etc.
>>>
>>> Perhaps a simple modification of the lucene demo that includes some
>>> sort of .. largish data source to index, and then a configured number
>>> of searches run against that corpus with known parameters? (yes you
>>> could in fact go alot further than this, but I figure its a start)
>>>
>>> All users would have to do is download the somewhat large tar, and
>>> then run some sort of test harness script that was included, and then
>>> post their results.
>>>
>>> I'm sure its not feasible to do something like this with the current
>>> release schedule, but I think it would be nice to have.
>>>
>>> Matt
>>>
>>> Chris Hostetter wrote:
>>>
>>>> : Prob want to run it on decent hardware as well (eg mabye I
>>>> shouldn't do
>>>> : it with my 5200 rpm laptop drives).
>>>>
>>>> as long as both are run on the same hardware, and the page lists the
>>>> hardware, it's the relative numbers that matter the most.
>>>>
>>>>
>>>>
>>>> -Hoss
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>>
>>>>
>>>
>>
>>
>>
>
>
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene website - benchmarks page
Posted by Matthew Hall <mh...@informatics.jax.org>.
That's great... perhaps it should be listed on the Benchmark page though ^^
Matt
Mark Miller wrote:
> We have it - its the benchmark contrib - just run the mirco or standard
> benchmark alg.
>
> Matthew Hall wrote:
>
>> Sorry to interject, but I think that the most useful thing that could
>> be done to this page would be the creation of a standard benchmark
>> test suite that someone could simply download and run on own systems.
>>
>> That way the benchmarking numbers for all users could be easily
>> compared between boxes etc... etc.
>>
>> Perhaps a simple modification of the lucene demo that includes some
>> sort of .. largish data source to index, and then a configured number
>> of searches run against that corpus with known parameters? (yes you
>> could in fact go alot further than this, but I figure its a start)
>>
>> All users would have to do is download the somewhat large tar, and
>> then run some sort of test harness script that was included, and then
>> post their results.
>>
>> I'm sure its not feasible to do something like this with the current
>> release schedule, but I think it would be nice to have.
>>
>> Matt
>>
>> Chris Hostetter wrote:
>>
>>> : Prob want to run it on decent hardware as well (eg mabye I
>>> shouldn't do
>>> : it with my 5200 rpm laptop drives).
>>>
>>> as long as both are run on the same hardware, and the page lists the
>>> hardware, it's the relative numbers that matter the most.
>>>
>>>
>>>
>>> -Hoss
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>>
>>>
>>>
>>
>
>
>
--
Matthew Hall
Software Engineer
Mouse Genome Informatics
mhall@informatics.jax.org
(207) 288-6012
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene website - benchmarks page
Posted by Mark Miller <ma...@gmail.com>.
We have it - its the benchmark contrib - just run the mirco or standard
benchmark alg.
Matthew Hall wrote:
> Sorry to interject, but I think that the most useful thing that could
> be done to this page would be the creation of a standard benchmark
> test suite that someone could simply download and run on own systems.
>
> That way the benchmarking numbers for all users could be easily
> compared between boxes etc... etc.
>
> Perhaps a simple modification of the lucene demo that includes some
> sort of .. largish data source to index, and then a configured number
> of searches run against that corpus with known parameters? (yes you
> could in fact go alot further than this, but I figure its a start)
>
> All users would have to do is download the somewhat large tar, and
> then run some sort of test harness script that was included, and then
> post their results.
>
> I'm sure its not feasible to do something like this with the current
> release schedule, but I think it would be nice to have.
>
> Matt
>
> Chris Hostetter wrote:
>> : Prob want to run it on decent hardware as well (eg mabye I
>> shouldn't do
>> : it with my 5200 rpm laptop drives).
>>
>> as long as both are run on the same hardware, and the page lists the
>> hardware, it's the relative numbers that matter the most.
>>
>>
>>
>> -Hoss
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-dev-help@lucene.apache.org
>>
>>
>
>
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene website - benchmarks page
Posted by Matthew Hall <mh...@informatics.jax.org>.
Sorry to interject, but I think that the most useful thing that could be
done to this page would be the creation of a standard benchmark test
suite that someone could simply download and run on own systems.
That way the benchmarking numbers for all users could be easily compared
between boxes etc... etc.
Perhaps a simple modification of the lucene demo that includes some sort
of .. largish data source to index, and then a configured number of
searches run against that corpus with known parameters? (yes you could
in fact go alot further than this, but I figure its a start)
All users would have to do is download the somewhat large tar, and then
run some sort of test harness script that was included, and then post
their results.
I'm sure its not feasible to do something like this with the current
release schedule, but I think it would be nice to have.
Matt
Chris Hostetter wrote:
> : Prob want to run it on decent hardware as well (eg mabye I shouldn't do
> : it with my 5200 rpm laptop drives).
>
> as long as both are run on the same hardware, and the page lists the
> hardware, it's the relative numbers that matter the most.
>
>
>
> -Hoss
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
--
Matthew Hall
Software Engineer
Mouse Genome Informatics
mhall@informatics.jax.org
(207) 288-6012
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene website - benchmarks page
Posted by Chris Hostetter <ho...@fucit.org>.
: Prob want to run it on decent hardware as well (eg mabye I shouldn't do
: it with my 5200 rpm laptop drives).
as long as both are run on the same hardware, and the page lists the
hardware, it's the relative numbers that matter the most.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene website - benchmarks page
Posted by Mark Miller <ma...@gmail.com>.
Sure - not a bad idea - need to remember to set autocommit to false on
2.4 though.
Not sure if there are any other fair settings -
Prob want to run it on decent hardware as well (eg mabye I shouldn't do
it with my 5200 rpm laptop drives).
note: I also think the wiki already has benchmark stuff too - and its
just as outdated :)
- Mark
Chris Hostetter wrote:
> pulling a crap doc fro mteh release seems sound to me.
>
> alternately: couldn't we just replace it with the output from the
> contrib/benchmarker on some of the bigger tests (the full wikipedia ones)
> comparing 2.4 with 2.9 ?
>
> then just make it a pre-release TODO item for the future: update that page
> to reflect the benchmarks of the current release.
>
> : I would suggest we move it to the wiki (I think we can simply remove
> : the 1.2 and 1.3 benchmarks) and try to get a more recent benchmark
> : soon. In other words a benchmark page on the wiki could be maintained
> : by all users and commiters and would encurage people to publish their
> : results as the hurdle is not as high as it is if you wanna get
> : something on the official website.
> : I'm happy to add the page and encurage people on the user list to add
> : their benchmarks and performance experiences.
>
>
>
> -Hoss
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene website - benchmarks page
Posted by Chris Hostetter <ho...@fucit.org>.
pulling a crap doc fro mteh release seems sound to me.
alternately: couldn't we just replace it with the output from the
contrib/benchmarker on some of the bigger tests (the full wikipedia ones)
comparing 2.4 with 2.9 ?
then just make it a pre-release TODO item for the future: update that page
to reflect the benchmarks of the current release.
: I would suggest we move it to the wiki (I think we can simply remove
: the 1.2 and 1.3 benchmarks) and try to get a more recent benchmark
: soon. In other words a benchmark page on the wiki could be maintained
: by all users and commiters and would encurage people to publish their
: results as the hurdle is not as high as it is if you wanna get
: something on the official website.
: I'm happy to add the page and encurage people on the user list to add
: their benchmarks and performance experiences.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene website - benchmarks page
Posted by Simon Willnauer <si...@googlemail.com>.
I would suggest we move it to the wiki (I think we can simply remove
the 1.2 and 1.3 benchmarks) and try to get a more recent benchmark
soon. In other words a benchmark page on the wiki could be maintained
by all users and commiters and would encurage people to publish their
results as the hurdle is not as high as it is if you wanna get
something on the official website.
I'm happy to add the page and encurage people on the user list to add
their benchmarks and performance experiences.
simon
On Wed, Aug 26, 2009 at 4:18 AM, Mark Miller<ma...@gmail.com> wrote:
> Mark Miller wrote:
>> This are very old and not very useful anymore. Should we pull this page?
>> Its kind of an embarrassment if we don't actually maintain it to be
>> remotely current. These all with Lucene 1.2, 1.3 ...
>>
>>
> Please excuse the small child like grammar :)
>
> --
> - Mark
>
> http://www.lucidimagination.com
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Lucene website - benchmarks page
Posted by Mark Miller <ma...@gmail.com>.
Mark Miller wrote:
> This are very old and not very useful anymore. Should we pull this page?
> Its kind of an embarrassment if we don't actually maintain it to be
> remotely current. These all with Lucene 1.2, 1.3 ...
>
>
Please excuse the small child like grammar :)
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org