You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Karl Wettin (JIRA)" <ji...@apache.org> on 2007/08/04 16:29:54 UTC
[jira] Updated: (LUCENE-550) InstantiatedIndex - faster but memory
consuming index
[ https://issues.apache.org/jira/browse/LUCENE-550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-550:
-------------------------------
Attachment: LUCENE-550_20070804_no_core_changes.txt
This is a small and completely isolated version of InstantiatedIndex, the results of my "last attempt" thread:
http://www.nabble.com/Last-attempt-tf4153815.html
It requires no changes to the Lucene core but hogs a bit more RAM and probably depends on your JIT to avoid wasting CPU. So prior required definalization and generalization is replaced by aggregation (strategy pattern). I also had to remove all the polymorphic index handling (IndexWriterInterface et c), and I have removed the IndexWriter in InstantiatedIndex. One now have to create a new InstantiatedIndex and pass down an IndexReader instead. So there is no appending allowed. Also, there are no locks no more, but that should not be needed anymore.
The port of the complete test suite from Lucene to the unison index handling has been removed. Ie there are no real test cases that demonstrate this patch. Anything but term vectors and payloads should work great though. The code base is over a year old and these are new features I did not have time to implement or test.
No new benchmarks. The greatest loss is the loss of features, not CPU and RAM. Perhaps it waste 15% more resources than the previous patch?
As I personally enjoy the features removed in this patch, I will keep on running Lucene 2.0 and the old version, but this should be easier to understand and maintain if anyone else wants to take a look at it.
> InstantiatedIndex - faster but memory consuming index
> -----------------------------------------------------
>
> Key: LUCENE-550
> URL: https://issues.apache.org/jira/browse/LUCENE-550
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Store
> Affects Versions: 2.0.0
> Reporter: Karl Wettin
> Assignee: Karl Wettin
> Attachments: HitCollectionBench.jpg, lucene-550.jpg, LUCENE-550_20070804_no_core_changes.txt, test-reports.zip, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2, trunk.diff.bz2
>
>
> An non file centrinc all in memory index. Consumes some 2x the memory of a RAMDirectory (in a term satured index) but is between 3x-60x faster depending on application and how one counts. Average query is about 8x faster. IndexWriter and IndexModifier have been realized in InterfaceIndexWriter and InterfaceIndexModifier.
> InstantiatedIndex is wrapped in a new top layer index facade (class Index) that comes with factory methods for writers, readers and searchers for unison index handeling. There are decorators with notification handling that can be used for automatically syncronizing searchers on updates, et.c.
> Index also comes with FS/RAMDirectory implementation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org