You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Andrzej Bialecki <ab...@getopt.org> on 2008/11/14 02:52:13 UTC
[ANN] Luke 0.9 released
Hi all,
I'm happy to announce that the new release of Luke - the Lucene Index
Toolbox - is available for download (or WebStart use) here:
http://www.getopt.org/luke
Changes in v. 0.9:
------------------
This release adds many functionality enhancements and advanced features
available in Lucene 2.4.
* New features and improvements:
o Added new tools:
+ Check Index - checks Lucene indexes for problems, and can fix some
of them. This is a GUI front-end to the Lucene CheckIndex tool.
+ Export to XML - exports index data and metadata to an XML file.
This is available both from the GUI and from the command-line.
o Significantly improved Optimize and Cleanup tools.
o Added ability to set norms on any indexed field in a document, or a
range of documents.
o Delete multiple documents by specifying ranges of document numbers.
o Added support for new field functionality: omitTF and binary fields.
o Improve the low-level information about the index, including format
version.
o Show interesting details about IndexCommit points and associated files.
o Add short explanations of index files' functions.
o Improve document reconstruction - now the information from
TermFreqVector can be used if available. Also, DocReconstructor can be
used outside of Luke.
o Significantly improved advanced search options - QueryParser
settings, Similarity and HitCollector settings.
o Read-only functionality is supported directly in IndexReader.
* Bug fixes:
A lot of effort went into refactoring the code, moving away if possible
from the spaghetti code influenced by Thinlet and into a modular design.
Still much needs to be done here. :(
This means that there are likely many more bugs than in the previous
release, although I tested all functionality to make sure that there is
no data loss.
HOWEVER, if you work with precious data, it's always a good idea to use
the "Read-only" option.
As usually, bug reports or suggestions for improvements, or even better
patches, are welcome!
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [ANN] Luke 0.9 released
Posted by mark harwood <ma...@yahoo.co.uk>.
>>BTW, if you have a small test index with multiple commit points could you please send it to me off the list?
See the "setup" method in the junit test "TestTransactionRollbackCapability2" attached here: https://issues.apache.org/jira/browse/LUCENE-1449
Cheers,
Mark
----- Original Message ----
From: Andrzej Bialecki <ab...@getopt.org>
To: java-user@lucene.apache.org
Sent: Friday, 14 November, 2008 10:47:03
Subject: Re: [ANN] Luke 0.9 released
mark harwood wrote:
> Hi Andrzej,
>
> Thanks for the update. Looks like you've been busy adding some great
> new features!
>
> I think you may have a bug in opening an index with prior commit
> points, though. I want to keep these in my index and so I opened it
> in Luke selecting the "open read only" and "keep all commit points"
> checkboxes. I can see in the code that under these conditions you
> take care to use a "KeepAllDeletionPolicy" but line 176 of
> IndexGate.java is hardcoded with a
> "KeepOnlyLastCommitDeletionPolicy". This wiped my prior commit points
> (No worries - it was only a test index).
Sorry about that - I'm glad it was disposable data. BTW, if you have a small test index with multiple commit points could you please send it to me off the list?
> The call stack leading to this code was: Luke.showFiles Luke.getIndexDeletableNames IndexGate.GetDeletableFiles
>
> The fix could be to pass the policy through this stack but may
> "opening read only" should mean just that and NO deletes at all are
> ever done - regardless?
Theoretically when you open a read-only IndexReader you can't delete anything using that IndexReader, but in practice Luke directly accesses the underlying Directory in many other places ... I forgot about the use of IndexFileDeleter - and indeed passing the read-only flag here can solve this, because then I can always use KeepAllDeletionPolicy when opening read-only.
Thanks for the report!
-- Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: [ANN] Luke 0.9 released
Posted by Andrzej Bialecki <ab...@getopt.org>.
mark harwood wrote:
> Hi Andrzej,
>
> Thanks for the update. Looks like you've been busy adding some great
> new features!
>
> I think you may have a bug in opening an index with prior commit
> points, though. I want to keep these in my index and so I opened it
> in Luke selecting the "open read only" and "keep all commit points"
> checkboxes. I can see in the code that under these conditions you
> take care to use a "KeepAllDeletionPolicy" but line 176 of
> IndexGate.java is hardcoded with a
> "KeepOnlyLastCommitDeletionPolicy". This wiped my prior commit points
> (No worries - it was only a test index).
Sorry about that - I'm glad it was disposable data. BTW, if you have a
small test index with multiple commit points could you please send it to
me off the list?
> The call stack leading to this code was: Luke.showFiles
> Luke.getIndexDeletableNames IndexGate.GetDeletableFiles
>
> The fix could be to pass the policy through this stack but may
> "opening read only" should mean just that and NO deletes at all are
> ever done - regardless?
Theoretically when you open a read-only IndexReader you can't delete
anything using that IndexReader, but in practice Luke directly accesses
the underlying Directory in many other places ... I forgot about the use
of IndexFileDeleter - and indeed passing the read-only flag here can
solve this, because then I can always use KeepAllDeletionPolicy when
opening read-only.
Thanks for the report!
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: [ANN] Luke 0.9 released
Posted by mark harwood <ma...@yahoo.co.uk>.
Hi Andrzej,
Thanks for the update. Looks like you've been busy adding some great new features!
I think you may have a bug in opening an index with prior commit points, though. I want to keep these in my index and so I opened it in Luke selecting the "open read only" and "keep all commit points" checkboxes.
I can see in the code that under these conditions you take care to use a "KeepAllDeletionPolicy" but line 176 of IndexGate.java is hardcoded with a "KeepOnlyLastCommitDeletionPolicy".
This wiped my prior commit points (No worries - it was only a test index).
The call stack leading to this code was:
Luke.showFiles
Luke.getIndexDeletableNames
IndexGate.GetDeletableFiles
The fix could be to pass the policy through this stack but may "opening read only" should mean just that and NO deletes at all are ever done - regardless?
Cheers,
Mark
----- Original Message ----
From: Andrzej Bialecki <ab...@getopt.org>
Sent: Friday, 14 November, 2008 1:52:13
Subject: [ANN] Luke 0.9 released
Hi all,
I'm happy to announce that the new release of Luke - the Lucene Index Toolbox - is available for download (or WebStart use) here:
http://www.getopt.org/luke
Changes in v. 0.9:
------------------
This release adds many functionality enhancements and advanced features available in Lucene 2.4.
* New features and improvements:
o Added new tools:
+ Check Index - checks Lucene indexes for problems, and can fix some
of them. This is a GUI front-end to the Lucene CheckIndex tool.
+ Export to XML - exports index data and metadata to an XML file.
This is available both from the GUI and from the command-line.
o Significantly improved Optimize and Cleanup tools.
o Added ability to set norms on any indexed field in a document, or a range of documents.
o Delete multiple documents by specifying ranges of document numbers.
o Added support for new field functionality: omitTF and binary fields.
o Improve the low-level information about the index, including format version.
o Show interesting details about IndexCommit points and associated files.
o Add short explanations of index files' functions.
o Improve document reconstruction - now the information from TermFreqVector can be used if available. Also, DocReconstructor can be used outside of Luke.
o Significantly improved advanced search options - QueryParser settings, Similarity and HitCollector settings.
o Read-only functionality is supported directly in IndexReader.
* Bug fixes:
A lot of effort went into refactoring the code, moving away if possible from the spaghetti code influenced by Thinlet and into a modular design. Still much needs to be done here. :(
This means that there are likely many more bugs than in the previous release, although I tested all functionality to make sure that there is no data loss.
HOWEVER, if you work with precious data, it's always a good idea to use the "Read-only" option.
As usually, bug reports or suggestions for improvements, or even better patches, are welcome!
-- Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org