You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Michael McCandless (JIRA)" <ji...@apache.org> on 2006/11/30 12:54:21 UTC
[jira] Created: (LUCENE-735) Simple tool to back-convert from
lockless to pre-lockless file format
Simple tool to back-convert from lockless to pre-lockless file format
---------------------------------------------------------------------
Key: LUCENE-735
URL: http://issues.apache.org/jira/browse/LUCENE-735
Project: Lucene - Java
Issue Type: Improvement
Components: Index
Affects Versions: 2.1
Reporter: Michael McCandless
Assigned To: Michael McCandless
Priority: Minor
Fix For: 2.1
Attachments: LUCENE-735.patch
Simple tool to back-convert from lockless to pre-lockless file format
The changes for lockless commits (LUCENE-701 ) are fairly minor and so
creating a tool to convert a lockless format index back to a
pre-lockless format index is 1) fairly simple, and 2) useful at least
for brave souls who want to try lockless but have the freedom to roll
back to Lucene 2.0, using the same index, if anything goes wrong.
I will attach an initial patch.
This has not yet received extensive testing so please be extremely
careful if you use this in production! I've only done minimal testing
so far: using IndexFiles to produce an index under lockless,
converting it to pre-lockless, and then doing searches against that
index with 2.0. More testing is clearly needed to ensure separate
deletes, separate norms, etc, are working correctly.
The tool prints details of what it did, eg:
>> java org.apache.lucene.index.ConvertPreLockless index
3 segments in index
segment 0: not compound file format
has deletions
rename _a_2.del to _a.del
no separate norms
segment 1: not compound file format
has deletions
rename _b_1.del to _b.del
no separate norms
segment 2: not compound file format
has deletions
rename _c_1.del to _c.del
no separate norms
wrote "segments" file
rename segments_8 to segments_8.old
Caveats:
* Tread very carefully! Test first in a sandox, etc.
* Make sure you only run this tool on an index that is not in use by
any reader/writers, else you could have problems: the tool
currently does not acquire the write lock even though it's
modifying the index.
* On Windows only: if your index has any un-referenced files (ie,
files that should have been deleted but were in use at the time)
at the time you run this tool, then they will never be deleted
(ie, pre-lockless Lucene won't know to delete them).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-735) Simple tool to back-convert from
lockless to pre-lockless file format
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-735?page=comments#action_12454591 ]
Michael McCandless commented on LUCENE-735:
-------------------------------------------
To use this, apply the patch to the Lucene trunk, then ant jar-core then run this:
java org.apache.lucene.index.ConvertPreLockless <path-to-index>
The conversion is in place, meaning, after this tool runs, your <path-to-index> should be in 2.0 file format.
> Simple tool to back-convert from lockless to pre-lockless file format
> ---------------------------------------------------------------------
>
> Key: LUCENE-735
> URL: http://issues.apache.org/jira/browse/LUCENE-735
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.1
> Reporter: Michael McCandless
> Assigned To: Michael McCandless
> Priority: Minor
> Fix For: 2.1
>
> Attachments: LUCENE-735.patch
>
>
> Simple tool to back-convert from lockless to pre-lockless file format
> The changes for lockless commits (LUCENE-701 ) are fairly minor and so
> creating a tool to convert a lockless format index back to a
> pre-lockless format index is 1) fairly simple, and 2) useful at least
> for brave souls who want to try lockless but have the freedom to roll
> back to Lucene 2.0, using the same index, if anything goes wrong.
> I will attach an initial patch.
> This has not yet received extensive testing so please be extremely
> careful if you use this in production! I've only done minimal testing
> so far: using IndexFiles to produce an index under lockless,
> converting it to pre-lockless, and then doing searches against that
> index with 2.0. More testing is clearly needed to ensure separate
> deletes, separate norms, etc, are working correctly.
> The tool prints details of what it did, eg:
> >> java org.apache.lucene.index.ConvertPreLockless index
> 3 segments in index
> segment 0: not compound file format
> has deletions
> rename _a_2.del to _a.del
> no separate norms
> segment 1: not compound file format
> has deletions
> rename _b_1.del to _b.del
> no separate norms
> segment 2: not compound file format
> has deletions
> rename _c_1.del to _c.del
> no separate norms
> wrote "segments" file
> rename segments_8 to segments_8.old
> Caveats:
> * Tread very carefully! Test first in a sandox, etc.
> * Make sure you only run this tool on an index that is not in use by
> any reader/writers, else you could have problems: the tool
> currently does not acquire the write lock even though it's
> modifying the index.
> * On Windows only: if your index has any un-referenced files (ie,
> files that should have been deleted but were in use at the time)
> at the time you run this tool, then they will never be deleted
> (ie, pre-lockless Lucene won't know to delete them).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-735) Simple tool to back-convert from
lockless to pre-lockless file format
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/LUCENE-735?page=all ]
Michael McCandless updated LUCENE-735:
--------------------------------------
Attachment: LUCENE-735.patch
> Simple tool to back-convert from lockless to pre-lockless file format
> ---------------------------------------------------------------------
>
> Key: LUCENE-735
> URL: http://issues.apache.org/jira/browse/LUCENE-735
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.1
> Reporter: Michael McCandless
> Assigned To: Michael McCandless
> Priority: Minor
> Fix For: 2.1
>
> Attachments: LUCENE-735.patch
>
>
> Simple tool to back-convert from lockless to pre-lockless file format
> The changes for lockless commits (LUCENE-701 ) are fairly minor and so
> creating a tool to convert a lockless format index back to a
> pre-lockless format index is 1) fairly simple, and 2) useful at least
> for brave souls who want to try lockless but have the freedom to roll
> back to Lucene 2.0, using the same index, if anything goes wrong.
> I will attach an initial patch.
> This has not yet received extensive testing so please be extremely
> careful if you use this in production! I've only done minimal testing
> so far: using IndexFiles to produce an index under lockless,
> converting it to pre-lockless, and then doing searches against that
> index with 2.0. More testing is clearly needed to ensure separate
> deletes, separate norms, etc, are working correctly.
> The tool prints details of what it did, eg:
> >> java org.apache.lucene.index.ConvertPreLockless index
> 3 segments in index
> segment 0: not compound file format
> has deletions
> rename _a_2.del to _a.del
> no separate norms
> segment 1: not compound file format
> has deletions
> rename _b_1.del to _b.del
> no separate norms
> segment 2: not compound file format
> has deletions
> rename _c_1.del to _c.del
> no separate norms
> wrote "segments" file
> rename segments_8 to segments_8.old
> Caveats:
> * Tread very carefully! Test first in a sandox, etc.
> * Make sure you only run this tool on an index that is not in use by
> any reader/writers, else you could have problems: the tool
> currently does not acquire the write lock even though it's
> modifying the index.
> * On Windows only: if your index has any un-referenced files (ie,
> files that should have been deleted but were in use at the time)
> at the time you run this tool, then they will never be deleted
> (ie, pre-lockless Lucene won't know to delete them).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-735) Simple tool to back-convert from
lockless to pre-lockless file format
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-735.
---------------------------------------
Resolution: Won't Fix
Fix Version/s: (was: 2.1)
Lucene Fields: [New, Patch Available] (was: [Patch Available, New])
I don't plan on making this tool generally available unless somehow the need suddenly arises (I hope not!).
> Simple tool to back-convert from lockless to pre-lockless file format
> ---------------------------------------------------------------------
>
> Key: LUCENE-735
> URL: https://issues.apache.org/jira/browse/LUCENE-735
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: 2.1
> Reporter: Michael McCandless
> Assigned To: Michael McCandless
> Priority: Minor
> Attachments: LUCENE-735.patch
>
>
> Simple tool to back-convert from lockless to pre-lockless file format
> The changes for lockless commits (LUCENE-701 ) are fairly minor and so
> creating a tool to convert a lockless format index back to a
> pre-lockless format index is 1) fairly simple, and 2) useful at least
> for brave souls who want to try lockless but have the freedom to roll
> back to Lucene 2.0, using the same index, if anything goes wrong.
> I will attach an initial patch.
> This has not yet received extensive testing so please be extremely
> careful if you use this in production! I've only done minimal testing
> so far: using IndexFiles to produce an index under lockless,
> converting it to pre-lockless, and then doing searches against that
> index with 2.0. More testing is clearly needed to ensure separate
> deletes, separate norms, etc, are working correctly.
> The tool prints details of what it did, eg:
> >> java org.apache.lucene.index.ConvertPreLockless index
> 3 segments in index
> segment 0: not compound file format
> has deletions
> rename _a_2.del to _a.del
> no separate norms
> segment 1: not compound file format
> has deletions
> rename _b_1.del to _b.del
> no separate norms
> segment 2: not compound file format
> has deletions
> rename _c_1.del to _c.del
> no separate norms
> wrote "segments" file
> rename segments_8 to segments_8.old
> Caveats:
> * Tread very carefully! Test first in a sandox, etc.
> * Make sure you only run this tool on an index that is not in use by
> any reader/writers, else you could have problems: the tool
> currently does not acquire the write lock even though it's
> modifying the index.
> * On Windows only: if your index has any un-referenced files (ie,
> files that should have been deleted but were in use at the time)
> at the time you run this tool, then they will never be deleted
> (ie, pre-lockless Lucene won't know to delete them).
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org