You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "John Wang (JIRA)" <ji...@apache.org> on 2009/12/15 11:09:18 UTC
[jira] Created: (LUCENE-2160) Tool to rename a field
Tool to rename a field
----------------------
Key: LUCENE-2160
URL: https://issues.apache.org/jira/browse/LUCENE-2160
Project: Lucene - Java
Issue Type: New Feature
Components: contrib/*
Affects Versions: 3.0
Reporter: John Wang
We found it useful to be able to rename a field.
It can save a lot of reindexing time/cost when being used in conjunction with ParallelReader to update partially a field.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2160) Tool to rename a field
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790660#action_12790660 ]
Michael McCandless commented on LUCENE-2160:
--------------------------------------------
Hmm... isn't there a danger here that a field rename would change the term sort order?
Ie, terms in the terms dict are sorted first by field and then by term text. Seems like this tool could break that, in the index?
> Tool to rename a field
> ----------------------
>
> Key: LUCENE-2160
> URL: https://issues.apache.org/jira/browse/LUCENE-2160
> Project: Lucene - Java
> Issue Type: New Feature
> Components: contrib/*
> Affects Versions: 3.0
> Reporter: John Wang
> Attachments: RenameField.java
>
>
> We found it useful to be able to rename a field.
> It can save a lot of reindexing time/cost when being used in conjunction with ParallelReader to update partially a field.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2160) Tool to rename a field
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Wang updated LUCENE-2160:
------------------------------
Attachment: RenameField.java
Fixed a problem with cfs files.
> Tool to rename a field
> ----------------------
>
> Key: LUCENE-2160
> URL: https://issues.apache.org/jira/browse/LUCENE-2160
> Project: Lucene - Java
> Issue Type: New Feature
> Components: contrib/*
> Affects Versions: 3.0
> Reporter: John Wang
> Attachments: RenameField.java, RenameField.java
>
>
> We found it useful to be able to rename a field.
> It can save a lot of reindexing time/cost when being used in conjunction with ParallelReader to update partially a field.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2160) Tool to rename a field
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790797#action_12790797 ]
John Wang commented on LUCENE-2160:
-----------------------------------
Good point. But do you ever sort across fields?
> Tool to rename a field
> ----------------------
>
> Key: LUCENE-2160
> URL: https://issues.apache.org/jira/browse/LUCENE-2160
> Project: Lucene - Java
> Issue Type: New Feature
> Components: contrib/*
> Affects Versions: 3.0
> Reporter: John Wang
> Attachments: RenameField.java
>
>
> We found it useful to be able to rename a field.
> It can save a lot of reindexing time/cost when being used in conjunction with ParallelReader to update partially a field.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2160) Tool to rename a field
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790809#action_12790809 ]
John Wang commented on LUCENE-2160:
-----------------------------------
Just did a test:
You are right, IndexReader.terms(Term) would no longer find the rename field name if the field name is out of order. If the order is preserved, it is ok, e.g. list of fields "a","c","f", if renaming "c" -> "d", it would be ok.
Our use case is however this:
We messed up our data in say, field "c", we rename it to "c_bak", and create a parallel index with one field and name if "c". merge the indexes. c_bak is then never accessed.
Would this work?
> Tool to rename a field
> ----------------------
>
> Key: LUCENE-2160
> URL: https://issues.apache.org/jira/browse/LUCENE-2160
> Project: Lucene - Java
> Issue Type: New Feature
> Components: contrib/*
> Affects Versions: 3.0
> Reporter: John Wang
> Attachments: RenameField.java, RenameField.java
>
>
> We found it useful to be able to rename a field.
> It can save a lot of reindexing time/cost when being used in conjunction with ParallelReader to update partially a field.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2160) Tool to rename a field
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791046#action_12791046 ]
John Wang commented on LUCENE-2160:
-----------------------------------
Did some more digging around the issue on field ordering. Is it possible to change FieldInfo file store to change the number in byNumber ArrayList along with the byName HashMap, and update the file? Or is the number already assumed to be in sort order from the tii file?
> Tool to rename a field
> ----------------------
>
> Key: LUCENE-2160
> URL: https://issues.apache.org/jira/browse/LUCENE-2160
> Project: Lucene - Java
> Issue Type: New Feature
> Components: contrib/*
> Affects Versions: 3.0
> Reporter: John Wang
> Attachments: RenameField.java, RenameField.java
>
>
> We found it useful to be able to rename a field.
> It can save a lot of reindexing time/cost when being used in conjunction with ParallelReader to update partially a field.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2160) Tool to rename a field
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791051#action_12791051 ]
John Wang commented on LUCENE-2160:
-----------------------------------
Looked at the file format wiki more closely, I see front-coding applies to all the terms in all fields. So my above comment would not work.
Do you think it makes sense to have a tii,tis file for each indexed field? Would the new codec allow for it?
-John
> Tool to rename a field
> ----------------------
>
> Key: LUCENE-2160
> URL: https://issues.apache.org/jira/browse/LUCENE-2160
> Project: Lucene - Java
> Issue Type: New Feature
> Components: contrib/*
> Affects Versions: 3.0
> Reporter: John Wang
> Attachments: RenameField.java, RenameField.java
>
>
> We found it useful to be able to rename a field.
> It can save a lot of reindexing time/cost when being used in conjunction with ParallelReader to update partially a field.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2160) Tool to rename a field
Posted by "John Wang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Wang updated LUCENE-2160:
------------------------------
Attachment: RenameField.java
part of the code was originally posted on nabble, but is not removed:
www.nabble.com/file/p15221929/fieldrename
> Tool to rename a field
> ----------------------
>
> Key: LUCENE-2160
> URL: https://issues.apache.org/jira/browse/LUCENE-2160
> Project: Lucene - Java
> Issue Type: New Feature
> Components: contrib/*
> Affects Versions: 3.0
> Reporter: John Wang
> Attachments: RenameField.java
>
>
> We found it useful to be able to rename a field.
> It can save a lot of reindexing time/cost when being used in conjunction with ParallelReader to update partially a field.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2160) Tool to rename a field
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791056#action_12791056 ]
Michael McCandless commented on LUCENE-2160:
--------------------------------------------
bq. Looked at the file format wiki more closely, I see front-coding applies to all the terms in all fields. So my above comment would not work.
Yeah I think you'll be in trouble in general if your new name doesn't "fit" in the same sort; you could check and allow a rename as long as it doesn't change the sort order?
bq. Do you think it makes sense to have a tii,tis file for each indexed field? Would the new codec allow for it?
A new codec would definitely be free to store however it wanted...
But even the standard codec (default codec for flex, most similar to the current index format) actually shouldn't mind if the fields are not in sorted order (though I haven't tested this!). It separately stores the seek position of each field... though we may need to fix the FieldsEnum to do the sorting if the index is no longer sorted.
> Tool to rename a field
> ----------------------
>
> Key: LUCENE-2160
> URL: https://issues.apache.org/jira/browse/LUCENE-2160
> Project: Lucene - Java
> Issue Type: New Feature
> Components: contrib/*
> Affects Versions: 3.0
> Reporter: John Wang
> Attachments: RenameField.java, RenameField.java
>
>
> We found it useful to be able to rename a field.
> It can save a lot of reindexing time/cost when being used in conjunction with ParallelReader to update partially a field.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org