You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (Created) (JIRA)" <ji...@apache.org> on 2011/12/01 13:32:40 UTC
[jira] [Created] (LUCENE-3613) remove .codecs.DefaultXXXX and split
into 3x/4x impls
remove .codecs.DefaultXXXX and split into 3x/4x impls
-----------------------------------------------------
Key: LUCENE-3613
URL: https://issues.apache.org/jira/browse/LUCENE-3613
Project: Lucene - Java
Issue Type: Task
Affects Versions: 4.0
Reporter: Robert Muir
Currently there is a lot of shared codec implementations for things like
field infos, segment infos, stored fields, term vectors under the .codecs
package with the name Default*
I think actually the .codecs package should be our abstract codec apis,
and the implementations should be under the respective codecs.
Also, lots of these implementations are still 'shared' across the 3.x and 4.x
formats, but shouldn't be: the 4.x versions should be "clean", and the 3.x
versions should have all the backwards cruft.
Additionally, the "PreFlexRW" should be writing this stuff in 3.x format,
but today it does not (it writes a 3.x index repackaged in a 4.x "container").
Really PreFlexRW needs to write indexes that are exactly like the 3.x format
(if you opened them up in lucene 3.x you could read them!). When this works
then we know that codecs will actually work for our backwards compatibility needs
and we know that we have good index backwards compatibility testing.
For starters we could just pick some 3.x version of lucene and wire PreFlexRW
to it... in the future maybe the writer should take a version param so it
can actually simulate different minor versions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3613) remove .codecs.DefaultXXXX and split
into 3x/4x impls
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-3613:
--------------------------------
Attachment: LUCENE-3613.patch
Here's a first patch, moving this Default* stuff to Lucene40 codec.
Some preflex stuff is obviously using Lucene40 impls still, but this just reflects reality.
> remove .codecs.DefaultXXXX and split into 3x/4x impls
> -----------------------------------------------------
>
> Key: LUCENE-3613
> URL: https://issues.apache.org/jira/browse/LUCENE-3613
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3613.patch
>
>
> Currently there is a lot of shared codec implementations for things like
> field infos, segment infos, stored fields, term vectors under the .codecs
> package with the name Default*
> I think actually the .codecs package should be our abstract codec apis,
> and the implementations should be under the respective codecs.
> Also, lots of these implementations are still 'shared' across the 3.x and 4.x
> formats, but shouldn't be: the 4.x versions should be "clean", and the 3.x
> versions should have all the backwards cruft.
> Additionally, the "PreFlexRW" should be writing this stuff in 3.x format,
> but today it does not (it writes a 3.x index repackaged in a 4.x "container").
> Really PreFlexRW needs to write indexes that are exactly like the 3.x format
> (if you opened them up in lucene 3.x you could read them!). When this works
> then we know that codecs will actually work for our backwards compatibility needs
> and we know that we have good index backwards compatibility testing.
> For starters we could just pick some 3.x version of lucene and wire PreFlexRW
> to it... in the future maybe the writer should take a version param so it
> can actually simulate different minor versions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3613) remove .codecs.DefaultXXXX and split
into 3x/4x impls
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-3613:
--------------------------------
Attachment: LUCENE-3613_vectors.patch
here's a pretty trivial patch splitting the 4.x and 3.x term vectors implementations.
I didn't do much refactoring except to remove shared docstore code from the 4.x version.
If there are no objects I would like to commit this, then separately add getComparator() to TermVectorsWriter, use it to sort the term vectors hash, and fix our 3.x backwards compatibility so preflex actually handles the sort order change in term vectors.
> remove .codecs.DefaultXXXX and split into 3x/4x impls
> -----------------------------------------------------
>
> Key: LUCENE-3613
> URL: https://issues.apache.org/jira/browse/LUCENE-3613
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3613.patch, LUCENE-3613_vectors.patch
>
>
> Currently there is a lot of shared codec implementations for things like
> field infos, segment infos, stored fields, term vectors under the .codecs
> package with the name Default*
> I think actually the .codecs package should be our abstract codec apis,
> and the implementations should be under the respective codecs.
> Also, lots of these implementations are still 'shared' across the 3.x and 4.x
> formats, but shouldn't be: the 4.x versions should be "clean", and the 3.x
> versions should have all the backwards cruft.
> Additionally, the "PreFlexRW" should be writing this stuff in 3.x format,
> but today it does not (it writes a 3.x index repackaged in a 4.x "container").
> Really PreFlexRW needs to write indexes that are exactly like the 3.x format
> (if you opened them up in lucene 3.x you could read them!). When this works
> then we know that codecs will actually work for our backwards compatibility needs
> and we know that we have good index backwards compatibility testing.
> For starters we could just pick some 3.x version of lucene and wire PreFlexRW
> to it... in the future maybe the writer should take a version param so it
> can actually simulate different minor versions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (LUCENE-3613) remove .codecs.DefaultXXXX and split
into 3x/4x impls
Posted by "Robert Muir (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-3613:
--------------------------------
Attachment: LUCENE-3613_vectors.patch
same patch: just with a bit more dead code removal: 3x reader doesn't need the raw methods used by bulk merging because PreFlexRW's vectors writer doesnt bulk merge (uses the default impl).
> remove .codecs.DefaultXXXX and split into 3x/4x impls
> -----------------------------------------------------
>
> Key: LUCENE-3613
> URL: https://issues.apache.org/jira/browse/LUCENE-3613
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
> Attachments: LUCENE-3613.patch, LUCENE-3613_vectors.patch, LUCENE-3613_vectors.patch
>
>
> Currently there is a lot of shared codec implementations for things like
> field infos, segment infos, stored fields, term vectors under the .codecs
> package with the name Default*
> I think actually the .codecs package should be our abstract codec apis,
> and the implementations should be under the respective codecs.
> Also, lots of these implementations are still 'shared' across the 3.x and 4.x
> formats, but shouldn't be: the 4.x versions should be "clean", and the 3.x
> versions should have all the backwards cruft.
> Additionally, the "PreFlexRW" should be writing this stuff in 3.x format,
> but today it does not (it writes a 3.x index repackaged in a 4.x "container").
> Really PreFlexRW needs to write indexes that are exactly like the 3.x format
> (if you opened them up in lucene 3.x you could read them!). When this works
> then we know that codecs will actually work for our backwards compatibility needs
> and we know that we have good index backwards compatibility testing.
> For starters we could just pick some 3.x version of lucene and wire PreFlexRW
> to it... in the future maybe the writer should take a version param so it
> can actually simulate different minor versions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (LUCENE-3613) remove .codecs.DefaultXXXX and
split into 3x/4x impls
Posted by "Robert Muir (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13161694#comment-13161694 ]
Robert Muir commented on LUCENE-3613:
-------------------------------------
I thought the fieldinfos would be a good one to start with here, but I think we are going to have to keep that DefaultXXX until 5.0
The problem is gaps in the field numbers, we have no easy way to simulate what we did in 3.x at write time,
unless we make stuff even more complicated just for the sake of simulating it properly... annoying.
So ill leave this one be for now and try to get some of this other stuff cleaned up.
> remove .codecs.DefaultXXXX and split into 3x/4x impls
> -----------------------------------------------------
>
> Key: LUCENE-3613
> URL: https://issues.apache.org/jira/browse/LUCENE-3613
> Project: Lucene - Java
> Issue Type: Task
> Affects Versions: 4.0
> Reporter: Robert Muir
>
> Currently there is a lot of shared codec implementations for things like
> field infos, segment infos, stored fields, term vectors under the .codecs
> package with the name Default*
> I think actually the .codecs package should be our abstract codec apis,
> and the implementations should be under the respective codecs.
> Also, lots of these implementations are still 'shared' across the 3.x and 4.x
> formats, but shouldn't be: the 4.x versions should be "clean", and the 3.x
> versions should have all the backwards cruft.
> Additionally, the "PreFlexRW" should be writing this stuff in 3.x format,
> but today it does not (it writes a 3.x index repackaged in a 4.x "container").
> Really PreFlexRW needs to write indexes that are exactly like the 3.x format
> (if you opened them up in lucene 3.x you could read them!). When this works
> then we know that codecs will actually work for our backwards compatibility needs
> and we know that we have good index backwards compatibility testing.
> For starters we could just pick some 3.x version of lucene and wire PreFlexRW
> to it... in the future maybe the writer should take a version param so it
> can actually simulate different minor versions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org