You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Mark Miller (JIRA)" <ji...@apache.org> on 2009/06/16 15:15:07 UTC
[jira] Created: (LUCENE-1695) Update the Highlighter to use the new
TokenStream API
Update the Highlighter to use the new TokenStream API
-----------------------------------------------------
Key: LUCENE-1695
URL: https://issues.apache.org/jira/browse/LUCENE-1695
Project: Lucene - Java
Issue Type: Improvement
Components: contrib/highlighter
Reporter: Mark Miller
Assignee: Mark Miller
Fix For: 2.9
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1695) Update the Highlighter to use the
new TokenStream API
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736100#action_12736100 ]
Robert Muir commented on LUCENE-1695:
-------------------------------------
Mark there is a related issue in some of the other contribs.
ShingleMatrix and Compound expose a token api in a similar way... I'm kinda not sure what to do with these.
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1695) Update the Highlighter to use the new
TokenStream API
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated LUCENE-1695:
--------------------------------
Attachment: LUCENE-1695.patch
Rough, non backward compat patch.
There is still an issue with testUnRewrittenQuery() - it passes in isolation, but not when run with the other tests:
java.io.IOException: Stream closed
at java.io.StringReader.ensureOpen(StringReader.java:56)
at java.io.StringReader.read(StringReader.java:90)
at org.apache.lucene.analysis.standard.StandardTokenizerImpl.zzRefill(StandardTokenizerImpl.java:451)
at org.apache.lucene.analysis.standard.StandardTokenizerImpl.getNextToken(StandardTokenizerImpl.java:637)
at org.apache.lucene.analysis.standard.StandardTokenizer.incrementToken(StandardTokenizer.java:153)
at org.apache.lucene.analysis.standard.StandardFilter.incrementToken(StandardFilter.java:50)
at org.apache.lucene.analysis.LowerCaseFilter.incrementToken(LowerCaseFilter.java:38)
at org.apache.lucene.analysis.StopFilter.incrementToken(StopFilter.java:222)
at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:242)
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1695) Update the Highlighter to use the
new TokenStream API
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737129#action_12737129 ]
Mark Miller commented on LUCENE-1695:
-------------------------------------
Alright - this is no idle threat. I'm gonna commit this. And break everyone's custom Highlighter plugin classes everywhere. Smash them. Obliterate them. Make them rewrite them. Or use the old Highlighter jar. Any moment now ...
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1695) Update the Highlighter to use the
new TokenStream API
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737887#action_12737887 ]
Mark Miller commented on LUCENE-1695:
-------------------------------------
Sorry about that Koji - just updated it.
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1695) Update the Highlighter to use the
new TokenStream API
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736114#action_12736114 ]
Uwe Schindler commented on LUCENE-1695:
---------------------------------------
bq. not to hijack this issue (sorry) but Uwe, this reminds me that there are things that extend SinkTokenizer in contrib (analysis/sinks) as well, so it would be great to have a migration plan for those also
Extend!? Very bad, all TokenStreams should be final :-) -- *should be* but may not. I think these classes should stay as they are and use Tee/Sink and we should deprecate them. But TeeSinkTokenStream is final (and this is important) and the corresponding Sinks are somehow abstract (without "known" implementation).
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1695) Update the Highlighter to use the
new TokenStream API
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736097#action_12736097 ]
Mark Miller commented on LUCENE-1695:
-------------------------------------
Actually, I guess I would rather still change the API's - otherwise there will have to be a lot of needless Token object creation. Anyone else have an opinion? If not, I'm going to completely break back compat with the Highlighter here.
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1695) Update the Highlighter to use the
new TokenStream API
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12735225#action_12735225 ]
Mark Miller commented on LUCENE-1695:
-------------------------------------
Turns out, if Token is not deprecated, we don't really have to change those Highlighter plugin API's after all. I'll revert them.
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1695) Update the Highlighter to use the
new TokenStream API
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736113#action_12736113 ]
Uwe Schindler commented on LUCENE-1695:
---------------------------------------
bq. I think you could probably continue using the 2.4 Highlighter jar as well? All of the classes should be intact and compatible I think.
Should be (from the TokenStream changes perspective). Are there any non-bw changes in query processing that may affect highlighter?
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1695) Update the Highlighter to use the new
TokenStream API
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated LUCENE-1695:
--------------------------------
Attachment: LUCENE-1695.patch
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1695) Update the Highlighter to use the
new TokenStream API
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller resolved LUCENE-1695.
---------------------------------
Resolution: Fixed
I've committed this. We can reopen if someone brings up a new argument. Putting it in will help me finish up making the SpanScorer the default Scorer.
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1695) Update the Highlighter to use the new
TokenStream API
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated LUCENE-1695:
--------------------------------
Attachment: LUCENE-1695.patch
To trunk
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1695) Update the Highlighter to use the
new TokenStream API
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736106#action_12736106 ]
Uwe Schindler commented on LUCENE-1695:
---------------------------------------
We had the same problem in core with Tee/sinkTokenizer. Both classes exported API using Token instances without any real use (I think it was because of tests). We deprecated the whole calss and created a new TeeSinkTokenFilter using States and do not export thier internal implementation (which is not needed for Tee/Sink usage).
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1695) Update the Highlighter to use the new
TokenStream API
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated LUCENE-1695:
--------------------------------
Attachment: LUCENE-1695.patch
Pretty much done, all tests pass. It breaks back compat, but frankly, straddling doesn't seem worth the effort here. Or even very possible. You can't really give new methods to use for the deprecated ones, and deprecating by class would be a real nuisance as we would lose class names I'd rather keep. We have no back compat policy, and I think its worth just pushing this to the new API.
I was also thinking about breaking back compat with changing the Highlighter to use the SpanScorer, so doing it all in one shot would be nice. The overall migration should be fairly simple once you understand the new TokenFilter API. I'll handle it for Solr.
Still needs either its own changes file to explain or could go in the contrib common changes file.
There is a change to the MemoryIndex to get around issues with the new/old API and CachingTokenFilters.
Ill have to see how the new TokenFilter API improvements issue works out before doing a final patch for this.
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1695) Update the Highlighter to use the
new TokenStream API
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736646#action_12736646 ]
Mark Miller commented on LUCENE-1695:
-------------------------------------
So without further objection, I'm going to commit this so that I can finish the 'make spanscorer the default' issue.
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1695) Update the Highlighter to use the new
TokenStream API
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated LUCENE-1695:
--------------------------------
Attachment: LUCENE-1695.patch
to trunk
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1695) Update the Highlighter to use the
new TokenStream API
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736111#action_12736111 ]
Robert Muir commented on LUCENE-1695:
-------------------------------------
not to hijack this issue (sorry) but Uwe, this reminds me that there are things that extend SinkTokenizer in contrib (analysis/sinks) as well, so it would be great to have a migration plan for those also.
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1695) Update the Highlighter to use the
new TokenStream API
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737885#action_12737885 ]
Koji Sekiguchi commented on LUCENE-1695:
----------------------------------------
Mark, can you remove System.out from TokenSources? I got a lot of "inc token" log messages when running highlight-profile.alg...
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1695) Update the Highlighter to use the
new TokenStream API
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12736112#action_12736112 ]
Mark Miller commented on LUCENE-1695:
-------------------------------------
Here we would need to deprecate the whole Highlighter or add some pretty nasty code to the current one to support both. Because the Highlighter has no back compat promise, I'd almost rather just jump to the new API. I think there are a couple other breaks that should be made (to make using the SpanScorer more rational when we make it the default), so doing everything at once with 2.9 seems somewhat reasonable to me. Updating code to work with the new API should be as straightforward as updating a TokenFilter/TokenStream. I think you could probably continue using the 2.4 Highlighter jar as well? All of the classes should be intact and compatible I think.
> Update the Highlighter to use the new TokenStream API
> -----------------------------------------------------
>
> Key: LUCENE-1695
> URL: https://issues.apache.org/jira/browse/LUCENE-1695
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/highlighter
> Reporter: Mark Miller
> Assignee: Mark Miller
> Fix For: 2.9
>
> Attachments: LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch, LUCENE-1695.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org