You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Ahmet Arslan (JIRA)" <ji...@apache.org> on 2014/05/04 01:41:16 UTC
[jira] [Updated] (SOLR-5426) org.apache.solr.common.SolrException;
org.apache.solr.common.SolrException:
org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0
exceeds length of provided text sized 840
[ https://issues.apache.org/jira/browse/SOLR-5426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ahmet Arslan updated SOLR-5426:
-------------------------------
Attachment: SOLR-5426.patch
Bring failing test case to trunk. Simplify schema and add 5 test methods to isolate problem. Problematic field type is given below. Exception occurs for only *stored* and *multiValued* field. TestCase demonstrates this.
Another interesting thing is test passes
* when WordDelimiterFilterFactory is removed from index analyzer
* when ReversedWildcardFilterFactory is removed from index analyzer
separately.
{code:xml}
<fieldType name="text_stx" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="0"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.ReversedWildcardFilterFactory" withOriginal="true"
maxPosAsterisk="3" maxPosQuestion="2" maxFractionAsterisk="0.33"/>
</analyzer>
<analyzer type="query">
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="0"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>
{code}
> org.apache.solr.common.SolrException; org.apache.solr.common.SolrException: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-5426
> URL: https://issues.apache.org/jira/browse/SOLR-5426
> Project: Solr
> Issue Type: Bug
> Components: highlighter
> Affects Versions: 4.4, 4.5.1
> Reporter: Nikolay
> Priority: Minor
> Attachments: SOLR-5426.patch, SOLR-5426.patch, highlighter.zip
>
>
> Highlighter does not work correctly on test-data.
> I added index- and config- files (see attached highlighter.zip) for reproducing this issue.
> Everything works fine if I search without highlighting:
> http://localhost:8983/solr/global/select?q=aa&wt=json&indent=true
> But if search with highlighting:
> http://localhost:8983/solr/global/select?q=aa&wt=json&indent=true&hl=true&hl.fl=*_stx&hl.simple.pre=<em>&hl.simple.post=<%2Fem>
> I'm get the error:
> ERROR - 2013-11-07 10:17:15.797; org.apache.solr.common.SolrException; null:org.apache.solr.common.SolrException: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840
> at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:542)
> at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:414)
> at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:139)
> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:208)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
> at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
> at org.eclipse.jetty.server.Server.handle(Server.java:368)
> at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
> at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
> at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
> at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
> at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
> at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> at java.lang.Thread.run(Unknown Source)
> Caused by: org.apache.lucene.search.highlight.InvalidTokenOffsetsException: Token 0 exceeds length of provided text sized 840
> at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:225)
> at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:527)
> ... 33 more
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org