You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Hoss Man (JIRA)" <ji...@apache.org> on 2015/08/25 01:40:45 UTC
[jira] [Updated] (SOLR-7954) ArrayIndexOutOfBoundsException from
distributed HLL serialization logic when using using
stats.field={!cardinality=1.0} in a distributed query
[ https://issues.apache.org/jira/browse/SOLR-7954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hoss Man updated SOLR-7954:
---------------------------
Description:
User reports indicate that using {{stats.field=\{!cardinality=1.0\}foo}} on a field that has extremely high cardinality on a single shard (example: 150K unique values) can lead to "ArrayIndexOutOfBoundsException: 3" on the shard during serialization of the HLL values.
using "cardinality=0.9" (or lower) doesn't produce the same symptoms, suggesting the problem is specific to large log2m and regwidth values.
was:
Following exception is thrown for the query :
bq. q=field1:*&stats=true&stats.field={!cardinality=1.0}field.
The exception is not seen once the cardinality is set to 0.9 or less.
The field is docValues enabled and indexed=false. The same exception I tried to reproduce on non docValues field but could not.
ERROR - 2015-08-11 12:24:00.222; [core] org.apache.solr.common.SolrException; null:java.lang.ArrayIndexOutOfBoundsException: 3
at net.agkn.hll.serialization.BigEndianAscendingWordSerializer.writeWord(BigEndianAscendingWordSerializer.java:152)
at net.agkn.hll.util.BitVector.getRegisterContents(BitVector.java:247)
at net.agkn.hll.HLL.toBytes(HLL.java:917)
at net.agkn.hll.HLL.toBytes(HLL.java:869)
at org.apache.solr.handler.component.AbstractStatsValues.getStatsValues(StatsValuesFactory.java:348)
at org.apache.solr.handler.component.StatsComponent.convertToResponse(StatsComponent.java:151)
at org.apache.solr.handler.component.StatsComponent.process(StatsComponent.java:62)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:255)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:497)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
Summary: ArrayIndexOutOfBoundsException from distributed HLL serialization logic when using using stats.field={!cardinality=1.0} in a distributed query (was: Exception while using {!cardinality=1.0}.)
cleaned up summary & description now that we have a better idea what the root symptoms/cause are.
> ArrayIndexOutOfBoundsException from distributed HLL serialization logic when using using stats.field={!cardinality=1.0} in a distributed query
> ----------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SOLR-7954
> URL: https://issues.apache.org/jira/browse/SOLR-7954
> Project: Solr
> Issue Type: Bug
> Affects Versions: 5.2.1
> Environment: SolrCloud 4 node cluster.
> Ubuntu 12.04
> OS Type 64 bit
> Reporter: Modassar Ather
> Assignee: Hoss Man
> Attachments: SOLR-7954.patch
>
>
> User reports indicate that using {{stats.field=\{!cardinality=1.0\}foo}} on a field that has extremely high cardinality on a single shard (example: 150K unique values) can lead to "ArrayIndexOutOfBoundsException: 3" on the shard during serialization of the HLL values.
> using "cardinality=0.9" (or lower) doesn't produce the same symptoms, suggesting the problem is specific to large log2m and regwidth values.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org