You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jay Potharaju <js...@gmail.com> on 2018/04/17 18:17:50 UTC
Re: solr 6.6.3 intermittent group faceting errors(Lucene54DocValuesProducer)
After digging into the error a bit more ..I see that the error messages
contain a call to lucenecodec54. I am using version solr 6.6.3. Any ideas
why is lucene54 being referred here??
Thanks
at org.apache.solr.request.SimpleFacets.lambda$getFacetFieldCounts$0(SimpleFacets.java:809)
at java.util.concurrent.FutureTask.run(Unknown Source)
at org.apache.solr.request.SimpleFacets$3.execute(SimpleFacets.java:742)
at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:818)
at org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetComponent.java:330)
at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:274)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:296)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkBounds(Unknown Source)
at java.nio.DirectByteBuffer.get(Unknown Source)
at org.apache.lucene.store.ByteBufferGuard.getBytes(ByteBufferGuard.java:93)
at org.apache.lucene.store.ByteBufferIndexInput.readBytes(ByteBufferIndexInput.java:89)
at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$CompressedBinaryDocValues$CompressedBinaryTermsEnum.readTerm(Lucene54DocValuesProducer.java:1349)
at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$CompressedBinaryDocValues$CompressedBinaryTermsEnum.next(Lucene54DocValuesProducer.java:1365)
at org.apache.lucene.search.grouping.term.TermGroupFacetCollector$MV$SegmentResult.nextTerm(TermGroupFacetCollector.java:367)
at org.apache.lucene.search.grouping.GroupFacetCollector.mergeSegmentResults(GroupFacetCollector.java:94)
at org.apache.solr.request.SimpleFacets.getGroupedCounts(SimpleFacets.java:696)
at org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:476)
at org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:405)
at org.apache.solr.request.SimpleFacets.lambda$getFacetFieldCounts$0(SimpleFacets.java:803)
Thanks
Jay Potharaju
On Tue, Apr 17, 2018 at 8:10 AM, Jay Potharaju <js...@gmail.com>
wrote:
> Hi
> Has anyone seen issues with group faceting on multivalued fields in solr
> 6x? Can any of the committers comment?
> Thanks
> Jay
>
> On Apr 16, 2018, at 1:44 PM, Jay Potharaju <js...@gmail.com> wrote:
>
> I deleted my collection and rebuilt it to check if there are any issues
> with indexing. I didn't see any errors during indexing. My collection is
> sharded and we use implicit routing...But after rebuilding my collection
> also I am getting errors on group faceting. This is not happening all the
> time but rather on small subset of data, which is fixed by reindexing.
>
> Any suggestions on what else to check for??
>
> Thanks
> Jay Potharaju
>
>
> On Mon, Apr 16, 2018 at 10:20 AM, Jay Potharaju <js...@gmail.com>
> wrote:
>
>> Hi,
>> I am testing solr 6.6.3 and have been running into intermittent group
>> faceting errors. I did some bulk indexing to initially setup the
>> collection I have multiple facet fields it only throws error on one of
>> the fields. The issue goes away when I reindex the data.
>>
>> <field name="category_id" type="tlong" indexed="true" stored="true"
>> required="false" multiValued="true" docValues="true"/>
>> I am upgrading from solr 5.3, didn't see this issue with the existing
>> version we are using. Any suggestions why this might be happening?
>>
>> Exception during facet.field: category_id
>> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMeth
>> od(HttpSolrClient.java:612)
>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(Htt
>> pSolrClient.java:279)
>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(Htt
>> pSolrClient.java:268)
>> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.j
>> ava:1219)
>> at org.apache.solr.handler.component.HttpShardHandler.lambda$
>> submit$0(HttpShardHandler.java:163)
>> at java.util.concurrent.FutureTask.run(Unknown Source)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>> at java.util.concurrent.FutureTask.run(Unknown Source)
>> at com.codahale.metrics.InstrumentedExecutorService$Instrumente
>> dRunnable.run(InstrumentedExecutorService.java:176)
>> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolE
>> xecutor.lambda$execute$0(ExecutorUtil.java:229)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>> Thanks
>> Jay
>>
>>
>
>
Re: solr 6.6.3 intermittent group faceting
errors(Lucene54DocValuesProducer)
Posted by Shawn Heisey <ap...@elyograg.org>.
On 4/17/2018 12:17 PM, Jay Potharaju wrote:
> After digging into the error a bit more ..I see that the error messages
> contain a call to lucenecodec54. I am using version solr 6.6.3. Any ideas
> why is lucene54 being referred here??
The 6.6 version uses index file formats that were last updated in
version 5.4.
It does tag the index as built by 6.6, though ... so you can't use it in
a 5.x version. There are almost certainly aspects of the written index
that go beyond any format in the 5.x versions. There is package named
org.apache.lucene.codecs.lucene60 in the 6.6.x versions.
The actual index built by 6.6.3 does have files with Lucene50 and
Lucene54 in the name:
Directory of
C:\Users\sheisey\Downloads\solr-6.6.3\example\techproducts\solr\techproducts\data\index
04/17/2018 03:43 PM <DIR> .
04/17/2018 03:43 PM <DIR> ..
04/17/2018 03:43 PM 165 segments_2
04/17/2018 03:43 PM 0 write.lock
04/17/2018 03:43 PM 85 _0.dii
04/17/2018 03:43 PM 768 _0.dim
04/17/2018 03:43 PM 4,792 _0.fdt
04/17/2018 03:43 PM 84 _0.fdx
04/17/2018 03:43 PM 2,546 _0.fnm
04/17/2018 03:43 PM 219 _0.nvd
04/17/2018 03:43 PM 118 _0.nvm
04/17/2018 03:43 PM 536 _0.si
04/17/2018 03:43 PM 178 _0.tvd
04/17/2018 03:43 PM 79 _0.tvx
04/17/2018 03:43 PM 941 _0_Lucene50_0.doc
04/17/2018 03:43 PM 77 _0_Lucene50_0.pay
04/17/2018 03:43 PM 2,247 _0_Lucene50_0.pos
04/17/2018 03:43 PM 12,550 _0_Lucene50_0.tim
04/17/2018 03:43 PM 579 _0_Lucene50_0.tip
04/17/2018 03:43 PM 254 _0_Lucene54_0.dvd
04/17/2018 03:43 PM 1,014 _0_Lucene54_0.dvm
Thanks,
Shawn
Re: solr 6.6.3 intermittent group faceting errors(Lucene54DocValuesProducer)
Posted by Jay Potharaju <js...@gmail.com>.
Thanks Eric & Shawn for chiming in ! In my solrconfig the lucene version is
set to 6.6.3. I do see that the index has lucene54 files.
With respect to the error regarding group faceting error.... it is similar
to what is being reported in SOLR-7867
<https://issues.apache.org/jira/browse/SOLR-7867>.
Thanks
Jay Potharaju
On Tue, Apr 17, 2018 at 8:17 PM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 4/17/2018 8:44 PM, Erick Erickson wrote:
>
>> The other possibility is that you have LuceneMatchVersion set to
>> 5-something in solrconfig.xml.
>>
>
> It's my understanding that luceneMatchVersion does NOT affect index format
> in any way, that about the only things that pay attention to this value are
> a subset of analysis components. Do I have an incorrect understanding?
>
> Does a Solr user even have the ability to influence the index format used
> without writing custom code?
>
> Thanks,
> Shawn
>
>
Re: solr 6.6.3 intermittent group faceting
errors(Lucene54DocValuesProducer)
Posted by Shawn Heisey <ap...@elyograg.org>.
On 4/17/2018 8:44 PM, Erick Erickson wrote:
> The other possibility is that you have LuceneMatchVersion set to
> 5-something in solrconfig.xml.
It's my understanding that luceneMatchVersion does NOT affect index
format in any way, that about the only things that pay attention to this
value are a subset of analysis components. Do I have an incorrect
understanding?
Does a Solr user even have the ability to influence the index format
used without writing custom code?
Thanks,
Shawn
Re: solr 6.6.3 intermittent group faceting errors(Lucene54DocValuesProducer)
Posted by Erick Erickson <er...@gmail.com>.
Those codecs only change their number when their behavior changes IIUC.
So lucenecodec54 may be there for Lucene50StoredFieldsFormat still
exists in master/8.0
IOW this is normal.
The other possibility is that you have LuceneMatchVersion set to
5-something in solrconfig.xml.
Best,
Erick
On Tue, Apr 17, 2018 at 11:17 AM, Jay Potharaju <js...@gmail.com> wrote:
> After digging into the error a bit more ..I see that the error messages
> contain a call to lucenecodec54. I am using version solr 6.6.3. Any ideas
> why is lucene54 being referred here??
> Thanks
>
> at org.apache.solr.request.SimpleFacets.lambda$getFacetFieldCounts$0(SimpleFacets.java:809)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at org.apache.solr.request.SimpleFacets$3.execute(SimpleFacets.java:742)
> at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:818)
> at org.apache.solr.handler.component.FacetComponent.getFacetCounts(FacetComponent.java:330)
> at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:274)
> at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:296)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
> at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:534)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IndexOutOfBoundsException
> at java.nio.Buffer.checkBounds(Unknown Source)
> at java.nio.DirectByteBuffer.get(Unknown Source)
> at org.apache.lucene.store.ByteBufferGuard.getBytes(ByteBufferGuard.java:93)
> at org.apache.lucene.store.ByteBufferIndexInput.readBytes(ByteBufferIndexInput.java:89)
> at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$CompressedBinaryDocValues$CompressedBinaryTermsEnum.readTerm(Lucene54DocValuesProducer.java:1349)
> at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$CompressedBinaryDocValues$CompressedBinaryTermsEnum.next(Lucene54DocValuesProducer.java:1365)
> at org.apache.lucene.search.grouping.term.TermGroupFacetCollector$MV$SegmentResult.nextTerm(TermGroupFacetCollector.java:367)
> at org.apache.lucene.search.grouping.GroupFacetCollector.mergeSegmentResults(GroupFacetCollector.java:94)
> at org.apache.solr.request.SimpleFacets.getGroupedCounts(SimpleFacets.java:696)
> at org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:476)
> at org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:405)
> at org.apache.solr.request.SimpleFacets.lambda$getFacetFieldCounts$0(SimpleFacets.java:803)
>
>
>
> Thanks
> Jay Potharaju
>
>
> On Tue, Apr 17, 2018 at 8:10 AM, Jay Potharaju <js...@gmail.com>
> wrote:
>
>> Hi
>> Has anyone seen issues with group faceting on multivalued fields in solr
>> 6x? Can any of the committers comment?
>> Thanks
>> Jay
>>
>> On Apr 16, 2018, at 1:44 PM, Jay Potharaju <js...@gmail.com> wrote:
>>
>> I deleted my collection and rebuilt it to check if there are any issues
>> with indexing. I didn't see any errors during indexing. My collection is
>> sharded and we use implicit routing...But after rebuilding my collection
>> also I am getting errors on group faceting. This is not happening all the
>> time but rather on small subset of data, which is fixed by reindexing.
>>
>> Any suggestions on what else to check for??
>>
>> Thanks
>> Jay Potharaju
>>
>>
>> On Mon, Apr 16, 2018 at 10:20 AM, Jay Potharaju <js...@gmail.com>
>> wrote:
>>
>>> Hi,
>>> I am testing solr 6.6.3 and have been running into intermittent group
>>> faceting errors. I did some bulk indexing to initially setup the
>>> collection I have multiple facet fields it only throws error on one of
>>> the fields. The issue goes away when I reindex the data.
>>>
>>> <field name="category_id" type="tlong" indexed="true" stored="true"
>>> required="false" multiValued="true" docValues="true"/>
>>> I am upgrading from solr 5.3, didn't see this issue with the existing
>>> version we are using. Any suggestions why this might be happening?
>>>
>>> Exception during facet.field: category_id
>>> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMeth
>>> od(HttpSolrClient.java:612)
>>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(Htt
>>> pSolrClient.java:279)
>>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(Htt
>>> pSolrClient.java:268)
>>> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.j
>>> ava:1219)
>>> at org.apache.solr.handler.component.HttpShardHandler.lambda$
>>> submit$0(HttpShardHandler.java:163)
>>> at java.util.concurrent.FutureTask.run(Unknown Source)
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>>> at java.util.concurrent.FutureTask.run(Unknown Source)
>>> at com.codahale.metrics.InstrumentedExecutorService$Instrumente
>>> dRunnable.run(InstrumentedExecutorService.java:176)
>>> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolE
>>> xecutor.lambda$execute$0(ExecutorUtil.java:229)
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>>> Thanks
>>> Jay
>>>
>>>
>>
>>