You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Srivatsan <ra...@gmail.com> on 2013/08/13 12:46:53 UTC
Split Shard Error - maxValue must be non-negative
Hi,
I am experimenting with solr 4.4.0 split shard feature. When i split the
shard i am getting the following exception.
/java.lang.IllegalArgumentException: maxValue must be non-negative (got: -1)
at
org.apache.lucene.util.packed.PackedInts.bitsRequired(PackedInts.java:1184)
at
org.apache.lucene.codecs.lucene42.Lucene42DocValuesConsumer.addNumericField(Lucene42DocValuesConsumer.java:140)
at
org.apache.lucene.codecs.lucene42.Lucene42DocValuesConsumer.addNumericField(Lucene42DocValuesConsumer.java:92)
at
org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:112)
at org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:221)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:119)
at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:2488)
at
org.apache.solr.update.SolrIndexSplitter.split(SolrIndexSplitter.java:125)
at
org.apache.solr.update.DirectUpdateHandler2.split(DirectUpdateHandler2.java:766)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleSplitAction(CoreAdminHandler.java:284)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:186)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at
org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:611)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:209)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
at
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:679)/
How to resolve this problem?
--
View this message in context: http://lucene.472066.n3.nabble.com/Split-Shard-Error-maxValue-must-be-non-negative-tp4084220.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Split Shard Error - maxValue must be non-negative
Posted by Greg Preston <gp...@marinsoftware.com>.
I'm running into the same issue using composite routing keys when all of
the shard keys end up in one of the subshards.
-Greg
On Tue, Aug 13, 2013 at 9:34 AM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:
> Scratch that. I obviously didn't pay attention to the stack trace.
> There is no workaround until 4.5 for this issue because we split the
> range by half and thus cannot guarantee that all segments will have
> numDocs > 0.
>
> On Tue, Aug 13, 2013 at 9:25 PM, Shalin Shekhar Mangar
> <sh...@gmail.com> wrote:
> > On Tue, Aug 13, 2013 at 9:15 PM, Robert Muir <rc...@gmail.com> wrote:
> >> On Tue, Aug 13, 2013 at 11:39 AM, Shalin Shekhar Mangar
> >> <sh...@gmail.com> wrote:
> >>> The splitting code calls commit before it starts the splitting. It
> creates
> >>> a LiveDocsReader using a bitset created by the split. This reader is
> merged
> >>> to an index using addIndexes.
> >>>
> >>> Shouldn't the addIndexes code then ignore all such 0-document segments?
> >>>
> >>>
> >>
> >> Not in 4.4: https://issues.apache.org/jira/browse/LUCENE-5116
> >
> >
> > Sorry, I didn't notice that. So 4.4 users must call commit/optimize
> > with expungeDeletes="true" until 4.5 is released if they run into this
> > problem.
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
Re: Split Shard Error - maxValue must be non-negative
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Scratch that. I obviously didn't pay attention to the stack trace.
There is no workaround until 4.5 for this issue because we split the
range by half and thus cannot guarantee that all segments will have
numDocs > 0.
On Tue, Aug 13, 2013 at 9:25 PM, Shalin Shekhar Mangar
<sh...@gmail.com> wrote:
> On Tue, Aug 13, 2013 at 9:15 PM, Robert Muir <rc...@gmail.com> wrote:
>> On Tue, Aug 13, 2013 at 11:39 AM, Shalin Shekhar Mangar
>> <sh...@gmail.com> wrote:
>>> The splitting code calls commit before it starts the splitting. It creates
>>> a LiveDocsReader using a bitset created by the split. This reader is merged
>>> to an index using addIndexes.
>>>
>>> Shouldn't the addIndexes code then ignore all such 0-document segments?
>>>
>>>
>>
>> Not in 4.4: https://issues.apache.org/jira/browse/LUCENE-5116
>
>
> Sorry, I didn't notice that. So 4.4 users must call commit/optimize
> with expungeDeletes="true" until 4.5 is released if they run into this
> problem.
>
> --
> Regards,
> Shalin Shekhar Mangar.
--
Regards,
Shalin Shekhar Mangar.
Re: Split Shard Error - maxValue must be non-negative
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Tue, Aug 13, 2013 at 9:15 PM, Robert Muir <rc...@gmail.com> wrote:
> On Tue, Aug 13, 2013 at 11:39 AM, Shalin Shekhar Mangar
> <sh...@gmail.com> wrote:
>> The splitting code calls commit before it starts the splitting. It creates
>> a LiveDocsReader using a bitset created by the split. This reader is merged
>> to an index using addIndexes.
>>
>> Shouldn't the addIndexes code then ignore all such 0-document segments?
>>
>>
>
> Not in 4.4: https://issues.apache.org/jira/browse/LUCENE-5116
Sorry, I didn't notice that. So 4.4 users must call commit/optimize
with expungeDeletes="true" until 4.5 is released if they run into this
problem.
--
Regards,
Shalin Shekhar Mangar.
Re: Split Shard Error - maxValue must be non-negative
Posted by Robert Muir <rc...@gmail.com>.
On Tue, Aug 13, 2013 at 11:39 AM, Shalin Shekhar Mangar
<sh...@gmail.com> wrote:
> The splitting code calls commit before it starts the splitting. It creates
> a LiveDocsReader using a bitset created by the split. This reader is merged
> to an index using addIndexes.
>
> Shouldn't the addIndexes code then ignore all such 0-document segments?
>
>
Not in 4.4: https://issues.apache.org/jira/browse/LUCENE-5116
Re: Split Shard Error - maxValue must be non-negative
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
The splitting code calls commit before it starts the splitting. It creates
a LiveDocsReader using a bitset created by the split. This reader is merged
to an index using addIndexes.
Shouldn't the addIndexes code then ignore all such 0-document segments?
On Tue, Aug 13, 2013 at 6:08 PM, Robert Muir <rc...@gmail.com> wrote:
> Well, i meant before, but i just took a look and this is implemented
> differently than the "merge" one.
>
> In any case, i think its the same bug, because I think the only way
> this can happen is if somehow this splitter is trying to create a
> 0-document "split" (or maybe a split containing all deletions).
>
> On Tue, Aug 13, 2013 at 8:22 AM, Srivatsan <ra...@gmail.com>
> wrote:
> > Ya i am performing commit after split request is submitted to server.
> >
> >
> >
> > --
> > View this message in context:
> http://lucene.472066.n3.nabble.com/Split-Shard-Error-maxValue-must-be-non-negative-tp4084220p4084256.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Regards,
Shalin Shekhar Mangar.
Re: Split Shard Error - maxValue must be non-negative
Posted by Robert Muir <rc...@gmail.com>.
Well, i meant before, but i just took a look and this is implemented
differently than the "merge" one.
In any case, i think its the same bug, because I think the only way
this can happen is if somehow this splitter is trying to create a
0-document "split" (or maybe a split containing all deletions).
On Tue, Aug 13, 2013 at 8:22 AM, Srivatsan <ra...@gmail.com> wrote:
> Ya i am performing commit after split request is submitted to server.
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Split-Shard-Error-maxValue-must-be-non-negative-tp4084220p4084256.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Split Shard Error - maxValue must be non-negative
Posted by Srivatsan <ra...@gmail.com>.
Ya i am performing commit after split request is submitted to server.
--
View this message in context: http://lucene.472066.n3.nabble.com/Split-Shard-Error-maxValue-must-be-non-negative-tp4084220p4084256.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Split Shard Error - maxValue must be non-negative
Posted by Robert Muir <rc...@gmail.com>.
did you do a (real) commit before trying to use this?
I am not sure how this splitting works, but at least the merge option
requires that.
i can't see this happening unless you are somehow splitting a 0
document index (or, if the splitter is creating 0 document splits)
so this is likely just a symptom of
https://issues.apache.org/jira/browse/LUCENE-5116
On Tue, Aug 13, 2013 at 6:46 AM, Srivatsan <ra...@gmail.com> wrote:
> Hi,
>
> I am experimenting with solr 4.4.0 split shard feature. When i split the
> shard i am getting the following exception.
>
> /java.lang.IllegalArgumentException: maxValue must be non-negative (got: -1)
> at
> org.apache.lucene.util.packed.PackedInts.bitsRequired(PackedInts.java:1184)
> at
> org.apache.lucene.codecs.lucene42.Lucene42DocValuesConsumer.addNumericField(Lucene42DocValuesConsumer.java:140)
> at
> org.apache.lucene.codecs.lucene42.Lucene42DocValuesConsumer.addNumericField(Lucene42DocValuesConsumer.java:92)
> at
> org.apache.lucene.codecs.DocValuesConsumer.mergeNumericField(DocValuesConsumer.java:112)
> at org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java:221)
> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:119)
> at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:2488)
> at
> org.apache.solr.update.SolrIndexSplitter.split(SolrIndexSplitter.java:125)
> at
> org.apache.solr.update.DirectUpdateHandler2.split(DirectUpdateHandler2.java:766)
> at
> org.apache.solr.handler.admin.CoreAdminHandler.handleSplitAction(CoreAdminHandler.java:284)
> at
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:186)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at
> org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:611)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:209)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
> at org.eclipse.jetty.server.Server.handle(Server.java:368)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
> at
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
> at
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
> at
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
> at
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
> at java.lang.Thread.run(Thread.java:679)/
>
>
> How to resolve this problem?
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Split-Shard-Error-maxValue-must-be-non-negative-tp4084220.html
> Sent from the Solr - User mailing list archive at Nabble.com.