You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@solr.apache.org by samurai blade <th...@gmail.com> on 2021/10/22 01:00:12 UTC

SplitShard Bug in Solr 8.4.1

Hi Dev team,


We have an issue for one of our teams in production.

When they fire a particular query say *id:xyz AND product:def AND
category:mno* they are getting an older document from say shard1.

When they fire the same query after removing category, i.e. *id:xyz AND
product:def*, they are getting the newly updated document from say shard2.

The document with *id:xyz* was recently updated.


This happened after there was a split shard fired on the shard
with splitMethod=link


Can you please tell us how to solve this as we have billions of such
documents.


Thanks for your help

Re: SplitShard Bug in Solr 8.4.1

Posted by David Smiley <ds...@apache.org>.
It's a little hard to follow what you're saying and I'm unsure how you came
to certain conclusions (e.g. that a doc was served from a particular
shard).  You could try to find an existing bug report, and failing that
then maybe file a new one, being very clear on what operations were
performed and what you observed.  But to be honest, debugging this would be
hard and I wouldn't anticipate any help.

FYI (and to fellow committers here): Where I work, we do shard splitting a
lot.  We've also customized Solr a ton for many reasons.  One thing in
particular that is pertinent here is that when a shard split starts, we
modified Solr to throw exceptions when updates come in until the shard
split has completed.  My choice would have been to block, but I digress.
This removes a whole class of complications within Solr to buffer and
replay these updates.  You probably ran into a bug over there and I'd
pity someone with the task of root-causing that.  Shard
splitting's first implication was splitMethod=rewrite (expensive) but now
it has "link" which I think is superior and can accomplish the important
part very fast.  Had Solr started with "link", I think we'd have blocked
updates and not gone through the complexity of buffering & replaying.  Sigh.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Thu, Oct 21, 2021 at 9:42 PM samurai blade <th...@gmail.com>
wrote:

> Hi Dev team,
>
>
> We have an issue for one of our teams in production.
>
> When they fire a particular query say *id:xyz AND product:def AND
> category:mno* they are getting an older document from say shard1.
>
> When they fire the same query after removing category, i.e. *id:xyz AND
> product:def*, they are getting the newly updated document from say shard2.
>
> The document with *id:xyz* was recently updated.
>
>
> This happened after there was a split shard fired on the shard
> with splitMethod=link
>
>
> Can you please tell us how to solve this as we have billions of such
> documents.
>
>
> Thanks for your help
>