Huh? What does this even mean? Not enough time left to update replicas. However, the schema is updated already.

[Michael Joyner <mi...@newsrx.com>]

Huh? What does this even mean? If the schema is updated already how can 
we be out of time to update it?

Not enough time left to update replicas. However, the schema is updated 
already.

Re: Huh? What does this even mean? Not enough time left to update replicas. However, the schema is updated already.

[Erick Erickson <er...@gmail.com>]

Well, managed schema in SolrCloud is a bit heavy-weight.
When you change the schema, two things need to happen:
1> the change has to be pushed to ZooKeeper
2> the replicas in the collection need to be reloaded to make
    the changes available to all replicas for the _next_ doc that
    comes in.

So what I think that message means is that the changes have
been propagated to ZooKeeper but the replicas haven't all
reloaded.

Best,
Erick

On Thu, Feb 9, 2017 at 11:29 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 2/9/2017 10:29 AM, Michael Joyner wrote:
>>
>> Huh? What does this even mean? If the schema is updated already how
>> can we be out of time to update it?
>>
>> Not enough time left to update replicas. However, the schema is
>> updated already.
>
> The code where the waitForOtherReplicasToUpdate method (that throws the
> exception) is called has this comment:
>
>       // Don't block further schema updates while waiting for a pending
> update to propagate to other replicas.
>       // This reduces the likelihood of a (time-limited) distributed
> deadlock during concurrent schema updates.
>
> Looks like it's part of the code that lets you manage your schema with
> HTTP calls.
>
> I don't really know what kinds of potential problems this code is
> designed to deal with, but it sounds like your Solr servers are having
> performance issues causing operations that are normally very fast to
> take a very long time instead.  One common cause for this is setting the
> heap too small, so that Java is forced to engage in nearly constant full
> garbage collections.
>
> The default timeout in this situation is ten minutes, unless you
> explicitly configure it to be different, which is probably done in the
> config for the managed schema.  In order for a ten minute timeout to be
> exceeded, the performance problem would be VERY severe ... to the point
> where I would be surprised if you can even get query results from Solr
> at all.
>
> Thanks,
> Shawn
>

Re: Huh? What does this even mean? Not enough time left to update replicas. However, the schema is updated already.

[Shawn Heisey <ap...@elyograg.org>]

On 2/9/2017 10:29 AM, Michael Joyner wrote:
>
> Huh? What does this even mean? If the schema is updated already how
> can we be out of time to update it?
>
> Not enough time left to update replicas. However, the schema is
> updated already.

The code where the waitForOtherReplicasToUpdate method (that throws the
exception) is called has this comment:

      // Don't block further schema updates while waiting for a pending
update to propagate to other replicas.
      // This reduces the likelihood of a (time-limited) distributed
deadlock during concurrent schema updates.

Looks like it's part of the code that lets you manage your schema with
HTTP calls.

I don't really know what kinds of potential problems this code is
designed to deal with, but it sounds like your Solr servers are having
performance issues causing operations that are normally very fast to
take a very long time instead.  One common cause for this is setting the
heap too small, so that Java is forced to engage in nearly constant full
garbage collections.

The default timeout in this situation is ten minutes, unless you
explicitly configure it to be different, which is probably done in the
config for the managed schema.  In order for a ten minute timeout to be
exceeded, the performance problem would be VERY severe ... to the point
where I would be surprised if you can even get query results from Solr
at all.

Thanks,
Shawn