You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Varun Thacker <va...@vthacker.in> on 2018/06/14 21:46:00 UTC
How do we interpret replicationFactor ?
While working on SOLR-11676
<https://issues.apache.org/jira/browse/SOLR-11676> a few questions came
that were't obvious
Should a user be allowed to specify replicationFactor and nrtReplicas ?
Today it's possible but my answer was it shouldn't be. What do others
think?
If everyone agrees the two shouldn't be specified then there is one problem
while fixing this - SolrJ
if (nrtReplicas != null) {
params.set( ZkStateReader.REPLICATION_FACTOR, nrtReplicas);// Keep
both for compatibility?
params.set( ZkStateReader.NRT_REPLICAS, nrtReplicas);
}
SolrJ sets both replicationFactor and nrtReplicas with the same value.
So if we simply put a check at the server saying "don't allow both
parameters" all SolrJ calls from older clients will fail
The compromise would be the server could check if both nrtReplicas and
replicationFactor are equal then don't error out
Second question, SolrJ doesn't allow a user to specify
replicationFactor but if you're using the API directly it's allowed.
Do we plan on deprecating replicationFactor eventually in favour of
nrtReplicas ? If yes would 7.5 be a good place to start throwing a
warning ?
Re: How do we interpret replicationFactor ?
Posted by Varun Thacker <va...@vthacker.in>.
If someone has 5 minutes could they please review my approach taken
in SOLR-11676
On Fri, Jun 15, 2018 at 12:24 PM, Varun Thacker <va...@vthacker.in> wrote:
> Thanks Tomás
>
> The approach I'm taking is SolrJ never sets replicationFactor and keep
> back-compat for older clients who would set both replicationFactor and
> nrtReplicas for the same thing
>
> I'm not going to remove it from cluster state just yet ( even with keeping
> back-compat ) . I'm thinking this parameter could mean an overarching
> replicationFactor ( used internally ) which would be a sum of all the
> replica types . We could use this info internally while external users
> would not be able to set it in the future
>
> On Fri, Jun 15, 2018 at 10:06 AM, Tomás Fernández Löbbe <
> tomasflobbe@gmail.com> wrote:
>
>> I think we should deprecate it. There were some concerns about this
>> because new users would understand quickly what "replicationFactor" is,
>> while "nrtReplicas" is not so intuitive, but I don't like having two ways
>> to do the same, and now that there are multiple types of replicas I think
>> it's better for the parameter to be explicit.
>> I would still keep accepting the parameter for backwards compatibility,
>> but maybe remove the internal use of it? Maybe even remove it from the
>> clusterstate (and again, make sure we can still read cluster states that
>> have it, for upgrades).
>>
>> On Thu, Jun 14, 2018 at 2:46 PM, Varun Thacker <va...@vthacker.in> wrote:
>>
>>> While working on SOLR-11676
>>> <https://issues.apache.org/jira/browse/SOLR-11676> a few questions came
>>> that were't obvious
>>>
>>> Should a user be allowed to specify replicationFactor and nrtReplicas ?
>>> Today it's possible but my answer was it shouldn't be. What do others
>>> think?
>>>
>>> If everyone agrees the two shouldn't be specified then there is one
>>> problem while fixing this - SolrJ
>>>
>>> if (nrtReplicas != null) {
>>> params.set( ZkStateReader.REPLICATION_FACTOR, nrtReplicas);// Keep both for compatibility?
>>> params.set( ZkStateReader.NRT_REPLICAS, nrtReplicas);
>>> }
>>>
>>> SolrJ sets both replicationFactor and nrtReplicas with the same value. So if we simply put a check at the server saying "don't allow both parameters" all SolrJ calls from older clients will fail
>>>
>>> The compromise would be the server could check if both nrtReplicas and replicationFactor are equal then don't error out
>>>
>>>
>>> Second question, SolrJ doesn't allow a user to specify replicationFactor but if you're using the API directly it's allowed.
>>>
>>> Do we plan on deprecating replicationFactor eventually in favour of nrtReplicas ? If yes would 7.5 be a good place to start throwing a warning ?
>>>
>>>
>>>
>>
>
Re: How do we interpret replicationFactor ?
Posted by Varun Thacker <va...@vthacker.in>.
Thanks Tomás
The approach I'm taking is SolrJ never sets replicationFactor and keep
back-compat for older clients who would set both replicationFactor and
nrtReplicas for the same thing
I'm not going to remove it from cluster state just yet ( even with keeping
back-compat ) . I'm thinking this parameter could mean an overarching
replicationFactor ( used internally ) which would be a sum of all the
replica types . We could use this info internally while external users
would not be able to set it in the future
On Fri, Jun 15, 2018 at 10:06 AM, Tomás Fernández Löbbe <
tomasflobbe@gmail.com> wrote:
> I think we should deprecate it. There were some concerns about this
> because new users would understand quickly what "replicationFactor" is,
> while "nrtReplicas" is not so intuitive, but I don't like having two ways
> to do the same, and now that there are multiple types of replicas I think
> it's better for the parameter to be explicit.
> I would still keep accepting the parameter for backwards compatibility,
> but maybe remove the internal use of it? Maybe even remove it from the
> clusterstate (and again, make sure we can still read cluster states that
> have it, for upgrades).
>
> On Thu, Jun 14, 2018 at 2:46 PM, Varun Thacker <va...@vthacker.in> wrote:
>
>> While working on SOLR-11676
>> <https://issues.apache.org/jira/browse/SOLR-11676> a few questions came
>> that were't obvious
>>
>> Should a user be allowed to specify replicationFactor and nrtReplicas ?
>> Today it's possible but my answer was it shouldn't be. What do others
>> think?
>>
>> If everyone agrees the two shouldn't be specified then there is one
>> problem while fixing this - SolrJ
>>
>> if (nrtReplicas != null) {
>> params.set( ZkStateReader.REPLICATION_FACTOR, nrtReplicas);// Keep both for compatibility?
>> params.set( ZkStateReader.NRT_REPLICAS, nrtReplicas);
>> }
>>
>> SolrJ sets both replicationFactor and nrtReplicas with the same value. So if we simply put a check at the server saying "don't allow both parameters" all SolrJ calls from older clients will fail
>>
>> The compromise would be the server could check if both nrtReplicas and replicationFactor are equal then don't error out
>>
>>
>> Second question, SolrJ doesn't allow a user to specify replicationFactor but if you're using the API directly it's allowed.
>>
>> Do we plan on deprecating replicationFactor eventually in favour of nrtReplicas ? If yes would 7.5 be a good place to start throwing a warning ?
>>
>>
>>
>
Re: How do we interpret replicationFactor ?
Posted by Tomás Fernández Löbbe <to...@gmail.com>.
I think we should deprecate it. There were some concerns about this because
new users would understand quickly what "replicationFactor" is, while
"nrtReplicas" is not so intuitive, but I don't like having two ways to do
the same, and now that there are multiple types of replicas I think it's
better for the parameter to be explicit.
I would still keep accepting the parameter for backwards compatibility, but
maybe remove the internal use of it? Maybe even remove it from the
clusterstate (and again, make sure we can still read cluster states that
have it, for upgrades).
On Thu, Jun 14, 2018 at 2:46 PM, Varun Thacker <va...@vthacker.in> wrote:
> While working on SOLR-11676
> <https://issues.apache.org/jira/browse/SOLR-11676> a few questions came
> that were't obvious
>
> Should a user be allowed to specify replicationFactor and nrtReplicas ?
> Today it's possible but my answer was it shouldn't be. What do others
> think?
>
> If everyone agrees the two shouldn't be specified then there is one
> problem while fixing this - SolrJ
>
> if (nrtReplicas != null) {
> params.set( ZkStateReader.REPLICATION_FACTOR, nrtReplicas);// Keep both for compatibility?
> params.set( ZkStateReader.NRT_REPLICAS, nrtReplicas);
> }
>
> SolrJ sets both replicationFactor and nrtReplicas with the same value. So if we simply put a check at the server saying "don't allow both parameters" all SolrJ calls from older clients will fail
>
> The compromise would be the server could check if both nrtReplicas and replicationFactor are equal then don't error out
>
>
> Second question, SolrJ doesn't allow a user to specify replicationFactor but if you're using the API directly it's allowed.
>
> Do we plan on deprecating replicationFactor eventually in favour of nrtReplicas ? If yes would 7.5 be a good place to start throwing a warning ?
>
>
>