You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Benyi Wang <be...@gmail.com> on 2016/09/08 18:42:42 UTC
Is it safe to change RF in this situation?
* I have a keyspace with RF=2;
* The client read the table using LOCAL_ONE;
* There is a batch job loading data into the tables using ALL.
I want to change RF to 3 and both the client and the batch job use
LOCAL_QUORUM.
My question is "Will the client still read the correct data when the repair
is running at the time my batch job loading is running too?"
Or should I change to LOCAL_QUORUM first?
Thanks.
Re: Is it safe to change RF in this situation?
Posted by Benyi Wang <be...@gmail.com>.
Thanks a lot. Will do as you suggested.
On Thu, Sep 8, 2016 at 3:08 PM, Hannu Kröger <hk...@gmail.com> wrote:
> Ok, so I have to say that I’m not 100% sure how many replicas of data is
> it trying to maintain but it should not blow up (if repair crashes or
> something, it’s ok). So it should be safe to do.
>
> When the repair has run you can start with the plan I suggested and run
> repairs afterwards.
>
> Hannu
>
> On 8 Sep 2016, at 18:01, Benyi Wang <be...@gmail.com> wrote:
>
> Thanks. What about this situation:
>
> * Change RF 2 => 3
> * Start repair
> * Roll back RF 3 => 2
> * repair is still running
>
> I'm wondering what the repair is trying to do? The repair is trying to fix
> as RF=2 or still trying to fix like RF=3?
>
> On Thu, Sep 8, 2016 at 2:53 PM, Hannu Kröger <hk...@gmail.com> wrote:
>
>> Yep, you can fix it by running repair or even faster by changing the
>> consistency level to local_quorum and deploying the new version of the app.
>>
>> Hannu
>>
>> On 8 Sep 2016, at 17:51, Benyi Wang <be...@gmail.com> wrote:
>>
>> Thanks Hannu,
>>
>> Unfortunately, we started changing RF from 2 to 3, and did see the empty
>> result rate is going higher. I assume that "If the LOCAL_ONE read hit the
>> new replica which is not there yet, the CQL query will return nothing." Is
>> my assumption correct?
>>
>> On Thu, Sep 8, 2016 at 11:49 AM, Hannu Kröger <hk...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> If you change RF=2 -> 3 first, the LOCAL_ONE reads might hit the new
>>> replica which is not there yet. So I would change LOCAL_ONE -> LOCAL_QUORUM
>>> first and then change the RF and then run the repair. LOCAL_QUORUM is
>>> effectively ALL in your case (RF=2) if you have just one DC, so you can
>>> change the batch CL later.
>>>
>>> Cheers,
>>> Hannu
>>>
>>> > On 8 Sep 2016, at 14:42, Benyi Wang <be...@gmail.com> wrote:
>>> >
>>> > * I have a keyspace with RF=2;
>>> > * The client read the table using LOCAL_ONE;
>>> > * There is a batch job loading data into the tables using ALL.
>>> >
>>> > I want to change RF to 3 and both the client and the batch job use
>>> LOCAL_QUORUM.
>>> >
>>> > My question is "Will the client still read the correct data when the
>>> repair is running at the time my batch job loading is running too?"
>>> >
>>> > Or should I change to LOCAL_QUORUM first?
>>> >
>>> > Thanks.
>>>
>>>
>>
>>
>
>
Re: Is it safe to change RF in this situation?
Posted by Hannu Kröger <hk...@gmail.com>.
Ok, so I have to say that I’m not 100% sure how many replicas of data is it trying to maintain but it should not blow up (if repair crashes or something, it’s ok). So it should be safe to do.
When the repair has run you can start with the plan I suggested and run repairs afterwards.
Hannu
> On 8 Sep 2016, at 18:01, Benyi Wang <be...@gmail.com> wrote:
>
> Thanks. What about this situation:
>
> * Change RF 2 => 3
> * Start repair
> * Roll back RF 3 => 2
> * repair is still running
>
> I'm wondering what the repair is trying to do? The repair is trying to fix as RF=2 or still trying to fix like RF=3?
>
> On Thu, Sep 8, 2016 at 2:53 PM, Hannu Kröger <hkroger@gmail.com <ma...@gmail.com>> wrote:
> Yep, you can fix it by running repair or even faster by changing the consistency level to local_quorum and deploying the new version of the app.
>
> Hannu
>
>> On 8 Sep 2016, at 17:51, Benyi Wang <bewang.tech@gmail.com <ma...@gmail.com>> wrote:
>>
>> Thanks Hannu,
>>
>> Unfortunately, we started changing RF from 2 to 3, and did see the empty result rate is going higher. I assume that "If the LOCAL_ONE read hit the new replica which is not there yet, the CQL query will return nothing." Is my assumption correct?
>>
>> On Thu, Sep 8, 2016 at 11:49 AM, Hannu Kröger <hkroger@gmail.com <ma...@gmail.com>> wrote:
>> Hi,
>>
>> If you change RF=2 -> 3 first, the LOCAL_ONE reads might hit the new replica which is not there yet. So I would change LOCAL_ONE -> LOCAL_QUORUM first and then change the RF and then run the repair. LOCAL_QUORUM is effectively ALL in your case (RF=2) if you have just one DC, so you can change the batch CL later.
>>
>> Cheers,
>> Hannu
>>
>> > On 8 Sep 2016, at 14:42, Benyi Wang <bewang.tech@gmail.com <ma...@gmail.com>> wrote:
>> >
>> > * I have a keyspace with RF=2;
>> > * The client read the table using LOCAL_ONE;
>> > * There is a batch job loading data into the tables using ALL.
>> >
>> > I want to change RF to 3 and both the client and the batch job use LOCAL_QUORUM.
>> >
>> > My question is "Will the client still read the correct data when the repair is running at the time my batch job loading is running too?"
>> >
>> > Or should I change to LOCAL_QUORUM first?
>> >
>> > Thanks.
>>
>>
>
>
Re: Is it safe to change RF in this situation?
Posted by Benyi Wang <be...@gmail.com>.
Thanks. What about this situation:
* Change RF 2 => 3
* Start repair
* Roll back RF 3 => 2
* repair is still running
I'm wondering what the repair is trying to do? The repair is trying to fix
as RF=2 or still trying to fix like RF=3?
On Thu, Sep 8, 2016 at 2:53 PM, Hannu Kröger <hk...@gmail.com> wrote:
> Yep, you can fix it by running repair or even faster by changing the
> consistency level to local_quorum and deploying the new version of the app.
>
> Hannu
>
> On 8 Sep 2016, at 17:51, Benyi Wang <be...@gmail.com> wrote:
>
> Thanks Hannu,
>
> Unfortunately, we started changing RF from 2 to 3, and did see the empty
> result rate is going higher. I assume that "If the LOCAL_ONE read hit the
> new replica which is not there yet, the CQL query will return nothing." Is
> my assumption correct?
>
> On Thu, Sep 8, 2016 at 11:49 AM, Hannu Kröger <hk...@gmail.com> wrote:
>
>> Hi,
>>
>> If you change RF=2 -> 3 first, the LOCAL_ONE reads might hit the new
>> replica which is not there yet. So I would change LOCAL_ONE -> LOCAL_QUORUM
>> first and then change the RF and then run the repair. LOCAL_QUORUM is
>> effectively ALL in your case (RF=2) if you have just one DC, so you can
>> change the batch CL later.
>>
>> Cheers,
>> Hannu
>>
>> > On 8 Sep 2016, at 14:42, Benyi Wang <be...@gmail.com> wrote:
>> >
>> > * I have a keyspace with RF=2;
>> > * The client read the table using LOCAL_ONE;
>> > * There is a batch job loading data into the tables using ALL.
>> >
>> > I want to change RF to 3 and both the client and the batch job use
>> LOCAL_QUORUM.
>> >
>> > My question is "Will the client still read the correct data when the
>> repair is running at the time my batch job loading is running too?"
>> >
>> > Or should I change to LOCAL_QUORUM first?
>> >
>> > Thanks.
>>
>>
>
>
Re: Is it safe to change RF in this situation?
Posted by Hannu Kröger <hk...@gmail.com>.
Yep, you can fix it by running repair or even faster by changing the consistency level to local_quorum and deploying the new version of the app.
Hannu
> On 8 Sep 2016, at 17:51, Benyi Wang <be...@gmail.com> wrote:
>
> Thanks Hannu,
>
> Unfortunately, we started changing RF from 2 to 3, and did see the empty result rate is going higher. I assume that "If the LOCAL_ONE read hit the new replica which is not there yet, the CQL query will return nothing." Is my assumption correct?
>
> On Thu, Sep 8, 2016 at 11:49 AM, Hannu Kröger <hkroger@gmail.com <ma...@gmail.com>> wrote:
> Hi,
>
> If you change RF=2 -> 3 first, the LOCAL_ONE reads might hit the new replica which is not there yet. So I would change LOCAL_ONE -> LOCAL_QUORUM first and then change the RF and then run the repair. LOCAL_QUORUM is effectively ALL in your case (RF=2) if you have just one DC, so you can change the batch CL later.
>
> Cheers,
> Hannu
>
> > On 8 Sep 2016, at 14:42, Benyi Wang <bewang.tech@gmail.com <ma...@gmail.com>> wrote:
> >
> > * I have a keyspace with RF=2;
> > * The client read the table using LOCAL_ONE;
> > * There is a batch job loading data into the tables using ALL.
> >
> > I want to change RF to 3 and both the client and the batch job use LOCAL_QUORUM.
> >
> > My question is "Will the client still read the correct data when the repair is running at the time my batch job loading is running too?"
> >
> > Or should I change to LOCAL_QUORUM first?
> >
> > Thanks.
>
>
Re: Is it safe to change RF in this situation?
Posted by Benyi Wang <be...@gmail.com>.
Thanks Hannu,
Unfortunately, we started changing RF from 2 to 3, and did see the empty
result rate is going higher. I assume that "If the LOCAL_ONE read hit the
new replica which is not there yet, the CQL query will return nothing." Is
my assumption correct?
On Thu, Sep 8, 2016 at 11:49 AM, Hannu Kröger <hk...@gmail.com> wrote:
> Hi,
>
> If you change RF=2 -> 3 first, the LOCAL_ONE reads might hit the new
> replica which is not there yet. So I would change LOCAL_ONE -> LOCAL_QUORUM
> first and then change the RF and then run the repair. LOCAL_QUORUM is
> effectively ALL in your case (RF=2) if you have just one DC, so you can
> change the batch CL later.
>
> Cheers,
> Hannu
>
> > On 8 Sep 2016, at 14:42, Benyi Wang <be...@gmail.com> wrote:
> >
> > * I have a keyspace with RF=2;
> > * The client read the table using LOCAL_ONE;
> > * There is a batch job loading data into the tables using ALL.
> >
> > I want to change RF to 3 and both the client and the batch job use
> LOCAL_QUORUM.
> >
> > My question is "Will the client still read the correct data when the
> repair is running at the time my batch job loading is running too?"
> >
> > Or should I change to LOCAL_QUORUM first?
> >
> > Thanks.
>
>
Re: Is it safe to change RF in this situation?
Posted by Hannu Kröger <hk...@gmail.com>.
Hi,
If you change RF=2 -> 3 first, the LOCAL_ONE reads might hit the new replica which is not there yet. So I would change LOCAL_ONE -> LOCAL_QUORUM first and then change the RF and then run the repair. LOCAL_QUORUM is effectively ALL in your case (RF=2) if you have just one DC, so you can change the batch CL later.
Cheers,
Hannu
> On 8 Sep 2016, at 14:42, Benyi Wang <be...@gmail.com> wrote:
>
> * I have a keyspace with RF=2;
> * The client read the table using LOCAL_ONE;
> * There is a batch job loading data into the tables using ALL.
>
> I want to change RF to 3 and both the client and the batch job use LOCAL_QUORUM.
>
> My question is "Will the client still read the correct data when the repair is running at the time my batch job loading is running too?"
>
> Or should I change to LOCAL_QUORUM first?
>
> Thanks.