You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Konrad <ko...@123mail.org> on 2018/07/26 12:04:03 UTC
Lost counter updates during Cassandra upgrade 2.2.11 to 3.11.2
Hi,
During rolling upgrade of our cluster we noticed that some updates on table with counters were not being applied. It looked as if it depended on whether coordinator handling request was already upgraded or not. I observed similar behavior while using cqlsh and executing queries manually. Sometimes it took several retries to see counter updated. There were no errors/warns in neither application nor Cassandra logs. The updates started working reliably once again when all nodes in dc have been upgraded. However, the lost updates did not reappear.
Our setup:
2 dc cluster, 5 + 5 nodes. However, only one is used for queries as client application is co-located in one region. I believe 1 dc is enough to reproduce it.
Replication factor 3+2
Consistency level LOCAL_QUORUM
Upgrading 2.2.11 to 3.11.2
I haven't found any report of similar issue on the internet. Has anyone heard about such behavior?
Thanks,
Konrad
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
For additional commands, e-mail: user-help@cassandra.apache.org
Re: Lost counter updates during Cassandra upgrade 2.2.11 to 3.11.2
Posted by Konrad <ko...@123mail.org>.
Hi,
I haven't investigated the issue further.
--
Konrad
On Sat, Nov 10, 2018, at 05:49, Laxmikant Upadhyay wrote:
> Hi,
>
> I have faced similar issue while upgrading from 2.1.16 -> 3.11.2 in a
> 3 node cluster.> I have raised jira ticket CASSANDRA-14881[1] for this issue , but have
> not got any response on this yet.> @Konrad did you get any resolution on this ?
>
> Regards,
> Laxmikant
>
>
>
>
> On Thu, Jul 26, 2018 at 5:34 PM Konrad <ko...@123mail.org> wrote:>> Hi,
>>
>> During rolling upgrade of our cluster we noticed that some updates
>> on table with counters were not being applied. It looked as if it
>> depended on whether coordinator handling request was already
>> upgraded or not. I observed similar behavior while using cqlsh and
>> executing queries manually. Sometimes it took several retries to see
>> counter updated. There were no errors/warns in neither application
>> nor Cassandra logs. The updates started working reliably once again
>> when all nodes in dc have been upgraded. However, the lost updates
>> did not reappear.>>
>> Our setup:
>> 2 dc cluster, 5 + 5 nodes. However, only one is used for queries as
>> client application is co-located in one region. I believe 1 dc is
>> enough to reproduce it.>> Replication factor 3+2
>> Consistency level LOCAL_QUORUM
>> Upgrading 2.2.11 to 3.11.2
>>
>> I haven't found any report of similar issue on the internet. Has
>> anyone heard about such behavior?>>
>> Thanks,
>> Konrad
>>
>> ---------------------------------------------------------------
>> ------>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
>> For additional commands, e-mail: user-help@cassandra.apache.org
>>
>
>
> --
>
> regards,
> Laxmikant Upadhyay
>
Links:
1. https://issues.apache.org/jira/browse/CASSANDRA-14881
Re: Lost counter updates during Cassandra upgrade 2.2.11 to 3.11.2
Posted by Laxmikant Upadhyay <la...@gmail.com>.
Hi,
I have faced similar issue while upgrading from 2.1.16 -> 3.11.2 in a 3
node cluster.
I have raised jira ticket CASSANDRA-14881
<https://issues.apache.org/jira/browse/CASSANDRA-14881> for this issue ,
but have not got any response on this yet.
@Konrad did you get any resolution on this ?
Regards,
Laxmikant
On Thu, Jul 26, 2018 at 5:34 PM Konrad <ko...@123mail.org> wrote:
> Hi,
>
> During rolling upgrade of our cluster we noticed that some updates on
> table with counters were not being applied. It looked as if it depended on
> whether coordinator handling request was already upgraded or not. I
> observed similar behavior while using cqlsh and executing queries manually.
> Sometimes it took several retries to see counter updated. There were no
> errors/warns in neither application nor Cassandra logs. The updates started
> working reliably once again when all nodes in dc have been upgraded.
> However, the lost updates did not reappear.
>
> Our setup:
> 2 dc cluster, 5 + 5 nodes. However, only one is used for queries as client
> application is co-located in one region. I believe 1 dc is enough to
> reproduce it.
> Replication factor 3+2
> Consistency level LOCAL_QUORUM
> Upgrading 2.2.11 to 3.11.2
>
> I haven't found any report of similar issue on the internet. Has anyone
> heard about such behavior?
>
> Thanks,
> Konrad
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org
> For additional commands, e-mail: user-help@cassandra.apache.org
>
>
--
regards,
Laxmikant Upadhyay