You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Javier Pareja <pa...@gmail.com> on 2018/03/05 11:55:14 UTC

How do counter updates work?

Hello everyone,

I am trying to understand how cassandra counter writes work in more detail
but all that I could find is this:
https://www.datastax.com/dev/blog/whats-new-in-cassandra-2-1-a-better-implementation-of-counters
From there I was able to extract the following process:
(click here to edit
<https://www.draw.io/?lightbox=1&highlight=0000ff&edit=_blank&layers=1&nav=1&title=Untitled%20Diagram.xml#R7VpRc6M2EP41nmkfcgPCYPKYOPHdTXudzCXtNY8yCKMGkCvk2LlfXwkkjATYjo0dp5O8BK2kZdldfftJ8sAZp6vPFM7jbyREyQBY4Wrg3AwAsIfA4%2F%2BE5KWU%2BP5lKZhRHMpBa8E9%2Fomk0JLSBQ5Rrg1khCQMz3VhQLIMBUyTQUrJUh8WkUR%2F6xzOUENwH8CkKf2BQxZLqW1Z644vCM9i%2BWrflR1TGDzNKFlk8n0D4ETFX9mdQqVLjs9jGJJlTeTcDpwxJYSVT%2BlqjBLhW%2BW2ct6ko7eym6KM7TRhhKDnjZDtBoCbGlwAXxrGXpQzUMh9I5uEspjMSAaT27X0uvhgJFTavBWzNJGPCZyi5LryyZgkhPKujGRiWs4gZVciXIZsghOhwVJtmSAub6MsVDOCBOY5Dh5inJUdcppdtmqT%2FkGMvcg2XDDCResP%2BZ2QuZyVM0qekLKSx84q%2FqoelQtibEQyNoEpTkSK%2F4VoCDMoxfJNvmy26ePhoS9%2Fi2%2F85Krmo%2FxktMKs1sVbj%2FKdzejKgOdkQQOVu45cMJDOkBwGSpGIZG2eTInPiKSIv58PoCiBDD%2FrqwDKxTSrxlVT7wjmlgBLLnwwlGktl%2F3Qt3QVpZ1yVj0tDUVDy1DkGorKj2so4qkBX2rD5mJA%2FgqDL62Ndjmbx%2FOH0gLVqjl3LSqWYPtydEvtzzBZIAUgB6xGS1%2BNuy6DTcnZmkjbc9KEGQXqtRy1vQNzshaFjU7uMqbm9dsVChYMceGYO5Mhyp%2F%2BnIdQiLyEG3w95SJvxiqP1CLEMX0uHmO0gty1fMgcUczNRHQtvVuLrpcxZuh%2BDgtvLXlF1cOmx7OqGNYO%2BPoKOOPYWRspi9YuMGeDDTgHEzzLBFajrPzWznR5RpSh1WATOqnekbEIZXNZq9RKFteLtHWE3AGN3PlBcZE5HDgI%2F%2FcNpQxOeW6YaaIXzS05cEjUvcBH02iXqIcQ%2BVFwnlH39aC7oCXqoCXqwx6ivjXIY5KmmCVk9nZRDkeX013XNvKC40W5EdKWwHdG%2BU3D7HxQ3zOnvq%2FmHS1cuB3Cz5gL24Yi7zy4sH1kLmxfnhcZtrVkrRK0z3QdtmSrc5TsHAI9eI6ZnR1JtUcchy1hlBxaUejvCIYDYVEp5xqrrmBBC5cpBcCKKEn5vxucPx1WbbvKYndp7aHAOZ7ueOCdsMB5jVBMyqOqSdOTMUmni3y7F%2Fso%2BsZuv43R%2B0ci9B%2FHXSep%2BT0D5aFHBR0V0DXIp7nmdi3ZZil1Lo%2BGrs0i%2BYjyRk7ztcmMQwWU45%2FFvrRMPMkO%2BGj3euDetNF7scZxAJMr2ZHiMCzWQ0fa148ROlFCnuBLSwbVwXg9AfyN6HFhffKA7%2BkOL1v75oMaQqIoR4dGSNG3D4g5CcTYNYAp4eacIcZx9oSYBo%2FYkcC9dldQGWi8p9OuzeMP3xU0D2v%2FIP8zvLOtwbsGPKcRoq9ZQFGq8%2Fip%2BCQWi8eFPFo3YTJJ8DxH2zkozOflLXCEVwI0j0FKHd89GSm1Rw0PHlRD9tz67gectszM%2Bi72ZFc7zR3OQ%2FGmxpaT7rHlXN8FjWEQH3ii33X03n1830NSm1cndtvus%2B3upJdTdKsRnO9X37jglwnM2a8b3GmdqTvNa2q%2FxZtHu5NoFsKbr%2Fe%2FCXfeJ2TZrztPczZiutNuu9k7mj%2FBdszlyPBcre8mb9ZdqBNcQURDmMfVdNG4g4zjSVZIgLVfGs8oDDHSOLE44EavuaxvRqTmcbfF4Up2IBu2DRJrbpN33m8bpLO6NeqZDJv2KrLbeURu3GYNnX7JMGgyrUahUwKxb9Ky2ft3QVTHRV6wgSuRhcP5at2ptNxdPXwpUnRdL0uF%2BkvqZXSHX0q8b5peAsQmmm6NvKFeIHpZNmJbW9d64QBdRS80vvXg%2Fq0BMYqCEPgbC1ELIEYRdK33AIgO6AkQvdMAommvve10wOCfYAuAuoZ%2B5eddxyv7Ov10aYzv%2BQ6z9Qd9xwRo8AHQNYAevhFAA03n6HBw5s31b8LL4esf3ju3%2FwE%3D>
).


PATH 1 will be much quicker than PATH 2 and its bottleneck (assuming HDD
drives) will be the commitlog
PATH 2 will need at least an access to disk to do a read (potentially even
in a different machine) and an access to disk to do a write to the
commitlog. This is at least twice as slow as PATH 1.

This is all the info that I could get from the internet but a lot is
missing. For example, there is no information about how the counter lock is
acquired, is there a shared lock across all the nodes?

Hope I am not oversimplifying things, but I think this will be useful to
better understand how to tune up the system.

Thanks in advance.

F Javier Pareja

Re: How do counter updates work?

Posted by Hannu Kröger <hk...@gmail.com>.
So just to clarify we have two different use cases:
- TIMEUUID is there for client side generation of unique row ids. It’s great for that.
- Cassandra counters are not very good for row id generation and suited better to e.g. those use cases I listed before

Hannu


> On 5 Mar 2018, at 16:34, Javier Pareja <pa...@gmail.com> wrote:
> 
> Doesn't cassandra have TIMEUUID for these use cases?
> 
> Anyways, hopefully someone can help me better understand possible delays when writing a counter.
> 
> F Javier Pareja
> 
> On Mon, Mar 5, 2018 at 1:54 PM, Hannu Kröger <hkroger@gmail.com <ma...@gmail.com>> wrote:
> Traditionally auto increment counters have been used to generate SQL row IDs. This is what Kyrylo probably is here referring to.
> 
> Cassandra counters are better tracking e.g. usage patterns, web site visitors, statistics, etc. 
> 
> For accurate counting (e.g. for generating IDs) those counters are not good because they are inaccurate in certain cases.
> 
> Hannu
> 
>> On 5 Mar 2018, at 15:50, Javier Pareja <pareja.javier@gmail.com <ma...@gmail.com>> wrote:
>> 
>> Hi Kyrulo,
>> 
>> I don't understand how UUIDs are related to counters, but I use counters to increment the value of a cell in an atomic manner. I could try reading the value and then writing to the cell but then I would lose the atomicity of the update.
>> 
>> F Javier Pareja
>> 
>> On Mon, Mar 5, 2018 at 1:32 PM, Kyrylo Lebediev <Kyrylo_Lebediev@epam.com <ma...@epam.com>> wrote:
>> Hello!
>> 
>> Can't answer your question but there is another one: "why do we need to maintain counters with their known limitations (and I've heard of some issues with implementation of counters in Cassandra), when there exist really effective uuid generation algorithms which allow us to generate unique values?"
>> https://begriffs.com/posts/2018-01-01-sql-keys-in-depth.html <https://begriffs.com/posts/2018-01-01-sql-keys-in-depth.html>
>>  <https://begriffs.com/posts/2018-01-01-sql-keys-in-depth.html>(the article is about keys in RDBMS's, but its statements are true for NoSQL as well)
>> 
>> Regards, 
>> Kyrill
>> From: jpare86@gmail.com <ma...@gmail.com> <jpare86@gmail.com <ma...@gmail.com>> on behalf of Javier Pareja <pareja.javier@gmail.com <ma...@gmail.com>>
>> Sent: Monday, March 5, 2018 1:55:14 PM
>> To: user@cassandra.apache.org <ma...@cassandra.apache.org>
>> Subject: How do counter updates work?
>>  
>> Hello everyone,
>> 
>> I am trying to understand how cassandra counter writes work in more detail but all that I could find is this: https://www.datastax.com/dev/blog/whats-new-in-cassandra-2-1-a-better-implementation-of-counters <https://www.datastax.com/dev/blog/whats-new-in-cassandra-2-1-a-better-implementation-of-counters>
>> From there I was able to extract the following process:
>> (click here to edit <https://www.draw.io/?lightbox=1&highlight=0000ff&edit=_blank&layers=1&nav=1&title=Untitled%20Diagram.xml#R7VpRc6M2EP41nmkfcgPCYPKYOPHdTXudzCXtNY8yCKMGkCvk2LlfXwkkjATYjo0dp5O8BK2kZdldfftJ8sAZp6vPFM7jbyREyQBY4Wrg3AwAsIfA4%2F%2BE5KWU%2BP5lKZhRHMpBa8E9%2Fomk0JLSBQ5Rrg1khCQMz3VhQLIMBUyTQUrJUh8WkUR%2F6xzOUENwH8CkKf2BQxZLqW1Z644vCM9i%2BWrflR1TGDzNKFlk8n0D4ETFX9mdQqVLjs9jGJJlTeTcDpwxJYSVT%2BlqjBLhW%2BW2ct6ko7eym6KM7TRhhKDnjZDtBoCbGlwAXxrGXpQzUMh9I5uEspjMSAaT27X0uvhgJFTavBWzNJGPCZyi5LryyZgkhPKujGRiWs4gZVciXIZsghOhwVJtmSAub6MsVDOCBOY5Dh5inJUdcppdtmqT%2FkGMvcg2XDDCResP%2BZ2QuZyVM0qekLKSx84q%2FqoelQtibEQyNoEpTkSK%2F4VoCDMoxfJNvmy26ePhoS9%2Fi2%2F85Krmo%2FxktMKs1sVbj%2FKdzejKgOdkQQOVu45cMJDOkBwGSpGIZG2eTInPiKSIv58PoCiBDD%2FrqwDKxTSrxlVT7wjmlgBLLnwwlGktl%2F3Qt3QVpZ1yVj0tDUVDy1DkGorKj2so4qkBX2rD5mJA%2FgqDL62Ndjmbx%2FOH0gLVqjl3LSqWYPtydEvtzzBZIAUgB6xGS1%2BNuy6DTcnZmkjbc9KEGQXqtRy1vQNzshaFjU7uMqbm9dsVChYMceGYO5Mhyp%2F%2BnIdQiLyEG3w95SJvxiqP1CLEMX0uHmO0gty1fMgcUczNRHQtvVuLrpcxZuh%2BDgtvLXlF1cOmx7OqGNYO%2BPoKOOPYWRspi9YuMGeDDTgHEzzLBFajrPzWznR5RpSh1WATOqnekbEIZXNZq9RKFteLtHWE3AGN3PlBcZE5HDgI%2F%2FcNpQxOeW6YaaIXzS05cEjUvcBH02iXqIcQ%2BVFwnlH39aC7oCXqoCXqwx6ivjXIY5KmmCVk9nZRDkeX013XNvKC40W5EdKWwHdG%2BU3D7HxQ3zOnvq%2FmHS1cuB3Cz5gL24Yi7zy4sH1kLmxfnhcZtrVkrRK0z3QdtmSrc5TsHAI9eI6ZnR1JtUcchy1hlBxaUejvCIYDYVEp5xqrrmBBC5cpBcCKKEn5vxucPx1WbbvKYndp7aHAOZ7ueOCdsMB5jVBMyqOqSdOTMUmni3y7F%2Fso%2BsZuv43R%2B0ci9B%2FHXSep%2BT0D5aFHBR0V0DXIp7nmdi3ZZil1Lo%2BGrs0i%2BYjyRk7ztcmMQwWU45%2FFvrRMPMkO%2BGj3euDetNF7scZxAJMr2ZHiMCzWQ0fa148ROlFCnuBLSwbVwXg9AfyN6HFhffKA7%2BkOL1v75oMaQqIoR4dGSNG3D4g5CcTYNYAp4eacIcZx9oSYBo%2FYkcC9dldQGWi8p9OuzeMP3xU0D2v%2FIP8zvLOtwbsGPKcRoq9ZQFGq8%2Fip%2BCQWi8eFPFo3YTJJ8DxH2zkozOflLXCEVwI0j0FKHd89GSm1Rw0PHlRD9tz67gectszM%2Bi72ZFc7zR3OQ%2FGmxpaT7rHlXN8FjWEQH3ii33X03n1830NSm1cndtvus%2B3upJdTdKsRnO9X37jglwnM2a8b3GmdqTvNa2q%2FxZtHu5NoFsKbr%2Fe%2FCXfeJ2TZrztPczZiutNuu9k7mj%2FBdszlyPBcre8mb9ZdqBNcQURDmMfVdNG4g4zjSVZIgLVfGs8oDDHSOLE44EavuaxvRqTmcbfF4Up2IBu2DRJrbpN33m8bpLO6NeqZDJv2KrLbeURu3GYNnX7JMGgyrUahUwKxb9Ky2ft3QVTHRV6wgSuRhcP5at2ptNxdPXwpUnRdL0uF%2BkvqZXSHX0q8b5peAsQmmm6NvKFeIHpZNmJbW9d64QBdRS80vvXg%2Fq0BMYqCEPgbC1ELIEYRdK33AIgO6AkQvdMAommvve10wOCfYAuAuoZ%2B5eddxyv7Ov10aYzv%2BQ6z9Qd9xwRo8AHQNYAevhFAA03n6HBw5s31b8LL4esf3ju3%2FwE%3D>). 
>> <image.png> <http:///>
>> 
>> PATH 1 will be much quicker than PATH 2 and its bottleneck (assuming HDD drives) will be the commitlog
>> PATH 2 will need at least an access to disk to do a read (potentially even in a different machine) and an access to disk to do a write to the commitlog. This is at least twice as slow as PATH 1.
>> 
>> This is all the info that I could get from the internet but a lot is missing. For example, there is no information about how the counter lock is acquired, is there a shared lock across all the nodes?
>> 
>> Hope I am not oversimplifying things, but I think this will be useful to better understand how to tune up the system.
>> 
>> Thanks in advance.
>> 
>> F Javier Pareja
>> 
> 
> 


Re: How do counter updates work?

Posted by Javier Pareja <pa...@gmail.com>.
Doesn't cassandra have TIMEUUID for these use cases?

Anyways, hopefully someone can help me better understand possible delays
when writing a counter.

F Javier Pareja

On Mon, Mar 5, 2018 at 1:54 PM, Hannu Kröger <hk...@gmail.com> wrote:

> Traditionally auto increment counters have been used to generate SQL row
> IDs. This is what Kyrylo probably is here referring to.
>
> Cassandra counters are better tracking e.g. usage patterns, web site
> visitors, statistics, etc.
>
> For accurate counting (e.g. for generating IDs) those counters are not
> good because they are inaccurate in certain cases.
>
> Hannu
>
> On 5 Mar 2018, at 15:50, Javier Pareja <pa...@gmail.com> wrote:
>
> Hi Kyrulo,
>
> I don't understand how UUIDs are related to counters, but I use counters
> to increment the value of a cell in an atomic manner. I could try reading
> the value and then writing to the cell but then I would lose the atomicity
> of the update.
>
> F Javier Pareja
>
> On Mon, Mar 5, 2018 at 1:32 PM, Kyrylo Lebediev <Ky...@epam.com>
> wrote:
>
>> Hello!
>>
>> Can't answer your question but there is another one: "why do we need to
>> maintain counters with their known limitations (and I've heard of some
>> issues with implementation of counters in Cassandra), when there exist
>> really effective uuid generation algorithms which allow us to generate
>> unique values?"
>> https://begriffs.com/posts/2018-01-01-sql-keys-in-depth.html
>> <https://begriffs.com/posts/2018-01-01-sql-keys-in-depth.html>(the
>> article is about keys in RDBMS's, but its statements are true for NoSQL as
>> well)
>>
>> Regards,
>> Kyrill
>> ------------------------------
>> *From:* jpare86@gmail.com <jp...@gmail.com> on behalf of Javier Pareja
>> <pa...@gmail.com>
>> *Sent:* Monday, March 5, 2018 1:55:14 PM
>> *To:* user@cassandra.apache.org
>> *Subject:* How do counter updates work?
>>
>> Hello everyone,
>>
>> I am trying to understand how cassandra counter writes work in more
>> detail but all that I could find is this: https://www.datastax.com/dev/b
>> log/whats-new-in-cassandra-2-1-a-better-implementation-of-counters
>> From there I was able to extract the following process:
>> (click here to edit
>> <https://www.draw.io/?lightbox=1&highlight=0000ff&edit=_blank&layers=1&nav=1&title=Untitled%20Diagram.xml#R7VpRc6M2EP41nmkfcgPCYPKYOPHdTXudzCXtNY8yCKMGkCvk2LlfXwkkjATYjo0dp5O8BK2kZdldfftJ8sAZp6vPFM7jbyREyQBY4Wrg3AwAsIfA4%2F%2BE5KWU%2BP5lKZhRHMpBa8E9%2Fomk0JLSBQ5Rrg1khCQMz3VhQLIMBUyTQUrJUh8WkUR%2F6xzOUENwH8CkKf2BQxZLqW1Z644vCM9i%2BWrflR1TGDzNKFlk8n0D4ETFX9mdQqVLjs9jGJJlTeTcDpwxJYSVT%2BlqjBLhW%2BW2ct6ko7eym6KM7TRhhKDnjZDtBoCbGlwAXxrGXpQzUMh9I5uEspjMSAaT27X0uvhgJFTavBWzNJGPCZyi5LryyZgkhPKujGRiWs4gZVciXIZsghOhwVJtmSAub6MsVDOCBOY5Dh5inJUdcppdtmqT%2FkGMvcg2XDDCResP%2BZ2QuZyVM0qekLKSx84q%2FqoelQtibEQyNoEpTkSK%2F4VoCDMoxfJNvmy26ePhoS9%2Fi2%2F85Krmo%2FxktMKs1sVbj%2FKdzejKgOdkQQOVu45cMJDOkBwGSpGIZG2eTInPiKSIv58PoCiBDD%2FrqwDKxTSrxlVT7wjmlgBLLnwwlGktl%2F3Qt3QVpZ1yVj0tDUVDy1DkGorKj2so4qkBX2rD5mJA%2FgqDL62Ndjmbx%2FOH0gLVqjl3LSqWYPtydEvtzzBZIAUgB6xGS1%2BNuy6DTcnZmkjbc9KEGQXqtRy1vQNzshaFjU7uMqbm9dsVChYMceGYO5Mhyp%2F%2BnIdQiLyEG3w95SJvxiqP1CLEMX0uHmO0gty1fMgcUczNRHQtvVuLrpcxZuh%2BDgtvLXlF1cOmx7OqGNYO%2BPoKOOPYWRspi9YuMGeDDTgHEzzLBFajrPzWznR5RpSh1WATOqnekbEIZXNZq9RKFteLtHWE3AGN3PlBcZE5HDgI%2F%2FcNpQxOeW6YaaIXzS05cEjUvcBH02iXqIcQ%2BVFwnlH39aC7oCXqoCXqwx6ivjXIY5KmmCVk9nZRDkeX013XNvKC40W5EdKWwHdG%2BU3D7HxQ3zOnvq%2FmHS1cuB3Cz5gL24Yi7zy4sH1kLmxfnhcZtrVkrRK0z3QdtmSrc5TsHAI9eI6ZnR1JtUcchy1hlBxaUejvCIYDYVEp5xqrrmBBC5cpBcCKKEn5vxucPx1WbbvKYndp7aHAOZ7ueOCdsMB5jVBMyqOqSdOTMUmni3y7F%2Fso%2BsZuv43R%2B0ci9B%2FHXSep%2BT0D5aFHBR0V0DXIp7nmdi3ZZil1Lo%2BGrs0i%2BYjyRk7ztcmMQwWU45%2FFvrRMPMkO%2BGj3euDetNF7scZxAJMr2ZHiMCzWQ0fa148ROlFCnuBLSwbVwXg9AfyN6HFhffKA7%2BkOL1v75oMaQqIoR4dGSNG3D4g5CcTYNYAp4eacIcZx9oSYBo%2FYkcC9dldQGWi8p9OuzeMP3xU0D2v%2FIP8zvLOtwbsGPKcRoq9ZQFGq8%2Fip%2BCQWi8eFPFo3YTJJ8DxH2zkozOflLXCEVwI0j0FKHd89GSm1Rw0PHlRD9tz67gectszM%2Bi72ZFc7zR3OQ%2FGmxpaT7rHlXN8FjWEQH3ii33X03n1830NSm1cndtvus%2B3upJdTdKsRnO9X37jglwnM2a8b3GmdqTvNa2q%2FxZtHu5NoFsKbr%2Fe%2FCXfeJ2TZrztPczZiutNuu9k7mj%2FBdszlyPBcre8mb9ZdqBNcQURDmMfVdNG4g4zjSVZIgLVfGs8oDDHSOLE44EavuaxvRqTmcbfF4Up2IBu2DRJrbpN33m8bpLO6NeqZDJv2KrLbeURu3GYNnX7JMGgyrUahUwKxb9Ky2ft3QVTHRV6wgSuRhcP5at2ptNxdPXwpUnRdL0uF%2BkvqZXSHX0q8b5peAsQmmm6NvKFeIHpZNmJbW9d64QBdRS80vvXg%2Fq0BMYqCEPgbC1ELIEYRdK33AIgO6AkQvdMAommvve10wOCfYAuAuoZ%2B5eddxyv7Ov10aYzv%2BQ6z9Qd9xwRo8AHQNYAevhFAA03n6HBw5s31b8LL4esf3ju3%2FwE%3D>
>> ).
>> <image.png>
>>
>> PATH 1 will be much quicker than PATH 2 and its bottleneck (assuming HDD
>> drives) will be the commitlog
>> PATH 2 will need at least an access to disk to do a read (potentially
>> even in a different machine) and an access to disk to do a write to the
>> commitlog. This is at least twice as slow as PATH 1.
>>
>> This is all the info that I could get from the internet but a lot is
>> missing. For example, there is no information about how the counter lock is
>> acquired, is there a shared lock across all the nodes?
>>
>> Hope I am not oversimplifying things, but I think this will be useful to
>> better understand how to tune up the system.
>>
>> Thanks in advance.
>>
>> F Javier Pareja
>>
>
>
>

Re: How do counter updates work?

Posted by Hannu Kröger <hk...@gmail.com>.
Traditionally auto increment counters have been used to generate SQL row IDs. This is what Kyrylo probably is here referring to.

Cassandra counters are better tracking e.g. usage patterns, web site visitors, statistics, etc. 

For accurate counting (e.g. for generating IDs) those counters are not good because they are inaccurate in certain cases.

Hannu

> On 5 Mar 2018, at 15:50, Javier Pareja <pa...@gmail.com> wrote:
> 
> Hi Kyrulo,
> 
> I don't understand how UUIDs are related to counters, but I use counters to increment the value of a cell in an atomic manner. I could try reading the value and then writing to the cell but then I would lose the atomicity of the update.
> 
> F Javier Pareja
> 
> On Mon, Mar 5, 2018 at 1:32 PM, Kyrylo Lebediev <Kyrylo_Lebediev@epam.com <ma...@epam.com>> wrote:
> Hello!
> 
> Can't answer your question but there is another one: "why do we need to maintain counters with their known limitations (and I've heard of some issues with implementation of counters in Cassandra), when there exist really effective uuid generation algorithms which allow us to generate unique values?"
> https://begriffs.com/posts/2018-01-01-sql-keys-in-depth.html <https://begriffs.com/posts/2018-01-01-sql-keys-in-depth.html>
>  <https://begriffs.com/posts/2018-01-01-sql-keys-in-depth.html>(the article is about keys in RDBMS's, but its statements are true for NoSQL as well)
> 
> Regards, 
> Kyrill
> From: jpare86@gmail.com <ma...@gmail.com> <jpare86@gmail.com <ma...@gmail.com>> on behalf of Javier Pareja <pareja.javier@gmail.com <ma...@gmail.com>>
> Sent: Monday, March 5, 2018 1:55:14 PM
> To: user@cassandra.apache.org <ma...@cassandra.apache.org>
> Subject: How do counter updates work?
>  
> Hello everyone,
> 
> I am trying to understand how cassandra counter writes work in more detail but all that I could find is this: https://www.datastax.com/dev/blog/whats-new-in-cassandra-2-1-a-better-implementation-of-counters <https://www.datastax.com/dev/blog/whats-new-in-cassandra-2-1-a-better-implementation-of-counters>
> From there I was able to extract the following process:
> (click here to edit <https://www.draw.io/?lightbox=1&highlight=0000ff&edit=_blank&layers=1&nav=1&title=Untitled%20Diagram.xml#R7VpRc6M2EP41nmkfcgPCYPKYOPHdTXudzCXtNY8yCKMGkCvk2LlfXwkkjATYjo0dp5O8BK2kZdldfftJ8sAZp6vPFM7jbyREyQBY4Wrg3AwAsIfA4%2F%2BE5KWU%2BP5lKZhRHMpBa8E9%2Fomk0JLSBQ5Rrg1khCQMz3VhQLIMBUyTQUrJUh8WkUR%2F6xzOUENwH8CkKf2BQxZLqW1Z644vCM9i%2BWrflR1TGDzNKFlk8n0D4ETFX9mdQqVLjs9jGJJlTeTcDpwxJYSVT%2BlqjBLhW%2BW2ct6ko7eym6KM7TRhhKDnjZDtBoCbGlwAXxrGXpQzUMh9I5uEspjMSAaT27X0uvhgJFTavBWzNJGPCZyi5LryyZgkhPKujGRiWs4gZVciXIZsghOhwVJtmSAub6MsVDOCBOY5Dh5inJUdcppdtmqT%2FkGMvcg2XDDCResP%2BZ2QuZyVM0qekLKSx84q%2FqoelQtibEQyNoEpTkSK%2F4VoCDMoxfJNvmy26ePhoS9%2Fi2%2F85Krmo%2FxktMKs1sVbj%2FKdzejKgOdkQQOVu45cMJDOkBwGSpGIZG2eTInPiKSIv58PoCiBDD%2FrqwDKxTSrxlVT7wjmlgBLLnwwlGktl%2F3Qt3QVpZ1yVj0tDUVDy1DkGorKj2so4qkBX2rD5mJA%2FgqDL62Ndjmbx%2FOH0gLVqjl3LSqWYPtydEvtzzBZIAUgB6xGS1%2BNuy6DTcnZmkjbc9KEGQXqtRy1vQNzshaFjU7uMqbm9dsVChYMceGYO5Mhyp%2F%2BnIdQiLyEG3w95SJvxiqP1CLEMX0uHmO0gty1fMgcUczNRHQtvVuLrpcxZuh%2BDgtvLXlF1cOmx7OqGNYO%2BPoKOOPYWRspi9YuMGeDDTgHEzzLBFajrPzWznR5RpSh1WATOqnekbEIZXNZq9RKFteLtHWE3AGN3PlBcZE5HDgI%2F%2FcNpQxOeW6YaaIXzS05cEjUvcBH02iXqIcQ%2BVFwnlH39aC7oCXqoCXqwx6ivjXIY5KmmCVk9nZRDkeX013XNvKC40W5EdKWwHdG%2BU3D7HxQ3zOnvq%2FmHS1cuB3Cz5gL24Yi7zy4sH1kLmxfnhcZtrVkrRK0z3QdtmSrc5TsHAI9eI6ZnR1JtUcchy1hlBxaUejvCIYDYVEp5xqrrmBBC5cpBcCKKEn5vxucPx1WbbvKYndp7aHAOZ7ueOCdsMB5jVBMyqOqSdOTMUmni3y7F%2Fso%2BsZuv43R%2B0ci9B%2FHXSep%2BT0D5aFHBR0V0DXIp7nmdi3ZZil1Lo%2BGrs0i%2BYjyRk7ztcmMQwWU45%2FFvrRMPMkO%2BGj3euDetNF7scZxAJMr2ZHiMCzWQ0fa148ROlFCnuBLSwbVwXg9AfyN6HFhffKA7%2BkOL1v75oMaQqIoR4dGSNG3D4g5CcTYNYAp4eacIcZx9oSYBo%2FYkcC9dldQGWi8p9OuzeMP3xU0D2v%2FIP8zvLOtwbsGPKcRoq9ZQFGq8%2Fip%2BCQWi8eFPFo3YTJJ8DxH2zkozOflLXCEVwI0j0FKHd89GSm1Rw0PHlRD9tz67gectszM%2Bi72ZFc7zR3OQ%2FGmxpaT7rHlXN8FjWEQH3ii33X03n1830NSm1cndtvus%2B3upJdTdKsRnO9X37jglwnM2a8b3GmdqTvNa2q%2FxZtHu5NoFsKbr%2Fe%2FCXfeJ2TZrztPczZiutNuu9k7mj%2FBdszlyPBcre8mb9ZdqBNcQURDmMfVdNG4g4zjSVZIgLVfGs8oDDHSOLE44EavuaxvRqTmcbfF4Up2IBu2DRJrbpN33m8bpLO6NeqZDJv2KrLbeURu3GYNnX7JMGgyrUahUwKxb9Ky2ft3QVTHRV6wgSuRhcP5at2ptNxdPXwpUnRdL0uF%2BkvqZXSHX0q8b5peAsQmmm6NvKFeIHpZNmJbW9d64QBdRS80vvXg%2Fq0BMYqCEPgbC1ELIEYRdK33AIgO6AkQvdMAommvve10wOCfYAuAuoZ%2B5eddxyv7Ov10aYzv%2BQ6z9Qd9xwRo8AHQNYAevhFAA03n6HBw5s31b8LL4esf3ju3%2FwE%3D>). 
> <image.png> <http:///>
> 
> PATH 1 will be much quicker than PATH 2 and its bottleneck (assuming HDD drives) will be the commitlog
> PATH 2 will need at least an access to disk to do a read (potentially even in a different machine) and an access to disk to do a write to the commitlog. This is at least twice as slow as PATH 1.
> 
> This is all the info that I could get from the internet but a lot is missing. For example, there is no information about how the counter lock is acquired, is there a shared lock across all the nodes?
> 
> Hope I am not oversimplifying things, but I think this will be useful to better understand how to tune up the system.
> 
> Thanks in advance.
> 
> F Javier Pareja
> 


Re: How do counter updates work?

Posted by Javier Pareja <pa...@gmail.com>.
Hi Kyrulo,

I don't understand how UUIDs are related to counters, but I use counters to
increment the value of a cell in an atomic manner. I could try reading the
value and then writing to the cell but then I would lose the atomicity of
the update.

F Javier Pareja

On Mon, Mar 5, 2018 at 1:32 PM, Kyrylo Lebediev <Ky...@epam.com>
wrote:

> Hello!
>
> Can't answer your question but there is another one: "why do we need to
> maintain counters with their known limitations (and I've heard of some
> issues with implementation of counters in Cassandra), when there exist
> really effective uuid generation algorithms which allow us to generate
> unique values?"
> https://begriffs.com/posts/2018-01-01-sql-keys-in-depth.html
>
> <https://begriffs.com/posts/2018-01-01-sql-keys-in-depth.html>(the
> article is about keys in RDBMS's, but its statements are true for NoSQL as
> well)
>
>
> Regards,
> Kyrill
> ------------------------------
> *From:* jpare86@gmail.com <jp...@gmail.com> on behalf of Javier Pareja <
> pareja.javier@gmail.com>
> *Sent:* Monday, March 5, 2018 1:55:14 PM
> *To:* user@cassandra.apache.org
> *Subject:* How do counter updates work?
>
> Hello everyone,
>
> I am trying to understand how cassandra counter writes work in more detail
> but all that I could find is this: https://www.datastax.com/dev/
> blog/whats-new-in-cassandra-2-1-a-better-implementation-of-counters
> From there I was able to extract the following process:
> (click here to edit
> <https://www.draw.io/?lightbox=1&highlight=0000ff&edit=_blank&layers=1&nav=1&title=Untitled%20Diagram.xml#R7VpRc6M2EP41nmkfcgPCYPKYOPHdTXudzCXtNY8yCKMGkCvk2LlfXwkkjATYjo0dp5O8BK2kZdldfftJ8sAZp6vPFM7jbyREyQBY4Wrg3AwAsIfA4%2F%2BE5KWU%2BP5lKZhRHMpBa8E9%2Fomk0JLSBQ5Rrg1khCQMz3VhQLIMBUyTQUrJUh8WkUR%2F6xzOUENwH8CkKf2BQxZLqW1Z644vCM9i%2BWrflR1TGDzNKFlk8n0D4ETFX9mdQqVLjs9jGJJlTeTcDpwxJYSVT%2BlqjBLhW%2BW2ct6ko7eym6KM7TRhhKDnjZDtBoCbGlwAXxrGXpQzUMh9I5uEspjMSAaT27X0uvhgJFTavBWzNJGPCZyi5LryyZgkhPKujGRiWs4gZVciXIZsghOhwVJtmSAub6MsVDOCBOY5Dh5inJUdcppdtmqT%2FkGMvcg2XDDCResP%2BZ2QuZyVM0qekLKSx84q%2FqoelQtibEQyNoEpTkSK%2F4VoCDMoxfJNvmy26ePhoS9%2Fi2%2F85Krmo%2FxktMKs1sVbj%2FKdzejKgOdkQQOVu45cMJDOkBwGSpGIZG2eTInPiKSIv58PoCiBDD%2FrqwDKxTSrxlVT7wjmlgBLLnwwlGktl%2F3Qt3QVpZ1yVj0tDUVDy1DkGorKj2so4qkBX2rD5mJA%2FgqDL62Ndjmbx%2FOH0gLVqjl3LSqWYPtydEvtzzBZIAUgB6xGS1%2BNuy6DTcnZmkjbc9KEGQXqtRy1vQNzshaFjU7uMqbm9dsVChYMceGYO5Mhyp%2F%2BnIdQiLyEG3w95SJvxiqP1CLEMX0uHmO0gty1fMgcUczNRHQtvVuLrpcxZuh%2BDgtvLXlF1cOmx7OqGNYO%2BPoKOOPYWRspi9YuMGeDDTgHEzzLBFajrPzWznR5RpSh1WATOqnekbEIZXNZq9RKFteLtHWE3AGN3PlBcZE5HDgI%2F%2FcNpQxOeW6YaaIXzS05cEjUvcBH02iXqIcQ%2BVFwnlH39aC7oCXqoCXqwx6ivjXIY5KmmCVk9nZRDkeX013XNvKC40W5EdKWwHdG%2BU3D7HxQ3zOnvq%2FmHS1cuB3Cz5gL24Yi7zy4sH1kLmxfnhcZtrVkrRK0z3QdtmSrc5TsHAI9eI6ZnR1JtUcchy1hlBxaUejvCIYDYVEp5xqrrmBBC5cpBcCKKEn5vxucPx1WbbvKYndp7aHAOZ7ueOCdsMB5jVBMyqOqSdOTMUmni3y7F%2Fso%2BsZuv43R%2B0ci9B%2FHXSep%2BT0D5aFHBR0V0DXIp7nmdi3ZZil1Lo%2BGrs0i%2BYjyRk7ztcmMQwWU45%2FFvrRMPMkO%2BGj3euDetNF7scZxAJMr2ZHiMCzWQ0fa148ROlFCnuBLSwbVwXg9AfyN6HFhffKA7%2BkOL1v75oMaQqIoR4dGSNG3D4g5CcTYNYAp4eacIcZx9oSYBo%2FYkcC9dldQGWi8p9OuzeMP3xU0D2v%2FIP8zvLOtwbsGPKcRoq9ZQFGq8%2Fip%2BCQWi8eFPFo3YTJJ8DxH2zkozOflLXCEVwI0j0FKHd89GSm1Rw0PHlRD9tz67gectszM%2Bi72ZFc7zR3OQ%2FGmxpaT7rHlXN8FjWEQH3ii33X03n1830NSm1cndtvus%2B3upJdTdKsRnO9X37jglwnM2a8b3GmdqTvNa2q%2FxZtHu5NoFsKbr%2Fe%2FCXfeJ2TZrztPczZiutNuu9k7mj%2FBdszlyPBcre8mb9ZdqBNcQURDmMfVdNG4g4zjSVZIgLVfGs8oDDHSOLE44EavuaxvRqTmcbfF4Up2IBu2DRJrbpN33m8bpLO6NeqZDJv2KrLbeURu3GYNnX7JMGgyrUahUwKxb9Ky2ft3QVTHRV6wgSuRhcP5at2ptNxdPXwpUnRdL0uF%2BkvqZXSHX0q8b5peAsQmmm6NvKFeIHpZNmJbW9d64QBdRS80vvXg%2Fq0BMYqCEPgbC1ELIEYRdK33AIgO6AkQvdMAommvve10wOCfYAuAuoZ%2B5eddxyv7Ov10aYzv%2BQ6z9Qd9xwRo8AHQNYAevhFAA03n6HBw5s31b8LL4esf3ju3%2FwE%3D>
> ).
>
>
> PATH 1 will be much quicker than PATH 2 and its bottleneck (assuming HDD
> drives) will be the commitlog
> PATH 2 will need at least an access to disk to do a read (potentially even
> in a different machine) and an access to disk to do a write to the
> commitlog. This is at least twice as slow as PATH 1.
>
> This is all the info that I could get from the internet but a lot is
> missing. For example, there is no information about how the counter lock is
> acquired, is there a shared lock across all the nodes?
>
> Hope I am not oversimplifying things, but I think this will be useful to
> better understand how to tune up the system.
>
> Thanks in advance.
>
> F Javier Pareja
>

Re: How do counter updates work?

Posted by Kyrylo Lebediev <Ky...@epam.com>.
Hello!

Can't answer your question but there is another one: "why do we need to maintain counters with their known limitations (and I've heard of some issues with implementation of counters in Cassandra), when there exist really effective uuid generation algorithms which allow us to generate unique values?"
https://begriffs.com/posts/2018-01-01-sql-keys-in-depth.html

<https://begriffs.com/posts/2018-01-01-sql-keys-in-depth.html>(the article is about keys in RDBMS's, but its statements are true for NoSQL as well)


Regards,
Kyrill

________________________________
From: jpare86@gmail.com <jp...@gmail.com> on behalf of Javier Pareja <pa...@gmail.com>
Sent: Monday, March 5, 2018 1:55:14 PM
To: user@cassandra.apache.org
Subject: How do counter updates work?

Hello everyone,

I am trying to understand how cassandra counter writes work in more detail but all that I could find is this: https://www.datastax.com/dev/blog/whats-new-in-cassandra-2-1-a-better-implementation-of-counters
From there I was able to extract the following process:
(click here to edit<https://www.draw.io/?lightbox=1&highlight=0000ff&edit=_blank&layers=1&nav=1&title=Untitled%20Diagram.xml#R7VpRc6M2EP41nmkfcgPCYPKYOPHdTXudzCXtNY8yCKMGkCvk2LlfXwkkjATYjo0dp5O8BK2kZdldfftJ8sAZp6vPFM7jbyREyQBY4Wrg3AwAsIfA4%2F%2BE5KWU%2BP5lKZhRHMpBa8E9%2Fomk0JLSBQ5Rrg1khCQMz3VhQLIMBUyTQUrJUh8WkUR%2F6xzOUENwH8CkKf2BQxZLqW1Z644vCM9i%2BWrflR1TGDzNKFlk8n0D4ETFX9mdQqVLjs9jGJJlTeTcDpwxJYSVT%2BlqjBLhW%2BW2ct6ko7eym6KM7TRhhKDnjZDtBoCbGlwAXxrGXpQzUMh9I5uEspjMSAaT27X0uvhgJFTavBWzNJGPCZyi5LryyZgkhPKujGRiWs4gZVciXIZsghOhwVJtmSAub6MsVDOCBOY5Dh5inJUdcppdtmqT%2FkGMvcg2XDDCResP%2BZ2QuZyVM0qekLKSx84q%2FqoelQtibEQyNoEpTkSK%2F4VoCDMoxfJNvmy26ePhoS9%2Fi2%2F85Krmo%2FxktMKs1sVbj%2FKdzejKgOdkQQOVu45cMJDOkBwGSpGIZG2eTInPiKSIv58PoCiBDD%2FrqwDKxTSrxlVT7wjmlgBLLnwwlGktl%2F3Qt3QVpZ1yVj0tDUVDy1DkGorKj2so4qkBX2rD5mJA%2FgqDL62Ndjmbx%2FOH0gLVqjl3LSqWYPtydEvtzzBZIAUgB6xGS1%2BNuy6DTcnZmkjbc9KEGQXqtRy1vQNzshaFjU7uMqbm9dsVChYMceGYO5Mhyp%2F%2BnIdQiLyEG3w95SJvxiqP1CLEMX0uHmO0gty1fMgcUczNRHQtvVuLrpcxZuh%2BDgtvLXlF1cOmx7OqGNYO%2BPoKOOPYWRspi9YuMGeDDTgHEzzLBFajrPzWznR5RpSh1WATOqnekbEIZXNZq9RKFteLtHWE3AGN3PlBcZE5HDgI%2F%2FcNpQxOeW6YaaIXzS05cEjUvcBH02iXqIcQ%2BVFwnlH39aC7oCXqoCXqwx6ivjXIY5KmmCVk9nZRDkeX013XNvKC40W5EdKWwHdG%2BU3D7HxQ3zOnvq%2FmHS1cuB3Cz5gL24Yi7zy4sH1kLmxfnhcZtrVkrRK0z3QdtmSrc5TsHAI9eI6ZnR1JtUcchy1hlBxaUejvCIYDYVEp5xqrrmBBC5cpBcCKKEn5vxucPx1WbbvKYndp7aHAOZ7ueOCdsMB5jVBMyqOqSdOTMUmni3y7F%2Fso%2BsZuv43R%2B0ci9B%2FHXSep%2BT0D5aFHBR0V0DXIp7nmdi3ZZil1Lo%2BGrs0i%2BYjyRk7ztcmMQwWU45%2FFvrRMPMkO%2BGj3euDetNF7scZxAJMr2ZHiMCzWQ0fa148ROlFCnuBLSwbVwXg9AfyN6HFhffKA7%2BkOL1v75oMaQqIoR4dGSNG3D4g5CcTYNYAp4eacIcZx9oSYBo%2FYkcC9dldQGWi8p9OuzeMP3xU0D2v%2FIP8zvLOtwbsGPKcRoq9ZQFGq8%2Fip%2BCQWi8eFPFo3YTJJ8DxH2zkozOflLXCEVwI0j0FKHd89GSm1Rw0PHlRD9tz67gectszM%2Bi72ZFc7zR3OQ%2FGmxpaT7rHlXN8FjWEQH3ii33X03n1830NSm1cndtvus%2B3upJdTdKsRnO9X37jglwnM2a8b3GmdqTvNa2q%2FxZtHu5NoFsKbr%2Fe%2FCXfeJ2TZrztPczZiutNuu9k7mj%2FBdszlyPBcre8mb9ZdqBNcQURDmMfVdNG4g4zjSVZIgLVfGs8oDDHSOLE44EavuaxvRqTmcbfF4Up2IBu2DRJrbpN33m8bpLO6NeqZDJv2KrLbeURu3GYNnX7JMGgyrUahUwKxb9Ky2ft3QVTHRV6wgSuRhcP5at2ptNxdPXwpUnRdL0uF%2BkvqZXSHX0q8b5peAsQmmm6NvKFeIHpZNmJbW9d64QBdRS80vvXg%2Fq0BMYqCEPgbC1ELIEYRdK33AIgO6AkQvdMAommvve10wOCfYAuAuoZ%2B5eddxyv7Ov10aYzv%2BQ6z9Qd9xwRo8AHQNYAevhFAA03n6HBw5s31b8LL4esf3ju3%2FwE%3D>).
[cid:ii_jee3zlby0_161f5cd7cfb1234b]</>

PATH 1 will be much quicker than PATH 2 and its bottleneck (assuming HDD drives) will be the commitlog
PATH 2 will need at least an access to disk to do a read (potentially even in a different machine) and an access to disk to do a write to the commitlog. This is at least twice as slow as PATH 1.

This is all the info that I could get from the internet but a lot is missing. For example, there is no information about how the counter lock is acquired, is there a shared lock across all the nodes?

Hope I am not oversimplifying things, but I think this will be useful to better understand how to tune up the system.

Thanks in advance.

F Javier Pareja