You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by narges saleh <sn...@gmail.com> on 2020/10/25 01:59:28 UTC
IgniteAtomicSequence
Hi All,
What is the consequence of data rebalancing across nodes as far as
IgniteAtomicSequence and the reserve on each node is concerned? For
example, if the last sequence number is 6000 in one node and the record
moves to a node whose last sequence number is 1000? Do the reserves on both
nodes get recalculated?
Are there best practices for generating and using these sequences?
Is IgniteAtomicSequence the right approach if I want to keep track of the
records on each node for a partitioned cache?
thanks.
Re: IgniteAtomicSequence
Posted by Pavel Vinokurov <vi...@gmail.com>.
IgniteAtomicLong shows poor results in case of non zero backups.
But IgniteAtomicSequence incremented 1M by ~0.4 second in the attached
example.
ср, 28 окт. 2020 г. в 13:45, narges saleh <sn...@gmail.com>:
> Thanks Pavel for the replies.
>
> I did a quick/basic benchmark of IgniteAtomicLong, and IgniteAtomicLong,
> with only two ignite nodes, with 1,000,000 iterations. Unless, I am doing
> something wrong, the result was very disappointing.
> With IgniteAtomicSequence, the iterations took about 30 seconds.
> With IgniteAtomicLong, it took over 3 minutes. I repeated the test 5 times.
>
> On Wed, Oct 28, 2020 at 5:33 AM Pavel Vinokurov <vi...@gmail.com>
> wrote:
>
>> The atomic sequence/long could be persisted if the default data region is
>> persistent[1].
>> One of the use cases for IgniteAtomicSequence is to generate unique IDs.
>> You could estimate the volume implicitly by getting metrics from the
>> default data region[2].
>>
>> [1]
>> https://ignite.apache.org/docs/latest/persistence/native-persistence#enabling-persistent-storage
>> [2]
>> https://apacheignite.readme.io/v2.8.0/docs/memory-metrics#getting-metrics
>>
>> вт, 27 окт. 2020 г. в 21:50, narges saleh <sn...@gmail.com>:
>>
>>> Questions:
>>> I know I should take into account the cluster size and the data volume,
>>> but how bad is the overhead of IgniteAtomicLong? Any benchmarking?
>>> What are the use cases for IgniteAtomicSequence if it does not
>>> guarantee ordering?
>>>
>>> thanks.
>>>
>>> On Tue, Oct 27, 2020 at 10:27 AM narges saleh <sn...@gmail.com>
>>> wrote:
>>>
>>>> Thank you Pavel, for the information.
>>>>
>>>> What happens to the atomic sequence/long value in case of a system
>>>> restart? Do I need to persist the value somewhere and pass it as the
>>>> initial value, across cluster restarts?
>>>>
>>>> On Tue, Oct 27, 2020 at 1:13 AM Pavel Vinokurov <
>>>> vinokurov.pasha@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> In case of rebalance the reserves aren't recalculated. So the next
>>>>> values will be in the range [1001,1000+reserveSize].
>>>>> After the local reserve exceeds, the node reserves a new range
>>>>> starting from the global last sequence number.
>>>>> The main idea IgniteAtomicSequence is to generate unique numbers but
>>>>> it doesn't guarantee ordering.
>>>>> If you need ordering you could set the reserve size to 1 or use
>>>>> IgniteAtomicLong.
>>>>>
>>>>> Thanks,
>>>>> Pavel
>>>>>
>>>>> вс, 25 окт. 2020 г. в 05:00, narges saleh <sn...@gmail.com>:
>>>>>
>>>>>> Hi All,
>>>>>> What is the consequence of data rebalancing across nodes as far as
>>>>>> IgniteAtomicSequence and the reserve on each node is concerned? For
>>>>>> example, if the last sequence number is 6000 in one node and the record
>>>>>> moves to a node whose last sequence number is 1000? Do the reserves on both
>>>>>> nodes get recalculated?
>>>>>>
>>>>>> Are there best practices for generating and using these sequences?
>>>>>>
>>>>>> Is IgniteAtomicSequence the right approach if I want to keep track of
>>>>>> the records on each node for a partitioned cache?
>>>>>>
>>>>>> thanks.
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Regards
>>>>>
>>>>> Pavel Vinokurov
>>>>>
>>>>
>>
>> --
>>
>> Regards
>>
>> Pavel Vinokurov
>>
>
--
Regards
Pavel Vinokurov
Re: IgniteAtomicSequence
Posted by narges saleh <sn...@gmail.com>.
Thanks Pavel for the replies.
I did a quick/basic benchmark of IgniteAtomicLong, and IgniteAtomicLong,
with only two ignite nodes, with 1,000,000 iterations. Unless, I am doing
something wrong, the result was very disappointing.
With IgniteAtomicSequence, the iterations took about 30 seconds.
With IgniteAtomicLong, it took over 3 minutes. I repeated the test 5 times.
On Wed, Oct 28, 2020 at 5:33 AM Pavel Vinokurov <vi...@gmail.com>
wrote:
> The atomic sequence/long could be persisted if the default data region is
> persistent[1].
> One of the use cases for IgniteAtomicSequence is to generate unique IDs.
> You could estimate the volume implicitly by getting metrics from the
> default data region[2].
>
> [1]
> https://ignite.apache.org/docs/latest/persistence/native-persistence#enabling-persistent-storage
> [2]
> https://apacheignite.readme.io/v2.8.0/docs/memory-metrics#getting-metrics
>
> вт, 27 окт. 2020 г. в 21:50, narges saleh <sn...@gmail.com>:
>
>> Questions:
>> I know I should take into account the cluster size and the data volume,
>> but how bad is the overhead of IgniteAtomicLong? Any benchmarking?
>> What are the use cases for IgniteAtomicSequence if it does not
>> guarantee ordering?
>>
>> thanks.
>>
>> On Tue, Oct 27, 2020 at 10:27 AM narges saleh <sn...@gmail.com>
>> wrote:
>>
>>> Thank you Pavel, for the information.
>>>
>>> What happens to the atomic sequence/long value in case of a system
>>> restart? Do I need to persist the value somewhere and pass it as the
>>> initial value, across cluster restarts?
>>>
>>> On Tue, Oct 27, 2020 at 1:13 AM Pavel Vinokurov <
>>> vinokurov.pasha@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> In case of rebalance the reserves aren't recalculated. So the next
>>>> values will be in the range [1001,1000+reserveSize].
>>>> After the local reserve exceeds, the node reserves a new range
>>>> starting from the global last sequence number.
>>>> The main idea IgniteAtomicSequence is to generate unique numbers but it
>>>> doesn't guarantee ordering.
>>>> If you need ordering you could set the reserve size to 1 or use
>>>> IgniteAtomicLong.
>>>>
>>>> Thanks,
>>>> Pavel
>>>>
>>>> вс, 25 окт. 2020 г. в 05:00, narges saleh <sn...@gmail.com>:
>>>>
>>>>> Hi All,
>>>>> What is the consequence of data rebalancing across nodes as far as
>>>>> IgniteAtomicSequence and the reserve on each node is concerned? For
>>>>> example, if the last sequence number is 6000 in one node and the record
>>>>> moves to a node whose last sequence number is 1000? Do the reserves on both
>>>>> nodes get recalculated?
>>>>>
>>>>> Are there best practices for generating and using these sequences?
>>>>>
>>>>> Is IgniteAtomicSequence the right approach if I want to keep track of
>>>>> the records on each node for a partitioned cache?
>>>>>
>>>>> thanks.
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Regards
>>>>
>>>> Pavel Vinokurov
>>>>
>>>
>
> --
>
> Regards
>
> Pavel Vinokurov
>
Re: IgniteAtomicSequence
Posted by Pavel Vinokurov <vi...@gmail.com>.
The atomic sequence/long could be persisted if the default data region is
persistent[1].
One of the use cases for IgniteAtomicSequence is to generate unique IDs.
You could estimate the volume implicitly by getting metrics from the
default data region[2].
[1]
https://ignite.apache.org/docs/latest/persistence/native-persistence#enabling-persistent-storage
[2]
https://apacheignite.readme.io/v2.8.0/docs/memory-metrics#getting-metrics
вт, 27 окт. 2020 г. в 21:50, narges saleh <sn...@gmail.com>:
> Questions:
> I know I should take into account the cluster size and the data volume,
> but how bad is the overhead of IgniteAtomicLong? Any benchmarking?
> What are the use cases for IgniteAtomicSequence if it does not
> guarantee ordering?
>
> thanks.
>
> On Tue, Oct 27, 2020 at 10:27 AM narges saleh <sn...@gmail.com>
> wrote:
>
>> Thank you Pavel, for the information.
>>
>> What happens to the atomic sequence/long value in case of a system
>> restart? Do I need to persist the value somewhere and pass it as the
>> initial value, across cluster restarts?
>>
>> On Tue, Oct 27, 2020 at 1:13 AM Pavel Vinokurov <
>> vinokurov.pasha@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> In case of rebalance the reserves aren't recalculated. So the next
>>> values will be in the range [1001,1000+reserveSize].
>>> After the local reserve exceeds, the node reserves a new range
>>> starting from the global last sequence number.
>>> The main idea IgniteAtomicSequence is to generate unique numbers but it
>>> doesn't guarantee ordering.
>>> If you need ordering you could set the reserve size to 1 or use
>>> IgniteAtomicLong.
>>>
>>> Thanks,
>>> Pavel
>>>
>>> вс, 25 окт. 2020 г. в 05:00, narges saleh <sn...@gmail.com>:
>>>
>>>> Hi All,
>>>> What is the consequence of data rebalancing across nodes as far as
>>>> IgniteAtomicSequence and the reserve on each node is concerned? For
>>>> example, if the last sequence number is 6000 in one node and the record
>>>> moves to a node whose last sequence number is 1000? Do the reserves on both
>>>> nodes get recalculated?
>>>>
>>>> Are there best practices for generating and using these sequences?
>>>>
>>>> Is IgniteAtomicSequence the right approach if I want to keep track of
>>>> the records on each node for a partitioned cache?
>>>>
>>>> thanks.
>>>>
>>>
>>>
>>> --
>>>
>>> Regards
>>>
>>> Pavel Vinokurov
>>>
>>
--
Regards
Pavel Vinokurov
Re: IgniteAtomicSequence
Posted by narges saleh <sn...@gmail.com>.
Questions:
I know I should take into account the cluster size and the data volume, but
how bad is the overhead of IgniteAtomicLong? Any benchmarking?
What are the use cases for IgniteAtomicSequence if it does not
guarantee ordering?
thanks.
On Tue, Oct 27, 2020 at 10:27 AM narges saleh <sn...@gmail.com> wrote:
> Thank you Pavel, for the information.
>
> What happens to the atomic sequence/long value in case of a system
> restart? Do I need to persist the value somewhere and pass it as the
> initial value, across cluster restarts?
>
> On Tue, Oct 27, 2020 at 1:13 AM Pavel Vinokurov <vi...@gmail.com>
> wrote:
>
>> Hi,
>>
>> In case of rebalance the reserves aren't recalculated. So the next values
>> will be in the range [1001,1000+reserveSize].
>> After the local reserve exceeds, the node reserves a new range
>> starting from the global last sequence number.
>> The main idea IgniteAtomicSequence is to generate unique numbers but it
>> doesn't guarantee ordering.
>> If you need ordering you could set the reserve size to 1 or use
>> IgniteAtomicLong.
>>
>> Thanks,
>> Pavel
>>
>> вс, 25 окт. 2020 г. в 05:00, narges saleh <sn...@gmail.com>:
>>
>>> Hi All,
>>> What is the consequence of data rebalancing across nodes as far as
>>> IgniteAtomicSequence and the reserve on each node is concerned? For
>>> example, if the last sequence number is 6000 in one node and the record
>>> moves to a node whose last sequence number is 1000? Do the reserves on both
>>> nodes get recalculated?
>>>
>>> Are there best practices for generating and using these sequences?
>>>
>>> Is IgniteAtomicSequence the right approach if I want to keep track of
>>> the records on each node for a partitioned cache?
>>>
>>> thanks.
>>>
>>
>>
>> --
>>
>> Regards
>>
>> Pavel Vinokurov
>>
>
Re: IgniteAtomicSequence
Posted by narges saleh <sn...@gmail.com>.
Thank you Pavel, for the information.
What happens to the atomic sequence/long value in case of a system restart?
Do I need to persist the value somewhere and pass it as the initial value,
across cluster restarts?
On Tue, Oct 27, 2020 at 1:13 AM Pavel Vinokurov <vi...@gmail.com>
wrote:
> Hi,
>
> In case of rebalance the reserves aren't recalculated. So the next values
> will be in the range [1001,1000+reserveSize].
> After the local reserve exceeds, the node reserves a new range
> starting from the global last sequence number.
> The main idea IgniteAtomicSequence is to generate unique numbers but it
> doesn't guarantee ordering.
> If you need ordering you could set the reserve size to 1 or use
> IgniteAtomicLong.
>
> Thanks,
> Pavel
>
> вс, 25 окт. 2020 г. в 05:00, narges saleh <sn...@gmail.com>:
>
>> Hi All,
>> What is the consequence of data rebalancing across nodes as far as
>> IgniteAtomicSequence and the reserve on each node is concerned? For
>> example, if the last sequence number is 6000 in one node and the record
>> moves to a node whose last sequence number is 1000? Do the reserves on both
>> nodes get recalculated?
>>
>> Are there best practices for generating and using these sequences?
>>
>> Is IgniteAtomicSequence the right approach if I want to keep track of the
>> records on each node for a partitioned cache?
>>
>> thanks.
>>
>
>
> --
>
> Regards
>
> Pavel Vinokurov
>
Re: IgniteAtomicSequence
Posted by Pavel Vinokurov <vi...@gmail.com>.
Hi,
In case of rebalance the reserves aren't recalculated. So the next values
will be in the range [1001,1000+reserveSize].
After the local reserve exceeds, the node reserves a new range
starting from the global last sequence number.
The main idea IgniteAtomicSequence is to generate unique numbers but it
doesn't guarantee ordering.
If you need ordering you could set the reserve size to 1 or use
IgniteAtomicLong.
Thanks,
Pavel
вс, 25 окт. 2020 г. в 05:00, narges saleh <sn...@gmail.com>:
> Hi All,
> What is the consequence of data rebalancing across nodes as far as
> IgniteAtomicSequence and the reserve on each node is concerned? For
> example, if the last sequence number is 6000 in one node and the record
> moves to a node whose last sequence number is 1000? Do the reserves on both
> nodes get recalculated?
>
> Are there best practices for generating and using these sequences?
>
> Is IgniteAtomicSequence the right approach if I want to keep track of the
> records on each node for a partitioned cache?
>
> thanks.
>
--
Regards
Pavel Vinokurov