You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@ignite.apache.org by Louis C <lc...@outlook.fr> on 2024/02/08 16:05:05 UTC

Question about REST/C++ client

Hello,

I am trying to use/test Apache Ignite for a particular use case. For background reference, my use case of Ignite is to do 100Ks (to begin with) of "Gets" and of "increments" of values that will be stored in probably multiple caches in Apache Ignite.
I read the Ignite documentation, but I couldn't figure out things in the C++ API side.

1.
I have read in the documentation<https://ignite.apache.org/docs/latest/key-value-api/basic-cache-operations> <https://ignite.apache.org/docs/latest/key-value-api/basic-cache-operations> that there is a "GetAsync" method in C++ client. But I could not find it in the code. Is it a deprecated API ? If so, let's imagine I want to do multiple calls in parallel (let's say 50 for instance), how can I achieve this ? Can I just call multiple "Get"s in parallel in my threads without any problem ? Must I create a client for each thread ? There does not seem to be anything related to thread safety of these methods in the doc...
2.
Does doing 100K "increments" of values in a cache seem achievable on an Ignite cluster of a single node (let's say the CPU is a last gen i7 with 8 physical cores)? The problem that I have is that I have very good performances (+100K "gets") using the batch methods of the Rest API, but no batch method exist for "increments", and the overhead of each http call cripples the performance to a few 1000s/s. What would be the "best" way of achieving this (preferably in C++ or Rest API, but I am open to Java too )?
3.
Related but a bit different : if I want to add a value to an existing one in the store (doing an "increment"), would it be (in general) faster to do it using the "transaction" mode or to use the "ReplaceIfEquals" methods (in general I do not update the same values in the same time) ? Is there a real overhead for transactions ?

Thanks in advance

Louis

RE: Re: Question about REST/C++ client

Posted by Louis C <lc...@outlook.fr>.

 Ah ok, I understand better.


Thanks for your answer.


On 2024/02/19 11:05:04 Stephen Darlington wrote:
> Ah, the trick you're missing is that the "incr" REST API doesn't (directly)
> use a cache. Instead it uses an Atomic Sequence:
> https://ignite.apache.org/docs/latest/data-structures/atomic-types
>
> On Wed, 14 Feb 2024 at 17:38, Louis C <lc...@outlook.fr> wrote:
>
> > Hello,
> >
> > Thanks for your answers Igor and Stephen.
> > Igor, to answer your question to the second point I used the same term as
> > used in the Rest API, but indeed it more or less what you said (even if
> > in this API we do not have to retrieve explicitely the base value).
> > Best regards,
> > Louis
> > ------------------------------
> > *De :* Stephen Darlington <sd...@apache.org>
> > *Envoyé :* mardi 13 février 2024 11:22
> > *À :* user@ignite.apache.org <us...@ignite.apache.org>
> > *Objet :* Re: Question about REST/C++ client
> >
> > The easy answer to the "50 gets in parallel" is to use GetAll.
> >
> > On Tue, 13 Feb 2024 at 09:54, Igor Sapego <is...@apache.org> wrote:
> >
> > 1. There is no such API. Seems like a weird error in documentation. We
> > will fix it. If you want to make 50 gets in parallel, I think it's better
> > to run 50 clients with the current API. You can use the same client too,
> > but it may be slower, depending on how many nodes you have in a cluster.
> > 2. What do you mean by increments? Do you mean getting value, modifying it
> > and putting it back?
> > 3. Atomics are faster in most cases (ReplaceIfEquals). Yes, there is a
> > real overhead for transactions, though how big it is depends on the
> > transaction isolation level.
> >
> > Best Regards,
> > Igor
> >
> >
> > On Thu, Feb 8, 2024 at 5:05 PM Louis C <lc...@outlook.fr> wrote:
> >
> >
> > Hello,
> >
> > I am trying to use/test Apache Ignite for a particular use case. For
> > background reference, my use case of Ignite is to do 100Ks (to begin with)
> > of "Gets" and of "increments" of values that will be stored in probably
> > multiple caches in Apache Ignite.
> > I read the Ignite documentation, but I couldn't figure out things in the
> > C++ API side.
> >
> >
> >    1. I have read in the documentation
> >    <https://ignite.apache.org/docs/latest/key-value-api/basic-cache-operations>
> >    <https://ignite.apache.org/docs/latest/key-value-api/basic-cache-operations>that
> >    there is a "GetAsync" method in C++ client. But I could not find it in the
> >    code. Is it a deprecated API ? If so, let's imagine I want to do multiple
> >    calls in parallel (let's say 50 for instance), how can I achieve this ? Can
> >    I just call multiple "Get"s in parallel in my threads without any problem ?
> >    Must I create a client for each thread ? There does not seem to be anything
> >    related to thread safety of these methods in the doc...
> >    2. Does doing 100K "increments" of values in a cache seem achievable
> >    on an Ignite cluster of a single node (let's say the CPU is a last gen i7
> >    with 8 physical cores)? The problem that I have is that I have very good
> >    performances (+100K "gets") using the batch methods of the Rest API, but no
> >    batch method exist for "increments", and the overhead of each http call
> >    cripples the performance to a few 1000s/s. What would be the "best" way of
> >    achieving this (preferably in C++ or Rest API, but I am open to Java too )?
> >    3. Related but a bit different : if I want to add a value to an
> >    existing one in the store (doing an "increment"), would it be (in general)
> >    faster to do it using the "transaction" mode or to use the
> >    "ReplaceIfEquals" methods (in general I do not update the same values in
> >    the same time) ? Is there a real overhead for transactions ?
> >
> >
> > Thanks in advance
> >
> > Louis
> >
> >
>

Re: Question about REST/C++ client

Posted by Stephen Darlington <sd...@apache.org>.

Ah, the trick you're missing is that the "incr" REST API doesn't (directly)
use a cache. Instead it uses an Atomic Sequence:
https://ignite.apache.org/docs/latest/data-structures/atomic-types

On Wed, 14 Feb 2024 at 17:38, Louis C <lc...@outlook.fr> wrote:

> Hello,
>
> Thanks for your answers Igor and Stephen.
> Igor, to answer your question to the second point I used the same term as
> used in the Rest API, but indeed it more or less what you said (even if
> in this API we do not have to retrieve explicitely the base value).
> Best regards,
> Louis
> ------------------------------
> *De :* Stephen Darlington <sd...@apache.org>
> *Envoyé :* mardi 13 février 2024 11:22
> *À :* user@ignite.apache.org <us...@ignite.apache.org>
> *Objet :* Re: Question about REST/C++ client
>
> The easy answer to the "50 gets in parallel" is to use GetAll.
>
> On Tue, 13 Feb 2024 at 09:54, Igor Sapego <is...@apache.org> wrote:
>
> 1. There is no such API. Seems like a weird error in documentation. We
> will fix it. If you want to make 50 gets in parallel, I think it's better
> to run 50 clients with the current API. You can use the same client too,
> but it may be slower, depending on how many nodes you have in a cluster.
> 2. What do you mean by increments? Do you mean getting value, modifying it
> and putting it back?
> 3. Atomics are faster in most cases (ReplaceIfEquals). Yes, there is a
> real overhead for transactions, though how big it is depends on the
> transaction isolation level.
>
> Best Regards,
> Igor
>
>
> On Thu, Feb 8, 2024 at 5:05 PM Louis C <lc...@outlook.fr> wrote:
>
>
> Hello,
>
> I am trying to use/test Apache Ignite for a particular use case. For
> background reference, my use case of Ignite is to do 100Ks (to begin with)
> of "Gets" and of "increments" of values that will be stored in probably
> multiple caches in Apache Ignite.
> I read the Ignite documentation, but I couldn't figure out things in the
> C++ API side.
>
>
>    1. I have read in the documentation
>    <https://ignite.apache.org/docs/latest/key-value-api/basic-cache-operations>
>    <https://ignite.apache.org/docs/latest/key-value-api/basic-cache-operations>that
>    there is a "GetAsync" method in C++ client. But I could not find it in the
>    code. Is it a deprecated API ? If so, let's imagine I want to do multiple
>    calls in parallel (let's say 50 for instance), how can I achieve this ? Can
>    I just call multiple "Get"s in parallel in my threads without any problem ?
>    Must I create a client for each thread ? There does not seem to be anything
>    related to thread safety of these methods in the doc...
>    2. Does doing 100K "increments" of values in a cache seem achievable
>    on an Ignite cluster of a single node (let's say the CPU is a last gen i7
>    with 8 physical cores)? The problem that I have is that I have very good
>    performances (+100K "gets") using the batch methods of the Rest API, but no
>    batch method exist for "increments", and the overhead of each http call
>    cripples the performance to a few 1000s/s. What would be the "best" way of
>    achieving this (preferably in C++ or Rest API, but I am open to Java too )?
>    3. Related but a bit different : if I want to add a value to an
>    existing one in the store (doing an "increment"), would it be (in general)
>    faster to do it using the "transaction" mode or to use the
>    "ReplaceIfEquals" methods (in general I do not update the same values in
>    the same time) ? Is there a real overhead for transactions ?
>
>
> Thanks in advance
>
> Louis
>
>

RE: Question about REST/C++ client

Posted by Louis C <lc...@outlook.fr>.

Hello,

Thanks for your answers Igor and Stephen.
Igor, to answer your question to the second point I used the same term as used in the Rest API[cid:ab48a0d9-3170-457d-a922-c37cacd9171f], but indeed it more or less what you said (even if in this API we do not have to retrieve explicitely the base value).
Best regards,
Louis
________________________________
De : Stephen Darlington <sd...@apache.org>
Envoyé : mardi 13 février 2024 11:22
À : user@ignite.apache.org <us...@ignite.apache.org>
Objet : Re: Question about REST/C++ client

The easy answer to the "50 gets in parallel" is to use GetAll.

On Tue, 13 Feb 2024 at 09:54, Igor Sapego <is...@apache.org>> wrote:
1. There is no such API. Seems like a weird error in documentation. We will fix it. If you want to make 50 gets in parallel, I think it's better to run 50 clients with the current API. You can use the same client too, but it may be slower, depending on how many nodes you have in a cluster.
2. What do you mean by increments? Do you mean getting value, modifying it and putting it back?
3. Atomics are faster in most cases (ReplaceIfEquals). Yes, there is a real overhead for transactions, though how big it is depends on the transaction isolation level.

Best Regards,
Igor

On Thu, Feb 8, 2024 at 5:05 PM Louis C <lc...@outlook.fr>> wrote:

Hello,

Thanks in advance

Louis

Re: Question about REST/C++ client

Posted by Stephen Darlington <sd...@apache.org>.

The easy answer to the "50 gets in parallel" is to use GetAll.

On Tue, 13 Feb 2024 at 09:54, Igor Sapego <is...@apache.org> wrote:

> 1. There is no such API. Seems like a weird error in documentation. We
> will fix it. If you want to make 50 gets in parallel, I think it's better
> to run 50 clients with the current API. You can use the same client too,
> but it may be slower, depending on how many nodes you have in a cluster.
> 2. What do you mean by increments? Do you mean getting value, modifying it
> and putting it back?
> 3. Atomics are faster in most cases (ReplaceIfEquals). Yes, there is a
> real overhead for transactions, though how big it is depends on the
> transaction isolation level.
>
> Best Regards,
> Igor
>
>
> On Thu, Feb 8, 2024 at 5:05 PM Louis C <lc...@outlook.fr> wrote:
>
>>
>> Hello,
>>
>> I am trying to use/test Apache Ignite for a particular use case. For
>> background reference, my use case of Ignite is to do 100Ks (to begin with)
>> of "Gets" and of "increments" of values that will be stored in probably
>> multiple caches in Apache Ignite.
>> I read the Ignite documentation, but I couldn't figure out things in the
>> C++ API side.
>>
>>
>>    1. I have read in the documentation
>>    <https://ignite.apache.org/docs/latest/key-value-api/basic-cache-operations>
>>    <https://ignite.apache.org/docs/latest/key-value-api/basic-cache-operations>that
>>    there is a "GetAsync" method in C++ client. But I could not find it in the
>>    code. Is it a deprecated API ? If so, let's imagine I want to do multiple
>>    calls in parallel (let's say 50 for instance), how can I achieve this ? Can
>>    I just call multiple "Get"s in parallel in my threads without any problem ?
>>    Must I create a client for each thread ? There does not seem to be anything
>>    related to thread safety of these methods in the doc...
>>    2. Does doing 100K "increments" of values in a cache seem achievable
>>    on an Ignite cluster of a single node (let's say the CPU is a last gen i7
>>    with 8 physical cores)? The problem that I have is that I have very good
>>    performances (+100K "gets") using the batch methods of the Rest API, but no
>>    batch method exist for "increments", and the overhead of each http call
>>    cripples the performance to a few 1000s/s. What would be the "best" way of
>>    achieving this (preferably in C++ or Rest API, but I am open to Java too )?
>>    3. Related but a bit different : if I want to add a value to an
>>    existing one in the store (doing an "increment"), would it be (in general)
>>    faster to do it using the "transaction" mode or to use the
>>    "ReplaceIfEquals" methods (in general I do not update the same values in
>>    the same time) ? Is there a real overhead for transactions ?
>>
>>
>> Thanks in advance
>>
>> Louis
>>
>

Re: Question about REST/C++ client

Posted by Igor Sapego <is...@apache.org>.

1. There is no such API. Seems like a weird error in documentation. We will
fix it. If you want to make 50 gets in parallel, I think it's better to run
50 clients with the current API. You can use the same client too, but it
may be slower, depending on how many nodes you have in a cluster.
2. What do you mean by increments? Do you mean getting value, modifying it
and putting it back?
3. Atomics are faster in most cases (ReplaceIfEquals). Yes, there is a real
overhead for transactions, though how big it is depends on the transaction
isolation level.

Best Regards,
Igor


On Thu, Feb 8, 2024 at 5:05 PM Louis C <lc...@outlook.fr> wrote:

>
> Hello,
>
> I am trying to use/test Apache Ignite for a particular use case. For
> background reference, my use case of Ignite is to do 100Ks (to begin with)
> of "Gets" and of "increments" of values that will be stored in probably
> multiple caches in Apache Ignite.
> I read the Ignite documentation, but I couldn't figure out things in the
> C++ API side.
>
>
>    1. I have read in the documentation
>    <https://ignite.apache.org/docs/latest/key-value-api/basic-cache-operations>
>    <https://ignite.apache.org/docs/latest/key-value-api/basic-cache-operations>that
>    there is a "GetAsync" method in C++ client. But I could not find it in the
>    code. Is it a deprecated API ? If so, let's imagine I want to do multiple
>    calls in parallel (let's say 50 for instance), how can I achieve this ? Can
>    I just call multiple "Get"s in parallel in my threads without any problem ?
>    Must I create a client for each thread ? There does not seem to be anything
>    related to thread safety of these methods in the doc...
>    2. Does doing 100K "increments" of values in a cache seem achievable
>    on an Ignite cluster of a single node (let's say the CPU is a last gen i7
>    with 8 physical cores)? The problem that I have is that I have very good
>    performances (+100K "gets") using the batch methods of the Rest API, but no
>    batch method exist for "increments", and the overhead of each http call
>    cripples the performance to a few 1000s/s. What would be the "best" way of
>    achieving this (preferably in C++ or Rest API, but I am open to Java too )?
>    3. Related but a bit different : if I want to add a value to an
>    existing one in the store (doing an "increment"), would it be (in general)
>    faster to do it using the "transaction" mode or to use the
>    "ReplaceIfEquals" methods (in general I do not update the same values in
>    the same time) ? Is there a real overhead for transactions ?
>
>
> Thanks in advance
>
> Louis
>