You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Surinder Mehra <re...@gmail.com> on 2022/03/08 03:56:16 UTC
getAllkeys Api
Hi,
I was looking for a way to fetch all cache keys in an efficient manner from
ignite cache. Looks like there is no such api yet.
Would it be correct to use a transformer in Scan query to just fetch the
key from entry being scanned. This will avoid fetching full cache entry
from server nodes
Or is there a better way to do this.
https://www.gridgain.com/docs/latest/developers-guide/key-value-api/using-scan-queries
Re: getAllkeys Api
Posted by Surinder Mehra <re...@gmail.com>.
Thanks for pointing out heap side view on this. Will try it
On Tue, 8 Mar 2022, 21:09 Stephen Darlington, <
stephen.darlington@gridgain.com> wrote:
> Yes, the transformer runs on the server side, so it should be functionally
> the same. However, a scan is going to copy the key and value to the heap,
> so it’s likely to use more memory than a SQL query.
>
> On 8 Mar 2022, at 10:04, Surinder Mehra <re...@gmail.com> wrote:
>
> Hi,
> Actually SQL is not enabled on these caches.
> We can try compute but I was thinking to use scan query with 'transformer'
> . I assume transformer runs on server node right so isn't it same as
> "select _key from cache"
>
> On Tue, 8 Mar 2022, 15:08 Stephen Darlington, <
> stephen.darlington@gridgain.com> wrote:
>
>> You could use SQL. “select _key from table”
>>
>> But really, copying all the data over the network is often not the best
>> strategy. Send a compute job to each node or partition and work on the data
>> “in place” on the data nodes.
>>
>> On 8 Mar 2022, at 03:56, Surinder Mehra <re...@gmail.com> wrote:
>>
>> Hi,
>> I was looking for a way to fetch all cache keys in an efficient manner
>> from ignite cache. Looks like there is no such api yet.
>> Would it be correct to use a transformer in Scan query to just fetch the
>> key from entry being scanned. This will avoid fetching full cache entry
>> from server nodes
>>
>> Or is there a better way to do this.
>>
>>
>> https://www.gridgain.com/docs/latest/developers-guide/key-value-api/using-scan-queries
>>
>>
>>
>>
>
Re: getAllkeys Api
Posted by Stephen Darlington <st...@gridgain.com>.
Yes, the transformer runs on the server side, so it should be functionally the same. However, a scan is going to copy the key and value to the heap, so it’s likely to use more memory than a SQL query.
> On 8 Mar 2022, at 10:04, Surinder Mehra <re...@gmail.com> wrote:
>
> Hi,
> Actually SQL is not enabled on these caches.
> We can try compute but I was thinking to use scan query with 'transformer' . I assume transformer runs on server node right so isn't it same as "select _key from cache"
>
> On Tue, 8 Mar 2022, 15:08 Stephen Darlington, <stephen.darlington@gridgain.com <ma...@gridgain.com>> wrote:
> You could use SQL. “select _key from table”
>
> But really, copying all the data over the network is often not the best strategy. Send a compute job to each node or partition and work on the data “in place” on the data nodes.
>
>> On 8 Mar 2022, at 03:56, Surinder Mehra <rednirus@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hi,
>> I was looking for a way to fetch all cache keys in an efficient manner from ignite cache. Looks like there is no such api yet.
>> Would it be correct to use a transformer in Scan query to just fetch the key from entry being scanned. This will avoid fetching full cache entry from server nodes
>>
>> Or is there a better way to do this.
>>
>> https://www.gridgain.com/docs/latest/developers-guide/key-value-api/using-scan-queries <https://www.gridgain.com/docs/latest/developers-guide/key-value-api/using-scan-queries>
>
Re: getAllkeys Api
Posted by Surinder Mehra <re...@gmail.com>.
Hi,
Actually SQL is not enabled on these caches.
We can try compute but I was thinking to use scan query with 'transformer'
. I assume transformer runs on server node right so isn't it same as
"select _key from cache"
On Tue, 8 Mar 2022, 15:08 Stephen Darlington, <
stephen.darlington@gridgain.com> wrote:
> You could use SQL. “select _key from table”
>
> But really, copying all the data over the network is often not the best
> strategy. Send a compute job to each node or partition and work on the data
> “in place” on the data nodes.
>
> On 8 Mar 2022, at 03:56, Surinder Mehra <re...@gmail.com> wrote:
>
> Hi,
> I was looking for a way to fetch all cache keys in an efficient manner
> from ignite cache. Looks like there is no such api yet.
> Would it be correct to use a transformer in Scan query to just fetch the
> key from entry being scanned. This will avoid fetching full cache entry
> from server nodes
>
> Or is there a better way to do this.
>
>
> https://www.gridgain.com/docs/latest/developers-guide/key-value-api/using-scan-queries
>
>
>
>
Re: getAllkeys Api
Posted by Stephen Darlington <st...@gridgain.com>.
You could use SQL. “select _key from table”
But really, copying all the data over the network is often not the best strategy. Send a compute job to each node or partition and work on the data “in place” on the data nodes.
> On 8 Mar 2022, at 03:56, Surinder Mehra <re...@gmail.com> wrote:
>
> Hi,
> I was looking for a way to fetch all cache keys in an efficient manner from ignite cache. Looks like there is no such api yet.
> Would it be correct to use a transformer in Scan query to just fetch the key from entry being scanned. This will avoid fetching full cache entry from server nodes
>
> Or is there a better way to do this.
>
> https://www.gridgain.com/docs/latest/developers-guide/key-value-api/using-scan-queries <https://www.gridgain.com/docs/latest/developers-guide/key-value-api/using-scan-queries>