Re: Pagination to fetch complete Cache data

[vkulichenko <va...@gmail.com>]

Hi,

Query results are already paginated by default. Page size is 1024 elements
by default and can be changed via Query.setPageSize() method. How many
entries in cache do you have? Iteration through the large data set is not
going to be fast anyway.

SQL should be used for indexed search, and they require additional
configuration. Refer to [1] for details.

[1] https://apacheignite.readme.io/docs/sql-queries

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Pagination-to-fetch-complete-Cache-data-tp3071p3073.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Pagination to fetch complete Cache data

["sujay_kb@ymail.com" <su...@ymail.com>]

Yes Val, let me try the options

I will get the Ignite from master and check it out

Thanks again Val for timely help



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Pagination-to-fetch-complete-Cache-data-tp3071p3111.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Pagination to fetch complete Cache data

[vkulichenko <va...@gmail.com>]

Hi,

Fetching all entries to the client will never be effective because it
involves transferring too much data across network. If you need to process
the whole data set, it's usually better to use Compute Grid [1]. Broadcast a
closure or execute a map-reduce task and send only results to the client. Is
it something that can work for you?

NPE in SQL is a known issue and already fixed. You can build Ignite from
master and check if it works for you.

[1] https://apacheignite.readme.io/docs/compute-grid

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Pagination-to-fetch-complete-Cache-data-tp3071p3087.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Pagination to fetch complete Cache data

["sujay_kb@ymail.com" <su...@ymail.com>]

Thanks for Info on Pagesize, i will try it out

I am having 3 million records i believe its small dataset, then also i am
facing this issue

Can you suggest which fetch approach is best for huge dataset (cache), 

Regarding previous question/post i have mentioned the exception(*Exception
in thread "main" javax.cache.CacheException: Failed to run reduce query
locally*.) while using the Query (select * [CacheName]) - is this a bug? -
because with the smaller dataset it works fine.



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Pagination-to-fetch-complete-Cache-data-tp3071p3084.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Pagination to fetch complete Cache data

[vkulichenko <va...@gmail.com>]

Hi,

Please properly subscribe to the mailing list so that the community can
receive email notifications for you messages. Here is the instruction:
http://apache-ignite-users.70518.x6.nabble.com/mailing_list/MailingListOptions.jtp?forum=1


begineer wrote
> Sorry for mistake, correction in below line. Set pageSize to 10. But
> output remains same. I still get 20 records in result.
> 
> Set
> <String>
>  res =
> cache.query(query.setPageSize(10)).getAll().stream().map(Cache.Entry::getValue).collect(toSet());

QueryCursor allows you to iterate through the whole result set, and the
pagination happens internally. This means that client will fetch rows in
pages discarding old ones, so that you never have more than 10 rows in local
memory at the same time.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Pagination-to-fetch-complete-Cache-data-tp3071p6394.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.