You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Surinder Mehra <re...@gmail.com> on 2022/04/28 06:59:29 UTC

Query performance

Hi,
We are running a sql field query to fetch 4million records from ignite
cache. We have created a group index for all fields used in where clause
and can see group index used. But the query takes 20 minutes to fetch all
records. If we provide more strict criteria to fetch only say 500 records,
it completes in less than 200 millis. This makes me wonder if I am missing
some configuration to make result fetching faster or I am not using ignite
correctly for this use case. Below log is printed during query execution.
Could you please advise me?

[11:46:30,694][WARNING][query-#8568][GridMapQueryExecutor] Query produced
big result set.  [fetched=100000, duration=120331ms, type=MAP,
distributedJoin=false, enforceJoinOrder=false, lazy=true,

Re[2]: Query performance

Posted by Zhenya Stanilovsky <ar...@mail.ru>.
Got it, can you show both sql requests (with strict and non strict criteria) and EXPLAIN output in both cases ?
Do you have indexes?
 
>Hey, it's enabled already. Please check the console log in my email   
>On Thu, 28 Apr 2022, 13:16 Zhenya Stanilovsky, < arzamas123@mail.ru > wrote:
>>
>>Hi, can you check the same with lazy [1] flag ? 
>>
>>[1]  https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/query/SqlFieldsQuery.html#setLazy-boolean-
>>   
>>>Hi,
>>>We are running a sql field query to fetch 4million records from ignite cache. We have created a group index for all fields used in where clause and can see group index used. But the query takes 20 minutes to fetch all records. If we provide more strict criteria to fetch only say 500 records, it completes in less than 200 millis. This makes me wonder if I am missing some configuration to make result fetching faster or I am not using ignite correctly for this use case. Below log is printed during query execution.  Could you please advise me?
>>>
>>>[11:46:30,694][WARNING][query-#8568][GridMapQueryExecutor] Query produced big result set.  [fetched=100000, duration=120331ms, type=MAP, distributedJoin=false, enforceJoinOrder=false, lazy=true, 
>> 
>> 
>> 
>>  
 
 
 
 

Re: Query performance

Posted by Surinder Mehra <re...@gmail.com>.
Hey, it's enabled already. Please check the console log in my email

On Thu, 28 Apr 2022, 13:16 Zhenya Stanilovsky, <ar...@mail.ru> wrote:

>
> Hi, can you check the same with lazy [1] flag ?
>
> [1]
> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/query/SqlFieldsQuery.html#setLazy-boolean-
>
>
> Hi,
> We are running a sql field query to fetch 4million records from ignite
> cache. We have created a group index for all fields used in where clause
> and can see group index used. But the query takes 20 minutes to fetch all
> records. If we provide more strict criteria to fetch only say 500 records,
> it completes in less than 200 millis. This makes me wonder if I am missing
> some configuration to make result fetching faster or I am not using ignite
> correctly for this use case. Below log is printed during query execution.
> Could you please advise me?
>
> [11:46:30,694][WARNING][query-#8568][GridMapQueryExecutor] Query produced
> big result set.  [fetched=100000, duration=120331ms, type=MAP,
> distributedJoin=false, enforceJoinOrder=false, lazy=true,
>
>
>
>
>
>

Re: Query performance

Posted by Zhenya Stanilovsky <ar...@mail.ru>.

Hi, can you check the same with lazy [1] flag ? 

[1]  https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/query/SqlFieldsQuery.html#setLazy-boolean-
 
>Hi,
>We are running a sql field query to fetch 4million records from ignite cache. We have created a group index for all fields used in where clause and can see group index used. But the query takes 20 minutes to fetch all records. If we provide more strict criteria to fetch only say 500 records, it completes in less than 200 millis. This makes me wonder if I am missing some configuration to make result fetching faster or I am not using ignite correctly for this use case. Below log is printed during query execution.  Could you please advise me?
>
>[11:46:30,694][WARNING][query-#8568][GridMapQueryExecutor] Query produced big result set.  [fetched=100000, duration=120331ms, type=MAP, distributedJoin=false, enforceJoinOrder=false, lazy=true,