You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Ray <ra...@cisco.com> on 2017/09/27 12:02:56 UTC
Full table scan query by ODBC causing node shutdown
I have a cache with 6 million rows.
The cache is configured with Partitioned cache mode and 3 backups.
And I'm running Ignite 2.1 on 4 nodes and each have a 8Gb heap size and 30Gb
non-heap size configured.
When I'm trying to fetch all the rows using "select * from mytable" by odbc
driver on windows, the node I'm querying will throw exception and eventually
shut down.
The debug log is in the attachment, the sql query begins at line 2579.
Please advice me how to solve this problem
Thanks
ignite-6593a74d.log
<http://apache-ignite-users.70518.x6.nabble.com/file/t1346/ignite-6593a74d.log>
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Full table scan query by ODBC causing node shutdown
Posted by vijayendra bhati <ve...@yahoo.com>.
Sent from Yahoo Mail on Android
On Wed, Sep 27, 2017 at 1:03 PM, Ray<ra...@cisco.com> wrote: I have a cache with 6 million rows.
The cache is configured with Partitioned cache mode and 3 backups.
And I'm running Ignite 2.1 on 4 nodes and each have a 8Gb heap size and 30Gb
non-heap size configured.
When I'm trying to fetch all the rows using "select * from mytable" by odbc
driver on windows, the node I'm querying will throw exception and eventually
shut down.
The debug log is in the attachment, the sql query begins at line 2579.
Please advice me how to solve this problem
Thanks
ignite-6593a74d.log
<http://apache-ignite-users.70518.x6.nabble.com/file/t1346/ignite-6593a74d.log>
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Full table scan query by ODBC causing node shutdown
Posted by Nikolai Tikhonov <nt...@apache.org>.
CacheConfiguration.queryParallelism parameter affects all queries which
perform under this cache. It doesn't matter which API you will use. I guess
that when you increase heap size and tune gc it will bring significant
performance improvement for the case.
Thanks,
Nikolai
On Wed, Sep 27, 2017 at 6:11 PM, Ray <ra...@cisco.com> wrote:
> Ok, I'll try increasing the heap size.
>
> One more question here, from the log it says the full table scan query is
> taking 30s.
> And I wonder is there any way to speed the query up?
> I found this article
> https://apacheignite.readme.io/v2.2/docs/sql-performance-
> and-debugging#section-query-parallelism
>
> And my question is if I set CacheConfiguration.queryParallelism parameter
> when I ingest the data, will it take effect when I query the data by ODBC?
> Or is it only effective querying the data by Java API with
> CacheConfiguration.queryParallelism specified?
>
> Thanks
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Full table scan query by ODBC causing node shutdown
Posted by Ray <ra...@cisco.com>.
Ok, I'll try increasing the heap size.
One more question here, from the log it says the full table scan query is
taking 30s.
And I wonder is there any way to speed the query up?
I found this article
https://apacheignite.readme.io/v2.2/docs/sql-performance-and-debugging#section-query-parallelism
And my question is if I set CacheConfiguration.queryParallelism parameter
when I ingest the data, will it take effect when I query the data by ODBC?
Or is it only effective querying the data by Java API with
CacheConfiguration.queryParallelism specified?
Thanks
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Full table scan query by ODBC causing node shutdown
Posted by Nikolai Tikhonov <nt...@apache.org>.
Hello!
Seems that you don't have enough memory for loading whole data set to
memory. Ignite move the whole ResultSet to memory when executing SQL query.
How I see from logs that this mertics before query execution:
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
...
^-- Heap [used=5054MB, free=35.88%, comm=7882MB]
I suppose that node was gone from topology due to long GC pauses. It's
known limitation and in 2.3 release this behaviour should be changed. More
details you can found there:
https://issues.apache.org/jira/browse/IGNITE-5991. As workaround you can
try to increase heap size on server nodes.
On Wed, Sep 27, 2017 at 3:02 PM, Ray <ra...@cisco.com> wrote:
> I have a cache with 6 million rows.
> The cache is configured with Partitioned cache mode and 3 backups.
> And I'm running Ignite 2.1 on 4 nodes and each have a 8Gb heap size and
> 30Gb
> non-heap size configured.
> When I'm trying to fetch all the rows using "select * from mytable" by odbc
> driver on windows, the node I'm querying will throw exception and
> eventually
> shut down.
>
> The debug log is in the attachment, the sql query begins at line 2579.
>
> Please advice me how to solve this problem
> Thanks
> ignite-6593a74d.log
> <http://apache-ignite-users.70518.x6.nabble.com/file/
> t1346/ignite-6593a74d.log>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>