You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by gvaidya <gv...@insynccloud.com> on 2021/01/15 19:30:51 UTC
Apache Ignite cluster node not loading all data from external DB
I am completely new to Apache Ignite and trying out some examples for
evaluating IMDG capabilities of Ignite. I am using GridGain web console to
create a simple project where I have imported a single table from my MS-SQL
external persistent database. This is a large table with ~9 million records
with over 100 columns. After downloading the project, compiling in VS Code,
and deploying the packaged JAR file in ignite installation folder, I start
the cluster (single node). So far so good, I can see the cluster and node in
my GridGain browser web console. However, every time I try to load the cache
(using Web Console Load on my cache), it only loads ~44K records which I am
able to query (e.g. using DBeaver). I tried setting the dataRegion mem size
to 12G. The machine has 32G RAM and over 70% is free. The cache does have
Read-Through enabled and I understand that key-value access via API can read
all data. However I am trying to run SQL on the cache to aggregate the data
for my evaluation. Why does the cacheLoad process not load all data from my
external MSSQL DB? Any thoughts? Thanks in advance.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Apache Ignite cluster node not loading all data from external
DB
Posted by gvaidya <gv...@insynccloud.com>.
Thank you for your response Ilya. I did not see any exceptions or errors in
the node logs even when running the ignite.bat process in verbose mode.
Having said that, I switched to another MS-SQL server and did not face the
same issue where only partial data was loaded. Facing other issues now and
will close this thread.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Apache Ignite cluster node not loading all data from external DB
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Do you see any exceptions in server nodes' logs? Can you please show your
cache store configuration?
Regards,
--
Ilya Kasnacheev
пт, 15 янв. 2021 г. в 22:30, gvaidya <gv...@insynccloud.com>:
> I am completely new to Apache Ignite and trying out some examples for
> evaluating IMDG capabilities of Ignite. I am using GridGain web console to
> create a simple project where I have imported a single table from my MS-SQL
> external persistent database. This is a large table with ~9 million records
> with over 100 columns. After downloading the project, compiling in VS Code,
> and deploying the packaged JAR file in ignite installation folder, I start
> the cluster (single node). So far so good, I can see the cluster and node
> in
> my GridGain browser web console. However, every time I try to load the
> cache
> (using Web Console Load on my cache), it only loads ~44K records which I am
> able to query (e.g. using DBeaver). I tried setting the dataRegion mem size
> to 12G. The machine has 32G RAM and over 70% is free. The cache does have
> Read-Through enabled and I understand that key-value access via API can
> read
> all data. However I am trying to run SQL on the cache to aggregate the data
> for my evaluation. Why does the cacheLoad process not load all data from my
> external MSSQL DB? Any thoughts? Thanks in advance.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>