You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Kurt Semba <ks...@extremenetworks.com> on 2019/09/10 15:01:19 UTC

Data from multiple MySQL DBs

Hi all,

I need to sync data from multiple MySQL databases into Ignite.

All those MySQL databases follow the same schema / same tables but obviously contain different data. They are separate instances of the same DB and we want to pull all that data into Ignite to have a central store to query against.

What would be the best strategy to solve this?

Create a dedicated Cache for each DB instance and then find a way to create a UNION SQL query over all those Caches?
Find a way to define all the MySQL DBs as data sources in the Spring XML cluster config file and hope that Ignite pulls the same type of data from all DBs?

I’m open for suggestions 😊
Thanks
Kurt

RE: Data from multiple MySQL DBs

Posted by Kurt Semba <ks...@extremenetworks.com>.
Hi Ilya,

Thanks for your suggestion. So what you are suggesting is:

  *   Run a Ignite single cluster which has the correct schemas
  *   Develop a tool which regularly pulls data from all these MySQL DBs and that uses the Ignite Data Streamer API to update the Caches in the cluster


  *   That way, all datasets from a specific schema & table from all MySQL DBs will end up in the same Ignite Cache and thus the data will be a UNION of all data from all MySQL DBs.
Kurt

From: Ilya Kasnacheev <il...@gmail.com>
Sent: Wednesday, September 11, 2019 5:10 PM
To: user@ignite.apache.org
Subject: Re: Data from multiple MySQL DBs

Hello!

I think the best course will be to read all DBs in user code and use data streamer to populate Ignite.

Regards,
--
Ilya Kasnacheev


вт, 10 сент. 2019 г. в 18:01, Kurt Semba <ks...@extremenetworks.com>>:
Hi all,

I need to sync data from multiple MySQL databases into Ignite.

All those MySQL databases follow the same schema / same tables but obviously contain different data. They are separate instances of the same DB and we want to pull all that data into Ignite to have a central store to query against.

What would be the best strategy to solve this?

Create a dedicated Cache for each DB instance and then find a way to create a UNION SQL query over all those Caches?
Find a way to define all the MySQL DBs as data sources in the Spring XML cluster config file and hope that Ignite pulls the same type of data from all DBs?

I’m open for suggestions 😊
Thanks
Kurt

Re: Data from multiple MySQL DBs

Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!

I think the best course will be to read all DBs in user code and use data
streamer to populate Ignite.

Regards,
-- 
Ilya Kasnacheev


вт, 10 сент. 2019 г. в 18:01, Kurt Semba <ks...@extremenetworks.com>:

> Hi all,
>
>
>
> I need to sync data from multiple MySQL databases into Ignite.
>
>
>
> All those MySQL databases follow the same schema / same tables but
> obviously contain different data. They are separate instances of the same
> DB and we want to pull all that data into Ignite to have a central store to
> query against.
>
>
>
> What would be the best strategy to solve this?
>
>
>
> Create a dedicated Cache for each DB instance and then find a way to
> create a UNION SQL query over all those Caches?
>
> Find a way to define all the MySQL DBs as data sources in the Spring XML
> cluster config file and hope that Ignite pulls the same type of data from
> all DBs?
>
>
>
> I’m open for suggestions 😊
>
> Thanks
>
> Kurt
>