You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by "Ch. Vishal Ratnam" <vi...@gmail.com> on 2022/01/24 05:19:53 UTC

Query - Apache Ignite Data Migration

Hi folks,

I'm a software developer in India at an e-commerce company (Tata Cliq). We
use apache ignite (version 2.7.5) as an in-memory data grid for one of our
core applications.

The problem:
We have a 2 node cluster in our current design. All caches are in
partitioned mode with 0 backups. We need to increase add backups to all the
caches and add more nodes to the same cluster.

From what we've tried, it looks like an existing cache cannot be modified
to have backups. Is this possible? Can an existing cache with 0 backups, be
modified to have 1 or more backups?
If the modification on an existing cache is not possible, what is the
solution for this?
We created a new cache with tried migrating the data from the old cache to
the new cache. But the data in the old cache is very large (Around 150
million rows). We used a client (Java thin client) to read an entire cache
and process each row, then used ignite streamer to write data to the new
cache. Since the data is huge, the Ignite cluster is crashing and only
around 3 million rows are getting processed and migrated to the new cache.

What is the solution for adding backups to an existing cache with 0 backups?
What is the optimal way to migrate 150 million records from one cache to
another?

Any help would be greatly appreciated.

Thanks & Regards,
Vishal Ratnam