You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Cihan Keser <ig...@ckeser.com> on 2019/10/24 11:52:32 UTC
Multiple embedded Ignite instances as replicated cache
Hi,
I have a case where I'm trying to use Ignite (embedded i.e.: Ignition.getOrStart(igniteConfiguration) called in a Spring bean definition) in a multi-node Spring Boot application with active-active setup. My goal is to utilize Ignite as a shared, atomic, and persistent key-value store.
For this test case I have two application nodes running the same code and below is my Spring configuration class; yet I keep getting "CacheInvalidStateException: Failed to execute cache operation (all partition owners have left the grid, partition data has been lost) [cacheName=sequenceNumberCache, part=187, key=XXX.SequenceNumberKey@f3c5d51c]".
Can anyone shed some light on what I'm doing wrong? I understand that manual topology management is preferred for rebalancing concerns but considering I'm using the in fully replicated mode with native persistence, I see no harm (though am still suspicious) in calling cluster.active(true) as soon as app inits. I really would like to avoid any manual topology management, is there a better way of doing this?
Thanks in advance,
Cihan Keser
```
@Configuration
public class SequenceNumberStoreConfig {
public static final String CACHE_NAME = "sequenceNumberCache";
@Bean
public Ignite igniteInstance(
@Value("${ignite.workDir}") String workDir
) {
CacheConfiguration<Integer, SequenceNumberKey> cacheConfiguration =
new CacheConfiguration<>(CACHE_NAME);
cacheConfiguration.setIndexedTypes(SequenceNumberKey.class, Integer.class);
cacheConfiguration.setAtomicityMode(CacheAtomicityMode.ATOMIC);
cacheConfiguration.setBackups(1);
cacheConfiguration.setWriteSynchronizationMode(CacheWriteSynchronizationMode.PRIMARY_SYNC);
cacheConfiguration.setCacheMode(CacheMode.REPLICATED);
cacheConfiguration.setPartitionLossPolicy(PartitionLossPolicy.READ_WRITE_SAFE);
cacheConfiguration.setReadFromBackup(true);
cacheConfiguration.setRebalanceMode(CacheRebalanceMode.ASYNC);
DataRegionConfiguration dataRegionConfiguration = new DataRegionConfiguration();
dataRegionConfiguration.setPersistenceEnabled(true);
DataStorageConfiguration dataStorageConfiguration = new DataStorageConfiguration();
dataStorageConfiguration.setDefaultDataRegionConfiguration(dataRegionConfiguration);
dataStorageConfiguration.setWalMode(WALMode.FSYNC);
TcpDiscoveryVmIpFinder tcpDiscoveryVmIpFinder = new TcpDiscoveryVmIpFinder();
tcpDiscoveryVmIpFinder.setShared(true);
tcpDiscoveryVmIpFinder.setAddresses(ImmutableList.of("node1", "node2"));
TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi();
discoverySpi.setIpFinder(tcpDiscoveryVmIpFinder);
IgniteConfiguration igniteConfiguration = new IgniteConfiguration();
igniteConfiguration.setCacheConfiguration(cacheConfiguration);
igniteConfiguration.setDiscoverySpi(discoverySpi);
igniteConfiguration.setDataStorageConfiguration(dataStorageConfiguration);
igniteConfiguration.setDeploymentMode(DeploymentMode.SHARED);
igniteConfiguration.setWorkDirectory(workDir);
igniteConfiguration.setGridLogger(new Slf4jLogger());
igniteConfiguration.setClientMode(false);
Ignite ignite = Ignition.getOrStart(igniteConfiguration);
ignite.cluster().active(true);
return ignite;
}
@Bean
public SequenceNumberProvider sequenceNumberStore(Ignite ignite) {
IgniteCache<SequenceNumberKey, Integer> igniteCache = ignite.getOrCreateCache(CACHE_NAME);
return new SequenceNumberProvider(igniteCache);
}
}
```
Re: Multiple embedded Ignite instances as replicated cache
Posted by Cihan Keser <ig...@ckeser.com>.
Thanks for the info, Ilya!
Do you think it would have any actual risk to use PartitionLossPolicy.IGNORE with my configuration? Because as I understand the data is fully replicated on each node, so it cannot really lose a particular partition; or do I misunderstand?
Is there a ticket/link where I can track the status topology auto-adjust?
> Until then, I guess you will have to fix topology on entry of every node.
And by that you mean activating the cluster (or resetting the lost-nodes) after all nodes join, right? Or is there any other way?
Cheers,
Cihan
On Thu, Oct 24, 2019, at 18:26, Ilya Kasnacheev wrote:
> Hello!
>
> You will soon be able to use topology auto-adjust. Until then, I guess you will have to fix topology on entry of every node.
> --
> Ilya Kasnacheev
>
>
> чт, 24 окт. 2019 г. в 14:52, Cihan Keser <ig...@ckeser.com>:
>> Hi,
>>
>> I have a case where I'm trying to use Ignite (embedded i.e.: Ignition.getOrStart(igniteConfiguration) called in a Spring bean definition) in a multi-node Spring Boot application with active-active setup. My goal is to utilize Ignite as a shared, atomic, and persistent key-value store.
>> For this test case I have two application nodes running the same code and below is my Spring configuration class; yet I keep getting "CacheInvalidStateException: Failed to execute cache operation (all partition owners have left the grid, partition data has been lost) [cacheName=sequenceNumberCache, part=187, key=XXX.SequenceNumberKey@f3c5d51c]".
>>
>> Can anyone shed some light on what I'm doing wrong? I understand that manual topology management is preferred for rebalancing concerns but considering I'm using the in fully replicated mode with native persistence, I see no harm (though am still suspicious) in calling cluster.active(true) as soon as app inits. I really would like to avoid any manual topology management, is there a better way of doing this?
>>
>> Thanks in advance,
>> Cihan Keser
>>
>>
>> ```
>> @Configuration
>> public class SequenceNumberStoreConfig {
>>
>> public static final String CACHE_NAME = "sequenceNumberCache";
>>
>> @Bean
>> public Ignite igniteInstance(
>> @Value("${ignite.workDir}") String workDir
>> ) {
>> CacheConfiguration<Integer, SequenceNumberKey> cacheConfiguration =
>> new CacheConfiguration<>(CACHE_NAME);
>> cacheConfiguration.setIndexedTypes(SequenceNumberKey.class, Integer.class);
>> cacheConfiguration.setAtomicityMode(CacheAtomicityMode.ATOMIC);
>> cacheConfiguration.setBackups(1);
>> cacheConfiguration.setWriteSynchronizationMode(CacheWriteSynchronizationMode.PRIMARY_SYNC);
>> cacheConfiguration.setCacheMode(CacheMode.REPLICATED);
>> cacheConfiguration.setPartitionLossPolicy(PartitionLossPolicy.READ_WRITE_SAFE);
>> cacheConfiguration.setReadFromBackup(true);
>> cacheConfiguration.setRebalanceMode(CacheRebalanceMode.ASYNC);
>>
>> DataRegionConfiguration dataRegionConfiguration = new DataRegionConfiguration();
>> dataRegionConfiguration.setPersistenceEnabled(true);
>>
>> DataStorageConfiguration dataStorageConfiguration = new DataStorageConfiguration();
>> dataStorageConfiguration.setDefaultDataRegionConfiguration(dataRegionConfiguration);
>> dataStorageConfiguration.setWalMode(WALMode.FSYNC);
>>
>> TcpDiscoveryVmIpFinder tcpDiscoveryVmIpFinder = new TcpDiscoveryVmIpFinder();
>> tcpDiscoveryVmIpFinder.setShared(true);
>> tcpDiscoveryVmIpFinder.setAddresses(ImmutableList.of("node1", "node2"));
>>
>> TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi();
>> discoverySpi.setIpFinder(tcpDiscoveryVmIpFinder);
>>
>> IgniteConfiguration igniteConfiguration = new IgniteConfiguration();
>> igniteConfiguration.setCacheConfiguration(cacheConfiguration);
>> igniteConfiguration.setDiscoverySpi(discoverySpi);
>> igniteConfiguration.setDataStorageConfiguration(dataStorageConfiguration);
>> igniteConfiguration.setDeploymentMode(DeploymentMode.SHARED);
>> igniteConfiguration.setWorkDirectory(workDir);
>> igniteConfiguration.setGridLogger(new Slf4jLogger());
>> igniteConfiguration.setClientMode(false);
>>
>> Ignite ignite = Ignition.getOrStart(igniteConfiguration);
>> ignite.cluster().active(true);
>>
>> return ignite;
>> }
>>
>> @Bean
>> public SequenceNumberProvider sequenceNumberStore(Ignite ignite) {
>> IgniteCache<SequenceNumberKey, Integer> igniteCache = ignite.getOrCreateCache(CACHE_NAME);
>> return new SequenceNumberProvider(igniteCache);
>> }
>>
>> }
>> ```
Re: Multiple embedded Ignite instances as replicated cache
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
You will soon be able to use topology auto-adjust. Until then, I guess you
will have to fix topology on entry of every node.
--
Ilya Kasnacheev
чт, 24 окт. 2019 г. в 14:52, Cihan Keser <ig...@ckeser.com>:
> Hi,
>
> I have a case where I'm trying to use Ignite (embedded i.e.:
> Ignition.getOrStart(igniteConfiguration) called in a Spring bean
> definition) in a multi-node Spring Boot application with active-active
> setup. My goal is to utilize Ignite as a shared, atomic, and persistent
> key-value store.
> For this test case I have two application nodes running the same code and
> below is my Spring configuration class; yet I keep getting
> "CacheInvalidStateException: Failed to execute cache operation (all
> partition owners have left the grid, partition data has been lost)
> [cacheName=sequenceNumberCache, part=187, key=XXX.SequenceNumberKey@f3c5d51c
> ]".
>
> Can anyone shed some light on what I'm doing wrong? I understand that
> manual topology management is preferred for rebalancing concerns but
> considering I'm using the in fully replicated mode with native persistence,
> I see no harm (though am still suspicious) in calling cluster.active(true)
> as soon as app inits. I really would like to avoid any manual topology
> management, is there a better way of doing this?
>
> Thanks in advance,
> Cihan Keser
>
>
> ```
> @Configuration
> public class SequenceNumberStoreConfig {
>
> public static final String CACHE_NAME = "sequenceNumberCache";
>
> @Bean
> public Ignite igniteInstance(
> @Value("${ignite.workDir}") String workDir
> ) {
> CacheConfiguration<Integer, SequenceNumberKey> cacheConfiguration =
> new CacheConfiguration<>(CACHE_NAME);
> cacheConfiguration.setIndexedTypes(SequenceNumberKey.class,
> Integer.class);
> cacheConfiguration.setAtomicityMode(CacheAtomicityMode.ATOMIC);
> cacheConfiguration.setBackups(1);
>
> cacheConfiguration.setWriteSynchronizationMode(CacheWriteSynchronizationMode.PRIMARY_SYNC);
> cacheConfiguration.setCacheMode(CacheMode.REPLICATED);
>
> cacheConfiguration.setPartitionLossPolicy(PartitionLossPolicy.READ_WRITE_SAFE);
> cacheConfiguration.setReadFromBackup(true);
> cacheConfiguration.setRebalanceMode(CacheRebalanceMode.ASYNC);
>
> DataRegionConfiguration dataRegionConfiguration = new
> DataRegionConfiguration();
> dataRegionConfiguration.setPersistenceEnabled(true);
>
> DataStorageConfiguration dataStorageConfiguration = new
> DataStorageConfiguration();
>
> dataStorageConfiguration.setDefaultDataRegionConfiguration(dataRegionConfiguration);
> dataStorageConfiguration.setWalMode(WALMode.FSYNC);
>
> TcpDiscoveryVmIpFinder tcpDiscoveryVmIpFinder = new
> TcpDiscoveryVmIpFinder();
> tcpDiscoveryVmIpFinder.setShared(true);
> tcpDiscoveryVmIpFinder.setAddresses(ImmutableList.of("node1",
> "node2"));
>
> TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi();
> discoverySpi.setIpFinder(tcpDiscoveryVmIpFinder);
>
> IgniteConfiguration igniteConfiguration = new
> IgniteConfiguration();
> igniteConfiguration.setCacheConfiguration(cacheConfiguration);
> igniteConfiguration.setDiscoverySpi(discoverySpi);
>
> igniteConfiguration.setDataStorageConfiguration(dataStorageConfiguration);
> igniteConfiguration.setDeploymentMode(DeploymentMode.SHARED);
> igniteConfiguration.setWorkDirectory(workDir);
> igniteConfiguration.setGridLogger(new Slf4jLogger());
> igniteConfiguration.setClientMode(false);
>
> Ignite ignite = Ignition.getOrStart(igniteConfiguration);
> ignite.cluster().active(true);
>
> return ignite;
> }
>
> @Bean
> public SequenceNumberProvider sequenceNumberStore(Ignite ignite) {
> IgniteCache<SequenceNumberKey, Integer> igniteCache =
> ignite.getOrCreateCache(CACHE_NAME);
> return new SequenceNumberProvider(igniteCache);
> }
>
> }
> ```
>
Re: Multiple embedded Ignite instances as replicated cache
Posted by Denis Magda <dm...@apache.org>.
Hi Cihan,
Is there any reason why you cannot separate applications from Ignite as a
store/database? The embedded mode makes sense for low-latency use cases
while the standard client-server deployment option is better in terms of
manageability, scalability:
https://www.gridgain.com/docs/latest/installation-guide/deployment-modes#application-deployment-modes
-
Denis
On Thu, Oct 24, 2019 at 4:53 AM Cihan Keser <ig...@ckeser.com>
wrote:
> Hi,
>
> I have a case where I'm trying to use Ignite (embedded i.e.:
> Ignition.getOrStart(igniteConfiguration) called in a Spring bean
> definition) in a multi-node Spring Boot application with active-active
> setup. My goal is to utilize Ignite as a shared, atomic, and persistent
> key-value store.
> For this test case I have two application nodes running the same code and
> below is my Spring configuration class; yet I keep getting
> "CacheInvalidStateException: Failed to execute cache operation (all
> partition owners have left the grid, partition data has been lost)
> [cacheName=sequenceNumberCache, part=187, key=XXX.SequenceNumberKey@f3c5d51c
> ]".
>
> Can anyone shed some light on what I'm doing wrong? I understand that
> manual topology management is preferred for rebalancing concerns but
> considering I'm using the in fully replicated mode with native persistence,
> I see no harm (though am still suspicious) in calling cluster.active(true)
> as soon as app inits. I really would like to avoid any manual topology
> management, is there a better way of doing this?
>
> Thanks in advance,
> Cihan Keser
>
>
> ```
> @Configuration
> public class SequenceNumberStoreConfig {
>
> public static final String CACHE_NAME = "sequenceNumberCache";
>
> @Bean
> public Ignite igniteInstance(
> @Value("${ignite.workDir}") String workDir
> ) {
> CacheConfiguration<Integer, SequenceNumberKey> cacheConfiguration =
> new CacheConfiguration<>(CACHE_NAME);
> cacheConfiguration.setIndexedTypes(SequenceNumberKey.class,
> Integer.class);
> cacheConfiguration.setAtomicityMode(CacheAtomicityMode.ATOMIC);
> cacheConfiguration.setBackups(1);
>
> cacheConfiguration.setWriteSynchronizationMode(CacheWriteSynchronizationMode.PRIMARY_SYNC);
> cacheConfiguration.setCacheMode(CacheMode.REPLICATED);
>
> cacheConfiguration.setPartitionLossPolicy(PartitionLossPolicy.READ_WRITE_SAFE);
> cacheConfiguration.setReadFromBackup(true);
> cacheConfiguration.setRebalanceMode(CacheRebalanceMode.ASYNC);
>
> DataRegionConfiguration dataRegionConfiguration = new
> DataRegionConfiguration();
> dataRegionConfiguration.setPersistenceEnabled(true);
>
> DataStorageConfiguration dataStorageConfiguration = new
> DataStorageConfiguration();
>
> dataStorageConfiguration.setDefaultDataRegionConfiguration(dataRegionConfiguration);
> dataStorageConfiguration.setWalMode(WALMode.FSYNC);
>
> TcpDiscoveryVmIpFinder tcpDiscoveryVmIpFinder = new
> TcpDiscoveryVmIpFinder();
> tcpDiscoveryVmIpFinder.setShared(true);
> tcpDiscoveryVmIpFinder.setAddresses(ImmutableList.of("node1",
> "node2"));
>
> TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi();
> discoverySpi.setIpFinder(tcpDiscoveryVmIpFinder);
>
> IgniteConfiguration igniteConfiguration = new
> IgniteConfiguration();
> igniteConfiguration.setCacheConfiguration(cacheConfiguration);
> igniteConfiguration.setDiscoverySpi(discoverySpi);
>
> igniteConfiguration.setDataStorageConfiguration(dataStorageConfiguration);
> igniteConfiguration.setDeploymentMode(DeploymentMode.SHARED);
> igniteConfiguration.setWorkDirectory(workDir);
> igniteConfiguration.setGridLogger(new Slf4jLogger());
> igniteConfiguration.setClientMode(false);
>
> Ignite ignite = Ignition.getOrStart(igniteConfiguration);
> ignite.cluster().active(true);
>
> return ignite;
> }
>
> @Bean
> public SequenceNumberProvider sequenceNumberStore(Ignite ignite) {
> IgniteCache<SequenceNumberKey, Integer> igniteCache =
> ignite.getOrCreateCache(CACHE_NAME);
> return new SequenceNumberProvider(igniteCache);
> }
>
> }
> ```
>