Data count decreases when node down

[Ashwin <as...@gmail.com>]

I'm new with Ignite and I'm trying to test data quality and availability of
Ignite cluster.

I use the below xml configuration for setting cluster,

<property name="discoverySpi">
        <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
            <property name="socketTimeout" value="50000" />
            <property name="networkTimeout" value="50000" />
            <property name="reconnectCount" value="5" />
            <property name="ipFinder">
                <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    <property name="addresses">
                        <list>
                            <value>x.x.x.1:47500..47509</value>
                            <value>x.x.x.2:47500..47509</value>
                        </list>
                    </property>
                </bean>
            </property>
        </bean>
    </property>
</bean>
Also and jthe CacheConfiguration is,

<bean id="cache-template-bean"
class="org.apache.ignite.configuration.CacheConfiguration">
                <property name="name" value="CACHE_TEMPLATE*"/>
                <property name="cacheMode" value="PARTITIONED" />
                <property name="backups" value="1" />
             
                <property name="atomicityMode" value="TRANSACTIONAL" />
                <property name="writeSynchronizationMode"
value="PRIMARY_SYNC" />
                <property name="rebalanceBatchSize" value="#{4 * 1024 *
1024}" />
                <property name="rebalanceMode" value="ASYNC" />
                <property name="statisticsEnabled" value="true" />
                <property name="rebalanceBatchesPrefetchCount" value="4" />
                <property name="defaultLockTimeout" value="5000" />
                <property name="readFromBackup" value="true" />
                <property name="queryParallelism" value="6" />
                <property name="nodeFilter">
My scenarios are,

Loaded the 5 million data when all the 3 nodes
Bring one node down
The count shows 3.75 million. (Data loss)
Bringing the node up counts 5 million again.
I tried backup 1,2,3 all resulted in the same data loss. As per Ignite
documents, appears the data loss should not happen. If this fixed, I can try
adding data when the node is down and check how it behaves.

Any suggestions, please?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Data count decreases when node down

[Denis Magda <dm...@apache.org>]

This shouldn't happen with the backups number set in your config and the
test you do. However, the PRIMARY_SYNC mode implies that an update/insert
will be replicated asynchronously to a backup node. It means that even
after the preloading is finished data still will be replicated from primary
to backup partitions in the background.

You have an option of using FULL_SYNC instead or if PRIMARY_SYNC is
preferred then try to preload data with IgniteDataStreamer - it can stream
straight to primary and backup partitions.


-
Denis


On Tue, Apr 9, 2019 at 6:34 AM Ashwin <as...@gmail.com> wrote:

> I'm new with Ignite and I'm trying to test data quality and availability of
> Ignite cluster.
>
> I use the below xml configuration for setting cluster,
>
> <property name="discoverySpi">
>         <bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
>             <property name="socketTimeout" value="50000" />
>             <property name="networkTimeout" value="50000" />
>             <property name="reconnectCount" value="5" />
>             <property name="ipFinder">
>                 <bean
>
> class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
>                     <property name="addresses">
>                         <list>
>                             <value>x.x.x.1:47500..47509</value>
>                             <value>x.x.x.2:47500..47509</value>
>                         </list>
>                     </property>
>                 </bean>
>             </property>
>         </bean>
>     </property>
> </bean>
> Also and jthe CacheConfiguration is,
>
> <bean id="cache-template-bean"
> class="org.apache.ignite.configuration.CacheConfiguration">
>                 <property name="name" value="CACHE_TEMPLATE*"/>
>                 <property name="cacheMode" value="PARTITIONED" />
>                 <property name="backups" value="1" />
>
>                 <property name="atomicityMode" value="TRANSACTIONAL" />
>                 <property name="writeSynchronizationMode"
> value="PRIMARY_SYNC" />
>                 <property name="rebalanceBatchSize" value="#{4 * 1024 *
> 1024}" />
>                 <property name="rebalanceMode" value="ASYNC" />
>                 <property name="statisticsEnabled" value="true" />
>                 <property name="rebalanceBatchesPrefetchCount" value="4" />
>                 <property name="defaultLockTimeout" value="5000" />
>                 <property name="readFromBackup" value="true" />
>                 <property name="queryParallelism" value="6" />
>                 <property name="nodeFilter">
> My scenarios are,
>
> Loaded the 5 million data when all the 3 nodes
> Bring one node down
> The count shows 3.75 million. (Data loss)
> Bringing the node up counts 5 million again.
> I tried backup 1,2,3 all resulted in the same data loss. As per Ignite
> documents, appears the data loss should not happen. If this fixed, I can
> try
> adding data when the node is down and check how it behaves.
>
> Any suggestions, please?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>