persistent store

[seyeony <se...@gmail.com>]

Hi,
I tested some examples related with persistent store. I tested with MariaDB
as persisten store.
Even though I tested with some Behind options like below, the performance
was not satisfied.
Maybe the performance of insert into the MariaDB table is not so good.
        ccfg.setReadThrough(true);
        ccfg.setWriteThrough(true);
        ccfg.setWriteBehindEnabled(true);
        ccfg.setWriteBehindFlushSize(2*10240);
        ccfg.setWriteBehindFlushFrequency(3000);
        ccfg.setWriteBehindFlushThreadCount(3);
        ccfg.setWriteBehindBatchSize(2048);
Is there any possibilities to use stream files as a persistene store,
instead DB?
Thanks,
-sy



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/persistent-store-tp7664.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: persistent store

[vkulichenko <va...@gmail.com>]

Hi,

Generally CacheStore is designed to be a single store shared between all
nodes. You can try to use write-behind to improve update performance:
https://apacheignite.readme.io/docs/persistent-store#write-behind-caching

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/persistent-store-tp7664p7753.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: persistent store

[seyeony <se...@gmail.com>]

Hi val,
Thank you for your response.
I mean 'stream file' is a linux regular file, in which a pojo can be stored.

I loaded 200,000 Person objects from MariaDB. This was fast. After load, I
deleted all object in MariaDB via 'delete from Person'
I modefied the salary value from example and tried to cache.put() for all
objects in other client node.
The performance was not satisfied for me. I guess this was because the
insert into MariaDB is not good enough.

My question was whether Apache Ignite provides functions to load/store
objects not in db but in regular file.

Configuration in example-default.xml which was included in
example-ignite.xml:
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xmlns:util="http://www.springframework.org/schema/util"
       xsi:schemaLocation="
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans.xsd
        http://www.springframework.org/schema/util
        http://www.springframework.org/schema/util/spring-util.xsd">

    <bean id="mariadbDataSource" class="org.mariadb.jdbc.MariaDbDataSource">
        <property name="url"
value="jdbc:mariadb://192.168.10.213:3306/ignite"/>
        <property name="user" value="ecube"/>
        <property name="password" value="obzcom1!"/>
    </bean>

    <bean abstract="true" id="ignite.cfg"
class="org.apache.ignite.configuration.IgniteConfiguration">
        
        <property name="peerClassLoadingEnabled" value="true"/>
        
        

        
        <property name="includeEventTypes">
            <list>
                
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_STARTED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_FINISHED"/>
                <util:constant
static-field="org.apache.ignite.events.EventType.EVT_TASK_FAILED"/>
            </list>
        </property>

        <property name="cacheConfiguration">
            <bean
class="org.apache.ignite.configuration.CacheConfiguration">
                <property name="cacheMode" value="PARTITIONED"/>
                <property name="backups" value="1"/>
                <property name="writeSynchronizationMode"
value="FULL_ASYNC"/>
                <property name="memoryMode" value="OFFHEAP_TIERED"/>
                <property name="startSize" value="#{100 * 1024 * 1024}"/>
                <property name="offHeapMaxMemory" value="#{4 * 1024L * 1024L
* 1024L}"/>
                
                <property name="swapEnabled" value="false"/> 
            </bean>
        </property>
        
        <property name="discoverySpi">
            <bean
class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
                <property name="ipFinder">
                    
                    
                    <bean
class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
                    
                        <property name="addresses">
                            <list>
                                
                                <value>192.168.10.57</value>
                                <value>192.168.10.58:47500..47509</value>
                                <value>192.168.10.59:47500..47509</value>
                                
                            </list>
                        </property>
                    </bean>
                </property>
            </bean>
        </property>
    </bean>
</beans>

I added the following options in CacheConfig.java which was generated by
ignite-schema-import.sh:
        ccfg.setReadThrough(true);
        ccfg.setWriteThrough(true);
        ccfg.setWriteBehindEnabled(true);
        ccfg.setWriteBehindFlushSize(2*10240);
        ccfg.setWriteBehindFlushFrequency(3000);
        ccfg.setWriteBehindFlushThreadCount(3);
        ccfg.setWriteBehindBatchSize(2048);

-sy



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/persistent-store-tp7664p7693.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: persistent store

[vkulichenko <va...@gmail.com>]

Hi,

You can use anything as persistent store as long as it can be implemented
within the CacheStore implementation. Can you clarify what you mean by
'stream files'?

As for performance, please provide more details. What and how are you
measuring? What is your configuration? How do you update the cache? What did
you try to resolve the issue?

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/persistent-store-tp7664p7687.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.