You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by "aaron@tophold.com" <aa...@tophold.com> on 2017/10/13 12:45:13 UTC
Some questions regard Cache Query and performance
hi All,
Regards ContinuousQuery, I notice there is a #setLocal will this meaning CQ only query against local cache? while this query only have #setRemoteFilterFactory method.
So the question is will my CQ's local listener only receive event updates from local cache or entire cluster?
Also the SqlFieldsQuery, we use this to batch update entries in cache, the frequent may be once several seconds.
The entry may be updated real time, sometime the data may inconsistent seem, will there any lock/transaction control of this?
We use the JDBC as back-end storage; and notice if we run the instance for several days, the memory usage keep increase, the performance downgrade very badly,
And eventually can not be used, from log we receive the ignite node stop event, but the memory not be reclaimed until we kill the JVM finally.
Our cache defined as:
<property name="cacheMode" value="PARTITIONED"/>
<property name="atomicityMode" value="ATOMIC"/>
<property name="readFromBackup" value="true"/>
<property name="backups" value="1"/>
<property name="readThrough" value="true"/>
<property name="writeThrough" value="true"/>
<property name="writeBehindEnabled" value="true"/>
<property name="writeBehindBatchSize" value="1024"/>
<property name="writeBehindFlushThreadCount" value="1"/>
<property name="storeKeepBinary" value="false"/>
<property name="onheapCacheEnabled" value="true"/>
<property name="memoryPolicyName" value="Asset_Region"/>
<property name="evictionPolicy">
<bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
<property name="maxSize" value="#{50000}"/>
</bean>
</property>
<property name="nearConfiguration">
<bean class="org.apache.ignite.configuration.NearCacheConfiguration">
<property name="nearEvictionPolicy">
<bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
<property name="maxSize" value="3000"/>
</bean>
</property>
<property name="nearStartSize" value="500"/>
</bean>
</property>
<property name="cacheStoreFactory" ref="storeagefactory"/>
<bean class="org.apache.ignite.configuration.MemoryPolicyConfiguration">
<property name="name" value="Asset_Region"/>
<property name="initialSize" value="#{2L * 1024L * 1024L * 1024L}"/>
<property name="maxSize" value="#{2L * 1024L * 1024L * 1024L}"/>
<property name="pageEvictionMode" value="RANDOM_2_LRU"/>
</bean>
Thanks for your time!
Regards
Aaron
aaron@tophold.com
Re: Some questions regard Cache Query and performance
Posted by Alexey Kukushkin <ku...@gmail.com>.
Aaron, let me forward this to DEV:
Ignite DEV community,
I tried to confirm setLocal(true) semantics for ContinuousQuery and I have
got not what I expected. I found that executing a ContinuousQuery with
setLocal(true):
- Still executes initial query remotely
- Listens for modifications locally
I expected that initial query shall also be local.
Is this by design or a bug?
On Fri, Oct 13, 2017 at 3:45 PM, aaron@tophold.com <aa...@tophold.com>
wrote:
> hi All,
>
> Regards ContinuousQuery, I notice there is a #setLocal will this meaning
> CQ only query against local cache? while this query only have #setRemoteFilterFactory
> method.
>
> So the question is will my CQ's local listener only receive event updates
> from local cache or entire cluster?
>
> Also the SqlFieldsQuery, we use this to batch update entries in cache, the
> frequent may be once several seconds.
>
> The entry may be updated real time, sometime the data may inconsistent
> seem, will there any lock/transaction control of this?
>
> We use the JDBC as back-end storage; and notice if we run the instance
> for several days, the memory usage keep increase, the performance downgrade
> very badly,
>
> And eventually can not be used, from log we receive the ignite node stop
> event, but the memory not be reclaimed until we kill the JVM finally.
>
> Our cache defined as:
>
> <property name="cacheMode" value="PARTITIONED"/>
> <property name="atomicityMode" value="ATOMIC"/>
> <property name="readFromBackup" value="true"/>
> <property name="backups" value="1"/>
>
> <property name="readThrough" value="true"/>
> <property name="writeThrough" value="true"/>
> <property name="writeBehindEnabled" value="true"/>
> <property name="writeBehindBatchSize" value="1024"/>
> <property name="writeBehindFlushThreadCount" value="1"/>
>
>
> <property name="storeKeepBinary" value="false"/>
> <property name="onheapCacheEnabled" value="true"/>
> <property name="memoryPolicyName" value="Asset_Region"/>
>
> <property name="evictionPolicy">
> <bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
> <property name="maxSize" value="#{50000}"/>
> </bean>
> </property>
>
> <property name="nearConfiguration">
> <bean class="org.apache.ignite.configuration.NearCacheConfiguration">
> <property name="nearEvictionPolicy">
> <bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
> <property name="maxSize" value="3000"/>
> </bean>
> </property>
> <property name="nearStartSize" value="500"/>
> </bean>
> </property>
>
> <property name="cacheStoreFactory" ref="storeagefactory"/>
>
> <bean class="org.apache.ignite.configuration.MemoryPolicyConfiguration">
> <property name="name" value="Asset_Region"/>
> <property name="initialSize" value="#{2L * 1024L * 1024L * 1024L}"/>
> <property name="maxSize" value="#{2L * 1024L * 1024L * 1024L}"/>
> <property name="pageEvictionMode" value="RANDOM_2_LRU"/>
> </bean>
>
>
> Thanks for your time!
>
> Regards
> Aaron
> ------------------------------
> aaron@tophold.com
>
--
Best regards,
Alexey
Re: Some questions regard Cache Query and performance
Posted by Alexey Kukushkin <ku...@gmail.com>.
Aaron, let me forward this to DEV:
Ignite DEV community,
I tried to confirm setLocal(true) semantics for ContinuousQuery and I have
got not what I expected. I found that executing a ContinuousQuery with
setLocal(true):
- Still executes initial query remotely
- Listens for modifications locally
I expected that initial query shall also be local.
Is this by design or a bug?
On Fri, Oct 13, 2017 at 3:45 PM, aaron@tophold.com <aa...@tophold.com>
wrote:
> hi All,
>
> Regards ContinuousQuery, I notice there is a #setLocal will this meaning
> CQ only query against local cache? while this query only have #setRemoteFilterFactory
> method.
>
> So the question is will my CQ's local listener only receive event updates
> from local cache or entire cluster?
>
> Also the SqlFieldsQuery, we use this to batch update entries in cache, the
> frequent may be once several seconds.
>
> The entry may be updated real time, sometime the data may inconsistent
> seem, will there any lock/transaction control of this?
>
> We use the JDBC as back-end storage; and notice if we run the instance
> for several days, the memory usage keep increase, the performance downgrade
> very badly,
>
> And eventually can not be used, from log we receive the ignite node stop
> event, but the memory not be reclaimed until we kill the JVM finally.
>
> Our cache defined as:
>
> <property name="cacheMode" value="PARTITIONED"/>
> <property name="atomicityMode" value="ATOMIC"/>
> <property name="readFromBackup" value="true"/>
> <property name="backups" value="1"/>
>
> <property name="readThrough" value="true"/>
> <property name="writeThrough" value="true"/>
> <property name="writeBehindEnabled" value="true"/>
> <property name="writeBehindBatchSize" value="1024"/>
> <property name="writeBehindFlushThreadCount" value="1"/>
>
>
> <property name="storeKeepBinary" value="false"/>
> <property name="onheapCacheEnabled" value="true"/>
> <property name="memoryPolicyName" value="Asset_Region"/>
>
> <property name="evictionPolicy">
> <bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
> <property name="maxSize" value="#{50000}"/>
> </bean>
> </property>
>
> <property name="nearConfiguration">
> <bean class="org.apache.ignite.configuration.NearCacheConfiguration">
> <property name="nearEvictionPolicy">
> <bean class="org.apache.ignite.cache.eviction.lru.LruEvictionPolicy">
> <property name="maxSize" value="3000"/>
> </bean>
> </property>
> <property name="nearStartSize" value="500"/>
> </bean>
> </property>
>
> <property name="cacheStoreFactory" ref="storeagefactory"/>
>
> <bean class="org.apache.ignite.configuration.MemoryPolicyConfiguration">
> <property name="name" value="Asset_Region"/>
> <property name="initialSize" value="#{2L * 1024L * 1024L * 1024L}"/>
> <property name="maxSize" value="#{2L * 1024L * 1024L * 1024L}"/>
> <property name="pageEvictionMode" value="RANDOM_2_LRU"/>
> </bean>
>
>
> Thanks for your time!
>
> Regards
> Aaron
> ------------------------------
> aaron@tophold.com
>
--
Best regards,
Alexey