You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Ali <al...@gmail.com> on 2017/12/04 13:36:31 UTC
Index performance
Hi!
I found poor sql index performance and CPU utilisation in ignite 2.3 version when try to benchmark cluster for performance.
Cluster configuration:
6 x 2 E5-2600v2 256GB 10GB link
1 x 4 E7 node (edge) from which tests are running
Ignite config
<property name="cacheConfiguration">
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value=“test"/>
<property name="backups" value="0"/>
<property name="atomicityMode" value="ATOMIC"/>
<property name="cacheMode" value="PARTITIONED"/>
<property name="copyOnRead" value="true"/>
<property name="readFromBackup" value="true"/>
<property name="queryParallelism" value=“48"/>
<property name="rebalanceMode" value="ASYNC"/>
<property name="statisticsEnabled" value="false"/>
<property name="queryEntities">
<list>
<bean class="org.apache.ignite.cache.QueryEntity">
<constructor-arg value="java.lang.String"/>
<constructor-arg value=“com.example.Test"/>
<property name="tableName" value=“Test"/>
<property name="fields">
<map>
<entry key=“field-1" value="java.lang.String"/>
<entry key=“field-2" value="java.lang.String"/>
<entry key=“field-3" value="java.lang.String"/>
<entry key=“field-4" value="java.lang.String"/>
<entry key=“field-5" value="java.lang.String"/>
<entry key=“field-6" value="java.lang.Long"/>
</map>
</property>
<property name="indexes">
<list>
<bean class="org.apache.ignite.cache.QueryIndex">
<property name="name" value="fields_idx"/>
<property name="fields">
<map>
<entry key=“field-2" value="true"/>
<entry key=“field-3" value="true"/>
<entry key=“field-4" value="true"/>
<entry key=“field-5" value="true"/>
<entry key=“field-6" value="true"/>
</map>
</property>
<property name="inlineSize" value="24"/>
<property name="indexType" value="SORTED"/>
</bean>
</list>
</property>
</bean>
</list>
</property>
</bean>
</property>
Query
public Collection<Test> find(String mcc, String mnc, String lac, String cid, long lastSeenDt, long offset, long limit) {
SqlQuery<String, Test> qry = new SqlQuery<>(
Entry.class,
"select * from Test where field-2 = ? and field-3 = ? and field-4 = ? and field-5 = ? and field-6 >= ? limit ? offset ?"
);
qry.setArgs(mcc, mnc, lac, cid, lastSeenDt, limit, offset);
try (QueryCursor<Cache.Entry<String, Test>> query = cache.query(qry)) {
return StreamSupport.stream(query.spliterator(), false)
.map(Cache.Entry::getValue)
.collect(Collectors.toList());
}
}
Benchmark
Benchmark consist of several java processes that call this query through RMI in {1..20 000} threads but maximum CPU utilisation doesn’t increase more than 9% on each node and doesn’t depend on load
simple IgniteCache#get test fully utilise CPU
With best regards
Alisher Alimov
alimovalisher@gmail.com
Re: Index performance
Posted by Evgenii Zhuravlev <e....@gmail.com>.
So, then share benchmarks code
Evgenii
2017-12-04 18:14 GMT+03:00 Ali <al...@gmail.com>:
> I would recommend using the default value for this property - 1. But, if
> you want to run a few of long queries, it could be increased.
>
>
> Doesn’t help
>
> I have already tried to execute test with configurations:
>
>
> - 1 request queryParalellism = 1
> - 100 concurrent request queryParalellism = 1
> - 1 request queryParalellism = 48
> - 100 concurrent request queryParalellism = 48
>
>
> For last three tests performance or CPU usage maximum change on 5%
>
> In parallel discusion here http://apache-ignite-users.70518.x6.nabble.com/
> Ignite-poor-performance-td18670.html one of users suggest to
> use parallelism
>
> With best regards
> Alisher Alimov
> alimovalisher@gmail.com
>
> On 4 Dec 2017, at 17:41, Evgenii Zhuravlev <e....@gmail.com>
> wrote:
>
> Hi,
>
> Please share benchmarks code, it's hard to understand why you can't load
> CPU for 100% without your code.
>
> But I see one thing, that definitely could lead to performance problems -
> you've set <property name="queryParallelism" value=“48"/> - this could
> lead to a lot of context switching since every query could be executed in
> 48 threads. I'm sure that you run a lot of queries in parallel, so, it's a
> very big value for this property, even if you have 48 CPU on a machine.
>
> I would recommend using the default value for this property - 1. But, if
> you want to run a few of long queries, it could be increased.
>
> Also, this property doesn't work for SQL queries, and, actually, for a
> default off-heap cache since version 2.0 at all:
>
> <property name="copyOnRead" value="true"/>
>
> Evgenii
>
> 2017-12-04 16:36 GMT+03:00 Ali <al...@gmail.com>:
>
>> Hi!
>>
>> I found poor sql index performance and CPU utilisation in ignite 2.3
>> version when try to benchmark cluster for performance.
>>
>> *Cluster configuration*:
>>
>> 6 x 2 E5-2600v2 256GB 10GB link
>> 1 x 4 E7 node (edge) from which tests are running
>>
>> *Ignite config*
>>
>>
>> <property name="cacheConfiguration">
>> <bean class="org.apache.ignite.configuration.CacheConfiguration">
>> <property name="name" value=“test"/>
>> <property name="backups" value="0"/>
>> <property name="atomicityMode" value="ATOMIC"/>
>> <property name="cacheMode" value="PARTITIONED"/>
>> <property name="copyOnRead" value="true"/>
>> <property name="readFromBackup" value="true"/>
>> <property name="queryParallelism" value=“48"/>
>> <property name="rebalanceMode" value="ASYNC"/>
>> <property name="statisticsEnabled" value="false"/>
>>
>> <property name="queryEntities">
>> <list>
>> <bean class="org.apache.ignite.cache.QueryEntity">
>>
>> <constructor-arg value="java.lang.String"/>
>> <constructor-arg value=“com.example.Test"/>
>>
>> <property name="tableName" value=“Test"/>
>> <property name="fields">
>> <map>
>> <entry key=“field-1" value="java.lang.String"/>
>> <entry key=“field-2" value="java.lang.String"/>
>> <entry key=“field-3" value="java.lang.String"/>
>> <entry key=“field-4" value="java.lang.String"/>
>> <entry key=“field-5" value="java.lang.String"/>
>> <entry key=“field-6" value="java.lang.Long"/>
>> </map>
>> </property>
>>
>> <property name="indexes">
>> <list>
>> <bean class="org.apache.ignite.cache.QueryIndex">
>> <property name="name" value="fields_idx"/>
>> <property name="fields">
>> <map>
>> <entry key=“field-2" value="true"/>
>> <entry key=“field-3" value="true"/>
>> <entry key=“field-4" value="true"/>
>> <entry key=“field-5" value="true"/>
>> <entry key=“field-6" value="true"/>
>> </map>
>> </property>
>> <property name="inlineSize" value="24"/>
>> <property name="indexType" value="SORTED"/>
>> </bean>
>> </list>
>> </property>
>> </bean>
>> </list>
>> </property>
>> </bean>
>> </property>
>>
>>
>> *Query*
>>
>> public Collection<Test> find(String mcc, String mnc, String lac, String cid, long lastSeenDt, long offset, long limit) {
>> SqlQuery<String, Test> qry = new SqlQuery<>(
>> Entry.class,
>> "select * from Test where field-2 = ? and field-3 = ? and field-4 = ? and field-5 = ? and field-6 >= ? limit ? offset ?"
>> );
>>
>> qry.setArgs(mcc, mnc, lac, cid, lastSeenDt, limit, offset);
>>
>> try (QueryCursor<Cache.Entry<String, Test>> query = cache.query(qry)) {
>> return StreamSupport.stream(query.spliterator(), false)
>> .map(Cache.Entry::getValue)
>> .collect(Collectors.toList());
>> }
>> }
>>
>>
>> *Benchmark*
>>
>>
>> Benchmark consist of several java processes that call this query through
>> RMI in {1..20 000} threads but maximum CPU utilisation doesn’t increase
>> more than 9% on each node and doesn’t depend on load
>>
>> simple IgniteCache#get test fully utilise CPU
>>
>>
>>
>> With best regards
>> Alisher Alimov
>> alimovalisher@gmail.com
>>
>>
>
>
Re: Index performance
Posted by Ali <al...@gmail.com>.
> I would recommend using the default value for this property - 1. But, if you want to run a few of long queries, it could be increased.
Doesn’t help
I have already tried to execute test with configurations:
1 request queryParalellism = 1
100 concurrent request queryParalellism = 1
1 request queryParalellism = 48
100 concurrent request queryParalellism = 48
For last three tests performance or CPU usage maximum change on 5%
In parallel discusion here http://apache-ignite-users.70518.x6.nabble.com/Ignite-poor-performance-td18670.html <http://apache-ignite-users.70518.x6.nabble.com/Ignite-poor-performance-td18670.html> one of users suggest to use parallelism
With best regards
Alisher Alimov
alimovalisher@gmail.com
> On 4 Dec 2017, at 17:41, Evgenii Zhuravlev <e....@gmail.com> wrote:
>
> Hi,
>
> Please share benchmarks code, it's hard to understand why you can't load CPU for 100% without your code.
>
> But I see one thing, that definitely could lead to performance problems - you've set <property name="queryParallelism" value=“48"/> - this could lead to a lot of context switching since every query could be executed in 48 threads. I'm sure that you run a lot of queries in parallel, so, it's a very big value for this property, even if you have 48 CPU on a machine.
>
> I would recommend using the default value for this property - 1. But, if you want to run a few of long queries, it could be increased.
>
> Also, this property doesn't work for SQL queries, and, actually, for a default off-heap cache since version 2.0 at all:
>
> <property name="copyOnRead" value="true"/>
>
> Evgenii
>
> 2017-12-04 16:36 GMT+03:00 Ali <alimovalisher@gmail.com <ma...@gmail.com>>:
> Hi!
>
> I found poor sql index performance and CPU utilisation in ignite 2.3 version when try to benchmark cluster for performance.
>
> Cluster configuration:
>
> 6 x 2 E5-2600v2 256GB 10GB link
> 1 x 4 E7 node (edge) from which tests are running
>
> Ignite config
>
>
> <property name="cacheConfiguration">
> <bean class="org.apache.ignite.configuration.CacheConfiguration">
> <property name="name" value=“test"/>
> <property name="backups" value="0"/>
> <property name="atomicityMode" value="ATOMIC"/>
> <property name="cacheMode" value="PARTITIONED"/>
> <property name="copyOnRead" value="true"/>
> <property name="readFromBackup" value="true"/>
> <property name="queryParallelism" value=“48"/>
> <property name="rebalanceMode" value="ASYNC"/>
> <property name="statisticsEnabled" value="false"/>
>
> <property name="queryEntities">
> <list>
> <bean class="org.apache.ignite.cache.QueryEntity">
>
> <constructor-arg value="java.lang.String"/>
> <constructor-arg value=“com.example.Test"/>
>
> <property name="tableName" value=“Test"/>
> <property name="fields">
> <map>
> <entry key=“field-1" value="java.lang.String"/>
> <entry key=“field-2" value="java.lang.String"/>
> <entry key=“field-3" value="java.lang.String"/>
> <entry key=“field-4" value="java.lang.String"/>
> <entry key=“field-5" value="java.lang.String"/>
> <entry key=“field-6" value="java.lang.Long"/>
> </map>
> </property>
>
> <property name="indexes">
> <list>
> <bean class="org.apache.ignite.cache.QueryIndex">
> <property name="name" value="fields_idx"/>
> <property name="fields">
> <map>
> <entry key=“field-2" value="true"/>
> <entry key=“field-3" value="true"/>
> <entry key=“field-4" value="true"/>
> <entry key=“field-5" value="true"/>
> <entry key=“field-6" value="true"/>
> </map>
> </property>
> <property name="inlineSize" value="24"/>
> <property name="indexType" value="SORTED"/>
> </bean>
> </list>
> </property>
> </bean>
> </list>
> </property>
> </bean>
> </property>
>
> Query
>
> public Collection<Test> find(String mcc, String mnc, String lac, String cid, long lastSeenDt, long offset, long limit) {
> SqlQuery<String, Test> qry = new SqlQuery<>(
> Entry.class,
> "select * from Test where field-2 = ? and field-3 = ? and field-4 = ? and field-5 = ? and field-6 >= ? limit ? offset ?"
> );
>
> qry.setArgs(mcc, mnc, lac, cid, lastSeenDt, limit, offset);
>
> try (QueryCursor<Cache.Entry<String, Test>> query = cache.query(qry)) {
> return StreamSupport.stream(query.spliterator(), false)
> .map(Cache.Entry::getValue)
> .collect(Collectors.toList());
> }
> }
>
> Benchmark
>
>
> Benchmark consist of several java processes that call this query through RMI in {1..20 000} threads but maximum CPU utilisation doesn’t increase more than 9% on each node and doesn’t depend on load
>
> simple IgniteCache#get test fully utilise CPU
>
>
>
> With best regards
> Alisher Alimov
> alimovalisher@gmail.com <ma...@gmail.com>
>
Re: Index performance
Posted by Evgenii Zhuravlev <e....@gmail.com>.
Hi,
Please share benchmarks code, it's hard to understand why you can't load
CPU for 100% without your code.
But I see one thing, that definitely could lead to performance problems -
you've set <property name="queryParallelism" value=“48"/> - this could
lead to a lot of context switching since every query could be executed in
48 threads. I'm sure that you run a lot of queries in parallel, so, it's a
very big value for this property, even if you have 48 CPU on a machine.
I would recommend using the default value for this property - 1. But, if
you want to run a few of long queries, it could be increased.
Also, this property doesn't work for SQL queries, and, actually, for a
default off-heap cache since version 2.0 at all:
<property name="copyOnRead" value="true"/>
Evgenii
2017-12-04 16:36 GMT+03:00 Ali <al...@gmail.com>:
> Hi!
>
> I found poor sql index performance and CPU utilisation in ignite 2.3
> version when try to benchmark cluster for performance.
>
> *Cluster configuration*:
>
> 6 x 2 E5-2600v2 256GB 10GB link
> 1 x 4 E7 node (edge) from which tests are running
>
> *Ignite config*
>
>
> <property name="cacheConfiguration">
> <bean class="org.apache.ignite.configuration.CacheConfiguration">
> <property name="name" value=“test"/>
> <property name="backups" value="0"/>
> <property name="atomicityMode" value="ATOMIC"/>
> <property name="cacheMode" value="PARTITIONED"/>
> <property name="copyOnRead" value="true"/>
> <property name="readFromBackup" value="true"/>
> <property name="queryParallelism" value=“48"/>
> <property name="rebalanceMode" value="ASYNC"/>
> <property name="statisticsEnabled" value="false"/>
>
> <property name="queryEntities">
> <list>
> <bean class="org.apache.ignite.cache.QueryEntity">
>
> <constructor-arg value="java.lang.String"/>
> <constructor-arg value=“com.example.Test"/>
>
> <property name="tableName" value=“Test"/>
> <property name="fields">
> <map>
> <entry key=“field-1" value="java.lang.String"/>
> <entry key=“field-2" value="java.lang.String"/>
> <entry key=“field-3" value="java.lang.String"/>
> <entry key=“field-4" value="java.lang.String"/>
> <entry key=“field-5" value="java.lang.String"/>
> <entry key=“field-6" value="java.lang.Long"/>
> </map>
> </property>
>
> <property name="indexes">
> <list>
> <bean class="org.apache.ignite.cache.QueryIndex">
> <property name="name" value="fields_idx"/>
> <property name="fields">
> <map>
> <entry key=“field-2" value="true"/>
> <entry key=“field-3" value="true"/>
> <entry key=“field-4" value="true"/>
> <entry key=“field-5" value="true"/>
> <entry key=“field-6" value="true"/>
> </map>
> </property>
> <property name="inlineSize" value="24"/>
> <property name="indexType" value="SORTED"/>
> </bean>
> </list>
> </property>
> </bean>
> </list>
> </property>
> </bean>
> </property>
>
>
> *Query*
>
> public Collection<Test> find(String mcc, String mnc, String lac, String cid, long lastSeenDt, long offset, long limit) {
> SqlQuery<String, Test> qry = new SqlQuery<>(
> Entry.class,
> "select * from Test where field-2 = ? and field-3 = ? and field-4 = ? and field-5 = ? and field-6 >= ? limit ? offset ?"
> );
>
> qry.setArgs(mcc, mnc, lac, cid, lastSeenDt, limit, offset);
>
> try (QueryCursor<Cache.Entry<String, Test>> query = cache.query(qry)) {
> return StreamSupport.stream(query.spliterator(), false)
> .map(Cache.Entry::getValue)
> .collect(Collectors.toList());
> }
> }
>
>
> *Benchmark*
>
>
> Benchmark consist of several java processes that call this query through
> RMI in {1..20 000} threads but maximum CPU utilisation doesn’t increase
> more than 9% on each node and doesn’t depend on load
>
> simple IgniteCache#get test fully utilise CPU
>
>
>
> With best regards
> Alisher Alimov
> alimovalisher@gmail.com
>
>