possible ignite bug around binary marshaller and data affinity

["Oleg Yurkivskyy (Luxoft)" <Ol...@tudor.com>]

Hi,

It appears that when using the default BinarySerialization, the @AffinityKeyMapped annotation doesn't seem to work and queries that rely on affinity (Using setLocal) aren't finding data which it would expect to find.

Please find below a project that can reproduce the issue (versions 2.7.5 and 2.7.6 are tested).
It is based on standard yardstick benchmark.
It can be run using this command after build:

./bin/benchmark-run-all.sh config/benchmark-company.properties

The tests run on 3 instances of ignite server and 1 instance of driver.
Test application populates the cache with some data using separate Java classes as a key and a value objects.
Key class EntityKey contains affinity key tid.
When cache is populated with test data benchmark runs affinityCall(ActionAffinityCallable) using affinity key.
ActionAffinityCallable executes query twice, on time with .setLocal(true) and second time .setLocal(false).
In normal situation both queries have to return the same data, because query returns the data using affinity key.
It works OK when entity and the key are Java Externalizable.
In case of binary marshaller these queries return different amount of data, that means that affinity key is not working as expected and data are spread across different nodes.
It doesn't deend on query type, SqlQuery and ScanQuery have the same problems, while SqlQuery is much faster.
It appears that when using the default BinarySerialization, the @AffinityKeyMapped annotation doesn't seem to work and queries that rely on affinity (Using setLocal) aren't finding data which it would expect to find. Please find below a project that can reproduce the issue
The project uses different permutations of BinarySerialization and standard serialization used with SqlQueries and ScanQueries. If Affinity DID work, then the results of both setLocal(true) and setLocal(false) should return the same results (as we use the affinityCall function. However as you can see below, that is not true for BinarySerialization

It can be seen in log files like this:

$  grep -r  "\*\*\* AffinityBinSql" output/ |wc -l
641153
$  grep -r  "\*\*\* AffinityBinScan" output/ |wc -l
40736
$  grep -r  "\*\*\* AffinityExtSql" output/ |wc -l
0
$  grep -r  "\*\*\* AffinityExtScan" output/ |wc -l
0

The workaround of this bug is standard Java serialization or using .setLocal(false) for queries.
Both workarounds result in higher CPU and network usage and slow things down.
For example, .setLocal(false) is 3 times slower than .setLocal(true) for the application attached.

Please recommend what can I do in this situation.
Do I need to create an issue in Ignite Jira for this bug?

Regards,
Oleg

_________________________________________________________

This communication is intended only for the addressee(s) and may contain confidential information. We do not waive any confidentiality by misdelivery. If you receive this communication in error, any use, dissemination, printing or copying is strictly prohibited; please destroy all electronic and paper copies and notify the sender immediately.

Re: possible ignite bug around binary marshaller and data affinity

[Ilya Kasnacheev <il...@gmail.com>]

Hello!

I have trouble contemplating your reproducer, but I think it's related to
the fact you are using QueryEntity.

QueryEntity means that your annotations are not used. In this case you
should use CacheKeyConfiguration to configure affinity for such caches:
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/CacheKeyConfiguration.html

Why it works for externalizable? It's hard to say, overall I think it's
related to https://issues.apache.org/jira/browse/IGNITE-9964 or similar.

Regards,



-- 
Ilya Kasnacheev


чт, 19 сент. 2019 г. в 17:26, Oleg Yurkivskyy (Luxoft) <
Oleg.Yurkivskyy@tudor.com>:

> Hi,
>
>
>
> It appears that when using the default BinarySerialization, the
> @AffinityKeyMapped annotation doesn’t seem to work and queries that rely on
> affinity (Using setLocal) aren’t finding data which it would expect to find.
>
>
>
> Please find below a project that can reproduce the issue (versions 2.7.5
> and 2.7.6 are tested).
>
> It is based on standard yardstick benchmark.
>
> It can be run using this command after build:
>
>
>
> ./bin/benchmark-run-all.sh config/benchmark-company.properties
>
>
>
> The tests run on 3 instances of ignite server and 1 instance of driver.
>
> Test application populates the cache with some data using separate Java
> classes as a key and a value objects.
>
> Key class EntityKey contains affinity key tid.
>
> When cache is populated with test data benchmark runs
> affinityCall(ActionAffinityCallable) using affinity key.
>
> ActionAffinityCallable executes query twice, on time with .setLocal(true)
> and second time .setLocal(false).
>
> In normal situation both queries have to return the same data, because
> query returns the data using affinity key.
>
> It works OK when entity and the key are Java Externalizable.
>
> In case of binary marshaller these queries return different amount of
> data, that means that affinity key is not working as expected and data are
> spread across different nodes.
>
> It doesn't deend on query type, SqlQuery and ScanQuery have the same
> problems, while SqlQuery is much faster.
>
> It appears that when using the default BinarySerialization, the
> @AffinityKeyMapped annotation doesn’t seem to work and queries that rely on
> affinity (Using setLocal) aren’t finding data which it would expect to
> find. Please find below a project that can reproduce the issue
>
> The project uses different permutations of BinarySerialization and
> standard serialization used with SqlQueries and ScanQueries. If Affinity
> DID work, then the results of both setLocal(true) and setLocal(false)
> should return the same results (as we use the affinityCall function.
> However as you can see below, that is not true for BinarySerialization
>
>
>
> It can be seen in log files like this:
>
>
>
> $  grep -r  "\*\*\* AffinityBinSql" output/ |wc -l
>
> 641153
>
> $  grep -r  "\*\*\* AffinityBinScan" output/ |wc -l
>
> 40736
>
> $  grep -r  "\*\*\* AffinityExtSql" output/ |wc -l
>
> 0
>
> $  grep -r  "\*\*\* AffinityExtScan" output/ |wc -l
>
> 0
>
>
>
> The workaround of this bug is standard Java serialization or using
> .setLocal(false) for queries.
>
> Both workarounds result in higher CPU and network usage and slow things
> down.
>
> For example, .setLocal(false) is 3 times slower than .setLocal(true) for
> the application attached.
>
>
>
> Please recommend what can I do in this situation.
>
> Do I need to create an issue in Ignite Jira for this bug?
>
>
>
> Regards,
>
> Oleg
>
> _________________________________________________________
>
> This communication is intended only for the addressee(s) and may contain
> confidential information. We do not waive any confidentiality by
> misdelivery. If you receive this communication in error, any use,
> dissemination, printing or copying is strictly prohibited; please destroy
> all electronic and paper copies and notify the sender immediately.
>