You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by oshevchenko <ol...@prudential.com> on 2019/01/06 17:44:15 UTC
SqlQuery retrieves same cache entry twice. ScanQuery results
conflicts with indentical SqlQuery
Met very strange SqlQuery when executing simple query on partitioned. The
problem that the same cache entry is retrieved twice. Looks like first time
entry gets retrieved from primary partition and second time it is taken from
back up. Another problem that same ScanQuery gives correct results. Code i
run to this behavior:
public class UnitDupKeyProblemTests {
public static void main(String[] args) {
try (Ignite ignite = Ignition.start("ignite-client-conf-unit.xml"))
{
Collection<Result> results = ignite.compute().broadcast(new
TestCallable("ENT_LST_0004_20180630_14009_999999_99999_9999_23_USD_99999_9999999999999_999999_2018_9_99999_LN00025_999999_999999"));
results.forEach(System.out::println);
}
}
private static class BiPredicateFilter implements
IgniteBiPredicate<String, Balance> {
@Override
public boolean apply(String string, Balance balance) {
return balance.getTransType().equals("23");
}
}
private static class Result implements Serializable {
int timesFound;
int nonPrimary;
Collection<String> addresses;
UUID nodeUid;
boolean primary;
boolean backUp;
public Result(int timesFound, int nonPrimary, Collection<String>
addresses, UUID nodeUid, boolean primary, boolean backUp) {
this.timesFound = timesFound;
this.addresses = addresses;
this.nodeUid = nodeUid;
this.primary = primary;
this.nonPrimary = nonPrimary;
this.backUp = backUp;
}
public int getTimesFound() {
return timesFound;
}
@Override
public String toString() {
return "Result{" +
"timesFound=" + timesFound +
", nonPrimary=" + nonPrimary +
", addresses=" + addresses +
", nodeUid=" + nodeUid +
", primary=" + primary +
", backUp=" + backUp +
'}';
}
}
private static class TestCallable implements IgniteCallable<Result> {
@IgniteInstanceResource
transient Ignite ignite;
private final String keyToTest;
public TestCallable(String keyToTest) {
this.keyToTest = keyToTest;
}
@Override
public Result call() throws Exception {
final IgniteCache<String, Balance> cache =
ignite.cache("BALANCE");
final ClusterNode clusterNode = ignite.cluster().localNode();
//final ScanQuery<String, Balance> query = new ScanQuery<>(new
BiPredicateFilter());
final SqlQuery<String, Balance> query = new
SqlQuery<>(Balance.class, "transType='23'");
query.setLocal(true);
int num = 0;
int nonPrimary = 0;
try (final QueryCursor<Cache.Entry<String, Balance>> cursor =
cache.query(query)) {
for (Cache.Entry<String, Balance> entry : cursor) {
if (cache.localPeek(entry.getKey(),
CachePeekMode.PRIMARY) == null) {
nonPrimary++;
}
if (keyToTest.equals(entry.getKey())) {
num++;
}
}
}
ignite.affinity("BALANCE").isPrimary(clusterNode, keyToTest);
return new Result(num, nonPrimary, clusterNode.addresses(),
clusterNode.id(),
ignite.affinity("BALANCE").isPrimary(clusterNode,
keyToTest),
ignite.affinity("BALANCE").isBackup(clusterNode,
keyToTest));
}
}
}
Output that proves my assumptions is:
Result{timesFound=1, nonPrimary=4, addresses=[127.0.0.1, 48.124.176.58],
nodeUid=0d45348c-2e94-4ac3-b9aa-b61fbdd56749, primary=false, backUp=true}
Result{timesFound=0, nonPrimary=5, addresses=[127.0.0.1, 48.124.184.19],
nodeUid=b5161591-7d62-4c0d-a866-7610b3665760, primary=false, backUp=false}
Result{timesFound=0, nonPrimary=2, addresses=[127.0.0.1, 48.124.176.57],
nodeUid=da3da1d3-a4f0-4273-8fac-f4ba479ae211, primary=false, backUp=false}
Result{timesFound=1, nonPrimary=2, addresses=[127.0.0.1, 48.124.184.20],
nodeUid=39203ddb-2c6b-4247-84bf-22f380130711, primary=true, backUp=false}
if switch to scanquery output looks correct:
Result{timesFound=0, nonPrimary=0, addresses=[127.0.0.1, 48.124.176.58],
nodeUid=0d45348c-2e94-4ac3-b9aa-b61fbdd56749, primary=false, backUp=true}
Result{timesFound=1, nonPrimary=0, addresses=[127.0.0.1, 48.124.184.20],
nodeUid=39203ddb-2c6b-4247-84bf-22f380130711, primary=true, backUp=false}
Result{timesFound=0, nonPrimary=0, addresses=[127.0.0.1, 48.124.176.57],
nodeUid=da3da1d3-a4f0-4273-8fac-f4ba479ae211, primary=false, backUp=false}
Result{timesFound=0, nonPrimary=0, addresses=[127.0.0.1, 48.124.184.19],
nodeUid=b5161591-7d62-4c0d-a866-7610b3665760, primary=false, backUp=false}
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: SqlQuery retrieves same cache entry twice. ScanQuery results
conflicts with indentical SqlQuery
Posted by oshevchenko <ol...@prudential.com>.
Hi Ilya,
Thanks a lot for your reply. I am running 2.5. Looks like my problem has to
do with IGNITE-8900 <https://issues.apache.org/jira/browse/IGNITE-8900>
which should be fixed for 2.7. Keep my fingers crossed that 2.7 fixes this
serious issue
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: SqlQuery retrieves same cache entry twice. ScanQuery results
conflicts with indentical SqlQuery
Posted by oshevchenko <ol...@prudential.com>.
Hi Ilya,
Thanks for quick reply on my problem. I am running 2.5. Looks like issue i
have has to do with IGNITE-8900
<https://issues.apache.org/jira/browse/IGNITE-8900> . I keep my fingers
crossed that this serious issue is gone with 2.7.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: SqlQuery retrieves same cache entry twice. ScanQuery results
conflicts with indentical SqlQuery
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
What is the Ignite version that you are using? Can you re-try using 2.7 if
you're using an earlier one?
Regards,
--
Ilya Kasnacheev
вс, 6 янв. 2019 г. в 20:54, oshevchenko <oleksandr.shevchenko@prudential.com
>:
> Met very strange SqlQuery when executing simple query on partitioned. The
> problem that the same cache entry is retrieved twice. Looks like first time
> entry gets retrieved from primary partition and second time it is taken
> from
> back up. Another problem that same ScanQuery gives correct results. Code i
> run to this behavior:
> public class UnitDupKeyProblemTests {
> public static void main(String[] args) {
> try (Ignite ignite = Ignition.start("ignite-client-conf-unit.xml"))
> {
> Collection<Result> results = ignite.compute().broadcast(new
>
> TestCallable("ENT_LST_0004_20180630_14009_999999_99999_9999_23_USD_99999_9999999999999_999999_2018_9_99999_LN00025_999999_999999"));
> results.forEach(System.out::println);
> }
> }
>
> private static class BiPredicateFilter implements
> IgniteBiPredicate<String, Balance> {
> @Override
> public boolean apply(String string, Balance balance) {
> return balance.getTransType().equals("23");
> }
> }
>
> private static class Result implements Serializable {
> int timesFound;
> int nonPrimary;
> Collection<String> addresses;
> UUID nodeUid;
> boolean primary;
> boolean backUp;
>
> public Result(int timesFound, int nonPrimary, Collection<String>
> addresses, UUID nodeUid, boolean primary, boolean backUp) {
> this.timesFound = timesFound;
> this.addresses = addresses;
> this.nodeUid = nodeUid;
> this.primary = primary;
> this.nonPrimary = nonPrimary;
> this.backUp = backUp;
> }
>
> public int getTimesFound() {
> return timesFound;
> }
>
> @Override
> public String toString() {
> return "Result{" +
> "timesFound=" + timesFound +
> ", nonPrimary=" + nonPrimary +
> ", addresses=" + addresses +
> ", nodeUid=" + nodeUid +
> ", primary=" + primary +
> ", backUp=" + backUp +
> '}';
> }
> }
>
> private static class TestCallable implements IgniteCallable<Result> {
>
> @IgniteInstanceResource
> transient Ignite ignite;
>
> private final String keyToTest;
>
> public TestCallable(String keyToTest) {
> this.keyToTest = keyToTest;
> }
>
> @Override
> public Result call() throws Exception {
> final IgniteCache<String, Balance> cache =
> ignite.cache("BALANCE");
> final ClusterNode clusterNode = ignite.cluster().localNode();
> //final ScanQuery<String, Balance> query = new ScanQuery<>(new
> BiPredicateFilter());
> final SqlQuery<String, Balance> query = new
> SqlQuery<>(Balance.class, "transType='23'");
>
>
> query.setLocal(true);
> int num = 0;
> int nonPrimary = 0;
> try (final QueryCursor<Cache.Entry<String, Balance>> cursor
> =
> cache.query(query)) {
> for (Cache.Entry<String, Balance> entry : cursor) {
> if (cache.localPeek(entry.getKey(),
> CachePeekMode.PRIMARY) == null) {
> nonPrimary++;
> }
>
> if (keyToTest.equals(entry.getKey())) {
> num++;
> }
> }
> }
>
> ignite.affinity("BALANCE").isPrimary(clusterNode, keyToTest);
> return new Result(num, nonPrimary, clusterNode.addresses(),
> clusterNode.id(),
> ignite.affinity("BALANCE").isPrimary(clusterNode,
> keyToTest),
> ignite.affinity("BALANCE").isBackup(clusterNode,
> keyToTest));
> }
> }
> }
>
> Output that proves my assumptions is:
> Result{timesFound=1, nonPrimary=4, addresses=[127.0.0.1, 48.124.176.58],
> nodeUid=0d45348c-2e94-4ac3-b9aa-b61fbdd56749, primary=false, backUp=true}
> Result{timesFound=0, nonPrimary=5, addresses=[127.0.0.1, 48.124.184.19],
> nodeUid=b5161591-7d62-4c0d-a866-7610b3665760, primary=false, backUp=false}
> Result{timesFound=0, nonPrimary=2, addresses=[127.0.0.1, 48.124.176.57],
> nodeUid=da3da1d3-a4f0-4273-8fac-f4ba479ae211, primary=false, backUp=false}
> Result{timesFound=1, nonPrimary=2, addresses=[127.0.0.1, 48.124.184.20],
> nodeUid=39203ddb-2c6b-4247-84bf-22f380130711, primary=true, backUp=false}
>
> if switch to scanquery output looks correct:
> Result{timesFound=0, nonPrimary=0, addresses=[127.0.0.1, 48.124.176.58],
> nodeUid=0d45348c-2e94-4ac3-b9aa-b61fbdd56749, primary=false, backUp=true}
> Result{timesFound=1, nonPrimary=0, addresses=[127.0.0.1, 48.124.184.20],
> nodeUid=39203ddb-2c6b-4247-84bf-22f380130711, primary=true, backUp=false}
> Result{timesFound=0, nonPrimary=0, addresses=[127.0.0.1, 48.124.176.57],
> nodeUid=da3da1d3-a4f0-4273-8fac-f4ba479ae211, primary=false, backUp=false}
> Result{timesFound=0, nonPrimary=0, addresses=[127.0.0.1, 48.124.184.19],
> nodeUid=b5161591-7d62-4c0d-a866-7610b3665760, primary=false, backUp=false}
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>