[jira] [Created] (IGNITE-2088) EntryProcessor invoke might not run in CLOCK write order

["Avihai Berkovitz (JIRA)" <ji...@apache.org>]

Avihai Berkovitz created IGNITE-2088:
----------------------------------------

             Summary: EntryProcessor invoke might not run in CLOCK write order
                 Key: IGNITE-2088
                 URL: https://issues.apache.org/jira/browse/IGNITE-2088
             Project: Ignite
          Issue Type: Bug
    Affects Versions: ignite-1.4
         Environment: Windows 8.1 64 bit
java version "1.8.0_51"
Java(TM) SE Runtime Environment (build 1.8.0_51-b16)
Java HotSpot(TM) 64-Bit Server VM (build 25.51-b03, mixed mode)
Ignite 1.4.0
            Reporter: Avihai Berkovitz


I have a program that runs several threads at the same time, all of them executing an invoke() on the same cache key to update it. I found that the EntryProcessor is not always executed, and in this case invoke() returns null. This happens when the cache is configured with CLOCK CacheAtomicWriteOrderMode, but not when using PRIMARY.
After some debugging I found out that it happens when the cache gets the invocations out of order, and dismisses the older ones, returning the result from GridCacheMapEntry line 1855.

As I understand it, this mechanism is in place to protect against out-of-order updates. Invokes, however, are a special case as they execute client code with unknown side effects that should not be discarded. If you believe the current behavior is correct, it should at least be documented.

Here is a sample program to illustrate the problem. Notice that "IN INVOKE" is printed less than 10 times (usually):
{code}
IgniteConfiguration igniteConfiguration = new IgniteConfiguration()
		.setFailoverSpi(new AlwaysFailoverSpi())
		.setGridLogger(new Slf4jLogger())
		.setPeerClassLoadingEnabled(false)
		.setDeploymentMode(DeploymentMode.CONTINUOUS);
Ignite ignite = Ignition.start(igniteConfiguration);

CacheConfiguration<String, Long> cacheConfiguration = new CacheConfiguration<String, Long>()
		.setName("test")
		.setCacheMode(CacheMode.PARTITIONED)
		.setAtomicityMode(CacheAtomicityMode.ATOMIC)
		.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);

IgniteCache<String, Long> cache = ignite.getOrCreateCache(cacheConfiguration);

Thread[] threads = new Thread[10];

for (int i = 0; i < threads.length; i++) {
	threads[i] = new Thread() {
		@Override
		public void run() {
			System.out.println(String.format("Thread %s running", getId()));

			Long result = cache.invoke("key", (entry, arguments) -> {
				Long value = entry.getValue();
				System.out.println(String.format("IN INVOKE: thread ID:%s exists:%s value:%s", getId(), entry.exists(), value));
				if (!entry.exists()) {
					entry.setValue(getId());
				}
				return value;
			});

			System.out.println(String.format("Thread %s got invoke result %s", getId(), result));
		}
	};
}

for (Thread thread : threads) {
	thread.start();
}

Thread.sleep(2_000);

for (Thread thread : threads) {
	thread.interrupt();
}
for (Thread thread : threads) {
	thread.join();
}
System.out.println("Done");
{code}

Output sample:
{noformat}
[13:51:41]    __________  ________________ 
[13:51:41]   /  _/ ___/ |/ /  _/_  __/ __/ 
[13:51:41]  _/ // (7 7    // /  / / / _/   
[13:51:41] /___/\___/_/|_/___/ /_/ /___/  
[13:51:41] 
[13:51:41] ver. 1.4.0#20150924-sha1:c2def5f6
[13:51:41] 2015 Copyright(C) Apache Software Foundation
[13:51:41] 
[13:51:41] Ignite documentation: http://ignite.apache.org
[13:51:41] 
[13:51:41] Quiet mode.
[13:51:41]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}
[13:51:41] 
[13:51:41] Initial heap size is 256MB (should be no less than 512MB, use -Xms512m -Xmx512m).
[13:51:43] Configured plugins:
[13:51:43]   ^-- None
[13:51:43] 
[13:52:04] Security status [authentication=off, communication encryption=off]
[13:52:10] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
[13:52:10] 
[13:52:10] Ignite node started OK (id=0ceced67)
[13:52:10] Topology snapshot [ver=1, servers=1, clients=0, CPUs=8, heap=3.5GB]
Thread 105 running
Thread 107 running
Thread 106 running
Thread 108 running
Thread 109 running
Thread 110 running
Thread 111 running
Thread 112 running
Thread 114 running
Thread 113 running
IN INVOKE: thread ID:107 exists:false value:null
IN INVOKE: thread ID:113 exists:true value:107
Thread 107 got invoke result null
IN INVOKE: thread ID:105 exists:true value:107
Thread 111 got invoke result null
Thread 113 got invoke result 107
Thread 105 got invoke result 107
IN INVOKE: thread ID:110 exists:true value:107
Thread 110 got invoke result 107
Thread 106 got invoke result null
IN INVOKE: thread ID:112 exists:true value:107
Thread 112 got invoke result 107
IN INVOKE: thread ID:108 exists:true value:107
Thread 108 got invoke result 107
Thread 109 got invoke result null
Thread 114 got invoke result null
Done
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)