Transactional cache in Atomic mode

[msuh <ms...@jobcase.com>]

Hi,

Two questions about transactionality and locking: 
1) I've been looking at https://apacheignite.readme.io/docs/transactions and
couldn't get a clear answer from the Ignite documentation so I hope to get
an answer here.
All of the caches we use are set to TRANSACTIONAL, as we will often need to
do a logical operation on a group of caches. However, there are cases we
want to have ATOMIC operations on a single cache to increase performance.

What I'm wondering is: if a cache is set to operate on a TRANSACTIONAL mode,
does it always need to be operated inside the IgniteTransactions
transactions = ignite.transactions(); Java transaction API? Would the
changes simply not commit if we were to update the cache without surrounding
it inside this IgniteTransactions context?

2) To increase performance of a ScanQuery over a test cache of ~1million
entries, I have set up a ThreadPool of 10 threads to perform ScanQuery over
each partition - where each thread would take over each partition - in a
cluster that has a single server node. Each transaction had a commit size of
100k.
However the entire operation failed because the threads could not acquire
the lock for the transaction, and the transactions of other threads
timed-out. What is the locking granularity for each transaction (seems like
it locks the entire cache from what I've witnessed)? Is it possible to
change the locking level so that I can set the transaction to lock on a
partition or any other granularity? 

Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Transactional cache in Atomic mode

[msuh <ms...@jobcase.com>]

Hi Mikhail,

Thank you for your response.

1) From my testing, a ScanQuery of 1 million entries in an explicit
transaction (with PESSIMISTIC, SERIALIZABLE) took 10 minutes
single-threaded, whereas implicit transaction took 1 minute with the same
environment and single-threaded. If we assume that implicit transaction
works the same implementation-wise as the ATOMIC mode, implicit should be
much faster as noted in the Ignite documentation as well. Unfortunately,
there seems to be very little documentation on how the implicit transaction
works. What is its TransactionConcurrency and TransactionIsolation modes?

2) I think by the definition of how explicit transaction should work,
entry-level locking is unlikely for explicit transaction (perhaps it's only
for atomic-mode or implicit?). Explicit transaction groups multiple cache
operations and since we also make it PESSIMISTIC and SERIALIZABLE, it seems
highly likely to lock on a larger granularity than an entry to prevent other
operations from accessing or writing to any of those caches we operate on. 
I presume the locking behavior and granularity would change based on the
transaction, TransactionConcurrency, and TransactionIsolation. Do you know
how the locking schema changes based on each of the choices for the modes? I
don't think there's any documentation available.

Here is a portion of the stacktrace which may or may not be useful.

Caused by: class org.apache.ignite.transactions.TransactionTimeoutException:
Failed to acquire lock within provided timeout for transaction
[timeout=60000,
tx=GridNearTxLocal[xid=a7a36ec5861-00000000-097d-8e11-0000-000000000001,
xidVersion=GridCacheVersion [topVer=159223313, order=1547746818682,
nodeOrder=1], concurrency=PESSIMISTIC, isolation=SERIALIZABLE, state=ACTIVE,
invalidate=false, rollbackOnly=false,
nodeId=78a6e3af-19e6-4b62-831e-fd27f1478afb, timeout=60000, duration=61497]]
    at
org.apache.ignite.internal.util.IgniteUtils$13.apply(IgniteUtils.java:899)
    at
org.apache.ignite.internal.util.IgniteUtils$13.apply(IgniteUtils.java:894)
    ... 27 more




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Transactional cache in Atomic mode

[Mikhail <mi...@gmail.com>]

>1)

Batching always works better, however, I would benchmark single update vs
batch update to prove this.

>Would ignite itself make the implicit transaction /only if/ the explicit
JAVA API for transaction has not been called? 

Right, if there's no definition for the transaction, there will be an
implicit transaction, otherwise, explicit one will be used.

>2)
ignite uses entry lock level, so in your case, there should be no crossing
between transactions, so I don't see the reason why transaction should fail.
Do you think you can share reproducer or at least full logs with
stacktraces?

Thanks,
Mike.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Transactional cache in Atomic mode

[msuh <ms...@jobcase.com>]

Hi Mikhail,

Thanks for your answer.

1) So if Ignite implicitly puts each cache update in a transaction, does
that mean it's just better performance-wise to wrap 100k updates in a
transaction (We found that 100k was the optimal number of updates to do in a
single transaction) than to not explicitly put the updates in a transaction?
Would ignite itself make the implicit transaction /only if/ the explicit
JAVA API for transaction has not been called?

2) I should've stated it more clearly. So I was testing code structured like
this:

ExecutorService executor = Executors.newFixedThreadPool(10);
for (each partition p in node) {
    executor.execute(
         try(Transaction tx =
            ignite.transactions().txStart(..)) {
          update the entires in p
       }
    )
}

So we're wondering what the locking granularity is when we enter an
explicitly called transaction. Does it lock on partition, node, entry, or
something else? And is this configurable for each execution?


   
   



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Transactional cache in Atomic mode

[Mikhail <mi...@gmail.com>]

Hi

>1)
you can do all operations on transaction caches without defining an explicit
transaction.
However, even if you don't start transaction and for example put some data
in transactional cache, ignite itself will make implicit transaction, so
transactionCache.put("key", "value") -> will update you caches without any
call to commit.

>2)

ScanQuery isn't transactional, so I don't understand your question, because
ScanQuery doesn't acquire any locks.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/