Proper use of ignite semaphore

[robbie <ro...@gmail.com>]

Hi guys,

I have a couple of questions regarding our team’s use of Ignite’s semaphore. 

Our team has used semaphores to throttle the number of compute jobs being
sent to the grid:
e.g. :
In ComputeTaskAdapter.map {
final IgniteSemaphore semaphore = ignite.semaphore(semaphoreName,
throttleSize, true, true);
taskSession.setAttribute(SEMAPHORE_KEY, semaphoreName);
for (ComputeJob job : jobs) {
    semaphore.acquire();
    mapper.send(job);
}
}

The semaphores are then released in ComputeJob.execute {

final String semaphoreName =
taskSession.getAttribute(AbstractComputeTaskAdapter.SEMAPHORE_KEY);
final IgniteSemaphore semaphore = semaphoreName != null ?
ignite.semaphore(semaphoreName, 0, true, false) : null;
try {
    return executeManaged();
}
finally {

    if (semaphore != null) {
        semaphore.release();
    }
    if (latch != null) {
        latch.countDown();
    }
    LOG.info(".. completed sequence #" + sequence + ": " + taskSessionId);
}
}


My question is, is this the proper use of Ignite semaphores? Is it safe to
use semaphores inside compute tasks? During my resiliency testing with this
code, it seems like the locks from semaphores don’t get released whenever I
start killing (kill -9) all but one of our server nodes.





--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Proper-use-of-ignite-semaphore-tp13509.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Proper use of ignite semaphore

[dkarachentsev <dk...@gridgain.com>]

Hi,

To make semaphore fail safe in your case you may set AtomicConfiguration to
IgniteConfiguration with REPLICATED cache mode, so semaphore data will be
stored on every node.
Additionally check FifoQueueCollisionSpi [1] that will control jobs
execution for you.

[1]
https://apacheignite.readme.io/v2.0/docs/job-scheduling#section-fifo-ordering

Thanks!
-Dmitry



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Proper-use-of-ignite-semaphore-tp13509p13527.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.