You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Hadoop QA (JIRA)" <ji...@apache.org> on 2016/07/29 23:51:21 UTC
[jira] [Commented] (OOZIE-2501) ZK reentrant lock doesn't work for
few cases
[ https://issues.apache.org/jira/browse/OOZIE-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15400259#comment-15400259 ]
Hadoop QA commented on OOZIE-2501:
----------------------------------
Testing JIRA OOZIE-2501
Cleaning local git workspace
----------------------------
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
. {color:green}+1{color} the patch does not introduce any @author tags
. {color:green}+1{color} the patch does not introduce any tabs
. {color:green}+1{color} the patch does not introduce any trailing spaces
. {color:green}+1{color} the patch does not introduce any line longer than 132
. {color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
. {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
. {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
{color:red}-1 COMPILE{color}
. {color:red}-1{color} HEAD does not compile
. {color:red}-1{color} patch does not compile
. {color:green}+1{color} the patch does not seem to introduce new javac warnings
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
. {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
. {color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
. Tests run: 1795
. Tests failed: 130
. Tests errors: 323
. The patch failed the following testcases:
. testCoordKillXCommandUniqueness(org.apache.oozie.command.coord.TestCoordKillXCommand)
. testCoordKillRemovePushMissingDeps(org.apache.oozie.command.coord.TestCoordKillXCommand)
. testTimeoutTimingOutWriteLockThreads(org.apache.oozie.service.TestZKLocksService)
. testTimeoutTimingOutWriteLockOozies(org.apache.oozie.service.TestZKLocksService)
. testReadLockThreads(org.apache.oozie.service.TestZKLocksService)
. testReadLockOozies(org.apache.oozie.service.TestZKLocksService)
. testReadWriteLockThreads(org.apache.oozie.service.TestZKLocksService)
. testReadWriteLockOozies(org.apache.oozie.service.TestZKLocksService)
. testWriteReadLockThreads(org.apache.oozie.service.TestZKLocksService)
. testWriteReadLockOozies(org.apache.oozie.service.TestZKLocksService)
. testReentrantMultipleCall(org.apache.oozie.service.TestZKLocksService)
. testWaitWriteLockThreads(org.apache.oozie.service.TestZKLocksService)
. testWaitWriteLockOozies(org.apache.oozie.service.TestZKLocksService)
. testNoWaitWriteLockThreads(org.apache.oozie.service.TestZKLocksService)
. testNoWaitWriteLockOozies(org.apache.oozie.service.TestZKLocksService)
. testTimeoutWaitingWriteLockThreads(org.apache.oozie.service.TestZKLocksService)
. testTimeoutWaitingWriteLockOozies(org.apache.oozie.service.TestZKLocksService)
. testInterrupt(org.apache.oozie.service.TestCallableQueueService)
. testInterruptsWithCompositeCallable(org.apache.oozie.service.TestCallableQueueService)
. testInterruptsInCompositeCallable(org.apache.oozie.service.TestCallableQueueService)
. testInterruptsWithDistinguishedLockKeys(org.apache.oozie.service.TestCallableQueueService)
. testMaxInterruptMapSize(org.apache.oozie.service.TestCallableQueueService)
. testFsFailover(org.apache.oozie.action.TestActionFailover)
. testTimeOutWithException1(org.apache.oozie.command.coord.TestCoordPushDependencyCheckXCommand)
. testRequeueOnException(org.apache.oozie.command.coord.TestCoordPushDependencyCheckXCommand)
. testEngine(org.apache.oozie.command.coord.TestFutureActionsTimeOut)
. testCoordStatusTransitServiceSuspendedBottomUp(org.apache.oozie.service.TestStatusTransitService)
. testCoordStatusTransitServiceBackwardSupport(org.apache.oozie.service.TestStatusTransitService)
. testCoordStatusTransitServiceRunning3(org.apache.oozie.service.TestStatusTransitService)
. testBundleStatusTransitServiceForTerminalStates(org.apache.oozie.service.TestStatusTransitService)
. testCoordStatusTransitServiceSuspendedByUser(org.apache.oozie.service.TestStatusTransitService)
. testCoordStatusTransitServiceRunning1(org.apache.oozie.service.TestStatusTransitService)
. testCoordStatusTransitServiceKilledByUser2(org.apache.oozie.service.TestStatusTransitService)
. testBundleStatusTransitServicePausedWithError(org.apache.oozie.service.TestStatusTransitService)
. testBundleStatusTransitServiceKilled2(org.apache.oozie.service.TestStatusTransitService)
. testBundleStatusTransitServiceSucceeded1(org.apache.oozie.service.TestStatusTransitService)
. testCoordStatusTransitServicePaused(org.apache.oozie.service.TestStatusTransitService)
. testBundleStatusTransitServicePaused(org.apache.oozie.service.TestStatusTransitService)
. testBundleStatusTransitServiceSuspendedWithError(org.apache.oozie.service.TestStatusTransitService)
. testCoordStatusTransitServiceRunning2(org.apache.oozie.service.TestStatusTransitService)
. testBundleStatusTransitServiceSucceeded2(org.apache.oozie.service.TestStatusTransitService)
. testCoordStatusTransitServicePausedWithError(org.apache.oozie.service.TestStatusTransitService)
. testCoordStatusTransitServiceSucceeded(org.apache.oozie.service.TestStatusTransitService)
. testCoordStatusTransitServiceDoneWithError(org.apache.oozie.service.TestStatusTransitService)
. testCoordStatusTransitServiceSuspendedWithError(org.apache.oozie.service.TestStatusTransitService)
. testCoordStatusTransitServiceForTimeout(org.apache.oozie.service.TestStatusTransitService)
. testCoordStatusTransitServiceStaleCoordActions(org.apache.oozie.service.TestStatusTransitService)
. testBundleStatusTransitRunningFromKilled(org.apache.oozie.service.TestStatusTransitService)
. testBundleStatusTransitRunningWithError(org.apache.oozie.service.TestStatusTransitService)
. testBundleStatusCoordSubmitFails(org.apache.oozie.service.TestStatusTransitService)
. testSLAUpdateWithHA(org.apache.oozie.service.TestHASLAService)
. testOnJobEvent(org.apache.oozie.sla.TestSLAJobEventListener)
. testXCommandLifecycleLockingFailingToLock(org.apache.oozie.command.TestXCommand)
. testActionCheckerServiceCoord(org.apache.oozie.service.TestActionCheckerService)
. testBundleChangeReport(org.apache.oozie.command.bundle.TestBundleChangeXCommand)
. testCheckBundleActionStatus(org.apache.oozie.command.bundle.TestBundleChangeXCommand)
. testLoadOnRestart(org.apache.oozie.sla.TestSLACalculatorMemory)
. testWorkflowJobSLAStatusOnRestart(org.apache.oozie.sla.TestSLACalculatorMemory)
. testEventMissOnRestart(org.apache.oozie.sla.TestSLACalculatorMemory)
. testSLAEvents1(org.apache.oozie.sla.TestSLACalculatorMemory)
. testSLAEvents2(org.apache.oozie.sla.TestSLACalculatorMemory)
. testDuplicateStartMiss(org.apache.oozie.sla.TestSLACalculatorMemory)
. testDuplicateEndMiss(org.apache.oozie.sla.TestSLACalculatorMemory)
. testDisablingAlertsEvents(org.apache.oozie.sla.TestSLACalculatorMemory)
. testEventOutOfOrder(org.apache.oozie.sla.TestSLACalculatorMemory)
. testChangeTimeDeleteRunning(org.apache.oozie.command.coord.TestCoordChangeXCommand)
. testCoordStatus_Failed(org.apache.oozie.command.coord.TestCoordChangeXCommand)
. testCoordStatus_Killed(org.apache.oozie.command.coord.TestCoordChangeXCommand)
. testCoordStatus_Changefailed(org.apache.oozie.command.coord.TestCoordChangeXCommand)
. testCoord_throwException(org.apache.oozie.command.coord.TestCoordChangeXCommand)
. testCoordChangeEndTime3(org.apache.oozie.command.coord.TestCoordChangeXCommand)
. testWorkflowActionRecoveryUserRetry(org.apache.oozie.service.TestRecoveryService)
. testBundleRecoveryCoordCreate(org.apache.oozie.service.TestRecoveryService)
. testCoordCreateNotifyParentFailed(org.apache.oozie.service.TestRecoveryService)
. testCoordActionRecoveryServiceForSubmitted(org.apache.oozie.service.TestRecoveryService)
. testCoordActionRecoveryServiceForWaiting(org.apache.oozie.service.TestRecoveryService)
. testCoordActionRecoveryServiceForSuspended(org.apache.oozie.service.TestRecoveryService)
. testCoordActionRecoveryServiceForKilled(org.apache.oozie.service.TestRecoveryService)
. testCoordActionRecoveryServiceForResume(org.apache.oozie.service.TestRecoveryService)
. testBundleEngineResume(org.apache.oozie.servlet.TestV1JobServletBundleEngine)
. testBundleEngineSuspend(org.apache.oozie.servlet.TestV1JobServletBundleEngine)
. testBundleEngineStart(org.apache.oozie.servlet.TestV1JobServletBundleEngine)
. testBundleEngineKill(org.apache.oozie.servlet.TestV1JobServletBundleEngine)
. testBundleEngineChange(org.apache.oozie.servlet.TestV1JobServletBundleEngine)
. testCoordActionInputCheckXCommandUniqueness(org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC)
. testResolveCoordConfiguration(org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC)
. testTimeoutWithException(org.apache.oozie.command.coord.TestCoordActionInputCheckXCommandNonUTC)
. testKill(org.apache.oozie.command.coord.TestAbandonedCoordChecker)
. testStartTime(org.apache.oozie.command.coord.TestAbandonedCoordChecker)
. testCatchupJob(org.apache.oozie.command.coord.TestAbandonedCoordChecker)
. testPauseUnpause1(org.apache.oozie.service.TestPauseTransitService)
. testPauseUnpause2(org.apache.oozie.service.TestPauseTransitService)
. testPauseBundleAndCoordinator(org.apache.oozie.service.TestPauseTransitService)
. testUnpauseBundleAndCoordinator(org.apache.oozie.service.TestPauseTransitService)
. testStart1(org.apache.oozie.service.TestPauseTransitService)
. testStart2(org.apache.oozie.service.TestPauseTransitService)
. testCoordDefinitionChangeError(org.apache.oozie.command.coord.TestCoordUpdateXCommand)
. testCoordDefUnsupportedChange(org.apache.oozie.command.coord.TestCoordUpdateXCommand)
. testActionMaterForHcatalogIncorrectURI(org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand)
. testResetSequence_withMultiThread(org.apache.oozie.service.TestZKUUIDService)
. testResetSequence(org.apache.oozie.service.TestZKUUIDService)
. testEngine(org.apache.oozie.command.coord.TestPastActionsTimeOut)
. testCoordMaterializeTriggerService2(org.apache.oozie.service.TestCoordMaterializeTriggerService)
. testCoordMaterializeTriggerService3(org.apache.oozie.service.TestCoordMaterializeTriggerService)
. testCoordMaterializeTriggerService1(org.apache.oozie.service.TestCoordMaterializeTriggerService)
. testWaitWriteLock(org.apache.oozie.lock.TestMemoryLocks)
. testNoWaitWriteLock(org.apache.oozie.lock.TestMemoryLocks)
. testTimeoutWaitingWriteLock(org.apache.oozie.lock.TestMemoryLocks)
. testTimeoutTimingOutWriteLock(org.apache.oozie.lock.TestMemoryLocks)
. testReadLock(org.apache.oozie.lock.TestMemoryLocks)
. testReadWriteLock(org.apache.oozie.lock.TestMemoryLocks)
. testWriteReadLock(org.apache.oozie.lock.TestMemoryLocks)
. testWriteLockSameThreadNoWait(org.apache.oozie.lock.TestMemoryLocks)
. testWriteLockSameThreadWait(org.apache.oozie.lock.TestMemoryLocks)
. testLockReentrant(org.apache.oozie.lock.TestMemoryLocks)
. testCoordRerunActionsNeg2(org.apache.oozie.command.coord.TestCoordRerunXCommand)
. testCoordRerunDateNeg(org.apache.oozie.command.coord.TestCoordRerunXCommand)
. testCoordRerunKilledCoord(org.apache.oozie.command.coord.TestCoordRerunXCommand)
. testCoordRerunFailedCoordAction(org.apache.oozie.command.coord.TestCoordRerunXCommand)
. testCustomDoneFlag(org.apache.oozie.TestCoordinatorEngine)
. testEngine(org.apache.oozie.TestCoordinatorEngine)
. testDoneFlag(org.apache.oozie.TestCoordinatorEngine)
. testEmptyDoneFlag(org.apache.oozie.TestCoordinatorEngine)
. testDoneFlagCreation(org.apache.oozie.TestCoordinatorEngine)
. testUpdateSLA(org.apache.oozie.sla.TestSLAService)
. testEndMissDBConfirm(org.apache.oozie.sla.TestSLAService)
. testCoordActionInputCheckXCommandUniqueness(org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand)
. testResolveCoordConfiguration(org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand)
. testTimeoutWithException(org.apache.oozie.command.coord.TestCoordActionInputCheckXCommand)
. testLocalOozieExampleEnd(org.apache.oozie.example.TestLocalOozieExample)
{color:green}+1 DISTRO{color}
. {color:green}+1{color} distro tarball builds with the patch
----------------------------
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3167/
> ZK reentrant lock doesn't work for few cases
> --------------------------------------------
>
> Key: OOZIE-2501
> URL: https://issues.apache.org/jira/browse/OOZIE-2501
> Project: Oozie
> Issue Type: Bug
> Reporter: Purshotam Shah
> Assignee: Purshotam Shah
> Attachments: OOZIE-2501-V1.patch
>
>
> We will have an issue when oozie trying to acquire a lock and at the same time, some other thread is releasing the same lock .
> acquireLock will wait for 5 sec to acquire the lock. It will bypass the synchronized block and get lockEntry from the hashmap.
> While it waiting for 5 sec to acquire the lock, other thread releases the lock and may execute the release code which will remove lockEntry from the map.
> If some other command from same thread tries to acquire the lock, it will create a new InterProcessReadWriteLock object and use that for acquiring the lock.
> Logic for lock acquiring.
> {code}
> public LockToken getWriteLock(String resource, long wait) throws InterruptedException {
> InterProcessReadWriteLock lockEntry;
> synchronized (zkLocks) {
> if (zkLocks.containsKey(resource)) {
> lockEntry = zkLocks.get(resource);
> }
> else {
> lockEntry = new InterProcessReadWriteLock(zk.getClient(), LOCKS_NODE + "/" + resource);
> zkLocks.put(resource, lockEntry);
> }
> }
> InterProcessMutex writeLock = lockEntry.writeLock();
> return acquireLock(wait, writeLock, resource);
> }
> {code}
> Logic for lock releasing
> {code}
> public void release() {
> try {
> lock.release();
> if (zkLocks.get(resource) == null) {
> return;
> }
> if (!isLockHeld()) {
> synchronized (zkLocks) {
> if (zkLocks.get(resource) != null) {
> if (!isLockHeld()) {
> zkLocks.remove(resource);
> }
> }
> }
> }
> }
> catch (Exception ex) {
> LOG.warn("Could not release lock: " + ex.getMessage(), ex);
> }
> }
> {code}
> Curator code to acquire lock.
> {code}
> private boolean internalLock(long time, TimeUnit unit) throws Exception
> {
> /*
> Note on concurrency: a given lockData instance
> can be only acted on by a single thread so locking isn't necessary
> */
> Thread currentThread = Thread.currentThread();
> LockData lockData = threadData.get(currentThread);
> if ( lockData != null )
> {
> // re-entering
> lockData.lockCount.incrementAndGet();
> return true;
> }
> String lockPath = internals.attemptLock(time, unit, getLockNodeBytes());
> if ( lockPath != null )
> {
> LockData newLockData = new LockData(currentThread, lockPath);
> threadData.put(currentThread, newLockData);
> return true;
> }
> return false;
> }
> {code}
> The approach we have followed is to use map with weakvalue. Once the lock is unreachable. GC will remove it from the map. We don't have to explicitly remove it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)