You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oozie.apache.org by "Dénes Bodó (Jira)" <ji...@apache.org> on 2022/02/08 10:08:00 UTC
[jira] [Commented] (OOZIE-3646) Possible dead-lock in SignalXCommand
[ https://issues.apache.org/jira/browse/OOZIE-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17488722#comment-17488722 ]
Dénes Bodó commented on OOZIE-3646:
-----------------------------------
{noformat}
16:10:35 Testing JIRA OOZIE-3646
16:10:35
16:10:35 Cleaning local git workspace
16:10:35
16:10:35 ----------------------------
16:10:35
16:10:35 +1 PATCH_APPLIES
16:10:35 +1 CLEAN
16:10:35 -1 RAW_PATCH_ANALYSIS
16:10:35 +1 the patch does not introduce any @author tags
16:10:35 +1 the patch does not introduce any tabs
16:10:35 +1 the patch does not introduce any trailing spaces
16:10:35 +1 the patch does not introduce any star imports
16:10:35 -1 the patch contains 6 line(s) longer than 132 characters
16:10:35 +1 the patch adds/modifies 1 testcase(s)
16:10:35 +1 RAT
16:10:35 +1 the patch does not seem to introduce new RAT warnings
16:10:35 +1 JAVADOC
16:10:35 +1 Javadoc generation succeeded with the patch
16:10:35 +1 the patch does not seem to introduce new Javadoc warning(s)
16:10:35 +1 COMPILE
16:10:35 +1 HEAD compiles
16:10:35 +1 patch compiles
16:10:35 +1 the patch does not seem to introduce new javac warnings
16:10:35 -1 There are [6] new bugs found below threshold in total that must be fixed.
16:10:35 +1 There are no new bugs found in [client].
16:10:35 +1 There are no new bugs found in [webapp].
16:10:35 +1 There are no new bugs found in [fluent-job/fluent-job-api].
16:10:35 +1 There are no new bugs found in [server].
16:10:35 -1 There are [5] new bugs found below threshold in [core] that must be fixed.
16:10:35 You can find the SpotBugs diff here (look for the red and orange ones): core/findbugs-new.html
16:10:35 The most important SpotBugs errors are:
16:10:35 At BulkJPAExecutor.java:[line 206]: This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection
16:10:35 At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175]
16:10:35 At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199]
16:10:35 This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206]
16:10:35 At BulkJPAExecutor.java:[line 111]: At BulkJPAExecutor.java:[line 127]
16:10:35 +1 There are no new bugs found in [sharelib/spark].
16:10:35 +1 There are no new bugs found in [sharelib/hive2].
16:10:35 +1 There are no new bugs found in [sharelib/distcp].
16:10:35 +1 There are no new bugs found in [sharelib/streaming].
16:10:35 +1 There are no new bugs found in [sharelib/hive].
16:10:35 +1 There are no new bugs found in [sharelib/git].
16:10:35 +1 There are no new bugs found in [sharelib/hcatalog].
16:10:35 +1 There are no new bugs found in [sharelib/pig].
16:10:35 -1 There are [1] new bugs found below threshold in [sharelib/oozie] that must be fixed.
16:10:35 You can find the SpotBugs diff here (look for the red and orange ones): sharelib/oozie/findbugs-new.html
16:10:35 The most important SpotBugs errors are:
16:10:35 At ShellMain.java:[line 93]: This usage of java/lang/ProcessBuilder.<init>(Ljava/util/List;)V can be vulnerable to Command Injection
16:10:35 At ShellMain.java:[line 91]: At ShellMain.java:[line 90]
16:10:35 At ShellMain.java:[line 92]
16:10:35 +1 There are no new bugs found in [sharelib/sqoop].
16:10:35 +1 There are no new bugs found in [tools].
16:10:35 +1 There are no new bugs found in [docs].
16:10:35 +1 There are no new bugs found in [examples].
16:10:35 +1 BACKWARDS_COMPATIBILITY
16:10:35 +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
16:10:35 +1 the patch does not modify JPA files
16:10:35 +1 TESTS
16:10:35 Tests run: 3216
16:10:35 Tests failed at first run:
16:10:35 TestCoordActionInputCheckXCommand#testNone
16:10:35 TestBlockingInputStream#testFailure
16:10:35 For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
16:10:35 +1 DISTRO
16:10:35 +1 distro tarball builds with the patch
16:10:35 +1 MODERNIZER
16:10:35
16:10:35 ----------------------------
16:10:35 -1 Overall result, please check the reported -1(s) {noformat}
Only {{-1 the patch contains 6 line(s) longer than 132 characters}} left from patch set 2. If those are fixed and uploaded as OOZIE-3646-003.patch I will commit your change, [~zuston] .
> Possible dead-lock in SignalXCommand
> ------------------------------------
>
> Key: OOZIE-3646
> URL: https://issues.apache.org/jira/browse/OOZIE-3646
> Project: Oozie
> Issue Type: Bug
> Reporter: Junfan Zhang
> Assignee: Junfan Zhang
> Priority: Major
> Attachments: OOZIE-3646-002.patch, OOZIE-3646.patch-1, OOZIE-3646.patch-2, a1.png
>
>
> The limited thread execution mechanism aims to solve the dead-lock when all active threads are executing the SignalXCommand's invokeAll method.
> h2. Dead-lock when to happen
> Assuming that Oozie CallableQueue thread pool size is 120, when all threads are executing the {{SignalXCommand.startForkedActions}} method, a deadlock occurs.
> Because in {{SignalXCommand.startForkedActions}}, the code of
> {code:java}
> List<Future<ActionExecutorContext>> futures = Services.get().get(CallableQueueService.class)
> .invokeAll(tasks);
> {code}
> will be sync executed, however now all callableQueue threads are busy.
> h2. Solution
> 1. Need to limit directly invokeAll call when the num of rest threads is less than the tasks
> 2. To obtain correct active threads number in callableQueue, the SignalXCommand.class lock is needed.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)