[jira] [Commented] (OOZIE-3132) Instrument SLACalculatorMemory

["Hadoop QA (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-3132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16258219#comment-16258219 ] 

Hadoop QA commented on OOZIE-3132:
----------------------------------

Testing JIRA OOZIE-3132

Cleaning local git workspace

----------------------------

{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.    {color:green}+1{color} the patch does not introduce any @author tags
.    {color:green}+1{color} the patch does not introduce any tabs
.    {color:green}+1{color} the patch does not introduce any trailing spaces
.    {color:green}+1{color} the patch does not introduce any line longer than 132
.    {color:green}+1{color} the patch does adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.    {color:green}+1{color} the patch does not seem to introduce new RAT warnings
{color:green}+1 JAVADOC{color}
.    {color:green}+1{color} the patch does not seem to introduce new Javadoc warnings
.    {color:red}WARNING{color}: the current HEAD has 77 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.    {color:green}+1{color} HEAD compiles
.    {color:green}+1{color} patch compiles
.    {color:green}+1{color} the patch does not seem to introduce new javac warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [server].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.    {color:green}+1{color} the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
.    {color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.    Tests run: 0
.    Tests rerun: 0
.    Tests failed at first run: org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand,org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.command.coord.TestCoordSubmitXCommand,org.apache.oozie.service.TestStatusTransitService,org.apache.oozie.util.TestMetricsInstrumentation,org.apache.oozie.command.coord.TestCoordActionMissingDependenciesXCommand,org.apache.oozie.service.TestConfigurationService,
{color:green}+1 DISTRO{color}
.    {color:green}+1{color} distro tarball builds with the patch 

----------------------------
{color:green}*+1 Overall result, good!, no -1s*{color}

{color:red}. There is at least one warning, please check{color}

The full output of the test-patch run is available at

. https://builds.apache.org/job/PreCommit-OOZIE-Build/241/

> Instrument SLACalculatorMemory
> ------------------------------
>
>                 Key: OOZIE-3132
>                 URL: https://issues.apache.org/jira/browse/OOZIE-3132
>             Project: Oozie
>          Issue Type: Improvement
>          Components: core
>    Affects Versions: 4.3.0
>            Reporter: Andras Piros
>            Assignee: Andras Piros
>             Fix For: 5.0.0b1
>
>         Attachments: OOZIE-3132.001.patch
>
>
> When there are lots of {{WorkflowJobBean}} and {{CoordinatorJobBean}} instances that have to be followed up on creating {{SLASummaryBean}} instances, following can occur:
> * we set {{oozie.sla.service.SLAService.capacity}} to a sane value like {{10000}} to preserve heap consumption
> * {{SLACalculatorMemory#addRegistration()}} and {{SLACalculatorMemory#updateRegistration}} would:
> ** either emit {{TRACE}} level logs like {{SLA Registration Event - Job:}} showing the add / update of {{SLARegistrationBean}} was successful
> ** or emit {{ERROR}} level logs like {{SLACalculator memory capacity reached. Cannot add or update new SLA Registration entry for job}} showing the add / update of {{SLARegistrationBean}} was not successful
> Since sometimes stale or already processed {{SLAEvent}} entries from {{SLACalculatorMemory#slaMap}} get removed, it's pretty hard to say what is its the actual size - that is, whether the next add or update command will succeed
> We need an {{Instrumentation.Counter}} instance that gets incremented when there is an {{SLACalculatorMemory#slaMap#put()}} with a new entry added, and gets decremented when there happens a {{SLACalculatorMemory#slaMap#remove()}} with an existing entry removed. This counter will be automatically present within REST interface, and Oozie client.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)