You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by zentol <gi...@git.apache.org> on 2017/11/14 14:06:30 UTC

[GitHub] flink pull request #5012: [FLINK-8070][yarn][tests] Print errors found in lo...

GitHub user zentol opened a pull request:

    https://github.com/apache/flink/pull/5012

    [FLINK-8070][yarn][tests] Print errors found in log files

    ## What is the purpose of the change
    
    This PR modifies the `YarnTestBase` to print found exceptions in the test failure message.
    
    The output now looks like this:
    
    ```
    java.lang.AssertionError(Found a file /home/Zento/flink/flink-yarn-tests/target/flink-yarn-tests-fifo/flink-yarn-tests-fifo-logDir-nm-0_0/application_1510667711263_0001/container_1510667711263_0001_01_000001/jobmanager.log with a prohibited string (one of [Exception, Started SelectChannelConnector@0.0.0.0:8081]). Excerpts:
    [
    java.lang.Exception: Could not create actor system
            at org.apache.flink.runtime.clusterframework.BootstrapTools.startActorSystem(BootstrapTools.java:171)
            at org.apache.flink.runtime.clusterframework.BootstrapTools.startActorSystem(BootstrapTools.java:115)
            at org.apache.flink.yarn.YarnApplicationMasterRunner.runApplicationMaster(YarnApplicationMasterRunner.java:313)
            at org.apache.flink.yarn.YarnApplicationMasterRunner$1.call(YarnApplicationMasterRunner.java:199)
            at org.apache.flink.yarn.YarnApplicationMasterRunner$1.call(YarnApplicationMasterRunner.java:196)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:422)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
            at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
            at org.apache.flink.yarn.YarnApplicationMasterRunner.run(YarnApplicationMasterRunner.java:196)
            at org.apache.flink.yarn.YarnApplicationMasterRunner.main(YarnApplicationMasterRunner.java:123)
    Caused by: java.lang.VerifyError: Inconsistent stackmap frames at branch target 152
    ])
            at org.junit.runners.model.MultipleFailureException.assertEmpty(MultipleFailureException.java:67)
            at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:39)
            at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
            at org.junit.rules.RunRules.evaluate(RunRules.java:20)
            at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
            at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
            at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
            at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
            at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
            at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
            at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
            at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
            at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
            at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
            at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
            at org.junit.rules.RunRules.evaluate(RunRules.java:20)
            at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
            at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
            at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
            at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
            at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
            at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
            at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
            at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
    ```
    
    ## Verifying this change
    
    Create some failure condition in the yarn tests and check the test output. (I _luckily_ got one on hand locally due to FLINK-8071)


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zentol/flink 8070

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5012.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5012
    
----
commit f563873f1a413c5410c565df318f01510c1a0620
Author: zentol <ch...@apache.org>
Date:   2017-11-14T10:55:33Z

    [FLINK-8070][yarn][tests] Print errors found in log files

----


---

[GitHub] flink issue #5012: [FLINK-8070][yarn][tests] Print errors found in log files

Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:

    https://github.com/apache/flink/pull/5012
  
    In this particular stack trace, you have the stack trace of the assertion error
    
    ```
    java.lang.AssertionError(... exception message ...)
            at org.junit.runners.model.MultipleFailureException.assertEmpty(MultipleFailureException.java:67)
            at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:39)
            at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
            at org.junit.rules.RunRules.evaluate(RunRules.java:20)
            at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
            at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
            at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
            at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
            at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
            at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
            at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
            at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
            at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
            at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
            at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
            at org.junit.rules.RunRules.evaluate(RunRules.java:20)
            at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
            at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
            at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
            at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
            at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)
            at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
            at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
            at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
    ```
    
    and the excerpts in the exception message of the AssertionError, as a list of exceptions
    
    ```
    Found a file /home/Zento/flink/flink-yarn-tests/target/flink-yarn-tests-fifo/flink-yarn-tests-fifo-logDir-nm-0_0/application_1510667711263_0001/container_1510667711263_0001_01_000001/jobmanager.log with a prohibited string (one of [Exception, Started SelectChannelConnector@0.0.0.0:8081]). Excerpts:
    [
    java.lang.Exception: Could not create actor system
            at org.apache.flink.runtime.clusterframework.BootstrapTools.startActorSystem(BootstrapTools.java:171)
            at org.apache.flink.runtime.clusterframework.BootstrapTools.startActorSystem(BootstrapTools.java:115)
            at org.apache.flink.yarn.YarnApplicationMasterRunner.runApplicationMaster(YarnApplicationMasterRunner.java:313)
            at org.apache.flink.yarn.YarnApplicationMasterRunner$1.call(YarnApplicationMasterRunner.java:199)
            at org.apache.flink.yarn.YarnApplicationMasterRunner$1.call(YarnApplicationMasterRunner.java:196)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:422)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
            at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
            at org.apache.flink.yarn.YarnApplicationMasterRunner.run(YarnApplicationMasterRunner.java:196)
            at org.apache.flink.yarn.YarnApplicationMasterRunner.main(YarnApplicationMasterRunner.java:123)
    Caused by: java.lang.VerifyError: Inconsistent stackmap frames at branch target 152
    ]
    ```
    
    The formatting isn't really ideal, but i don't know of an easy way to change it. (And it's still better than nothing)



---

[GitHub] flink issue #5012: [FLINK-8070][yarn][tests] Print errors found in log files

Posted by greghogan <gi...@git.apache.org>.
Github user greghogan commented on the issue:

    https://github.com/apache/flink/pull/5012
  
    LGTM. I'm puzzled why the `])` is printed out-of-order in the middle of the stack trace.


---

[GitHub] flink pull request #5012: [FLINK-8070][yarn][tests] Print errors found in lo...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/flink/pull/5012


---

[GitHub] flink issue #5012: [FLINK-8070][yarn][tests] Print errors found in log files

Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:

    https://github.com/apache/flink/pull/5012
  
    merging.


---

[GitHub] flink issue #5012: [FLINK-8070][yarn][tests] Print errors found in log files

Posted by zentol <gi...@git.apache.org>.
Github user zentol commented on the issue:

    https://github.com/apache/flink/pull/5012
  
    Also, this particular exception stops at after `Caused by: java.lang.VerifyError: Inconsistent stackmap frames at branch target 152` because the next lines don't look like a stack trace.
    
    Here's a snippet:
    ```
      at org.apache.flink.yarn.YarnApplicationMasterRunner.run(YarnApplicationMasterRunner.java:196)
            at org.apache.flink.yarn.YarnApplicationMasterRunner.main(YarnApplicationMasterRunner.java:123)
    Caused by: java.lang.VerifyError: Inconsistent stackmap frames at branch target 152
    Exception Details:
      Location:
        akka/dispatch/Mailbox.processAllSystemMessages()V @152: getstatic
      Reason:
    ```


---

[GitHub] flink issue #5012: [FLINK-8070][yarn][tests] Print errors found in log files

Posted by greghogan <gi...@git.apache.org>.
Github user greghogan commented on the issue:

    https://github.com/apache/flink/pull/5012
  
    Ah, thanks @zentol for the clarification. Very nice to have this!


---