You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by "Adams, Joshua" <ja...@owlcyberdefense.com> on 2023/02/21 16:33:52 UTC
Issues with runtime2 tests
I've been sitting on this a while as I think it's most likely just a missing dependency on my system, but I've been getting some errors on a few of the runtime2 tests. For example:
error] Test org.apache.daffodil.runtime2.TestExNums.c_ex_nums failed: org.apache.daffodil.tdml.TDMLExceptionImpl: (Implementation: daffodilC) UnparseError: Unparse Error: Result of /tmp/daffodilC14041106780301072313/c/daffodil…: 132
[error] , took 4.447 sec
[error] at org.apache.daffodil.tdml.TDMLException$.apply(TDMLException.scala:36)
[error] at org.apache.daffodil.tdml.ParserTestCase.doOnePassRoundTripUnparseExpectSuccess(TDMLRunner.scala:1204)
[error] at org.apache.daffodil.tdml.ParserTestCase.runParseExpectSuccess(TDMLRunner.scala:1354)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2(TDMLRunner.scala:989)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2$adapted(TDMLRunner.scala:979)
[error] at scala.util.Either$RightProjection.foreach(Either.scala:652)
[error] at org.apache.daffodil.tdml.ParserTestCase.runProcessor(TDMLRunner.scala:979)
[error] at org.apache.daffodil.tdml.TestCase.run(TDMLRunner.scala:922)
[error] at org.apache.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:452)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:229)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:235)
[error] at org.apache.daffodil.runtime2.TestExNums.c_ex_nums(TestExNums.scala:48)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] at java.lang.reflect.Method.invoke(Method.java:566)
[error] ...
[error] Caused by: Unparse Error: Result of /tmp/daffodilC14041106780301072313/c/daffodil…: 132
[error] Caused by: os.SubprocessException: Result of /tmp/daffodilC14041106780301072313/c/daffodil…: 132
[error] at os.proc.call(ProcessOps.scala:87)
[error] at org.apache.daffodil.processor.tdml.Runtime2TDMLDFDLProcessor.unparse(Runtime2TDMLDFDLProcessor.scala:197)
[error] at org.apache.daffodil.processor.tdml.Runtime2TDMLDFDLProcessor.unparse(Runtime2TDMLDFDLProcessor.scala:217)
[error] at org.apache.daffodil.tdml.ParserTestCase.doOnePassRoundTripUnparseExpectSuccess(TDMLRunner.scala:1201)
[error] at org.apache.daffodil.tdml.ParserTestCase.runParseExpectSuccess(TDMLRunner.scala:1354)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2(TDMLRunner.scala:989)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2$adapted(TDMLRunner.scala:979)
[error] at scala.util.Either$RightProjection.foreach(Either.scala:652)
[error] at org.apache.daffodil.tdml.ParserTestCase.runProcessor(TDMLRunner.scala:979)
[error] at org.apache.daffodil.tdml.TestCase.run(TDMLRunner.scala:922)
[error] at org.apache.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:452)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:229)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:235)
[error] at org.apache.daffodil.runtime2.TestExNums.c_ex_nums(TestExNums.scala:48)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] at java.lang.reflect.Method.invoke(Method.java:566)
Any idea's what I'm missing? I've tested on a different system and everything works fine so I'm leaning towards some dependency that I'm missing that we may need to document.
Josh
RE: Issues with runtime2 tests
Posted by "McGann, Mike" <mm...@owlcyberdefense.com>.
I'm not sure if this applies, but I've seen similar issues in the past when security software decides to quarantine the binary.
// Mike
-----Original Message-----
From: Interrante, John A (GE Research, US) <Jo...@ge.com>
Sent: Tuesday, February 21, 2023 12:55
To: dev@daffodil.apache.org
Subject: RE: Issues with runtime2 tests
Josh,
Your stack trace's error message tells me that when you ran the ex_nums test case with TDMLImplementation.DaffodilC, the C-compiled executable exited abnormally with the numeric exit code 132 instead of the normal (successful) exit code 0. When a program dies because of a signal, its exit code is encoded as 128 + signal-number, which tells me that the executable died because it received the SIGILL (illegal instruction) signal. Getting a SIGILL signal indicates something strange must have happened (https://stackoverflow.com/questions/7901867/what-causes-signal-sigill).
Is there anything unusual about your machine or is it a standard Intel/AMD 64-bit processor running standard Ubuntu 20.04 or 22.04 LTS with the system C compiler, libs, etc.? You may have to hand-generate the C code from the ex_nums schema (daffodil generate c -s ex_nums.dfd,xsd), hand-compile the executable (make -C c), and see if you can reproduce the SIGILL yourself (c/daffodil unparse infosets/ex_nums.dat.xml -o c/ex_num.dat). If you do reproduce the signal, then you can run the executable under the gdb debugger and try to determine how it got the signal.
John
-----Original Message-----
From: Adams, Joshua <ja...@owlcyberdefense.com>
Sent: Tuesday, February 21, 2023 8:34 AM
To: dev@daffodil.apache.org
Subject: EXT: Issues with runtime2 tests
WARNING: This email originated from outside of GE. Please validate the sender's email address before clicking on links or attachments as they may not be safe.
I've been sitting on this a while as I think it's most likely just a missing dependency on my system, but I've been getting some errors on a few of the runtime2 tests. For example:
error] Test org.apache.daffodil.runtime2.TestExNums.c_ex_nums failed: org.apache.daffodil.tdml.TDMLExceptionImpl: (Implementation: daffodilC) UnparseError: Unparse Error: Result of /tmp/daffodilC14041106780301072313/c/daffodil...: 132 [error] , took 4.447 sec
[error] at org.apache.daffodil.tdml.TDMLException$.apply(TDMLException.scala:36)
[error] at org.apache.daffodil.tdml.ParserTestCase.doOnePassRoundTripUnparseExpectSuccess(TDMLRunner.scala:1204)
[error] at org.apache.daffodil.tdml.ParserTestCase.runParseExpectSuccess(TDMLRunner.scala:1354)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2(TDMLRunner.scala:989)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2$adapted(TDMLRunner.scala:979)
[error] at scala.util.Either$RightProjection.foreach(Either.scala:652)
[error] at org.apache.daffodil.tdml.ParserTestCase.runProcessor(TDMLRunner.scala:979)
[error] at org.apache.daffodil.tdml.TestCase.run(TDMLRunner.scala:922)
[error] at org.apache.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:452)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:229)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:235)
[error] at org.apache.daffodil.runtime2.TestExNums.c_ex_nums(TestExNums.scala:48)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] at java.lang.reflect.Method.invoke(Method.java:566)
[error] ...
[error] Caused by: Unparse Error: Result of /tmp/daffodilC14041106780301072313/c/daffodil...: 132 [error] Caused by: os.SubprocessException: Result of /tmp/daffodilC14041106780301072313/c/daffodil...: 132
[error] at os.proc.call(ProcessOps.scala:87)
[error] at org.apache.daffodil.processor.tdml.Runtime2TDMLDFDLProcessor.unparse(Runtime2TDMLDFDLProcessor.scala:197)
[error] at org.apache.daffodil.processor.tdml.Runtime2TDMLDFDLProcessor.unparse(Runtime2TDMLDFDLProcessor.scala:217)
[error] at org.apache.daffodil.tdml.ParserTestCase.doOnePassRoundTripUnparseExpectSuccess(TDMLRunner.scala:1201)
[error] at org.apache.daffodil.tdml.ParserTestCase.runParseExpectSuccess(TDMLRunner.scala:1354)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2(TDMLRunner.scala:989)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2$adapted(TDMLRunner.scala:979)
[error] at scala.util.Either$RightProjection.foreach(Either.scala:652)
[error] at org.apache.daffodil.tdml.ParserTestCase.runProcessor(TDMLRunner.scala:979)
[error] at org.apache.daffodil.tdml.TestCase.run(TDMLRunner.scala:922)
[error] at org.apache.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:452)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:229)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:235)
[error] at org.apache.daffodil.runtime2.TestExNums.c_ex_nums(TestExNums.scala:48)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] at java.lang.reflect.Method.invoke(Method.java:566)
Any idea's what I'm missing? I've tested on a different system and everything works fine so I'm leaning towards some dependency that I'm missing that we may need to document.
Josh
Re: Issues with runtime2 tests
Posted by "Adams, Joshua" <ja...@owlcyberdefense.com>.
Doing everything by had seems to work fine and I simply get a return code of 0 when running the unparse command. I'm running on a standard AMD64 chip running Fedora 37 Workstation. The other machine that I've tested on is also running Fedora 37 Workstation with what should be a very similar software setup and it works fine.
I probably need to spend some time digging around to get a better idea of why this is failing when being run as part of the sbt tests.
Josh
________________________________
From: Interrante, John A (GE Research, US) <Jo...@ge.com>
Sent: Tuesday, February 21, 2023 12:54 PM
To: dev@daffodil.apache.org <de...@daffodil.apache.org>
Subject: RE: Issues with runtime2 tests
Josh,
Your stack trace's error message tells me that when you ran the ex_nums test case with TDMLImplementation.DaffodilC, the C-compiled executable exited abnormally with the numeric exit code 132 instead of the normal (successful) exit code 0. When a program dies because of a signal, its exit code is encoded as 128 + signal-number, which tells me that the executable died because it received the SIGILL (illegal instruction) signal. Getting a SIGILL signal indicates something strange must have happened (https://stackoverflow.com/questions/7901867/what-causes-signal-sigill).
Is there anything unusual about your machine or is it a standard Intel/AMD 64-bit processor running standard Ubuntu 20.04 or 22.04 LTS with the system C compiler, libs, etc.? You may have to hand-generate the C code from the ex_nums schema (daffodil generate c -s ex_nums.dfd,xsd), hand-compile the executable (make -C c), and see if you can reproduce the SIGILL yourself (c/daffodil unparse infosets/ex_nums.dat.xml -o c/ex_num.dat). If you do reproduce the signal, then you can run the executable under the gdb debugger and try to determine how it got the signal.
John
-----Original Message-----
From: Adams, Joshua <ja...@owlcyberdefense.com>
Sent: Tuesday, February 21, 2023 8:34 AM
To: dev@daffodil.apache.org
Subject: EXT: Issues with runtime2 tests
WARNING: This email originated from outside of GE. Please validate the sender's email address before clicking on links or attachments as they may not be safe.
I've been sitting on this a while as I think it's most likely just a missing dependency on my system, but I've been getting some errors on a few of the runtime2 tests. For example:
error] Test org.apache.daffodil.runtime2.TestExNums.c_ex_nums failed: org.apache.daffodil.tdml.TDMLExceptionImpl: (Implementation: daffodilC) UnparseError: Unparse Error: Result of /tmp/daffodilC14041106780301072313/c/daffodil...: 132 [error] , took 4.447 sec
[error] at org.apache.daffodil.tdml.TDMLException$.apply(TDMLException.scala:36)
[error] at org.apache.daffodil.tdml.ParserTestCase.doOnePassRoundTripUnparseExpectSuccess(TDMLRunner.scala:1204)
[error] at org.apache.daffodil.tdml.ParserTestCase.runParseExpectSuccess(TDMLRunner.scala:1354)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2(TDMLRunner.scala:989)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2$adapted(TDMLRunner.scala:979)
[error] at scala.util.Either$RightProjection.foreach(Either.scala:652)
[error] at org.apache.daffodil.tdml.ParserTestCase.runProcessor(TDMLRunner.scala:979)
[error] at org.apache.daffodil.tdml.TestCase.run(TDMLRunner.scala:922)
[error] at org.apache.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:452)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:229)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:235)
[error] at org.apache.daffodil.runtime2.TestExNums.c_ex_nums(TestExNums.scala:48)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] at java.lang.reflect.Method.invoke(Method.java:566)
[error] ...
[error] Caused by: Unparse Error: Result of /tmp/daffodilC14041106780301072313/c/daffodil...: 132 [error] Caused by: os.SubprocessException: Result of /tmp/daffodilC14041106780301072313/c/daffodil...: 132
[error] at os.proc.call(ProcessOps.scala:87)
[error] at org.apache.daffodil.processor.tdml.Runtime2TDMLDFDLProcessor.unparse(Runtime2TDMLDFDLProcessor.scala:197)
[error] at org.apache.daffodil.processor.tdml.Runtime2TDMLDFDLProcessor.unparse(Runtime2TDMLDFDLProcessor.scala:217)
[error] at org.apache.daffodil.tdml.ParserTestCase.doOnePassRoundTripUnparseExpectSuccess(TDMLRunner.scala:1201)
[error] at org.apache.daffodil.tdml.ParserTestCase.runParseExpectSuccess(TDMLRunner.scala:1354)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2(TDMLRunner.scala:989)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2$adapted(TDMLRunner.scala:979)
[error] at scala.util.Either$RightProjection.foreach(Either.scala:652)
[error] at org.apache.daffodil.tdml.ParserTestCase.runProcessor(TDMLRunner.scala:979)
[error] at org.apache.daffodil.tdml.TestCase.run(TDMLRunner.scala:922)
[error] at org.apache.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:452)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:229)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:235)
[error] at org.apache.daffodil.runtime2.TestExNums.c_ex_nums(TestExNums.scala:48)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] at java.lang.reflect.Method.invoke(Method.java:566)
Any idea's what I'm missing? I've tested on a different system and everything works fine so I'm leaning towards some dependency that I'm missing that we may need to document.
Josh
RE: Issues with runtime2 tests
Posted by "Interrante, John A (GE Research, US)" <Jo...@ge.com>.
Josh,
Your stack trace's error message tells me that when you ran the ex_nums test case with TDMLImplementation.DaffodilC, the C-compiled executable exited abnormally with the numeric exit code 132 instead of the normal (successful) exit code 0. When a program dies because of a signal, its exit code is encoded as 128 + signal-number, which tells me that the executable died because it received the SIGILL (illegal instruction) signal. Getting a SIGILL signal indicates something strange must have happened (https://stackoverflow.com/questions/7901867/what-causes-signal-sigill).
Is there anything unusual about your machine or is it a standard Intel/AMD 64-bit processor running standard Ubuntu 20.04 or 22.04 LTS with the system C compiler, libs, etc.? You may have to hand-generate the C code from the ex_nums schema (daffodil generate c -s ex_nums.dfd,xsd), hand-compile the executable (make -C c), and see if you can reproduce the SIGILL yourself (c/daffodil unparse infosets/ex_nums.dat.xml -o c/ex_num.dat). If you do reproduce the signal, then you can run the executable under the gdb debugger and try to determine how it got the signal.
John
-----Original Message-----
From: Adams, Joshua <ja...@owlcyberdefense.com>
Sent: Tuesday, February 21, 2023 8:34 AM
To: dev@daffodil.apache.org
Subject: EXT: Issues with runtime2 tests
WARNING: This email originated from outside of GE. Please validate the sender's email address before clicking on links or attachments as they may not be safe.
I've been sitting on this a while as I think it's most likely just a missing dependency on my system, but I've been getting some errors on a few of the runtime2 tests. For example:
error] Test org.apache.daffodil.runtime2.TestExNums.c_ex_nums failed: org.apache.daffodil.tdml.TDMLExceptionImpl: (Implementation: daffodilC) UnparseError: Unparse Error: Result of /tmp/daffodilC14041106780301072313/c/daffodil...: 132 [error] , took 4.447 sec
[error] at org.apache.daffodil.tdml.TDMLException$.apply(TDMLException.scala:36)
[error] at org.apache.daffodil.tdml.ParserTestCase.doOnePassRoundTripUnparseExpectSuccess(TDMLRunner.scala:1204)
[error] at org.apache.daffodil.tdml.ParserTestCase.runParseExpectSuccess(TDMLRunner.scala:1354)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2(TDMLRunner.scala:989)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2$adapted(TDMLRunner.scala:979)
[error] at scala.util.Either$RightProjection.foreach(Either.scala:652)
[error] at org.apache.daffodil.tdml.ParserTestCase.runProcessor(TDMLRunner.scala:979)
[error] at org.apache.daffodil.tdml.TestCase.run(TDMLRunner.scala:922)
[error] at org.apache.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:452)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:229)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:235)
[error] at org.apache.daffodil.runtime2.TestExNums.c_ex_nums(TestExNums.scala:48)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] at java.lang.reflect.Method.invoke(Method.java:566)
[error] ...
[error] Caused by: Unparse Error: Result of /tmp/daffodilC14041106780301072313/c/daffodil...: 132 [error] Caused by: os.SubprocessException: Result of /tmp/daffodilC14041106780301072313/c/daffodil...: 132
[error] at os.proc.call(ProcessOps.scala:87)
[error] at org.apache.daffodil.processor.tdml.Runtime2TDMLDFDLProcessor.unparse(Runtime2TDMLDFDLProcessor.scala:197)
[error] at org.apache.daffodil.processor.tdml.Runtime2TDMLDFDLProcessor.unparse(Runtime2TDMLDFDLProcessor.scala:217)
[error] at org.apache.daffodil.tdml.ParserTestCase.doOnePassRoundTripUnparseExpectSuccess(TDMLRunner.scala:1201)
[error] at org.apache.daffodil.tdml.ParserTestCase.runParseExpectSuccess(TDMLRunner.scala:1354)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2(TDMLRunner.scala:989)
[error] at org.apache.daffodil.tdml.ParserTestCase.$anonfun$runProcessor$2$adapted(TDMLRunner.scala:979)
[error] at scala.util.Either$RightProjection.foreach(Either.scala:652)
[error] at org.apache.daffodil.tdml.ParserTestCase.runProcessor(TDMLRunner.scala:979)
[error] at org.apache.daffodil.tdml.TestCase.run(TDMLRunner.scala:922)
[error] at org.apache.daffodil.tdml.DFDLTestSuite.runOneTest(TDMLRunner.scala:452)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:229)
[error] at org.apache.daffodil.tdml.Runner.runOneTest(RunnerFactory.scala:235)
[error] at org.apache.daffodil.runtime2.TestExNums.c_ex_nums(TestExNums.scala:48)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] at java.lang.reflect.Method.invoke(Method.java:566)
Any idea's what I'm missing? I've tested on a different system and everything works fine so I'm leaning towards some dependency that I'm missing that we may need to document.
Josh