You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by "Claude Mamo (Jira)" <ji...@apache.org> on 2023/06/15 16:46:00 UTC
[jira] [Updated] (DAFFODIL-2827) Error: Unsupported encoding: ISO-8859-1. Supported encodings:
[ https://issues.apache.org/jira/browse/DAFFODIL-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claude Mamo updated DAFFODIL-2827:
----------------------------------
Description:
I've consistently observed the following error when compiling several DFDL schemas concurrently on OpenJDK 11 and 17:
{noformat}
Caused by: org.apache.daffodil.dsom.SchemaDefinitionError: Schema Definition Error: Unsupported encoding: ISO-8859-1. Supported encodings:
Schema context: E1153 Location line 1524 column 14 in file:/home/smooks-edi-cartridge/edifact-schemas/target/classes/d94b/EDIFACT-Segments.dfdl.xsd
at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE (SDE.scala:179)
at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE$ (SDE.scala:178)
at org.apache.daffodil.dsom.DPathCompileInfo.SDE (CompiledExpression1.scala:210)
at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError (ThrowsSDE.scala:58)
at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError$ (ThrowsSDE.scala:58)
at org.apache.daffodil.dsom.DPathCompileInfo.schemaDefinitionError (CompiledExpression1.scala:210)
at org.apache.daffodil.processors.CharsetEvBase.compute (EvEncoding.scala:99)
at org.apache.daffodil.processors.CharsetEvBase.compute (EvEncoding.scala:81)
at org.apache.daffodil.processors.Evaluatable.evaluate (Evaluatable.scala:281)
at org.apache.daffodil.processors.Evaluatable.compileTimeEvaluate (Evaluatable.scala:212)
at org.apache.daffodil.processors.Evaluatable.compile (Evaluatable.scala:317)
at org.apache.daffodil.processors.Evaluatable.compile (Evaluatable.scala:326)
at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv (RuntimePropertyMixins.scala:122)
at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv$ (RuntimePropertyMixins.scala:119)
at org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv (RuntimePropertyMixins.scala:115)
at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv$ (RuntimePropertyMixins.scala:114)
at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo (TermEncodingMixin.scala:104)
at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo$ (TermEncodingMixin.scala:102)
at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo (LocalElementDecl.scala:25)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:180)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2 (ElementBaseRuntime1Mixin.scala:170)
at scala.collection.immutable.List.map (List.scala:293)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1 (ElementBaseRuntime1Mixin.scala:169)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (ElementBaseRuntime1Mixin.scala:169)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$ (ElementBaseRuntime1Mixin.scala:168)
at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (LocalElementDecl.scala:25)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:177)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2 (ElementBaseRuntime1Mixin.scala:170)
at scala.collection.immutable.List.map (List.scala:293)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1 (ElementBaseRuntime1Mixin.scala:169)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (ElementBaseRuntime1Mixin.scala:169)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$ (ElementBaseRuntime1Mixin.scala:168)
at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (LocalElementDecl.scala:25)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:177)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
at org.apache.daffodil.runtime1.LocalElementDeclBaseRuntime1Mixin.$anonfun$$init$$1 (LocalElementDeclBaseRuntime1Mixin.scala:24)
at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
at org.apache.daffodil.oolag.OOLAG$OOLAGHost.$anonfun$checkErrors$2 (OOLAG.scala:427)
at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23)
at org.apache.daffodil.oolag.OOLAG$.keepGoing (OOLAG.scala:65)
at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors (OOLAG.scala:427)
at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors$ (OOLAG.scala:413)
at org.apache.daffodil.dsom.SchemaComponentImpl.checkErrors (SchemaComponent.scala:34)
at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError (OOLAG.scala:484)
at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError$ (OOLAG.scala:483)
at org.apache.daffodil.dsom.SchemaSet.isError (SchemaSet.scala:543)
at org.apache.daffodil.compiler.ProcessorFactory.isError (Compiler.scala:136)
at org.apache.daffodil.compiler.Compiler.org$apache$daffodil$compiler$Compiler$$compileSourceInternal (Compiler.scala:419)
at org.apache.daffodil.compiler.Compiler$.org$apache$daffodil$compiler$Compiler$$compileSourceSynchronizer (Compiler.scala:461)
at org.apache.daffodil.compiler.Compiler.compileSource (Compiler.scala:404)
at org.apache.daffodil.compiler.Compiler.compileFile (Compiler.scala:388)
at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:146)
at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:126)
at org.smooks.edi.edg.EdifactDfdlSchemaGenerator.generateDfdlSchemas (EdifactDfdlSchemaGenerator.java:141){noformat}
Precisely, the error occurs on the line of code:
{code:java}
Daffodil.compiler().compileFile(...)
{code}
The error doesn't always occur on the same DFDL schema or the same schema location line for that matter. Curiously, this error, which I presume is happening because of a race-condition, doesn't show up on OpenJDK 8 so it could be a bug in the Java runtime. Note that a new Daffodil compiler is created to compile each schema. Moving {{Daffodil.compiler().compileFile(...)}} inside a {{synchronized}} block doesn't make a difference.
I had a hunch and added the following code before forking the thread to concurrently compile the schemas:
{code:java}
org.apache.daffodil.processors.charset.CharsetUtils.supportedEncodingsString(){code}
The above makes the error go away and I'm using it as a workaround. Can't really say why so but I suspect the race-condition is caused by cache reading that takes place when checking whether the encoding exists during schema compilation.
I've created a [branch|https://github.com/claudemamo/smooks-edi-cartridge/tree/daffodil-unsupported-encoding-issue] where you should be able to reproduce the problem. Steps to reproduce:
# git clone [https://github.com/claudemamo/smooks-edi-cartridge/tree/daffodil-unsupported-encoding-issue] --branch daffodil-unsupported-encoding-issue
# cd smooks-edi-cartridge
# mvn clean install -DskipTests
# The build should fail on the {{edifact-schemas}} branch because of this issue. The error should occur at this point: [https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L139.] Uncomment the following line to enable the workaround [https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L95]
was:
I've consistently observed the following error when compiling several DFDL schemas concurrently on OpenJDK 11 and 17:
{noformat}
Caused by: org.apache.daffodil.dsom.SchemaDefinitionError: Schema Definition Error: Unsupported encoding: ISO-8859-1. Supported encodings:
Schema context: E1153 Location line 1524 column 14 in file:/home/smooks-edi-cartridge/edifact-schemas/target/classes/d94b/EDIFACT-Segments.dfdl.xsd
at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE (SDE.scala:179)
at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE$ (SDE.scala:178)
at org.apache.daffodil.dsom.DPathCompileInfo.SDE (CompiledExpression1.scala:210)
at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError (ThrowsSDE.scala:58)
at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError$ (ThrowsSDE.scala:58)
at org.apache.daffodil.dsom.DPathCompileInfo.schemaDefinitionError (CompiledExpression1.scala:210)
at org.apache.daffodil.processors.CharsetEvBase.compute (EvEncoding.scala:99)
at org.apache.daffodil.processors.CharsetEvBase.compute (EvEncoding.scala:81)
at org.apache.daffodil.processors.Evaluatable.evaluate (Evaluatable.scala:281)
at org.apache.daffodil.processors.Evaluatable.compileTimeEvaluate (Evaluatable.scala:212)
at org.apache.daffodil.processors.Evaluatable.compile (Evaluatable.scala:317)
at org.apache.daffodil.processors.Evaluatable.compile (Evaluatable.scala:326)
at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv (RuntimePropertyMixins.scala:122)
at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv$ (RuntimePropertyMixins.scala:119)
at org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv (RuntimePropertyMixins.scala:115)
at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv$ (RuntimePropertyMixins.scala:114)
at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo (TermEncodingMixin.scala:104)
at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo$ (TermEncodingMixin.scala:102)
at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo (LocalElementDecl.scala:25)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:180)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2 (ElementBaseRuntime1Mixin.scala:170)
at scala.collection.immutable.List.map (List.scala:293)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1 (ElementBaseRuntime1Mixin.scala:169)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (ElementBaseRuntime1Mixin.scala:169)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$ (ElementBaseRuntime1Mixin.scala:168)
at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (LocalElementDecl.scala:25)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:177)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2 (ElementBaseRuntime1Mixin.scala:170)
at scala.collection.immutable.List.map (List.scala:293)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1 (ElementBaseRuntime1Mixin.scala:169)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (ElementBaseRuntime1Mixin.scala:169)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$ (ElementBaseRuntime1Mixin.scala:168)
at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (LocalElementDecl.scala:25)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:177)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
at org.apache.daffodil.runtime1.LocalElementDeclBaseRuntime1Mixin.$anonfun$$init$$1 (LocalElementDeclBaseRuntime1Mixin.scala:24)
at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
at org.apache.daffodil.oolag.OOLAG$OOLAGHost.$anonfun$checkErrors$2 (OOLAG.scala:427)
at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23)
at org.apache.daffodil.oolag.OOLAG$.keepGoing (OOLAG.scala:65)
at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors (OOLAG.scala:427)
at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors$ (OOLAG.scala:413)
at org.apache.daffodil.dsom.SchemaComponentImpl.checkErrors (SchemaComponent.scala:34)
at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError (OOLAG.scala:484)
at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError$ (OOLAG.scala:483)
at org.apache.daffodil.dsom.SchemaSet.isError (SchemaSet.scala:543)
at org.apache.daffodil.compiler.ProcessorFactory.isError (Compiler.scala:136)
at org.apache.daffodil.compiler.Compiler.org$apache$daffodil$compiler$Compiler$$compileSourceInternal (Compiler.scala:419)
at org.apache.daffodil.compiler.Compiler$.org$apache$daffodil$compiler$Compiler$$compileSourceSynchronizer (Compiler.scala:461)
at org.apache.daffodil.compiler.Compiler.compileSource (Compiler.scala:404)
at org.apache.daffodil.compiler.Compiler.compileFile (Compiler.scala:388)
at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:146)
at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:126)
at org.smooks.edi.edg.EdifactDfdlSchemaGenerator.generateDfdlSchemas (EdifactDfdlSchemaGenerator.java:141){noformat}
Precisely, the error occurs on the line of code:
{code:java}
Daffodil.compiler().compileFile(...)
{code}
The error doesn't always occur on the same DFDL schema or the same schema location line for that matter. Curiously, this error, which I presume is happening because of a race-condition, doesn't show up on OpenJDK 8 so it could be a bug in the Java runtime. Note that a new Daffodil compiler is created to compile each schema. Moving {{Daffodil.compiler().compileFile(...)}} inside a {{synchronized}} block doesn't make a difference.
I had a hunch and added the following code before forking the thread to concurrently compile the schemas:
{code:java}
org.apache.daffodil.processors.charset.CharsetUtils.supportedEncodingsString(){code}
The above makes the error go away and I'm using it as a workaround. Can't really say why so but I suspect the race-condition is caused by cache reading that takes place when checking whether the encoding exists during schema compilation.
I've created a [branch|https://github.com/claudemamo/smooks-edi-cartridge/tree/daffodil-unsupported-encoding-issue] where you should be able to reproduce the problem. Steps to reproduce:
# git clone [https://github.com/claudemamo/smooks-edi-cartridge/tree/daffodil-unsupported-encoding-issue] --branch daffodil-unsupported-encoding-issue
# cd smooks-edi-cartridge
# mvn clean install
# The build should fail on the {{edifact-schemas}} branch because of this issue. The error should occur at this point: [https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L139.] Uncomment the following line to enable the workaround [https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L95]
> Error: Unsupported encoding: ISO-8859-1. Supported encodings:
> --------------------------------------------------------------
>
> Key: DAFFODIL-2827
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2827
> Project: Daffodil
> Issue Type: Bug
> Components: DFDL Schemas
> Affects Versions: 3.4.0
> Environment: openjdk version "11.0.19" 2023-04-18
> OpenJDK Runtime Environment (build 11.0.19+7-post-Ubuntu-0ubuntu122.04.1)
> OpenJDK 64-Bit Server VM (build 11.0.19+7-post-Ubuntu-0ubuntu122.04.1, mixed mode, sharing)
> Reporter: Claude Mamo
> Priority: Minor
> Labels: race-condition
>
> I've consistently observed the following error when compiling several DFDL schemas concurrently on OpenJDK 11 and 17:
> {noformat}
> Caused by: org.apache.daffodil.dsom.SchemaDefinitionError: Schema Definition Error: Unsupported encoding: ISO-8859-1. Supported encodings:
> Schema context: E1153 Location line 1524 column 14 in file:/home/smooks-edi-cartridge/edifact-schemas/target/classes/d94b/EDIFACT-Segments.dfdl.xsd
> at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE (SDE.scala:179)
> at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE$ (SDE.scala:178)
> at org.apache.daffodil.dsom.DPathCompileInfo.SDE (CompiledExpression1.scala:210)
> at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError (ThrowsSDE.scala:58)
> at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError$ (ThrowsSDE.scala:58)
> at org.apache.daffodil.dsom.DPathCompileInfo.schemaDefinitionError (CompiledExpression1.scala:210)
> at org.apache.daffodil.processors.CharsetEvBase.compute (EvEncoding.scala:99)
> at org.apache.daffodil.processors.CharsetEvBase.compute (EvEncoding.scala:81)
> at org.apache.daffodil.processors.Evaluatable.evaluate (Evaluatable.scala:281)
> at org.apache.daffodil.processors.Evaluatable.compileTimeEvaluate (Evaluatable.scala:212)
> at org.apache.daffodil.processors.Evaluatable.compile (Evaluatable.scala:317)
> at org.apache.daffodil.processors.Evaluatable.compile (Evaluatable.scala:326)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv (RuntimePropertyMixins.scala:122)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv$ (RuntimePropertyMixins.scala:119)
> at org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv (RuntimePropertyMixins.scala:115)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv$ (RuntimePropertyMixins.scala:114)
> at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo (TermEncodingMixin.scala:104)
> at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo$ (TermEncodingMixin.scala:102)
> at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:180)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2 (ElementBaseRuntime1Mixin.scala:170)
> at scala.collection.immutable.List.map (List.scala:293)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1 (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$ (ElementBaseRuntime1Mixin.scala:168)
> at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:177)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2 (ElementBaseRuntime1Mixin.scala:170)
> at scala.collection.immutable.List.map (List.scala:293)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1 (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$ (ElementBaseRuntime1Mixin.scala:168)
> at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:177)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.LocalElementDeclBaseRuntime1Mixin.$anonfun$$init$$1 (LocalElementDeclBaseRuntime1Mixin.scala:24)
> at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.$anonfun$checkErrors$2 (OOLAG.scala:427)
> at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23)
> at org.apache.daffodil.oolag.OOLAG$.keepGoing (OOLAG.scala:65)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors (OOLAG.scala:427)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors$ (OOLAG.scala:413)
> at org.apache.daffodil.dsom.SchemaComponentImpl.checkErrors (SchemaComponent.scala:34)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError (OOLAG.scala:484)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError$ (OOLAG.scala:483)
> at org.apache.daffodil.dsom.SchemaSet.isError (SchemaSet.scala:543)
> at org.apache.daffodil.compiler.ProcessorFactory.isError (Compiler.scala:136)
> at org.apache.daffodil.compiler.Compiler.org$apache$daffodil$compiler$Compiler$$compileSourceInternal (Compiler.scala:419)
> at org.apache.daffodil.compiler.Compiler$.org$apache$daffodil$compiler$Compiler$$compileSourceSynchronizer (Compiler.scala:461)
> at org.apache.daffodil.compiler.Compiler.compileSource (Compiler.scala:404)
> at org.apache.daffodil.compiler.Compiler.compileFile (Compiler.scala:388)
> at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:146)
> at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:126)
> at org.smooks.edi.edg.EdifactDfdlSchemaGenerator.generateDfdlSchemas (EdifactDfdlSchemaGenerator.java:141){noformat}
> Precisely, the error occurs on the line of code:
> {code:java}
> Daffodil.compiler().compileFile(...)
> {code}
> The error doesn't always occur on the same DFDL schema or the same schema location line for that matter. Curiously, this error, which I presume is happening because of a race-condition, doesn't show up on OpenJDK 8 so it could be a bug in the Java runtime. Note that a new Daffodil compiler is created to compile each schema. Moving {{Daffodil.compiler().compileFile(...)}} inside a {{synchronized}} block doesn't make a difference.
> I had a hunch and added the following code before forking the thread to concurrently compile the schemas:
> {code:java}
> org.apache.daffodil.processors.charset.CharsetUtils.supportedEncodingsString(){code}
> The above makes the error go away and I'm using it as a workaround. Can't really say why so but I suspect the race-condition is caused by cache reading that takes place when checking whether the encoding exists during schema compilation.
> I've created a [branch|https://github.com/claudemamo/smooks-edi-cartridge/tree/daffodil-unsupported-encoding-issue] where you should be able to reproduce the problem. Steps to reproduce:
> # git clone [https://github.com/claudemamo/smooks-edi-cartridge/tree/daffodil-unsupported-encoding-issue] --branch daffodil-unsupported-encoding-issue
> # cd smooks-edi-cartridge
> # mvn clean install -DskipTests
> # The build should fail on the {{edifact-schemas}} branch because of this issue. The error should occur at this point: [https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L139.] Uncomment the following line to enable the workaround [https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L95]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)