You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by "Steve Lawrence (Jira)" <ji...@apache.org> on 2023/06/15 17:51:00 UTC
[jira] [Commented] (DAFFODIL-2827) Error: Unsupported encoding: ISO-8859-1. Supported encodings:
[ https://issues.apache.org/jira/browse/DAFFODIL-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733187#comment-17733187 ]
Steve Lawrence commented on DAFFODIL-2827:
------------------------------------------
Based on the "Supported Encodings" error message being empty, it sounds like no encodings are loaded, which is done using our SimpleNamedServiceLoader:
https://github.com/apache/daffodil/blob/main/daffodil-io/src/main/scala/org/apache/daffodil/io/processors/charset/BitsCharsetDefinitionRegistry.scala#L27-L28
The resulting map is stored in a lazy val, which is thread safe, so I wouldn't think there would be any race conditions related loading charsets. And since you still hit this by putting compileFile in a synchronized block, it feels like there's something else going on.
The SimpleNamedServiceLoader is a helper class that uses a Java ServiceLoader to find encoding implementation classes on the classpath. I'm not sure why the ServiceLoader wouldn't be able to find the classes though. I've only skimmed the code you linked, but nothing jumps out at me that might cause an issue. I assume you don't have a custom ClassLoader or are messing with the classpath in any way?
> Error: Unsupported encoding: ISO-8859-1. Supported encodings:
> --------------------------------------------------------------
>
> Key: DAFFODIL-2827
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2827
> Project: Daffodil
> Issue Type: Bug
> Components: DFDL Schemas
> Affects Versions: 3.4.0
> Environment: openjdk version "11.0.19" 2023-04-18
> OpenJDK Runtime Environment (build 11.0.19+7-post-Ubuntu-0ubuntu122.04.1)
> OpenJDK 64-Bit Server VM (build 11.0.19+7-post-Ubuntu-0ubuntu122.04.1, mixed mode, sharing)
> Reporter: Claude Mamo
> Priority: Minor
> Labels: race-condition
>
> I've consistently observed the following error when compiling several DFDL schemas concurrently on OpenJDK 11 and 17:
> {noformat}
> Caused by: org.apache.daffodil.dsom.SchemaDefinitionError: Schema Definition Error: Unsupported encoding: ISO-8859-1. Supported encodings:
> Schema context: E1153 Location line 1524 column 14 in file:/home/smooks-edi-cartridge/edifact-schemas/target/classes/d94b/EDIFACT-Segments.dfdl.xsd
> at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE (SDE.scala:179)
> at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE$ (SDE.scala:178)
> at org.apache.daffodil.dsom.DPathCompileInfo.SDE (CompiledExpression1.scala:210)
> at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError (ThrowsSDE.scala:58)
> at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError$ (ThrowsSDE.scala:58)
> at org.apache.daffodil.dsom.DPathCompileInfo.schemaDefinitionError (CompiledExpression1.scala:210)
> at org.apache.daffodil.processors.CharsetEvBase.compute (EvEncoding.scala:99)
> at org.apache.daffodil.processors.CharsetEvBase.compute (EvEncoding.scala:81)
> at org.apache.daffodil.processors.Evaluatable.evaluate (Evaluatable.scala:281)
> at org.apache.daffodil.processors.Evaluatable.compileTimeEvaluate (Evaluatable.scala:212)
> at org.apache.daffodil.processors.Evaluatable.compile (Evaluatable.scala:317)
> at org.apache.daffodil.processors.Evaluatable.compile (Evaluatable.scala:326)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv (RuntimePropertyMixins.scala:122)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv$ (RuntimePropertyMixins.scala:119)
> at org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv (RuntimePropertyMixins.scala:115)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv$ (RuntimePropertyMixins.scala:114)
> at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo (TermEncodingMixin.scala:104)
> at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo$ (TermEncodingMixin.scala:102)
> at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:180)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2 (ElementBaseRuntime1Mixin.scala:170)
> at scala.collection.immutable.List.map (List.scala:293)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1 (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$ (ElementBaseRuntime1Mixin.scala:168)
> at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:177)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2 (ElementBaseRuntime1Mixin.scala:170)
> at scala.collection.immutable.List.map (List.scala:293)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1 (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$ (ElementBaseRuntime1Mixin.scala:168)
> at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:177)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.LocalElementDeclBaseRuntime1Mixin.$anonfun$$init$$1 (LocalElementDeclBaseRuntime1Mixin.scala:24)
> at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.$anonfun$checkErrors$2 (OOLAG.scala:427)
> at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23)
> at org.apache.daffodil.oolag.OOLAG$.keepGoing (OOLAG.scala:65)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors (OOLAG.scala:427)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors$ (OOLAG.scala:413)
> at org.apache.daffodil.dsom.SchemaComponentImpl.checkErrors (SchemaComponent.scala:34)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError (OOLAG.scala:484)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError$ (OOLAG.scala:483)
> at org.apache.daffodil.dsom.SchemaSet.isError (SchemaSet.scala:543)
> at org.apache.daffodil.compiler.ProcessorFactory.isError (Compiler.scala:136)
> at org.apache.daffodil.compiler.Compiler.org$apache$daffodil$compiler$Compiler$$compileSourceInternal (Compiler.scala:419)
> at org.apache.daffodil.compiler.Compiler$.org$apache$daffodil$compiler$Compiler$$compileSourceSynchronizer (Compiler.scala:461)
> at org.apache.daffodil.compiler.Compiler.compileSource (Compiler.scala:404)
> at org.apache.daffodil.compiler.Compiler.compileFile (Compiler.scala:388)
> at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:146)
> at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:126)
> at org.smooks.edi.edg.EdifactDfdlSchemaGenerator.generateDfdlSchemas (EdifactDfdlSchemaGenerator.java:141){noformat}
> Precisely, the error occurs on the line of code:
> {code:java}
> Daffodil.compiler().compileFile(...)
> {code}
> The error doesn't always occur on the same DFDL schema or the same schema location line for that matter. Curiously, this error, which I presume is happening because of a race-condition, doesn't show up on OpenJDK 8 so it could be a bug in the Java runtime. Note that a new Daffodil compiler is created to compile each schema. Moving {{Daffodil.compiler().compileFile(...)}} inside a {{synchronized}} block doesn't make a difference.
> I had a hunch and added the following code before forking the thread to concurrently compile the schemas:
> {code:java}
> org.apache.daffodil.processors.charset.CharsetUtils.supportedEncodingsString(){code}
> The above makes the error go away and I'm using it as a workaround. Can't really say why so but I suspect the race-condition is caused by cache reading that takes place when checking whether the encoding exists during schema compilation.
> I've created a [branch|https://github.com/claudemamo/smooks-edi-cartridge/tree/daffodil-unsupported-encoding-issue] where you should be able to reproduce the problem. Steps to reproduce:
> # git clone [https://github.com/claudemamo/smooks-edi-cartridge/tree/daffodil-unsupported-encoding-issue] --branch daffodil-unsupported-encoding-issue
> # cd smooks-edi-cartridge
> # mvn clean install -DskipTests
> # The build should fail on the {{edifact-schemas}} branch because of this issue. The error should occur at this point: [https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L139.] Uncomment the following line to apply the workaround: [https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L95]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)