You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@daffodil.apache.org by "Steve Lawrence (Jira)" <ji...@apache.org> on 2023/06/16 14:37:00 UTC
[jira] [Comment Edited] (DAFFODIL-2827) Error: Unsupported encoding: ISO-8859-1. Supported encodings:
[ https://issues.apache.org/jira/browse/DAFFODIL-2827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733527#comment-17733527 ]
Steve Lawrence edited comment on DAFFODIL-2827 at 6/16/23 2:36 PM:
-------------------------------------------------------------------
I think the {{parallel()}} call is doing something funky with the class loader.
I've found that if I remove the {{parallel()}} call, the classloader uses by our {{SimpleNamedServiceLoader}} is a {{java.net.URLClassLoader}}, but with the {{parallel()}} call it uses a {{jdk.internal.loader.ClassLoaders$AppClassLoader}}.
Note that on Java 8 it still uses the URLClassLoader with or without the parallel call so works regardless.
I found this thread on stack overflow that seems about a similar issue and looks to have some explanations and references for what's going on and potential fixes.
https://stackoverflow.com/questions/55452778/parallelstream-causing-classnotfoundexception-with-jaxb-api
Sounds like it's not something Daffodil can control and needs to be changed by the calling application to ensure the correct class loader is used.
This also explains why calling {{CharsetUtils.supportedEncodingsString}} acts as a workaround--the ServiceLoader ends up being called with the the correct ClassLoader, the results are cached, and then the parallel calls have access to the cache without needing to worry about the ServiceLoader/ClassLoader being used.
was (Author: slawrence):
I think the {{parallel()}} call is doing something funky with the class loader.
I've found that if I remove the {{parallel()}} call, the classloader uses by our {{SimpleNamedServiceLoader}} is a {{java.net.URLClassLoader}}, but with the {{parallel()}} call it uses a {{jdk.internal.loader.ClassLoaders$AppClassLoader}}.
Note that on Java 8 it still uses the URLClassLoader with or without the parallel call so works regardless.
I found this thread on stack overflow that seems about a similar issue and looks to have some explanations and references for what's going on and potential fixes.
Sounds like it's not something Daffodil can control and needs to be changed by the calling application to ensure the correct class loader is used.
This also explains why calling {{CharsetUtils.supportedEncodingsString}} acts as a workaround--the ServiceLoader ends up being called with the the correct ClassLoader, the results are cached, and then the parallel calls have access to the cache without needing to worry about the ServiceLoader/ClassLoader being used.
> Error: Unsupported encoding: ISO-8859-1. Supported encodings:
> --------------------------------------------------------------
>
> Key: DAFFODIL-2827
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2827
> Project: Daffodil
> Issue Type: Bug
> Components: DFDL Schemas
> Affects Versions: 3.4.0
> Environment: openjdk version "11.0.19" 2023-04-18
> OpenJDK Runtime Environment (build 11.0.19+7-post-Ubuntu-0ubuntu122.04.1)
> OpenJDK 64-Bit Server VM (build 11.0.19+7-post-Ubuntu-0ubuntu122.04.1, mixed mode, sharing)
> Reporter: Claude Mamo
> Priority: Minor
> Labels: race-condition
>
> I've consistently observed the following error when compiling several DFDL schemas concurrently on OpenJDK 11 and 17:
> {noformat}
> Caused by: org.apache.daffodil.dsom.SchemaDefinitionError: Schema Definition Error: Unsupported encoding: ISO-8859-1. Supported encodings:
> Schema context: E1153 Location line 1524 column 14 in file:/home/smooks-edi-cartridge/edifact-schemas/target/classes/d94b/EDIFACT-Segments.dfdl.xsd
> at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE (SDE.scala:179)
> at org.apache.daffodil.dsom.ImplementsThrowsSDE.SDE$ (SDE.scala:178)
> at org.apache.daffodil.dsom.DPathCompileInfo.SDE (CompiledExpression1.scala:210)
> at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError (ThrowsSDE.scala:58)
> at org.apache.daffodil.exceptions.ThrowsSDE.schemaDefinitionError$ (ThrowsSDE.scala:58)
> at org.apache.daffodil.dsom.DPathCompileInfo.schemaDefinitionError (CompiledExpression1.scala:210)
> at org.apache.daffodil.processors.CharsetEvBase.compute (EvEncoding.scala:99)
> at org.apache.daffodil.processors.CharsetEvBase.compute (EvEncoding.scala:81)
> at org.apache.daffodil.processors.Evaluatable.evaluate (Evaluatable.scala:281)
> at org.apache.daffodil.processors.Evaluatable.compileTimeEvaluate (Evaluatable.scala:212)
> at org.apache.daffodil.processors.Evaluatable.compile (Evaluatable.scala:317)
> at org.apache.daffodil.processors.Evaluatable.compile (Evaluatable.scala:326)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv (RuntimePropertyMixins.scala:122)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.maybeCharsetEv$ (RuntimePropertyMixins.scala:119)
> at org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.maybeCharsetEv (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv (RuntimePropertyMixins.scala:115)
> at org.apache.daffodil.dsom.TermRuntimeValuedPropertiesMixin.charsetEv$ (RuntimePropertyMixins.scala:114)
> at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.charsetEv (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo (TermEncodingMixin.scala:104)
> at org.apache.daffodil.dsom.TermEncodingMixin.encodingInfo$ (TermEncodingMixin.scala:102)
> at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.encodingInfo (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:180)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2 (ElementBaseRuntime1Mixin.scala:170)
> at scala.collection.immutable.List.map (List.scala:293)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1 (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$ (ElementBaseRuntime1Mixin.scala:168)
> at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:177)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$2 (ElementBaseRuntime1Mixin.scala:170)
> at scala.collection.immutable.List.map (List.scala:293)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$childrenERDs$1 (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (ElementBaseRuntime1Mixin.scala:169)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$ (ElementBaseRuntime1Mixin.scala:168)
> at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.org$apache$daffodil$runtime1$ElementBaseRuntime1Mixin$$childrenERDs (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.$anonfun$elementRuntimeData$1 (ElementBaseRuntime1Mixin.scala:177)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute (OOLAG.scala:724)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValue.value (OOLAG.scala:724)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.runtime1.ElementBaseRuntime1Mixin.elementRuntimeData$ (ElementBaseRuntime1Mixin.scala:174)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData$lzycompute (LocalElementDecl.scala:25)
> at org.apache.daffodil.dsom.LocalElementDeclBase.elementRuntimeData (LocalElementDecl.scala:25)
> at org.apache.daffodil.runtime1.LocalElementDeclBaseRuntime1Mixin.$anonfun$$init$$1 (LocalElementDeclBaseRuntime1Mixin.scala:24)
> at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.liftedTree1$1 (OOLAG.scala:679)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny$lzycompute (OOLAG.scala:677)
> at org.apache.daffodil.oolag.OOLAG$OOLAGValueBase.valueAsAny (OOLAG.scala:674)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.$anonfun$checkErrors$2 (OOLAG.scala:427)
> at scala.runtime.java8.JFunction0$mcV$sp.apply (JFunction0$mcV$sp.java:23)
> at org.apache.daffodil.oolag.OOLAG$.keepGoing (OOLAG.scala:65)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors (OOLAG.scala:427)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.checkErrors$ (OOLAG.scala:413)
> at org.apache.daffodil.dsom.SchemaComponentImpl.checkErrors (SchemaComponent.scala:34)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError (OOLAG.scala:484)
> at org.apache.daffodil.oolag.OOLAG$OOLAGHost.isError$ (OOLAG.scala:483)
> at org.apache.daffodil.dsom.SchemaSet.isError (SchemaSet.scala:543)
> at org.apache.daffodil.compiler.ProcessorFactory.isError (Compiler.scala:136)
> at org.apache.daffodil.compiler.Compiler.org$apache$daffodil$compiler$Compiler$$compileSourceInternal (Compiler.scala:419)
> at org.apache.daffodil.compiler.Compiler$.org$apache$daffodil$compiler$Compiler$$compileSourceSynchronizer (Compiler.scala:461)
> at org.apache.daffodil.compiler.Compiler.compileSource (Compiler.scala:404)
> at org.apache.daffodil.compiler.Compiler.compileFile (Compiler.scala:388)
> at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:146)
> at org.apache.daffodil.japi.Compiler.compileFile (Daffodil.scala:126)
> at org.smooks.edi.edg.EdifactDfdlSchemaGenerator.generateDfdlSchemas (EdifactDfdlSchemaGenerator.java:141){noformat}
> Precisely, the error occurs on the line of code:
> {code:java}
> Daffodil.compiler().compileFile(...)
> {code}
> The error doesn't always occur on the same DFDL schema or the same schema location line for that matter. Curiously, this error, which I presume is happening because of a race-condition, doesn't show up on OpenJDK 8 so it could be a bug in the Java runtime. Note that a new Daffodil compiler is created to compile each schema. Moving {{Daffodil.compiler().compileFile(...)}} inside a {{synchronized}} block doesn't make a difference.
> I had a hunch and added the following code before forking the thread to concurrently compile the schemas:
> {code:java}
> org.apache.daffodil.processors.charset.CharsetUtils.supportedEncodingsString(){code}
> The above makes the error go away and I'm using it as a workaround. Can't really say why so but I suspect the race-condition is caused by cache reading that takes place when checking whether the encoding exists during schema compilation.
> I've created a [branch|https://github.com/claudemamo/smooks-edi-cartridge/tree/daffodil-unsupported-encoding-issue] where you should be able to reproduce the problem. Steps to reproduce:
> # git clone [https://github.com/claudemamo/smooks-edi-cartridge/tree/daffodil-unsupported-encoding-issue] --branch daffodil-unsupported-encoding-issue
> # cd smooks-edi-cartridge
> # mvn clean install -DskipTests
> # The build should fail on the {{edifact-schemas}} Maven module because of this issue. The error should occur at this point: [https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L139.] Uncomment the following line to apply the workaround: [https://github.com/claudemamo/smooks-edi-cartridge/blob/daffodil-unsupported-encoding-issue/edg/src/main/java/org/smooks/edi/edg/EdifactDfdlSchemaGenerator.java#L95]
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)