You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by "Steve Lawrence (JIRA)" <ji...@apache.org> on 2019/01/10 13:25:00 UTC
[jira] [Commented] (DAFFODIL-2049) UnsupportedCharsetException:
X-IBM-1047-S390
[ https://issues.apache.org/jira/browse/DAFFODIL-2049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16739382#comment-16739382 ]
Steve Lawrence commented on DAFFODIL-2049:
------------------------------------------
If you're using Daffodil 2.1.0 and earlier (which I think this error comes from based on the backtrace), you should be able to use "IBM-1047" as the encoding name. It looks like Java just doesn't know the "x-ibm-1047-s390" alias for the character set.
However, in Daffodil 2.2.0 we switched to custom implementations for decoders instead of relying on Java decoders. So now we only support charsets that we explicitly allow and have implementations for, which is fairly small and does not include IBM-1047. Fortunately, new decoders are not too difficult to imlementation for these kinds of character sets. And we do implement IBM-037, which I believe is very similar. The source for that decoder is here:
[https://github.com/apache/incubator-daffodil/blob/master/daffodil-io/src/main/scala/org/apache/daffodil/processors/charset/IBM037.scala]
If you'd like to create a new decoder for IBM-1407 based off of our IBM-037 decoder so it can be available for Daffodil 2.3.0 and later, we would be happy to guide you and review the changes.
> UnsupportedCharsetException: X-IBM-1047-S390
> --------------------------------------------
>
> Key: DAFFODIL-2049
> URL: https://issues.apache.org/jira/browse/DAFFODIL-2049
> Project: Daffodil
> Issue Type: Bug
> Components: CLI
> Affects Versions: 2.0.0, 2.1.0, 2.2.0
> Reporter: Ed Jordan
> Priority: Major
>
> In the old Daffodil bug reporting system, I was asked to transfer this problem forward here, which I have not done until now (sorry).
> My sub-command in this case was _save-parser_, but in the past the exception has occurred with _parse_ and _unparse_ as well.
> The DFDL schema contains the following dfdl-format tag:
> {quote}<dfdl:format xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/" textStringPadCharacter="%SP;" textTrimKind="padChar" textPadKind="padChar" encoding="x-ibm-1047-s390" nilKind="literalCharacter" nilValue="%NUL;" ref="tpfbase:tpfBaseFmt"/>
> {quote}
> I received the following exception in the output:
> {quote}!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !! An unexpected exception occurred. This is a bug! !! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Please report this bug and help us fix it: https://opensource.ncsa.illinois.edu/confluence/display/DFDL/How+to+Report+a+Bug Please include the following exception, the command you ran, and any input, schema, or tdml files used that led to this bug. java.nio.charset.UnsupportedCharsetException: X-IBM-1047-S390 at java.nio.charset.Charset.forName(Unknown Source) at edu.illinois.ncsa.daffodil.processors.charset.CharsetUtils$.getCharset(CharsetUtils.scala:85) at edu.illinois.ncsa.daffodil.processors.CharsetEv.compute(EvEncoding.scala:99) at edu.illinois.ncsa.daffodil.processors.CharsetEv.compute(EvEncoding.scala:91) at edu.illinois.ncsa.daffodil.processors.Evaluatable.apply(Evaluatable.scala:330) at edu.illinois.ncsa.daffodil.processors.EvaluatableBase$class.compile(Evaluatable.scala:170) at edu.illinois.ncsa.daffodil.processors.Evaluatable.compile(Evaluatable.scala:358) at edu.illinois.ncsa.daffodil.processors.EvaluatableBase$class.compile(Evaluatable.scala:143) at edu.illinois.ncsa.daffodil.processors.Evaluatable.compile(Evaluatable.scala:301) at edu.illinois.ncsa.daffodil.dsom.TermRuntimeValuedPropertiesMixin$class.maybeCharsetEv(RuntimePropertyMixins.scala:130) at edu.illinois.ncsa.daffodil.dsom.Term.maybeCharsetEv$lzycompute(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.Term.maybeCharsetEv(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.TermRuntimeValuedPropertiesMixin$class.charsetEv(RuntimePropertyMixins.scala:121) at edu.illinois.ncsa.daffodil.dsom.Term.charsetEv$lzycompute(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.Term.charsetEv(Term.scala:51) at edu.illinois.ncsa.daffodil.processors.KnownEncodingMixin$class.knownEncodingName(EncodingRuntimeData.scala:78) at edu.illinois.ncsa.daffodil.dsom.Term.knownEncodingName$lzycompute(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.Term.knownEncodingName(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.TermEncodingMixin$class.knownEncodingAlignmentInBits(TermEncodingMixin.scala:84) at edu.illinois.ncsa.daffodil.dsom.Term.knownEncodingAlignmentInBits$lzycompute(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.Term.knownEncodingAlignmentInBits(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.TermEncodingMixin$class.hasTextAlignment(TermEncodingMixin.scala:226) at edu.illinois.ncsa.daffodil.dsom.Term.hasTextAlignment$lzycompute(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.Term.hasTextAlignment(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.TermEncodingMixin$class.isLocallyTextOnly(TermEncodingMixin.scala:109) at edu.illinois.ncsa.daffodil.dsom.Term.isLocallyTextOnly$lzycompute(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.Term.isLocallyTextOnly(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.TermEncodingMixin$class.summaryEncoding(TermEncodingMixin.scala:198) at edu.illinois.ncsa.daffodil.dsom.Term.summaryEncoding$lzycompute(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.Term.summaryEncoding(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.TermEncodingMixin$class.encodingInfo(TermEncodingMixin.scala:97) at edu.illinois.ncsa.daffodil.dsom.Term.encodingInfo$lzycompute(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.Term.encodingInfo(Term.scala:51) at edu.illinois.ncsa.daffodil.dsom.ElementBase$$anonfun$41.apply(ElementBase.scala:533) at edu.illinois.ncsa.daffodil.dsom.ElementBase$$anonfun$41.apply(ElementBase.scala:533) at edu.illinois.ncsa.daffodil.processors.TermRuntimeData.encodingInfo$lzycompute(RuntimeData.scala:161) at edu.illinois.ncsa.daffodil.processors.TermRuntimeData.encodingInfo(RuntimeData.scala:161) at edu.illinois.ncsa.daffodil.processors.TermRuntimeData.preSerialization(RuntimeData.scala:174) at edu.illinois.ncsa.daffodil.processors.ElementRuntimeData.preSerialization(RuntimeData.scala:712) at edu.illinois.ncsa.daffodil.dsom.ElementBase$$anonfun$2.apply$mcV$sp(ElementBase.scala:110) at edu.illinois.ncsa.daffodil.dsom.ElementBase$$anonfun$2.apply(ElementBase.scala:110) at edu.illinois.ncsa.daffodil.dsom.ElementBase$$anonfun$2.apply(ElementBase.scala:110) at edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.liftedTree1$1(OOLAG.scala:579) at edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.value$lzycompute(OOLAG.scala:577) at edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.value(OOLAG.scala:575) at edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGValue.valueAsAny(OOLAG.scala:573) at edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$$anonfun$checkErrors$2.apply$mcV$sp(OOLAG.scala:281) at edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$$anonfun$checkErrors$2.apply(OOLAG.scala:281) at edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost$$anonfun$checkErrors$2.apply(OOLAG.scala:281) at edu.illinois.ncsa.daffodil.oolag.OOLAG$.keepGoing(OOLAG.scala:74) at edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost.checkErrors(OOLAG.scala:281) at edu.illinois.ncsa.daffodil.oolag.OOLAG$OOLAGHost.isError(OOLAG.scala:340) at edu.illinois.ncsa.daffodil.compiler.ProcessorFactory.edu$illinois$ncsa$daffodil$compiler$ProcessorFactory$$super$isError(Compiler.scala:150) at edu.illinois.ncsa.daffodil.compiler.ProcessorFactory$$anonfun$isError$1$$anonfun$apply$mcZ$sp$2.apply$mcZ$sp(Compiler.scala:150) at edu.illinois.ncsa.daffodil.compiler.ProcessorFactory$$anonfun$isError$1$$anonfun$apply$mcZ$sp$2.apply(Compiler.scala:141) at edu.illinois.ncsa.daffodil.compiler.ProcessorFactory$$anonfun$isError$1$$anonfun$apply$mcZ$sp$2.apply(Compiler.scala:141) at edu.illinois.ncsa.daffodil.oolag.OOLAG$.keepGoing(OOLAG.scala:74) at edu.illinois.ncsa.daffodil.compiler.ProcessorFactory$$anonfun$isError$1.apply$mcZ$sp(Compiler.scala:141) at edu.illinois.ncsa.daffodil.compiler.ProcessorFactory$$anonfun$isError$1.apply(Compiler.scala:141) at edu.illinois.ncsa.daffodil.compiler.ProcessorFactory$$anonfun$isError$1.apply(Compiler.scala:141) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58) at edu.illinois.ncsa.daffodil.ExecutionMode$$anonfun$usingCompilerMode$1.apply(ExecutionMode.scala:79) at edu.illinois.ncsa.daffodil.ExecutionMode$$anonfun$usingCompilerMode$1.apply(ExecutionMode.scala:79) at edu.illinois.ncsa.daffodil.compiler.ProcessorFactory.isError(Compiler.scala:141) at edu.illinois.ncsa.daffodil.compiler.Compiler.compileSource(Compiler.scala:332) at edu.illinois.ncsa.daffodil.Main$$anonfun$71.apply(Main.scala:728) at edu.illinois.ncsa.daffodil.Main$$anonfun$71.apply(Main.scala:727) at edu.illinois.ncsa.daffodil.util.Timer$.getTimeResult(Timer.scala:91) at edu.illinois.ncsa.daffodil.util.Timer$.getResult(Timer.scala:50) at edu.illinois.ncsa.daffodil.Main$.createProcessorFromSchema(Main.scala:727) at edu.illinois.ncsa.daffodil.Main$.run(Main.scala:1140) at edu.illinois.ncsa.daffodil.Main$.main(Main.scala:1316) at edu.illinois.ncsa.daffodil.Main.main(Main.scala)
> {quote}
> My workaround to the exception has been to substitute "ebcdic-cp-us" for "x-ibm-1047-s390", but this will no longer work for users.
> Thanks for your help.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)