[jira] [Comment Edited] (DAFFODIL-2957) dfdl:decimalSigned="no" bugs with negative values and data representations

["Mike Beckerle (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DAFFODIL-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17899219#comment-17899219 ] 

Mike Beckerle edited comment on DAFFODIL-2957 at 11/18/24 5:50 PM:
-------------------------------------------------------------------

Basically, Daffodil is not doing a good job with xs:decimal when dfdl:decimalSigned="no".

See the files TestDecimalSigned.scala and TestDecimalSigned.tdml in 

[https://github.com/apache/daffodil/pull/1376/files]

That PR doesn't attempt to fix the issue. Just adds test coverage.

As suspected, for both packed and ibm4690Packed, Daffodil happily creates negative values when parsing despite dfdl:decimalSigned="no", and when unparsing happily converts negative values into negative representations. This means just "fixing" this to detect these errors would potentially break schemas that users are currently using. So for daffodil, fixing these to check will perhaps require a switch to enforce decimalSigned="no" checking both when parsing and unparsing. 

For BCD, there is no case of parsing a negative rep as BCD is inherently unsigned. When unparsing a negative value to BCD, Daffodil puts a negative sign nibble first, the same as for ibm4690Packed. This is simply broken data being output. This needs a check that is NOT controlled by the above mentioned switch. 

For binary, parsing properly interprets all the bits as a positive value. For unparsing, a negative value results in an Assertion failure in the code. A user will get a bug traceback that says "This is a bug, please report...". This needs a bug fix which will NOT need to have any switched behavior. 

 


was (Author: mbeckerle):
Basically, Daffodil is not doing a good job with xs:decimal when dfdl:decimalSigned="no".

See the files TestDecimalSigned.scala and TestDecimalSigned.tdml in 

[https://github.com/apache/daffodil/pull/1376/files]

As suspected, for both packed and ibm4690Packed, Daffodil happily creates negative values when parsing despite dfdl:decimalSigned="no", and when unparsing happily converts negative values into negative representations. This means just "fixing" this to detect these errors would potentially break schemas that users are currently using. So for daffodil, fixing these to check will perhaps require a switch to enforce decimalSigned="no" checking both when parsing and unparsing. 

For BCD, there is no case of parsing a negative rep as BCD is inherently unsigned. When unparsing a negative value to BCD, Daffodil puts a negative sign nibble first, the same as for ibm4690Packed. This is simply broken data being output. This needs a check that is NOT controlled by the above mentioned switch. 

For binary, parsing properly interprets all the bits as a positive value. For unparsing, a negative value results in an Assertion failure in the code. A user will get a bug traceback that says "This is a bug, please report...". This needs a bug fix which will NOT need to have any switched behavior. 

 

> dfdl:decimalSigned="no" bugs with negative values and data representations
> --------------------------------------------------------------------------
>
>                 Key: DAFFODIL-2957
>                 URL: https://issues.apache.org/jira/browse/DAFFODIL-2957
>             Project: Daffodil
>          Issue Type: Bug
>          Components: Back End
>    Affects Versions: 3.9.0
>            Reporter: Mike Beckerle
>            Assignee: Mike Beckerle
>            Priority: Minor
>
> Tests are needed for whether unparsing a negative value as an element of type xs:decimal with dfdl:decimalSigned="no" detects an error or not. 
> When dfdl:binaryNumberRep="bcd" this is definitely an error and should produce an Unparse error. (IBM DFDL does detect this.)
> When dfdl:binaryNumberRep is 'binary', 'packed', or 'ibm4790Packed', the data stream can represent a negative sign, the question is whether this error is detected or not. 
> The DFDL Workgroup wants all implementations to test if this error is detected currently because if this was clarified in the spec as causing an unparse error, but current implementations are not detecting this currently, then this would be a breaking change that would require more careful analysis, and possibly switches to provide backward compatibility. Or the behavior would just become "implementation defined". 
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)