[jira] [Created] (DAFFODIL-2384) textNumberCheckPolicy="strict" does not require grouping separators

["Steve Lawrence (Jira)" <ji...@apache.org>]

Steve Lawrence created DAFFODIL-2384:
----------------------------------------

             Summary: textNumberCheckPolicy="strict" does not require grouping separators 
                 Key: DAFFODIL-2384
                 URL: https://issues.apache.org/jira/browse/DAFFODIL-2384
             Project: Daffodil
          Issue Type: Bug
          Components: ICU
            Reporter: Steve Lawrence


Say we have this schema snippet:

{code:xml}
<xs:element name="SimpleDataFormat">
    <xs:complexType>
        <xs:sequence>
            <xs:element name="NumStudents" type="xs:nonNegativeInteger" 
                dfdl:textNumberCheckPolicy="strict"
                dfdl:textNumberPattern="#,###"
                dfdl:textStandardGroupingSeparator=","
                dfdl:textStandardDecimalSeparator="."
            />
        </xs:sequence>
    </xs:complexType>
</xs:element>
{code}

This successfully parses the data
{code}
1234
{code}

Even though textNumberCheckPolicy="strict" and the pattern contains a grouping separator, it still allows data that does not contain grouping separators. This feels like bug.

That said, we just use the ICU library for parsing numbers based on the textNumberPattern. This library has this to say about strict parsing of numbers:

{quote}
The following conditions cause a parse failure relative to [lax] mode
(examples use the pattern "#,##0.#"):

* The presence and position of special symbols, including currency, 
must match the pattern.
    
  '+123' fails (there is no plus sign in the pattern)
* Leading or doubled grouping separators
      
  ',123' and '1,,234" fail
* Groups of incorrect length when grouping is used
        
  '1,23' and '1234,567' fail, but '1234' passes
* Grouping separators used in numbers followed by exponents
        
  '1,234E5' fails, but '1234E5' and '1,234E' pass ('E' is not an 
   exponent when not followed by a number)
{quote}

So bsaed on ICU's description of strict, this is the expected behavior. It doesn't say anything about missing grouping separators causing an error. Only that if they do exist then they must be in the right spot.

The only thing the DFDL specification mentions regarding strict numbers is this:

{quote}
If 'strict' and dfdl:textNumberRep is 'standard' then the data must 
follow the pattern with the exceptions that digits 0-9, decimal 
separator and exponent separator are always recognised and parsed
{quote}

To me, that reads like the decimal separator should always be required in strict mode, so this feels like the ICU behavior and the behavior described in the DFDL specification do not match. And I believe the DFDL behavior was intended to match match ICU behavior, so it's possible the DFDL specification needs to be updated.

We need to determine if which specification is correct, and if Daffodil needs to change it's behavior.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)