You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@daffodil.apache.org by Jim Welch <ji...@gmail.com> on 2018/09/14 17:07:58 UTC

dateTime calendarPattern Errors

   1. binaryCalendarRep being set to binarySeconds gave an error that EPOCH
   was not set. No default to unix as the standard seems to imply.
   2. Setting the Epoch to  1970-01-01T00:00:00.0000 as per the standard is
   rejected. The  *calendarPattern*  seems to be  yyyy-MM-dd'T'HH:mm:ssSSSS
   without the last decimal point. Yet ISO8601 seems to have the decimal point
   for milliseconds.
   3. Also since this is binarySeconds, there should be no milliseconds
   since they do not exist.
   4. Setting the calendarPattern="yyyy-MM-dd'T'HH:mm:ss", does not seem to
   change the XML output. I even tried to use  "yyyy-MM-dd'T'HH:mm:", no
   change BIG problem.


*SCHEMA*:
   <xs:annotation>
        <xs:appinfo source="http://www.ogf.org/dfdl/">
            <!-- Set common defaults -->
            <dfdl:format
                calendarPatternKind="implicit"
                binaryCalendarRep="binarySeconds"
                binaryCalendarEpoch="1970-01-01T00:00:00"
...
      <xs:simpleType name="DateTime" dfdl:lengthKind="explicit"
dfdl:length="4">
        <xs:restriction base="xs:dateTime" />
      </xs:simpleType>
...
                    <xs:element name="Time_Last"           type="DateTime"
/>


*OUTPUT - XML*
    <Time_Last>2018-08-21T17:26:00.000000</Time_Last>

*SCHEMA*: edited:
calendarPatternKind="implicit"
binaryCalendarRep="binarySeconds"
binaryCalendarEpoch="1970-01-01T00:00:00"

*calendarPatternKind="explicit"*
*calendarPattern="yyyy-MM-dd'T'HH:mm:ss"*

*No change in the output!*
    <Time_Last>2018-08-21T17:26:00.00*0000*</Time_Last>

Re: dateTime calendarPattern Errors

Posted by Jim Welch <ji...@gmail.com>.
>
> Here are test files for these bugs.
>
xbat.txt --> x.bat to avoid mail filters.

Re: dateTime calendarPattern Errors

Posted by Jim Welch <ji...@gmail.com>.
>
> I tried putting the *calendarPattern on the *xs:simpleType
> name="DateTime" and on the <xs:element name="Time_Last". Still no change to
> the output!
>

Re: dateTime calendarPattern Errors

Posted by Mike Beckerle <mb...@tresys.com>.
The fact that DFDLGeneralFormat.dfdl.xsd doesn't provide a value for binaryCalendarEpoch is also a bug. That format is intended to provide useful defaults for most properties while at the same time being explicit about them, as DFDL doesn't provide default values itself (intentionally).


I'll create a JIRA ticket for this also.

________________________________
From: Steve Lawrence <sl...@apache.org>
Sent: Friday, September 14, 2018 3:38:09 PM
To: users@daffodil.apache.org; Jim Welch
Subject: Re: dateTime calendarPattern Errors

1. Very few properties in DFDL have default values--if a property is
needed then the schema must provide a value. So if
representation="binary", and binaryCalendarRep is binarySecond or
binaryMilliseconds, and the type is xs:date/xs:time/xs:dateTime, then
binaryCalendarEpoch must be provided.

2. Daffodil will accept dateTimes for the binaryCalendarEpoch property
matching either of these patterns:

   uuuu-MM-dd'T'HH:mm:ssZZZZ
   uuuu-MM-dd'T'HH:mm:ss

So the milliseconds should not be provided and the timezone is
optional. I suspect ICU (which we use for parsing this value) is being
lax and is accepting more precision in the form of ss.SSSSSS. From
what I can tell, the DFDL spec appears to be silent about what the
correct format for this property is and if milliseconds should be
allowed or not.

3. Milliseconds shouldn't be necessary, in fact, I wouldn't be
surprised if the extra seconds precision is just ignored.

4. The calendarPattern field does not affect how the dateTime is
output in the infoset. That property only plays a role for date/times
when representation="text" or when binaryCalendarRep is
packed/bcd/ibm4690Packed. And when it is used, it only describes the
format of the pattern *in the data*. The format in the infoset follows
the canonical XML representation for date times, which is the format
you are seeing. Though, looking at the XSD w3c spec, it looks like the
canonical representation shouldn't include the extra precision if the
precision is just zero. So this is just a bug in how Daffodil outputs
the infoset for date/times. I'll open a bug to track this issue.

- Steve

On 09/14/2018 01:07 PM, Jim Welch wrote:
>  1. binaryCalendarRep being set to binarySeconds gave an error that EPOCH was
>     not set. No default to unix as the standard seems to imply.
>  2. Setting the Epoch to 1970-01-01T00:00:00.0000 as per the standard is
>     rejected. The *calendarPattern*  seems to be yyyy-MM-dd'T'HH:mm:ssSSSS
>     without the last decimal point. Yet ISO8601 seems to have the decimal point
>     for milliseconds.
>  3. Also since this is binarySeconds, there should be no milliseconds since they
>     do not exist.
>  4. Setting the calendarPattern="yyyy-MM-dd'T'HH:mm:ss", does not seem to change
>     the XML output. I even tried to use "yyyy-MM-dd'T'HH:mm:", no change BIG
>     problem.
>
> *
> *
> *SCHEMA*:
>     <xs:annotation>
>          <xs:appinfo source="http://www.ogf.org/dfdl/">
>              <!-- Set common defaults -->
>              <dfdl:format
>                  calendarPatternKind="implicit"
>                  binaryCalendarRep="binarySeconds"
>                  binaryCalendarEpoch="1970-01-01T00:00:00"
> ...
>        <xs:simpleType name="DateTime" dfdl:lengthKind="explicit" dfdl:length="4">
>          <xs:restriction base="xs:dateTime" />
>        </xs:simpleType>
> ...
>                      <xs:element name="Time_Last"           type="DateTime" />
>
>
> *OUTPUT - XML*
>      <Time_Last>2018-08-21T17:26:00.000000</Time_Last>
>
> *SCHEMA*: edited:
> calendarPatternKind="implicit"
> binaryCalendarRep="binarySeconds"
> binaryCalendarEpoch="1970-01-01T00:00:00"
> *calendarPatternKind="explicit"
> *
> *calendarPattern="yyyy-MM-dd'T'HH:mm:ss"*
>
> *No change in the output!*
>      <Time_Last>2018-08-21T17:26:00.00*0000*</Time_Last>
>


Re: dateTime calendarPattern Errors

Posted by Steve Lawrence <sl...@apache.org>.
1. Very few properties in DFDL have default values--if a property is
needed then the schema must provide a value. So if
representation="binary", and binaryCalendarRep is binarySecond or
binaryMilliseconds, and the type is xs:date/xs:time/xs:dateTime, then
binaryCalendarEpoch must be provided.

2. Daffodil will accept dateTimes for the binaryCalendarEpoch property
matching either of these patterns:

   uuuu-MM-dd'T'HH:mm:ssZZZZ
   uuuu-MM-dd'T'HH:mm:ss

So the milliseconds should not be provided and the timezone is
optional. I suspect ICU (which we use for parsing this value) is being
lax and is accepting more precision in the form of ss.SSSSSS. From
what I can tell, the DFDL spec appears to be silent about what the
correct format for this property is and if milliseconds should be
allowed or not.

3. Milliseconds shouldn't be necessary, in fact, I wouldn't be
surprised if the extra seconds precision is just ignored.

4. The calendarPattern field does not affect how the dateTime is
output in the infoset. That property only plays a role for date/times
when representation="text" or when binaryCalendarRep is
packed/bcd/ibm4690Packed. And when it is used, it only describes the
format of the pattern *in the data*. The format in the infoset follows
the canonical XML representation for date times, which is the format
you are seeing. Though, looking at the XSD w3c spec, it looks like the
canonical representation shouldn't include the extra precision if the
precision is just zero. So this is just a bug in how Daffodil outputs
the infoset for date/times. I'll open a bug to track this issue.

- Steve

On 09/14/2018 01:07 PM, Jim Welch wrote:
>  1. binaryCalendarRep being set to binarySeconds gave an error that EPOCH was
>     not set. No default to unix as the standard seems to imply.
>  2. Setting the Epoch to 1970-01-01T00:00:00.0000 as per the standard is
>     rejected. The *calendarPattern*  seems to be yyyy-MM-dd'T'HH:mm:ssSSSS
>     without the last decimal point. Yet ISO8601 seems to have the decimal point
>     for milliseconds.
>  3. Also since this is binarySeconds, there should be no milliseconds since they
>     do not exist.
>  4. Setting the calendarPattern="yyyy-MM-dd'T'HH:mm:ss", does not seem to change
>     the XML output. I even tried to use "yyyy-MM-dd'T'HH:mm:", no change BIG
>     problem.
> 
> *
> *
> *SCHEMA*:
>     <xs:annotation>
>          <xs:appinfo source="http://www.ogf.org/dfdl/">
>              <!-- Set common defaults -->
>              <dfdl:format
>                  calendarPatternKind="implicit"
>                  binaryCalendarRep="binarySeconds"
>                  binaryCalendarEpoch="1970-01-01T00:00:00"
> ...
>        <xs:simpleType name="DateTime" dfdl:lengthKind="explicit" dfdl:length="4">
>          <xs:restriction base="xs:dateTime" />
>        </xs:simpleType>
> ...
>                      <xs:element name="Time_Last"           type="DateTime" />
> 
> 
> *OUTPUT - XML*
>      <Time_Last>2018-08-21T17:26:00.000000</Time_Last>
> 
> *SCHEMA*: edited:
> calendarPatternKind="implicit"
> binaryCalendarRep="binarySeconds"
> binaryCalendarEpoch="1970-01-01T00:00:00"
> *calendarPatternKind="explicit"
> *
> *calendarPattern="yyyy-MM-dd'T'HH:mm:ss"*
> 
> *No change in the output!*
>      <Time_Last>2018-08-21T17:26:00.00*0000*</Time_Last>
>