[VALIDATOR] Java 9 change to use CLDR formats cause API break - what to do?

[sebb <se...@gmail.com>]

Java 9 now uses Locale strings from the Unicode consortium by default.

This has cause several of the Validator unit tests to fail. For example:

validator.validate("31/Dez/05 21-05", "dd/MMM/yy HH-mm", Locale.GERMAN);

no longer parses OK, because the short version of December is now
"Dez." (with trailing fullstop)

I have temporarily suppressed the errors by defining the following for Java9+

java.locale.providers=COMPAT,CLDR

However this is not ideal.

Whilst the tests could be updated to use the new formats, e.g. by
generating the test data at run time, it is not clear that this is the
correct approach, as it would hide changes in behaviour.

Ideally the code would continue to accept old style dates, but it's
not at all clear how to do this.

Anyone have any bright ideas?

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

RE: [VALIDATOR] Java 9 change to use CLDR formats cause API break - what to do?

[Jason Pyeron <jp...@pdinc.us>]

> -----Original Message-----
> From: sebb <se...@gmail.com>
> Sent: Sunday, May 17, 2020 11:39 AM
> To: Commons Developers List <de...@commons.apache.org>
> Subject: Re: [VALIDATOR] Java 9 change to use CLDR formats cause API break - what to do?
> 
> On Sun, 17 May 2020 at 15:22, Jason Pyeron <jp...@pdinc.us> wrote:
> >
> > > -----Original Message-----
> > > From: sebb [mailto:sebbaz@gmail.com]
> > > Sent: Sunday, May 17, 2020 9:05 AM
> > > To: CommonsDev <de...@commons.apache.org>
> > > Subject: [VALIDATOR] Java 9 change to use CLDR formats cause API break - what to do?
> > >
> > > Java 9 now uses Locale strings from the Unicode consortium by default.
> > >
> > > This has cause several of the Validator unit tests to fail. For example:
> > >
> > > validator.validate("31/Dez/05 21-05", "dd/MMM/yy HH-mm", Locale.GERMAN);
> > >
> > > no longer parses OK, because the short version of December is now
> > > "Dez." (with trailing fullstop)
> > >
> > > I have temporarily suppressed the errors by defining the following for Java9+
> > >
> > > java.locale.providers=COMPAT,CLDR
> > >
> > > However this is not ideal.
> > >
> > > Whilst the tests could be updated to use the new formats, e.g. by
> > > generating the test data at run time, it is not clear that this is the
> > > correct approach, as it would hide changes in behaviour.
> >
> > No, the test should be duplicated and then modified.
> >
> > Legacy tests should have a conditional skip if Java >= 9
> >
> > New tests should have a conditional skip, only if needed, if Java < 9
> 
> I don't see how that is different from what I have done -- it will
> still hide the change in behaviour.
> 
> > >
> > > Ideally the code would continue to accept old style dates, but it's
> > > not at all clear how to do this.
> >
> > This is what I mean by "only if needed". Clearly the "intention" of Java 9 is to change the
> defaults, but if a "legacy" configuration / compatibility mode is enabled then ...
> 
> The test failures are caused by the change in Java 9 defaults, however
> the issue is what to do about the change in behaviour.
> 
> Date strings that were previously valid are no longer valid when using Java 9+
> The source data doesn't change when the Java version is updated.
> That does not seem right.
> 
> There are some other changes to do with how Java interprets strings such as
> -<cc>20.00
> and
> (<cc>20.00)
> 
> where <cc> is a currency such as £ (GBP) or $ (USD).
> 
> Again, I don't think it's right that these should pass validation with
> Java 8 and fail with Java 9 or vice-versa.
> 
> Also, Java 9 changes how SHORT etc. dates/times are treated.
> The changes are extensive.
> 
> I don't believe that Validator should only apply to output created by
> the same version of Java.
> 
> The question is how to ensure that Validator continues to support
> existing formats?

Java 9 is not backwards compatible here. To make Validator "behave" on Java 9+ as it did previously, should require code changes. At the same time API updates can be made to either expose Java 9 behavior by request -or- propagate the change of defaults to the consumers Validator.

Keeping in sync with changes like this is an exercise in chasing a dragon. The best one can hope for is to document the behavior for the two versions as split by the Java 9 SDK update and continue to build both environments in CI. 



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [VALIDATOR] Java 9 change to use CLDR formats cause API break - what to do?

[sebb <se...@gmail.com>]

On Sun, 17 May 2020 at 15:22, Jason Pyeron <jp...@pdinc.us> wrote:
>
> > -----Original Message-----
> > From: sebb [mailto:sebbaz@gmail.com]
> > Sent: Sunday, May 17, 2020 9:05 AM
> > To: CommonsDev <de...@commons.apache.org>
> > Subject: [VALIDATOR] Java 9 change to use CLDR formats cause API break - what to do?
> >
> > Java 9 now uses Locale strings from the Unicode consortium by default.
> >
> > This has cause several of the Validator unit tests to fail. For example:
> >
> > validator.validate("31/Dez/05 21-05", "dd/MMM/yy HH-mm", Locale.GERMAN);
> >
> > no longer parses OK, because the short version of December is now
> > "Dez." (with trailing fullstop)
> >
> > I have temporarily suppressed the errors by defining the following for Java9+
> >
> > java.locale.providers=COMPAT,CLDR
> >
> > However this is not ideal.
> >
> > Whilst the tests could be updated to use the new formats, e.g. by
> > generating the test data at run time, it is not clear that this is the
> > correct approach, as it would hide changes in behaviour.
>
> No, the test should be duplicated and then modified.
>
> Legacy tests should have a conditional skip if Java >= 9
>
> New tests should have a conditional skip, only if needed, if Java < 9

I don't see how that is different from what I have done -- it will
still hide the change in behaviour.

> >
> > Ideally the code would continue to accept old style dates, but it's
> > not at all clear how to do this.
>
> This is what I mean by "only if needed". Clearly the "intention" of Java 9 is to change the defaults, but if a "legacy" configuration / compatibility mode is enabled then ...

The test failures are caused by the change in Java 9 defaults, however
the issue is what to do about the change in behaviour.

Date strings that were previously valid are no longer valid when using Java 9+
The source data doesn't change when the Java version is updated.
That does not seem right.

There are some other changes to do with how Java interprets strings such as
-<cc>20.00
and
(<cc>20.00)

where <cc> is a currency such as £ (GBP) or $ (USD).

Again, I don't think it's right that these should pass validation with
Java 8 and fail with Java 9 or vice-versa.

Also, Java 9 changes how SHORT etc. dates/times are treated.
The changes are extensive.

I don't believe that Validator should only apply to output created by
the same version of Java.

The question is how to ensure that Validator continues to support
existing formats?

>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

RE: [VALIDATOR] Java 9 change to use CLDR formats cause API break - what to do?

[Jason Pyeron <jp...@pdinc.us>]

> -----Original Message-----
> From: sebb [mailto:sebbaz@gmail.com]
> Sent: Sunday, May 17, 2020 9:05 AM
> To: CommonsDev <de...@commons.apache.org>
> Subject: [VALIDATOR] Java 9 change to use CLDR formats cause API break - what to do?
> 
> Java 9 now uses Locale strings from the Unicode consortium by default.
> 
> This has cause several of the Validator unit tests to fail. For example:
> 
> validator.validate("31/Dez/05 21-05", "dd/MMM/yy HH-mm", Locale.GERMAN);
> 
> no longer parses OK, because the short version of December is now
> "Dez." (with trailing fullstop)
> 
> I have temporarily suppressed the errors by defining the following for Java9+
> 
> java.locale.providers=COMPAT,CLDR
> 
> However this is not ideal.
> 
> Whilst the tests could be updated to use the new formats, e.g. by
> generating the test data at run time, it is not clear that this is the
> correct approach, as it would hide changes in behaviour.

No, the test should be duplicated and then modified.

Legacy tests should have a conditional skip if Java >= 9

New tests should have a conditional skip, only if needed, if Java < 9

> 
> Ideally the code would continue to accept old style dates, but it's
> not at all clear how to do this.

This is what I mean by "only if needed". Clearly the "intention" of Java 9 is to change the defaults, but if a "legacy" configuration / compatibility mode is enabled then ...



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org