Re: [lang3] FastDateFormat fails on some locales?

[Kevin Risden <kr...@apache.org>]

I did some further testing and found out of the ~160 locales that my
JDK8 had, only ja-JP-u-ca-japanese-x-lvariant-JP failed with the
ArrayIndexOutOfBoundsException

Kevin Risden

On Wed, Jan 30, 2019 at 1:36 PM Kevin Risden <kr...@apache.org> wrote:
>
> I found this while looking at Apache Lucene/Solr and Hadoop 3. Hadoop
> uses FastDateFormat to format the current timestamp. Apache
> Lucene/Solr randomizes locales to ensure that things behave correctly
> even when there are different locales being used. There have been a
> few failures that have the following stack trace:
>
> ava.lang.ArrayIndexOutOfBoundsException: 4
>    [junit4]   2> at
> org.apache.commons.lang3.time.FastDatePrinter$TextField.appendTo(FastDatePrinter.java:901)
> ~[commons-lang3-3.7.jar:3.7]
>    [junit4]   2> at
> org.apache.commons.lang3.time.FastDatePrinter.applyRules(FastDatePrinter.java:573)
> ~[commons-lang3-3.7.jar:3.7]
>    [junit4]   2> at
> org.apache.commons.lang3.time.FastDatePrinter.applyRulesToString(FastDatePrinter.java:455)
> ~[commons-lang3-3.7.jar:3.7]
>    [junit4]   2> at
> org.apache.commons.lang3.time.FastDatePrinter.format(FastDatePrinter.java:446)
> ~[commons-lang3-3.7.jar:3.7]
>    [junit4]   2> at
> org.apache.commons.lang3.time.FastDateFormat.format(FastDateFormat.java:428)
> ~[commons-lang3-3.7.jar:3.7]
>    [junit4]   2> at
> org.apache.hadoop.hdfs.server.datanode.DirectoryScanner.start(DirectoryScanner.java:281)
> ~[hadoop-hdfs-3.2.0.jar:?]
>    [junit4]   2> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.initDirectoryScanner(DataNode.java:1090)
> ~[hadoop-hdfs-3.2.0.jar:?]
>    [junit4]   2> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1686)
> ~[hadoop-hdfs-3.2.0.jar:?]
>    [junit4]   2> at
> org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:390)
> ~[hadoop-hdfs-3.2.0.jar:?]
>    [junit4]   2> at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:280)
> ~[hadoop-hdfs-3.2.0.jar:?]
>    [junit4]   2> at
> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:819)
> [hadoop-hdfs-3.2.0.jar:?]
>    [junit4]   2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
>
> I was also able to reproduce this with a simple test case:
>
> long timestamp = System.currentTimeMillis();
> Locale.setDefault(Locale.forLanguageTag("ja-JP-u-ca-japanese-x-lvariant-JP"));
> Assert.assertEquals(SimpleDateFormat.getInstance().format(timestamp),
> FastDateFormat.getInstance().format(timestamp));
>
> Showing that the issue isn't with Hadoop but with commons-lang3
> specifically. SimpleDateFormat has no issue formatting the timestamp
> with the given locale. The FastDateFormat javadoc doesn't state any
> issues with locales.
>
> Is this to be expected?
>
> Kevin Risden

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org