You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by David Waitzman <dj...@bbn.com> on 2006/04/26 15:28:27 UTC
[commons-net] listFiles failing to parse Apr 26 02:45 like dates
We have used the FTPClient library for years. Today it has started to fail
to parse a specific time of day pattern even though nothing has changed in
our code. This happens across multiple ftp sites.
Example, in the following listings, the files marked on the right with !!
fail to parse (e.g. the array returned by listFiles has a null for them)
...
-rw-r--r-- 1 0 0 14892 Apr 26 07:11 gt.db.gz
-rw-r--r-- 1 0 0 447 Apr 26 00:52 gts.db.gz
-rw-r--r-- 1 0 0 3208 Apr 26 09:30 gw.db.gz
-rw-r--r-- 1 0 0 60310 Apr 26 02:49 jpirr.db.gz !!
-rw-r--r-- 1 0 0 3972 Apr 25 16:37 koren.db.gz
-rw-r--r-- 1 0 0 2773841 Apr 25 12:29 level3.db.gz
-rw-r--r-- 1 0 0 1822 Apr 25 16:20 look.db.gz
-rw-r--r-- 1 0 0 602 Apr 25 20:00 nestegg.db.gz
-rw-r--r-- 1 0 0 1775 Apr 26 02:34 openface.db.gz !!
-rw-r--r-- 1 0 0 4576 Apr 26 01:20 ottix.db.gz
-rw-r--r-- 1 0 0 844 Apr 25 17:43 panix.db.gz
-rw-r--r-- 1 0 0 3715586 Apr 26 10:24 radb.db.gz
-rw-r--r-- 1 0 0 158425 Apr 25 18:59 reach.db.gz
-rw-r--r-- 1 0 0 1633 Apr 26 07:53 retina.db.gz
-rw-r--r-- 1 0 0 9547 Apr 26 06:55 rgnet.db.gz
-rw-r--r-- 1 0 0 17817 Apr 26 06:32 risq.db.gz
-rw-r--r-- 1 0 0 49 Apr 26 09:26 semaphore.db.gz
-rw-r--r-- 1 0 0 41727 Apr 26 08:14 sinet.db.gz
-rw-r--r-- 1 0 0 538 Apr 25 17:41 soundinternet.db.gz
-rw-r--r-- 1 0 0 12581 Apr 26 02:16 sprint.db.gz !!
-rw-r--r-- 1 0 0 423 Apr 26 03:16 univali.db.gz
-rw-r--r-- 1 0 0 1233 Apr 26 00:06 vdn.db.gz
-rw-r--r-- 1 0 0 415370 Apr 25 18:58 verio.db.gz
-rw-r--r-- 1 ftp ftp 13969315 Apr 25 22:25
oix-full-snapshot-2006-04-25-2200.dat.bz2
-rw-r--r-- 1 ftp ftp 72 Apr 25 22:25
oix-full-snapshot-2006-04-25-2200.dat.md5
-rw-r--r-- 1 ftp ftp 13968226 Apr 26 00:18
oix-full-snapshot-2006-04-26-0000.dat.bz2
-rw-r--r-- 1 ftp ftp 72 Apr 26 00:18
oix-full-snapshot-2006-04-26-0000.dat.md5
-rw-r--r-- 1 ftp ftp 13979565 Apr 26 02:18
oix-full-snapshot-2006-04-26-0200.dat.bz2 !!
-rw-r--r-- 1 ftp ftp 72 Apr 26 02:18
oix-full-snapshot-2006-04-26-0200.dat.md5 !!
-rw-r--r-- 1 ftp ftp 13977077 Apr 26 04:17
oix-full-snapshot-2006-04-26-0400.dat.bz2
-rw-r--r-- 1 ftp ftp 72 Apr 26 04:18
It seems to be a sensitive to "Apr 26 02". Very strange.
I am using both commons-net 1.4.0 and 1.4.1 from multiple OSs with sun 1.5.0_06.
--
David Waitzman
BBN Technologies
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org
Re: [commons-net] listFiles failing to parse Apr 26 02:45 like dates
Posted by Stephen Colebourne <sc...@btopenworld.com>.
David Waitzman wrote:
> The fix I made was to force the FTP timezone to be UTC using:
> FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_UNIX);
> conf.setServerTimeZoneId("UTC");
> ftpClient.configure(conf);
> The UTC timezone has no daylight savings time so all the times are valid.
> I'd rather keep my default timezone but turn off daylight savings time
> transitions, but I don't know how to do that. (Any suggestions?)
The only way to turn off time zone transitions is to:
a) pick either summer or winter time (usually the one without daylight
savings)
b) find out the offset
c) use the offset as a fixed offset, eg 'GMT+02:00'
Of course the time will only be accurate half the year...
Stephen
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org
Re: [commons-net] listFiles failing to parse Apr 26 02:45 like dates
Posted by David Waitzman <dj...@bbn.com>.
To follow up on my own message, I found a workaround and here is a detailed
analysis:
The issue is why does FTPClient.listFiles fail on just 4/26 02:xx times?
listFiles uses SimpleDateFormatter.parse which uses the Java date classes.
Parsing fails when lenient=false for an invalid time according to daylight
savings time. According to a time zone information file, from 1967 to 1973
the last Sunday in April was a transition date. The date parsing code
assumes the year is the unix epoch, e.g. 1970, when no year is explicit in
the date string being parsed. So Apr 26 1970 is a daylight savings
spring-forward date and 2:00am to 2:59am does NOT exist on that date.
The fix I made was to force the FTP timezone to be UTC using:
FTPClientConfig conf = new FTPClientConfig(FTPClientConfig.SYST_UNIX);
conf.setServerTimeZoneId("UTC");
ftpClient.configure(conf);
The UTC timezone has no daylight savings time so all the times are valid.
I'd rather keep my default timezone but turn off daylight savings time
transitions, but I don't know how to do that. (Any suggestions?)
I made a test program which illustrates the problem
Lenient=true, TZ=Eastern
Trying date Apr 26 02:04
Parsed into Sun Apr 26 03:04:00 EDT 1970
Trying date Apr 26 07:04
Parsed into Sun Apr 26 07:04:00 EDT 1970
Trying date Jul 26 02:04
Parsed into Sun Jul 26 02:04:00 EDT 1970
Lenient=true, TZ=UTC
Trying date Apr 26 02:04
Parsed into Sat Apr 25 21:04:00 EST 1970
Trying date Apr 26 07:04
Parsed into Sun Apr 26 03:04:00 EDT 1970
Trying date Jul 26 02:04
Parsed into Sat Jul 25 22:04:00 EDT 1970
Lenient=false, TZ=Eastern
Trying date Apr 26 02:04
Parse failed with java.lang.NullPointerException
Trying date Apr 26 07:04
Parsed into Sun Apr 26 07:04:00 EDT 1970
Trying date Jul 26 02:04
Parsed into Sun Jul 26 02:04:00 EDT 1970
Lenient=false, TZ=UTC
Trying date Apr 26 02:04
Parsed into Sat Apr 25 21:04:00 EST 1970
Trying date Apr 26 07:04
Parsed into Sun Apr 26 03:04:00 EDT 1970
Trying date Jul 26 02:04
Parsed into Sat Jul 25 22:04:00 EDT 1970
The real bug is probably that listFiles Unix parser can't force the java
data parsing code to assume the year is the current year when the
UnixFTPEntryParser has to use the format string DEFAULT_RECENT_DATE_FORMAT.
Edge cases are complex here (ex. on January 2nd parsing December 31).
========== the test code =========
import java.text.*;
import java.util.Date;
import java.util.TimeZone;
public class date {
static final String format1 = "MMM d HH:mm"; //Nov 9 20:06
static final SimpleDateFormat formatter = new SimpleDateFormat(format1);
public static void test(String str) {
try {
System.out.println(" Trying date " + str);
ParsePosition pp = new ParsePosition(0);
Date parsed = formatter.parse(str, pp);
System.out.println(" Parsed into " + parsed.toString());
} catch (Exception e) {
System.out.println(" Parse failed with " + e);
}
}
public static void testMultiple() {
test("Apr 26 02:04");
test("Apr 26 07:04");
test("Jul 26 02:04");
}
public static void main(String [] args) {
formatter.setLenient(true);
TimeZone tz = TimeZone.getTimeZone("US/Eastern");
formatter.setTimeZone(tz);
System.out.println("Lenient=true, TZ=Eastern");
testMultiple();
tz = TimeZone.getTimeZone("UTC");
formatter.setTimeZone(tz);
System.out.println("Lenient=true, TZ=UTC");
testMultiple();
formatter.setLenient(false);
tz = TimeZone.getTimeZone("US/Eastern");
formatter.setTimeZone(tz);
System.out.println("Lenient=false, TZ=Eastern");
testMultiple();
tz = TimeZone.getTimeZone("UTC");
formatter.setTimeZone(tz);
System.out.println("Lenient=false, TZ=UTC");
testMultiple();
}
};
========== =========
I previously wrote:
> We have used the FTPClient library for years. Today it has started to
> fail to parse a specific time of day pattern even though nothing has
> changed in our code. This happens across multiple ftp sites.
>
> Example, in the following listings, the files marked on the right with
> !! fail to parse (e.g. the array returned by listFiles has a null for them)
> ...
> -rw-r--r-- 1 0 0 3208 Apr 26 09:30 gw.db.gz
> -rw-r--r-- 1 0 0 60310 Apr 26 02:49 jpirr.db.gz !!
> -rw-r--r-- 1 0 0 3972 Apr 25 16:37 koren.db.gz
...
> -rw-r--r-- 1 0 0 1775 Apr 26 02:34 openface.db.gz !!
...
> -rw-r--r-- 1 0 0 12581 Apr 26 02:16 sprint.db.gz !!
> -rw-r--r-- 1 ftp ftp 13968226 Apr 26 00:18
> oix-full-snapshot-2006-04-26-0000.dat.bz2
> oix-full-snapshot-2006-04-26-0200.dat.bz2 !!
> -rw-r--r-- 1 ftp ftp 72 Apr 26 02:18
> oix-full-snapshot-2006-04-26-0200.dat.md5 !!
>
> It seems to be a sensitive to "Apr 26 02". Very strange.
> I am using both commons-net 1.4.0 and 1.4.1 from multiple OSs with sun
> 1.5.0_06.
--
David Waitzman
BBN Technologies
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-user-help@jakarta.apache.org