You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Nuutti Kotivuori <na...@iki.fi> on 2002/05/04 17:07:35 UTC
ISO-8601 dates and issue 614
I'd like to tackle this issue on ASAP - though I might get busy from
work in the near future. But this manages to really annoy me in svn
right now :)
OK, first let's see if I understand the related matters correctly.
The pressure onto moving to ISO-8601 comes from two issues:
First, the WebDAV creationdate - RFC2518 specifies that
creationdate must be an ISO-8601 date - and not only any ISO-8601
date, but a very specific format. So, to comply with the RFC, we
must change the date format that travels over the wire.
Second, user representation - it would be nice if the date
presented to users would be a standard date instead of something
that is only near it. This second point is what got me interested
about the issue - I want the dates to be ISO-8601.
If I understand the current situation correctly, all time to string
conversion happens with two functions: svn_time_from_nts,
svn_time_to_nts - and these make an exact representation of the
apr_time_exp_t structure - even if the structure contains very weird
information. But the result is long and not too well suited for human
representation in it's entirety.
But you are saying that there's actually no need for these functions
to behave that way for subversion, it's just nice that they exist and
behave that way - and hence should not be scrapped? Well quite
reasonable I guess.
So, what needs to be done. I guess we need the functions to handle
both of the issues mentioned - on the wire and user presentation. On
the wire format needs to be able to translate the timestamp both ways
- to string and from string. And hopefully it would atleast be
non-lossy with apr_time_t, even if not containing all the cases
apr_time_exp_t can represent. The user representation needs only
conversion to string - and the representation can be lossy. Actually,
it might even be possible to have configurable timestamp formats - but
we'll come to that later.
OK, the format specified by RFC2518, which would be used on the wire,
when containing all items and assuming we wish the timestamps
travelling over the wire to be UTC, would be either one of these:
2002-05-04T12:34:56.999999Z
2002-05-04T12:34:56.999999-00:00
The difference is mentioned as:
,----
| If the time in UTC is known, but the offset to local time is
| unknown, this can be represented with an offset of "-00:00". This
| differs from an offset of "Z" which implies that UTC is the
| preferred reference point for the specified time.
`----
I'm unsure which one I would prefer - but I would possibly go for the
"Z". Although we do not know the offset to the local time, the UTC is
the preferred reference point for all times going over the wire. But I
don't really know.
For the user represented format, there's a bit more issues. Right now
there's that something humanize date which was used in svn log - and
hand cooked printfs in svn info. Both of those should probably be
scrapped.
But what do we really want to do? Do we want to print timestamps in
UTC? Or do we perhaps want to honor the users timezone and print them
accordingly? Or would we even want to specify a configuration option
for time stamp format that the user would see in svn log and svn info
atleast?
Then another issue - it would appear from the comments on the issue,
that there is yet another place where ISO-8601 times would be wanted -
repository. Or actually everywhere where dates go to string form and
back. Well this is a bit of a bigger issue I guess :)
Are svn:date properties the only place where dates are in string form
in the repository? Would we want to convert dates somewhere, or just
remain backward compatible? We do want to remain backward compatible
without special mangling to the repository do we? What about the over
the wire format - do we need to be backwards compatible with clients
that send old style dates? Do we need to convert them before putting
them into the repository? What about sending dates to old clients? I
hope we don't have to convert dates to old format for old clients?
Oof, this was a bit uncoherent - but that's because I'm in a rush.
Help me on this, tell me what to do - this might not be a small
matter, but I personally think it should go in before alpha or
whatnot. And I'm willing to put in effort to make it happen.
TIA,
-- Naked
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
brane@xbc.nu writes:
> Argh, had this on my todo list but forgot ...
>
> I really don't see why you'd want to avoid a wc conversion for the
> timestamps in particular. We already know we'll have to convert when
> the private URLs change from using node ids to using rev/path, so we
> might as well do both changes at the same time. The "always use HEAD"
> motto of the pre-Alpha stage should hold for working copies, too.
Actually, it turns out we're not going to need a grand wc conversion
for the version-resource-url change (see gstein's change of rev 1863).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by br...@xbc.nu.
Karl Fogel wrote:
>Branko Čibej <br...@xbc.nu> writes:
>
>
>>>Though I would really like to get some kind of an answer that what do
>>>we want to do with the old style svn_time_to_nts things - is there
>>>something worth keeping there?
>>>
>>>
>>Assuming you provide a script to convert the WC, I don't think
>>so. It's not worth keeping the month/day name parsing there -- that
>>should go into apr_strptime.
>>
>>
>
>I think we can do this without a grand wc conversion.
>
>The format currently in .svn/entries is instantly distinguishable from
>the format we want in the future. Thus, the conversion function that
>*reads* .svn/entries timestamps merely needs some compatibility code
>for a while (a few months, whatever). The writers, of course, would
>just start using the new format as soon as the two-format-accepting
>reader is in place.
>
>
Argh, had this on my todo list but forgot ...
I really don't see why you'd want to avoid a wc conversion for the
timestamps in particular. We already know we'll have to convert when the
private URLs change from using node ids to using rev/path, so we might
as well do both changes at the same time. The "always use HEAD" motto of
the pre-Alpha stage should hold for working copies, too.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Nuutti Kotivuori <na...@iki.fi> writes:
> 1) Old repository, new client
>
> Will repository get some date string from the client that it cannot
> parse after the change - if so, where?
Do repositories *ever* get dates from clients? I think only revision
numbers and paths are transmitted in that direction.
> 2) New repository, old client
>
> Will the new repository send dates to the client it cannot parse? Will
> it do this immediately and is it fatal - eg. upgrading repos breaks
> all old clients? Or does it just appear on new commits? Or is it not a
> problem at all?
It's okay for new repositories to break old clients, as long as the
code to deal with the new repository is checked in well before the
live server is actually upgraded. Being forced to upgrade your client
is normal around here :-). Being forced to re-checkout your working
copy is not (or shouldn't be, anyway, heh).
> I'll start testing my already changed client sometime soon - but I do
> not have a WebDAV server to play with for the test.
Have you / can you set up httpd-2.0 w/ mod_dav_svn locally?
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Nuutti Kotivuori <na...@iki.fi>.
brane@xbc.nu wrote:
> Don't worry about that sort of version skew, we're still pre-alpha
> and can afford to tell people to upgrade their servers and (soon)
> dump/load their repos.
Well this is all fine and dandy, as long as the users _can_ still
update - either via 'svn up' or a new bootstrap tarball :)
-- Naked
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Branko Čibej <br...@xbc.nu>.
Nuutti Kotivuori wrote:
>Karl Fogel wrote:
>
>
>>Branko Čibej <br...@xbc.nu> writes:
>>
>>
>>>>Though I would really like to get some kind of an answer that what
>>>>do we want to do with the old style svn_time_to_nts things - is
>>>>there something worth keeping there?
>>>>
>>>>
>>>Assuming you provide a script to convert the WC, I don't think
>>>so. It's not worth keeping the month/day name parsing there -- that
>>>should go into apr_strptime.
>>>
>>>
>>I think we can do this without a grand wc conversion.
>>
>>The format currently in .svn/entries is instantly distinguishable
>>from the format we want in the future. Thus, the conversion
>>function that *reads* .svn/entries timestamps merely needs some
>>compatibility code for a while (a few months, whatever). The
>>writers, of course, would just start using the new format as soon as
>>the two-format-accepting reader is in place.
>>
>>
>
>Exactly.
>
>I believe the working directories are no problem - since no "new"
>working directory should ever be accessed with an "old" client, so a
>simple reading compatibility will suffice.
>
>But we must remember that the change is supposed to change the
>timestamp format transmitted through WebDAV as well - so the problem
>is a bit bigger.
>
>If we, for example, change all the date to string functions so that
>they write the new style dates, but parse both old style and new style
>- what will happen in these cases:
>
Don't worry about that sort of version skew, we're still pre-alpha and
can afford to tell people to upgrade their servers and (soon) dump/load
their repos.
>1) Old repository, new client
>
>Will repository get some date string from the client that it cannot
>parse after the change - if so, where?
>
>2) New repository, old client
>
>Will the new repository send dates to the client it cannot parse? Will
>it do this immediately and is it fatal - eg. upgrading repos breaks
>all old clients? Or does it just appear on new commits? Or is it not a
>problem at all?
>
>If push comes to shove, we can probably upgrade both server and
>clients to understand the new format, but still write out the old
>format - and then wait a while (possibly a new tarball) for them to
>spread everywhere - and then change the writer to writing the new
>version. But let's hope we don't have to go into that.
>
>I'll start testing my already changed client sometime soon - but I do
>not have a WebDAV server to play with for the test.
>
>But hopefully these issues can be mapped by just looking at the code,
>not empirically finding out problem spots.
>
>-- Naked
>
>
>
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Nuutti Kotivuori <na...@iki.fi>.
Karl Fogel wrote:
> Branko Čibej <br...@xbc.nu> writes:
>> >Though I would really like to get some kind of an answer that what
>> >do we want to do with the old style svn_time_to_nts things - is
>> >there something worth keeping there?
>>
>> Assuming you provide a script to convert the WC, I don't think
>> so. It's not worth keeping the month/day name parsing there -- that
>> should go into apr_strptime.
>
> I think we can do this without a grand wc conversion.
>
> The format currently in .svn/entries is instantly distinguishable
> from the format we want in the future. Thus, the conversion
> function that *reads* .svn/entries timestamps merely needs some
> compatibility code for a while (a few months, whatever). The
> writers, of course, would just start using the new format as soon as
> the two-format-accepting reader is in place.
Exactly.
I believe the working directories are no problem - since no "new"
working directory should ever be accessed with an "old" client, so a
simple reading compatibility will suffice.
But we must remember that the change is supposed to change the
timestamp format transmitted through WebDAV as well - so the problem
is a bit bigger.
If we, for example, change all the date to string functions so that
they write the new style dates, but parse both old style and new style
- what will happen in these cases:
1) Old repository, new client
Will repository get some date string from the client that it cannot
parse after the change - if so, where?
2) New repository, old client
Will the new repository send dates to the client it cannot parse? Will
it do this immediately and is it fatal - eg. upgrading repos breaks
all old clients? Or does it just appear on new commits? Or is it not a
problem at all?
If push comes to shove, we can probably upgrade both server and
clients to understand the new format, but still write out the old
format - and then wait a while (possibly a new tarball) for them to
spread everywhere - and then change the writer to writing the new
version. But let's hope we don't have to go into that.
I'll start testing my already changed client sometime soon - but I do
not have a WebDAV server to play with for the test.
But hopefully these issues can be mapped by just looking at the code,
not empirically finding out problem spots.
-- Naked
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Branko Čibej <br...@xbc.nu> writes:
> >Though I would really like to get some kind of an answer that what do
> >we want to do with the old style svn_time_to_nts things - is there
> >something worth keeping there?
>
> Assuming you provide a script to convert the WC, I don't think
> so. It's not worth keeping the month/day name parsing there -- that
> should go into apr_strptime.
I think we can do this without a grand wc conversion.
The format currently in .svn/entries is instantly distinguishable from
the format we want in the future. Thus, the conversion function that
*reads* .svn/entries timestamps merely needs some compatibility code
for a while (a few months, whatever). The writers, of course, would
just start using the new format as soon as the two-format-accepting
reader is in place.
?
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Branko Čibej <br...@xbc.nu>.
Nuutti Kotivuori wrote:
>brane@xbc.nu wrote:
>
>>Ben Collins-Sussman wrote:
>>
>>>Branko Čibej <br...@xbc.nu> writes:
>>>
>>>>And yes, whatever timestamps we store in .svn/entries or send over
>>>>the wire should be UTC; svn_time_to_nts and svn_time_from_nts are
>>>>the places where this should be fixed.
>>>>
>>>Yes, this is the correct approach.
>>>
>>>But the implementor should be careful that they don't break existing
>>>working copies. We all have old-style dates sitting in our entries
>>>files; svn_time_from_nts() should be able to recognize *either*
>>>format of date-string, even though svn_time_to_nts() will be writing
>>>out ISO dates.
>>>
>>Ah yes, good point. Either that, or provide a script that converts
>>the WC (like we did some time ago when we changed
>>I-can't-remember-what).
>>
>
>Yes, this is basically what I though as well - so I'll get the job
>underway.
>
>Though I would really like to get some kind of an answer that what do
>we want to do with the old style svn_time_to_nts things - is there
>something worth keeping there?
>
Assuming you provide a script to convert the WC, I don't think so. It's
not worth keeping the month/day name parsing there -- that should go
into apr_strptime.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Nuutti Kotivuori <na...@iki.fi>.
brane@xbc.nu wrote:
> Ben Collins-Sussman wrote:
>> Branko Čibej <br...@xbc.nu> writes:
>>> And yes, whatever timestamps we store in .svn/entries or send over
>>> the wire should be UTC; svn_time_to_nts and svn_time_from_nts are
>>> the places where this should be fixed.
>>
>> Yes, this is the correct approach.
>>
>> But the implementor should be careful that they don't break existing
>> working copies. We all have old-style dates sitting in our entries
>> files; svn_time_from_nts() should be able to recognize *either*
>> format of date-string, even though svn_time_to_nts() will be writing
>> out ISO dates.
>
> Ah yes, good point. Either that, or provide a script that converts
> the WC (like we did some time ago when we changed
> I-can't-remember-what).
Yes, this is basically what I though as well - so I'll get the job
underway.
Though I would really like to get some kind of an answer that what do
we want to do with the old style svn_time_to_nts things - is there
something worth keeping there?
-- Naked
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Branko Čibej <br...@xbc.nu>.
Ben Collins-Sussman wrote:
>Branko Čibej <br...@xbc.nu> writes:
>
>>And yes, whatever timestamps we store in .svn/entries or send over the
>>wire should be UTC; svn_time_to_nts and svn_time_from_nts are the
>>places where this should be fixed.
>>
>
>Yes, this is the correct approach.
>
>But the implementor should be careful that they don't break existing
>working copies. We all have old-style dates sitting in our entries
>files; svn_time_from_nts() should be able to recognize *either*
>format of date-string, even though svn_time_to_nts() will be writing
>out ISO dates.
>
Ah yes, good point. Either that, or provide a script that converts the
WC (like we did some time ago when we changed I-can't-remember-what).
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Ben Collins-Sussman <su...@collab.net>.
Branko Čibej <br...@xbc.nu> writes:
> And yes, whatever timestamps we store in .svn/entries or send over the
> wire should be UTC; svn_time_to_nts and svn_time_from_nts are the
> places where this should be fixed.
Yes, this is the correct approach.
But the implementor should be careful that they don't break existing
working copies. We all have old-style dates sitting in our entries
files; svn_time_from_nts() should be able to recognize *either*
format of date-string, even though svn_time_to_nts() will be writing
out ISO dates.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Branko Čibej <br...@xbc.nu>.
Barry Pederson wrote:
> Tim Waugh wrote:
>
>
> > Unix-based systems of course use UTC, as any sane system would.
> >
> > The NT bug, as I recall, is that stat() will return a different
> > structure before and after a DST change. The time_t fields are
> > actually adjusted, and can't be interpreted as 'seconds since the
> > Epoch'. (Perhaps I'm misremembering.)
>
> That's about right, it has to do with the way date/time is stored in
> NTFS partitions. Here's a Microsoft description:
>
> http://support.microsoft.com/default.aspx?scid=kb;en-us;Q158588
>
> I just got burned by this a month or so ago with another program.
> Basically, it seems that Windows stores GMT/UTC, and then stat()
> shifts the time according to what your offset from GMT is *now* (not
> what the offset should have been on the particular date associated
> with the time).
>
> So it shouldn't be too difficult to compensate for that shift and get
> back to GMT 'seconds since the epoch'. IIUC, for example on my
> machine right now (in May), I'm at GMT+5, so if I subtract (5* 60 *
> 60) from the stat values, I should get a GMT value. And come
> November, when DST is off, for the very same file, subtracting (6* 60
> * 60) from the stat should give the same GMT value.
>
> The other workaround would be to not use stat(), and instead use the
> Win32 API call mentioned in the link above that gets UTC directly.
>
> Is this a problem in SVN? It looks like APR tries to take care of all
> that stuff (using the Win32 API instead of stat())
We use APR for that, and APR is (should be) consistent across platforms,
so that's not an issue. (Specifically, on Windows, it uses the Win32 APIs.)
And yes, whatever timestamps we store in .svn/entries or send over the
wire should be UTC; svn_time_to_nts and svn_time_from_nts are the places
where this should be fixed.
Parsing timsetamps from the -D option, and what goes into "svn log" and
"svn info", is a different matter. There used to be a patch floating
around that implements apr_strptime (an strptime replacement) for the
parser -- I think I still have it lying around. I think Mark Benedetto
King (bking@answerfriend.com, mbk@tigris.org) is/was working on that, too.
For everything else, we should be using apr_strftime to format the
output. Whether the format should be locale-specific, ISO-8601, or
perhaps even configurable via the config file, is something we have do
discuss and decide. I suppose a user-friendly version of ISO-8601 (space
instead of the T separator, etc.) would be a reasonable default.
--
Brane Čibej <br...@xbc.nu> http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Barry Pederson <bp...@barryp.org>.
Tim Waugh wrote:
> Unix-based systems of course use UTC, as any sane system would.
>
> The NT bug, as I recall, is that stat() will return a different
> structure before and after a DST change. The time_t fields are
> actually adjusted, and can't be interpreted as 'seconds since the
> Epoch'. (Perhaps I'm misremembering.)
That's about right, it has to do with the way date/time is stored in NTFS
partitions. Here's a Microsoft description:
http://support.microsoft.com/default.aspx?scid=kb;en-us;Q158588
I just got burned by this a month or so ago with another program. Basically,
it seems that Windows stores GMT/UTC, and then stat() shifts the time
according to what your offset from GMT is *now* (not what the offset should
have been on the particular date associated with the time).
So it shouldn't be too difficult to compensate for that shift and get back to
GMT 'seconds since the epoch'. IIUC, for example on my machine right now (in
May), I'm at GMT+5, so if I subtract (5* 60 * 60) from the stat values, I
should get a GMT value. And come November, when DST is off, for the very same
file, subtracting (6* 60 * 60) from the stat should give the same GMT value.
The other workaround would be to not use stat(), and instead use the Win32 API
call mentioned in the link above that gets UTC directly.
Is this a problem in SVN? It looks like APR tries to take care of all that
stuff (using the Win32 API instead of stat())
Barry
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Alan Langford <ja...@ambitonline.com>.
At 2002/05/06 13:36 +0100, Tim Waugh wrote:
>Unix-based systems of course use UTC, as any sane system would.
It sure seemed illogical that Unix would mess that up!
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: ISO-8601 dates and issue 614
Posted by Tim Waugh <tw...@redhat.com>.
On Mon, May 06, 2002 at 08:27:11AM -0400, Alan Langford wrote:
> You will find an interesting platform dependency here. There are lots of
> Windows/CVS complaints about file modification date changes when a system
> switches from standard to daylight savings time and back. In fact (ahem)
> most versions of Windows seem to write the modification date to the disk as
> UTC. This means that when the user changes time zones, or moves from or to
> daylight savings, the file modification times appear to change when in fact
> the OS is reporting an accurate date. The many rants against Windows and
> suggestions that that this is a "bug" suggest that most Unix based systems
> write local time to the disk.
Unix-based systems of course use UTC, as any sane system would.
The NT bug, as I recall, is that stat() will return a different
structure before and after a DST change. The time_t fields are
actually adjusted, and can't be interpreted as 'seconds since the
Epoch'. (Perhaps I'm misremembering.)
Tim.
*/
Re: ISO-8601 dates and issue 614
Posted by Alan Langford <ja...@ambitonline.com>.
You will find an interesting platform dependency here. There are lots of
Windows/CVS complaints about file modification date changes when a system
switches from standard to daylight savings time and back. In fact (ahem)
most versions of Windows seem to write the modification date to the disk as
UTC. This means that when the user changes time zones, or moves from or to
daylight savings, the file modification times appear to change when in fact
the OS is reporting an accurate date. The many rants against Windows and
suggestions that that this is a "bug" suggest that most Unix based systems
write local time to the disk.
It's also a problem since Windows is inconsistent. Win95 appears to write
local time to the disk, whereas WinNT and Win2000 write UTC.
I'm not sure how deep the implications of these differences is, but though
I'd make you aware of them just in case.
At 2002/05/04 20:07 +0300, Nuutti Kotivuori wrote:
>Help me on this, tell me what to do - this might not be a small
>matter, but I personally think it should go in before alpha or
>whatnot. And I'm willing to put in effort to make it happen.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org