You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Ori Avtalion <oa...@bezeqint.net> on 2007/03/27 21:59:20 UTC
Revisions with wrong chronologic order
Hi,
I have dumped two subversion repositories (with unrelated,
non-overlapping trees) and then loaded them into a new repository one
after the other.
I now have a repository where revisions 1-100 are dated 2005-2007 and
revisions 101-200 are dated 2005-2007 as well.
svn commands work, but is this a healthy, recommended state for a
repository?
I haven't tried it yet, but is it possible that running
"svn up -r {DATE}" will break something?
To prevent it, is there a way to merge the two dumps with correctly
sorted revisions, before loading them into a repository?
Thanks,
Ori
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Eli Carter <el...@commprove.com>.
On Wednesday 11 April 2007, Kylo Ginsberg wrote:
> I have a scenario where I would like to dump a small repository and
> load it into a large one. Essentially I want to tack all the
> revisions of the small repo on to the tail end of the large one with
> their date being "at load time".
>
> More specifically: I want the original large repository to keep
> revision #s (so merge comments in log entries are coherent) but I
> don't want to preserve revision #s from the small repository. I want
> the -r DATE syntax to work, with the understanding that all revs from
> the small repository now appear to be from the time of the load.
>
> Is this possible with the svnadmin dump/load, svndumpfilter, or these
> other scripts/tools that have been mentioned.
Since the date is a revprop, you should be able to use the svndumptool
transform-revprop command to change them.
Eli
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Kylo Ginsberg <ky...@gmail.com>.
On 4/11/07, Ryan Schmidt <su...@ryandesign.com> wrote:
> On Apr 11, 2007, at 10:29, Kylo Ginsberg wrote:
>
> > I have a scenario where I would like to dump a small repository and
> > load it into a large one. Essentially I want to tack all the
> > revisions of the small repo on to the tail end of the large one with
> > their date being "at load time".
> >
> But you can change a revision's date and time. You may be able to
> manipulate the dumpfile in some way. However, I recommend just using
> "svn propset --revprop -r 123 svn:date foo" where "123" is the
> revision number and "foo" is a date in Subversion format (which you
> can discover by running "svn propget --revprop -r 123 svn:date"). You
Ok, this would be easy to script. The svnbook says the svn:date is in
"ISO format" but doesn't reference a particular standard. However, it
appears to be a UTC timecode with microsecond resolution. I can ape
it with date (slightly ugly since date doesn't have a microsecond
option):
echo `date -u +%Y-%m-%dT%H:%M:%S.%N | cut -c-26`Z
Does that format look right?
And given that microsecond resolution, might subversion get confused
if several hundred revisions had exactly the same time? I can also
write the script such that it pulls a unique time for each revision.
Thanks,
Kylo
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Apr 11, 2007, at 10:29, Kylo Ginsberg wrote:
> I have a scenario where I would like to dump a small repository and
> load it into a large one. Essentially I want to tack all the
> revisions of the small repo on to the tail end of the large one with
> their date being "at load time".
>
> More specifically: I want the original large repository to keep
> revision #s (so merge comments in log entries are coherent) but I
> don't want to preserve revision #s from the small repository. I want
> the -r DATE syntax to work, with the understanding that all revs from
> the small repository now appear to be from the time of the load.
>
> Is this possible with the svnadmin dump/load, svndumpfilter, or these
> other scripts/tools that have been mentioned.
If you dump the small repo and load it into the big repo, the
revisions from the small repo will not suddenly get today's date and
time. They will still have the same date and time that they had in
the small repo.
But you can change a revision's date and time. You may be able to
manipulate the dumpfile in some way. However, I recommend just using
"svn propset --revprop -r 123 svn:date foo" where "123" is the
revision number and "foo" is a date in Subversion format (which you
can discover by running "svn propget --revprop -r 123 svn:date"). You
could do these propset commands in the small repo for all revisions
of the small repo before dumping it. (If you set the date to "now,"
you would need to be sure that nobody commits anything else to the
big repo between "now" and whenever you're done loading, so that the
revisions don't get out of order. Or you could set the date to some
time in the future, and plan to load them into the big repo at that
time.) Or you could set the revisions' dates in the big repo after
you load them in. (You would have to figure out which range of
revisions of the big repo came from the small repo.) Either way, you
will need a pre-revprop-change hook in one repo or the other to
enable you to change the svn:date property. You can remove the pre-
revprop-change hook when you're done changing the dates.
--
To reply to the mailing list, please use your mailer's Reply To All
function
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Kylo Ginsberg <ky...@gmail.com>.
I have a scenario where I would like to dump a small repository and
load it into a large one. Essentially I want to tack all the
revisions of the small repo on to the tail end of the large one with
their date being "at load time".
More specifically: I want the original large repository to keep
revision #s (so merge comments in log entries are coherent) but I
don't want to preserve revision #s from the small repository. I want
the -r DATE syntax to work, with the understanding that all revs from
the small repository now appear to be from the time of the load.
Is this possible with the svnadmin dump/load, svndumpfilter, or these
other scripts/tools that have been mentioned.
Thanks,
Kylo
On 4/11/07, Eli Carter <el...@commprove.com> wrote:
> On Tuesday 10 April 2007, Ryan Schmidt wrote:
> > I'm surprised that this didn't come up sooner in this thread, but
> > someone has already written a script to load multiple repository
> > dumps into a single new one in interleaved order so that the -r
> > {DATE} syntax works.
> >
> > That is the only reason why you would need to load interleaved, by
> > the way: So that the -r {DATE} syntax works. If you don't care about
> > that, then you can load each dump sequentially, as you tried (and
> > succeeded with) initially.
> >
> > The script to load multiple dumps interleaved is repomerge.py. See here:
> >
> > http://svn.haxx.se/users/archive-2006-01/1168.shtml
>
> Or better yet, use svndumptool. It does a lot more than just merge
> repositories....
>
> http://svn.borg.ch/svndumptool/
>
> Eli
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Eli Carter <el...@commprove.com>.
On Tuesday 10 April 2007, Ryan Schmidt wrote:
> I'm surprised that this didn't come up sooner in this thread, but
> someone has already written a script to load multiple repository
> dumps into a single new one in interleaved order so that the -r
> {DATE} syntax works.
>
> That is the only reason why you would need to load interleaved, by
> the way: So that the -r {DATE} syntax works. If you don't care about
> that, then you can load each dump sequentially, as you tried (and
> succeeded with) initially.
>
> The script to load multiple dumps interleaved is repomerge.py. See here:
>
> http://svn.haxx.se/users/archive-2006-01/1168.shtml
Or better yet, use svndumptool. It does a lot more than just merge
repositories....
http://svn.borg.ch/svndumptool/
Eli
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Ryan Schmidt <su...@ryandesign.com>.
I'm surprised that this didn't come up sooner in this thread, but
someone has already written a script to load multiple repository
dumps into a single new one in interleaved order so that the -r
{DATE} syntax works.
That is the only reason why you would need to load interleaved, by
the way: So that the -r {DATE} syntax works. If you don't care about
that, then you can load each dump sequentially, as you tried (and
succeeded with) initially.
The script to load multiple dumps interleaved is repomerge.py. See here:
http://svn.haxx.se/users/archive-2006-01/1168.shtml
--
To reply to the mailing list, please use your mailer's Reply To All
function
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Ori Avtalion <oa...@bezeqint.net>.
Nathan Kidd wrote:
> Jared Silva wrote:
>> Oefelein, Martina wrote:
>>> I think that's hard to do in a situation like Ori's where you have
>>> two repositories with 2 years commit history that you want to combine
>>> into one. You would have to dump each revision individually, and then
>>> load the dump files in date order.
>>
>> Yes, it may be tedious. However, if this is what he wants, this is
>> what he needs to do.
>>
>> Shame there is not a script to take care of this...
>
> Actually, this would be very simple to write a script for.
>
> for (each repository)
> for (each revision)
> dump > dumpdir/{revisiondate}-{revisionnumber}-{reponame}.dump
> done
> done
>
> for (alphabetically sorted list of files in dumpdir)
> load < {file}
> done
I have tried this. Didn't work quite well.
My procedure (which was only pseudo-automated - no script. sorry):
* Dump all revisions into seperate files using the command "svnadmin
dump -r # --incremental" - I got some warnings that some revisions were
of an earlier date than the first revision, and they cannot be imported
into an empty repos - that makes sense.
* Produce a list of revisions using "svn log | grep "^r.*|"
* Sort the list of revisions with the timestamp as the first field and
the revision as the second (There are cases where two timestamps are the
same since I used the vss2svn tool).
* Now I created a new repository and executed "svnadmin load" for each
revision in the sorted order.
The resulting repository had less revisions than the original and had
missing revisions.
During the conversion, I noticed that not every "svnadmin load" command
produced a commit. Some just printed something like "started new
transaction from old revision..." and only after one or two more "load"s
came the commit message.
Why should that happen, and how can I avoid it?
I'm using svnadmin v1.3.1
-Ori
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Ori Avtalion <oa...@bezeqint.net>.
Nathan Kidd wrote:
> Erik Huelsmann wrote:
>> On 3/28/07, Nathan Kidd <na...@spicycrypto.ca> wrote:
>>> > Shame there is not a script to take care of this...
>>>
>>> Actually, this would be very simple to write a script for.
>>>
>>> for (each repository)
>>> for (each revision)
>>> dump > dumpdir/{revisiondate}-{revisionnumber}-{reponame}.dump
>>> done
>>> done
>>>
>>> for (alphabetically sorted list of files in dumpdir)
>>> load < {file}
>>> done
>>
>> Are you sure all copy sources will be ordered correctly that way?
What are copy sources?
> Hmmm, no, I'm not sure. That pseudo code was the product of 5s thought
> on getting the order right, but I don't understand the dump/load
> internals enough to know if there would be side-effects. It would be an
> interesting experiment though...
I'll try implementing this tomorrow - see how it goes :)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Nathan Kidd <na...@spicycrypto.ca>.
Erik Huelsmann wrote:
> On 3/28/07, Nathan Kidd <na...@spicycrypto.ca> wrote:
>> > Shame there is not a script to take care of this...
>>
>> Actually, this would be very simple to write a script for.
>>
>> for (each repository)
>> for (each revision)
>> dump > dumpdir/{revisiondate}-{revisionnumber}-{reponame}.dump
>> done
>> done
>>
>> for (alphabetically sorted list of files in dumpdir)
>> load < {file}
>> done
>
> Are you sure all copy sources will be ordered correctly that way?
Hmmm, no, I'm not sure. That pseudo code was the product of 5s thought
on getting the order right, but I don't understand the dump/load
internals enough to know if there would be side-effects. It would be an
interesting experiment though...
-Nathan
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Erik Huelsmann <eh...@gmail.com>.
On 3/28/07, Nathan Kidd <na...@spicycrypto.ca> wrote:
> Jared Silva wrote:
> > Oefelein, Martina wrote:
> >> I think that's hard to do in a situation like Ori's where you have two
> >> repositories with 2 years commit history that you want to combine into
> >> one. You would have to dump each revision individually, and then load
> >> the dump files in date order.
> >
> > Yes, it may be tedious. However, if this is what he wants, this is
> > what he needs to do.
> >
> > Shame there is not a script to take care of this...
>
> Actually, this would be very simple to write a script for.
>
> for (each repository)
> for (each revision)
> dump > dumpdir/{revisiondate}-{revisionnumber}-{reponame}.dump
> done
> done
>
> for (alphabetically sorted list of files in dumpdir)
> load < {file}
> done
Are you sure all copy sources will be ordered correctly that way?
bye,
Erik.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Nathan Kidd <na...@spicycrypto.ca>.
Jared Silva wrote:
> Oefelein, Martina wrote:
>> I think that's hard to do in a situation like Ori's where you have two
>> repositories with 2 years commit history that you want to combine into
>> one. You would have to dump each revision individually, and then load
>> the dump files in date order.
>
> Yes, it may be tedious. However, if this is what he wants, this is
> what he needs to do.
>
> Shame there is not a script to take care of this...
Actually, this would be very simple to write a script for.
for (each repository)
for (each revision)
dump > dumpdir/{revisiondate}-{revisionnumber}-{reponame}.dump
done
done
for (alphabetically sorted list of files in dumpdir)
load < {file}
done
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Jared Silva <ja...@gmail.com>.
Oefelein, Martina wrote:
> Hi Jared,
>
> I think that's hard to do in a situation like Ori's where you have two repositories with 2 years commit history that you want to combine into one. You would have to dump each revision individually, and then load the dump files in date order.
>
> ciao
> Martina
Yes, it may be tedious. However, if this is what he wants, this is
what he needs to do.
Shame there is not a script to take care of this...
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
RE: Revisions with wrong chronologic order
Posted by "Oefelein, Martina" <Ma...@dionex.com>.
Hi Jared,
I think that's hard to do in a situation like Ori's where you have two repositories with 2 years commit history that you want to combine into one. You would have to dump each revision individually, and then load the dump files in date order.
ciao
Martina
-----Original Message-----
From: Jared Silva [mailto:jayrod@gmail.com]
Sent: Wednesday, March 28, 2007 3:46 PM
To: users
Subject: Re: Revisions with wrong chronologic order
Ori, you need to load in chronological order.
You can use datecheck.py to verify chronological order of the load.
http://svn.collab.net/repos/svn/trunk/tools/dev/datecheck.py
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Gerco Ballintijn <ge...@ballintijn.com>.
Erik Huelsmann wrote:
>
> On 3/28/07, Ori Avtalion <oa...@bezeqint.net> wrote:
>> Jared Silva wrote:
>> > Ori, you need to load in chronological order.
>>
>> Why is that important?
>> (besides the svn up -r {DATE} feature being broken for the overlapping
>> periods of time)
>
> It's not.
>
Are you sure?
I seem to recall reading two year back that the date specification
uses a binary search algorithm to find the correct revision. This
algorithm will fail if the dates are not in order. I have experienced
this as well when converting a CVS repository one module at a time...
Gerco.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Erik Huelsmann <eh...@gmail.com>.
On 3/28/07, Ori Avtalion <oa...@bezeqint.net> wrote:
> Jared Silva wrote:
> > Ori, you need to load in chronological order.
>
> Why is that important?
> (besides the svn up -r {DATE} feature being broken for the overlapping
> periods of time)
It's not.
bye,
Erik.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Jared Silva <ja...@gmail.com>.
Ori Avtalion wrote:
> Jared Silva wrote:
> > Ori, you need to load in chronological order.
>
> Why is that important?
> (besides the svn up -r {DATE} feature being broken for the overlapping
> periods of time)
For the reason you just stated...
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Ori Avtalion <oa...@bezeqint.net>.
Jared Silva wrote:
> Ori, you need to load in chronological order.
Why is that important?
(besides the svn up -r {DATE} feature being broken for the overlapping
periods of time)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Jared Silva <ja...@gmail.com>.
Ori, you need to load in chronological order.
You can use datecheck.py to verify chronological order of the load.
http://svn.collab.net/repos/svn/trunk/tools/dev/datecheck.py
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: Revisions with wrong chronologic order
Posted by Erik Huelsmann <eh...@gmail.com>.
On 3/27/07, Ori Avtalion <oa...@bezeqint.net> wrote:
> Hi,
>
> I have dumped two subversion repositories (with unrelated,
> non-overlapping trees) and then loaded them into a new repository one
> after the other.
>
> I now have a repository where revisions 1-100 are dated 2005-2007 and
> revisions 101-200 are dated 2005-2007 as well.
The repository at the ASF (apache.org) has the same thing. No problem
at all, but ...
> svn commands work, but is this a healthy, recommended state for a
> repository?
> I haven't tried it yet, but is it possible that running
> "svn up -r {DATE}" will break something?
... the {DATE} thing doesn't work correctly anymore.
> To prevent it, is there a way to merge the two dumps with correctly
> sorted revisions, before loading them into a repository?
Only if you write such a tool. I'm not aware there currently is. (This
isn't impossible though!)
bye,
Erik.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org