You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Stefan Sperling <st...@elego.de> on 2011/09/05 11:41:53 UTC

question for FSFS gurus (was: Re: FSFS successor ID design draft)

On Thu, Sep 01, 2011 at 12:06:40AM +0200, Stefan Sperling wrote:
> On Tue, Aug 30, 2011 at 12:43:29PM +0200, Stefan Sperling wrote:
> > I'll try to tweak my proposal such that successor ID updates become
> > transactional and happen as part of every commit.
> 
> Here's a first shot at this. Comments welcome.

FSFS gurus:

Are any of you looking at this?
Do you think this is worth writing a prototype implementation for?

I have so far only received feedback from danielsh. This makes me very
happy but if anyone with a couple more years of FSFS experience under
their belt could comment I would be even happier.

Thanks.

Re: question for FSFS gurus (was: Re: FSFS successor ID design draft)

Posted by Ivan Zhakov <iv...@visualsvn.com>.
On Mon, Sep 5, 2011 at 21:55, Stefan Sperling <st...@elego.de> wrote:
> On Mon, Sep 05, 2011 at 07:38:51PM +0200, Bert Huijben wrote:
>> > Also slightly OT (no FSFS-guruness here), but I think another
>> > important use-case is being able to quickly answer the question "in
>> > which revision was $URL@$REV deleted?" Or "give me the log of
>> > $URL@$REV up and until it was deleted."
>>
>> svn_ra_get_deleted_rev(), which answers this question was introduced in 1.6.
>> (But I don't know where it is used.)
>
> The guts of this are in svn_repos_deleted_rev().
> It runs a binary search across a specific revision range to find the
> delete event. During this search it repeatedly scans history backwards
> to find a related node and has to employ a complicated set of rules to
> deal with copies and replacements.
>
> This could be simplified with successor support. Just scan successors
> upwards from the start revision until either the end revision is reached
> or the node disappears. The only complication are multiple copies of the
> node. If more than one copy event is found they will all need to be
> scanned upwards but only one will match the node we're looking for.
>
It looks like just performance optimization. We can do it later, after
fixing tree-conflicts on moves.

-- 
Ivan Zhakov

Re: question for FSFS gurus (was: Re: FSFS successor ID design draft)

Posted by Stefan Sperling <st...@elego.de>.
On Mon, Sep 05, 2011 at 07:38:51PM +0200, Bert Huijben wrote:
> > Also slightly OT (no FSFS-guruness here), but I think another
> > important use-case is being able to quickly answer the question "in
> > which revision was $URL@$REV deleted?" Or "give me the log of
> > $URL@$REV up and until it was deleted."
> 
> svn_ra_get_deleted_rev(), which answers this question was introduced in 1.6.
> (But I don't know where it is used.)

The guts of this are in svn_repos_deleted_rev().
It runs a binary search across a specific revision range to find the
delete event. During this search it repeatedly scans history backwards
to find a related node and has to employ a complicated set of rules to
deal with copies and replacements.

This could be simplified with successor support. Just scan successors
upwards from the start revision until either the end revision is reached
or the node disappears. The only complication are multiple copies of the
node. If more than one copy event is found they will all need to be
scanned upwards but only one will match the node we're looking for.

RE: question for FSFS gurus (was: Re: FSFS successor ID design draft)

Posted by Bert Huijben <be...@qqmail.nl>.

> -----Original Message-----
> From: Johan Corveleyn [mailto:jcorvel@gmail.com]
> Sent: maandag 5 september 2011 13:15
> To: Ivan Zhakov; dev@subversion.apache.org
> Subject: Re: question for FSFS gurus (was: Re: FSFS successor ID design
draft)
> 
> On Mon, Sep 05, 2011 at 01:46:09PM +0400, Ivan Zhakov wrote:
> [...]
> > I'm not FSFS guru, but I still feel that FSFS successor ID doesn't
> > worth to be implemented because there is no strong reasons/usage for
> > it. For me it looks like bottom-up design approach.
> 
> Also slightly OT (no FSFS-guruness here), but I think another
> important use-case is being able to quickly answer the question "in
> which revision was $URL@$REV deleted?" Or "give me the log of
> $URL@$REV up and until it was deleted."

svn_ra_get_deleted_rev(), which answers this question was introduced in 1.6.
(But I don't know where it is used.)

> This question comes up in practice once in a while (has been asked a
> couple of times on the users-list, and to me personally by some of my
> dev-colleagues). The workaround is usually to script around it for
> example by doing a "log -v" of an ancestor and looking for the first
> deletion after $REV (or something similar).
> 
> When it is suggested that this kind of search could also be
> implemented as a feature inside svn (even if it's slow, at least it
> would be part of the core), that request is always refused by saying
> that it wouldn't be performant enough (and consequently eat too much
> resources of an SVN server). I hope an FSFS successor ID storage could
> help in this regard.

	Bert
> 
> --
> Johan


Re: question for FSFS gurus (was: Re: FSFS successor ID design draft)

Posted by Johan Corveleyn <jc...@gmail.com>.
On Mon, Sep 05, 2011 at 01:46:09PM +0400, Ivan Zhakov wrote:
[...]
> I'm not FSFS guru, but I still feel that FSFS successor ID doesn't
> worth to be implemented because there is no strong reasons/usage for
> it. For me it looks like bottom-up design approach.

Also slightly OT (no FSFS-guruness here), but I think another
important use-case is being able to quickly answer the question "in
which revision was $URL@$REV deleted?" Or "give me the log of
$URL@$REV up and until it was deleted."

This question comes up in practice once in a while (has been asked a
couple of times on the users-list, and to me personally by some of my
dev-colleagues). The workaround is usually to script around it for
example by doing a "log -v" of an ancestor and looking for the first
deletion after $REV (or something similar).

When it is suggested that this kind of search could also be
implemented as a feature inside svn (even if it's slow, at least it
would be part of the core), that request is always refused by saying
that it wouldn't be performant enough (and consequently eat too much
resources of an SVN server). I hope an FSFS successor ID storage could
help in this regard.

-- 
Johan

Re: question for FSFS gurus (was: Re: FSFS successor ID design draft)

Posted by Stefan Sperling <st...@elego.de>.
On Mon, Sep 05, 2011 at 12:12:31PM +0200, Stefan Sperling wrote:
> > Anyway we can implement top-level part of handling moves and then
> > optimize it using FSFS successor ID storage or something else.
> 
> We need the bottom parts first.
> It won't perform well enough without successor-IDs.

I should have been clearer:

"It" in my quote above is not the algorithm I posted pseudo-code for.
"It" is a different algorithm that finds nodes which were deleted in
the history of the target branch of a merge (again,
see http://svn.haxx.se/dev/archive-2011-08/0624.shtml).

Re: question for FSFS gurus (was: Re: FSFS successor ID design draft)

Posted by Stefan Sperling <st...@elego.de>.
On Mon, Sep 05, 2011 at 01:57:51PM +0400, Ivan Zhakov wrote:
> No, I think that auto-resolving tree-conflicts involving moves is most
> important task for Subversion 1.8. But I feel it could be implemented
> without implementing FSFS successor ID storage. It seems that
> algorithm that you posted could be reversed.

I have caused some confusion here, sorry.
Contrary to how I initially presented it, That algorithm has *nothing*
to do with successor IDs.  

Please see this post where I corrected my mistake:
http://svn.haxx.se/dev/archive-2011-08/0624.shtml

> Anyway we can implement top-level part of handling moves and then
> optimize it using FSFS successor ID storage or something else.

We need the bottom parts first.
It won't perform well enough without successor-IDs.

Re: question for FSFS gurus (was: Re: FSFS successor ID design draft)

Posted by Ivan Zhakov <iv...@visualsvn.com>.
On Mon, Sep 5, 2011 at 14:43, Daniel Shahaf <da...@elego.de> wrote:
> Ivan Zhakov wrote on Mon, Sep 05, 2011 at 13:57:51 +0400:
>> On Mon, Sep 5, 2011 at 13:52, Stefan Sperling <st...@elego.de> wrote:
>> > On Mon, Sep 05, 2011 at 01:46:09PM +0400, Ivan Zhakov wrote:
>> >> On Mon, Sep 5, 2011 at 13:41, Stefan Sperling <st...@elego.de> wrote:
>> >> > On Thu, Sep 01, 2011 at 12:06:40AM +0200, Stefan Sperling wrote:
>> >> >> On Tue, Aug 30, 2011 at 12:43:29PM +0200, Stefan Sperling wrote:
>> >> >> > I'll try to tweak my proposal such that successor ID updates become
>> >> >> > transactional and happen as part of every commit.
>> >> >>
>> >> >> Here's a first shot at this. Comments welcome.
>> >> >
>> >> > FSFS gurus:
>> >> >
>> >> > Are any of you looking at this?
>> >> > Do you think this is worth writing a prototype implementation for?
>> >> >
>> >> > I have so far only received feedback from danielsh. This makes me very
>> >> > happy but if anyone with a couple more years of FSFS experience under
>> >> > their belt could comment I would be even happier.
>> >> >
>> >> I'm not FSFS guru, but I still feel that FSFS successor ID doesn't
>> >> worth to be implemented because there is no strong reasons/usage for
>> >> it. For me it looks like bottom-up design approach.
>> >
>> > Hmmm... you don't think that auto-resolving tree-conflicts involving
>> > moves during merges is worth implementing?
>> >
>> No, I think that auto-resolving tree-conflicts involving moves is most
>> important task for Subversion 1.8. But I feel it could be implemented
>> without implementing FSFS successor ID storage. It seems that
>> algorithm that you posted could be reversed.
>>
>
> What do you mean by 'reversed'?
>
I mean to iterate over added_nodes (instead of deleted_nodes).

-- 
Ivan Zhakov

Re: question for FSFS gurus (was: Re: FSFS successor ID design draft)

Posted by Daniel Shahaf <da...@elego.de>.
Ivan Zhakov wrote on Mon, Sep 05, 2011 at 13:57:51 +0400:
> On Mon, Sep 5, 2011 at 13:52, Stefan Sperling <st...@elego.de> wrote:
> > On Mon, Sep 05, 2011 at 01:46:09PM +0400, Ivan Zhakov wrote:
> >> On Mon, Sep 5, 2011 at 13:41, Stefan Sperling <st...@elego.de> wrote:
> >> > On Thu, Sep 01, 2011 at 12:06:40AM +0200, Stefan Sperling wrote:
> >> >> On Tue, Aug 30, 2011 at 12:43:29PM +0200, Stefan Sperling wrote:
> >> >> > I'll try to tweak my proposal such that successor ID updates become
> >> >> > transactional and happen as part of every commit.
> >> >>
> >> >> Here's a first shot at this. Comments welcome.
> >> >
> >> > FSFS gurus:
> >> >
> >> > Are any of you looking at this?
> >> > Do you think this is worth writing a prototype implementation for?
> >> >
> >> > I have so far only received feedback from danielsh. This makes me very
> >> > happy but if anyone with a couple more years of FSFS experience under
> >> > their belt could comment I would be even happier.
> >> >
> >> I'm not FSFS guru, but I still feel that FSFS successor ID doesn't
> >> worth to be implemented because there is no strong reasons/usage for
> >> it. For me it looks like bottom-up design approach.
> >
> > Hmmm... you don't think that auto-resolving tree-conflicts involving
> > moves during merges is worth implementing?
> >
> No, I think that auto-resolving tree-conflicts involving moves is most
> important task for Subversion 1.8. But I feel it could be implemented
> without implementing FSFS successor ID storage. It seems that
> algorithm that you posted could be reversed.
> 

What do you mean by 'reversed'?

> Anyway we can implement top-level part of handling moves and then
> optimize it using FSFS successor ID storage or something else.
> 
> 
> -- 
> Ivan Zhakov

Re: question for FSFS gurus (was: Re: FSFS successor ID design draft)

Posted by Ivan Zhakov <iv...@visualsvn.com>.
On Mon, Sep 5, 2011 at 13:52, Stefan Sperling <st...@elego.de> wrote:
> On Mon, Sep 05, 2011 at 01:46:09PM +0400, Ivan Zhakov wrote:
>> On Mon, Sep 5, 2011 at 13:41, Stefan Sperling <st...@elego.de> wrote:
>> > On Thu, Sep 01, 2011 at 12:06:40AM +0200, Stefan Sperling wrote:
>> >> On Tue, Aug 30, 2011 at 12:43:29PM +0200, Stefan Sperling wrote:
>> >> > I'll try to tweak my proposal such that successor ID updates become
>> >> > transactional and happen as part of every commit.
>> >>
>> >> Here's a first shot at this. Comments welcome.
>> >
>> > FSFS gurus:
>> >
>> > Are any of you looking at this?
>> > Do you think this is worth writing a prototype implementation for?
>> >
>> > I have so far only received feedback from danielsh. This makes me very
>> > happy but if anyone with a couple more years of FSFS experience under
>> > their belt could comment I would be even happier.
>> >
>> I'm not FSFS guru, but I still feel that FSFS successor ID doesn't
>> worth to be implemented because there is no strong reasons/usage for
>> it. For me it looks like bottom-up design approach.
>
> Hmmm... you don't think that auto-resolving tree-conflicts involving
> moves during merges is worth implementing?
>
No, I think that auto-resolving tree-conflicts involving moves is most
important task for Subversion 1.8. But I feel it could be implemented
without implementing FSFS successor ID storage. It seems that
algorithm that you posted could be reversed.

Anyway we can implement top-level part of handling moves and then
optimize it using FSFS successor ID storage or something else.


-- 
Ivan Zhakov

Re: question for FSFS gurus (was: Re: FSFS successor ID design draft)

Posted by Stefan Sperling <st...@elego.de>.
On Mon, Sep 05, 2011 at 01:46:09PM +0400, Ivan Zhakov wrote:
> On Mon, Sep 5, 2011 at 13:41, Stefan Sperling <st...@elego.de> wrote:
> > On Thu, Sep 01, 2011 at 12:06:40AM +0200, Stefan Sperling wrote:
> >> On Tue, Aug 30, 2011 at 12:43:29PM +0200, Stefan Sperling wrote:
> >> > I'll try to tweak my proposal such that successor ID updates become
> >> > transactional and happen as part of every commit.
> >>
> >> Here's a first shot at this. Comments welcome.
> >
> > FSFS gurus:
> >
> > Are any of you looking at this?
> > Do you think this is worth writing a prototype implementation for?
> >
> > I have so far only received feedback from danielsh. This makes me very
> > happy but if anyone with a couple more years of FSFS experience under
> > their belt could comment I would be even happier.
> >
> I'm not FSFS guru, but I still feel that FSFS successor ID doesn't
> worth to be implemented because there is no strong reasons/usage for
> it. For me it looks like bottom-up design approach.

Hmmm... you don't think that auto-resolving tree-conflicts involving
moves during merges is worth implementing?

Re: question for FSFS gurus (was: Re: FSFS successor ID design draft)

Posted by Ivan Zhakov <iv...@visualsvn.com>.
On Mon, Sep 5, 2011 at 13:41, Stefan Sperling <st...@elego.de> wrote:
> On Thu, Sep 01, 2011 at 12:06:40AM +0200, Stefan Sperling wrote:
>> On Tue, Aug 30, 2011 at 12:43:29PM +0200, Stefan Sperling wrote:
>> > I'll try to tweak my proposal such that successor ID updates become
>> > transactional and happen as part of every commit.
>>
>> Here's a first shot at this. Comments welcome.
>
> FSFS gurus:
>
> Are any of you looking at this?
> Do you think this is worth writing a prototype implementation for?
>
> I have so far only received feedback from danielsh. This makes me very
> happy but if anyone with a couple more years of FSFS experience under
> their belt could comment I would be even happier.
>
I'm not FSFS guru, but I still feel that FSFS successor ID doesn't
worth to be implemented because there is no strong reasons/usage for
it. For me it looks like bottom-up design approach.

-- 
Ivan Zhakov