Revision graph functionality?

[Kylo Ginsberg <ky...@gmail.com>]

We've just converted a cvs repository to svn, and the resultant
repository is 5 GB with about 79k revs.  A number of cvs users were
accustomed to using TortoiseCVS to view revision graphs of salient
files.  Of course, this is dead easy and lightning fast with cvs since
every file's genealogy is represented in a single file in the
repository.  TortoiseSVN will produce revision graphs, but since that
requires parsing the log history for the entire repository, it is
impractically slow to put it mildly.

Although I reference the Tortoise guis as examples, my real questions
are wrt subversion proper:
* Are changes that could enable rapid construction of revision graphs
on the svn future feature list?  I imagine this would be quite an
undertaking, but it can't hurt to ask.
* Are there 3rd party scripts that will cook up revision graphs, or
perhaps, a database which could be queried dynamically to make
revision graphs? I'm imagining a tool that could be run nightly to
build the database indices.
* We're on fsfs now, but would the performance be dramatically
different with bdb?

Thanks for any comments,
Kylo

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Revision graph functionality?

[Kylo Ginsberg <ky...@gmail.com>]

On 3/22/07, Ryan Schmidt <su...@ryandesign.com> wrote:
> On Mar 22, 2007, at 23:18, Kylo Ginsberg wrote:
>
> > We've just converted a cvs repository to svn, and the resultant
> > repository is 5 GB with about 79k revs.  A number of cvs users were
> > accustomed to using TortoiseCVS to view revision graphs of salient
> > files.  Of course, this is dead easy and lightning fast with cvs since
> > every file's genealogy is represented in a single file in the
> > repository.  TortoiseSVN will produce revision graphs, but since that
> > requires parsing the log history for the entire repository, it is
> > impractically slow to put it mildly.
> >
> > Although I reference the Tortoise guis as examples, my real questions
> > are wrt subversion proper:
> > * Are changes that could enable rapid construction of revision graphs
> > on the svn future feature list?  I imagine this would be quite an
> > undertaking, but it can't hurt to ask.
> > * Are there 3rd party scripts that will cook up revision graphs, or
> > perhaps, a database which could be queried dynamically to make
> > revision graphs? I'm imagining a tool that could be run nightly to
> > build the database indices.
> > * We're on fsfs now, but would the performance be dramatically
> > different with bdb?
>
> I don't know if any of the existing tools do this, but it sounds to
> me like it should be possible for a post-commit hook to record
> relevant information about the commit in a database, and for a
> revision graph maker to query that database more quickly than
> querying the repository directly.
>
> I don't have experience with revision graphs... what does one look
> like? Can you send me a sample?
>

Hi Ryan

Here's a sample revision graph from the Tortoise site.  It's actually
kind of a weak example, since it doesn't show a heavily branched node
but it gives you an idea:

http://tortoisesvn.net/docs/release/TortoiseSVN_en/tsvn-dug-revgraph.html

It looks like the tortoise graph only follows copies/branches, but
with svn (perhaps post merge-tracking?) it could potentially follow
merges as well.  Powerful visualization aid.

And yes, a post-commit hook could commit add entries to a database
queriable by a visualization tool.  That would be slick.

Kylo

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Revision graph functionality?

[Rajko Albrecht <ra...@alwins-world.de>]

Quoting Kylo Ginsberg <ky...@gmail.com>:


>> So, what would help: a log-call where only log entries belonging to an item
>> existing at a specific revision would send.
>
> Can you elaborate?  Do you mean an svn log command which would show
> not just ancestry, but descent and "cousins" for a given file?  That
> is quite literally what one wants, but I'm not sure if that's what you
> mean or something else?

I want: svn history -p <rev> <item>

and I get log where <item> and revision <rev> belongs to. eg. copies,  
moving, modifications, deletions. Nothing else. Eg., not a range of  
revision like in "svn log" now but a range of associated items.


>> TortoiseSvn and kdesvn must get
>> the whole log, parsing it twice (from newest to oldest and back) to detect
>> copies/moves and so on.
>
> Why parsing it twice?  Once, newest to oldest to find the root parent,
> then back from the root to find all descendents?

Yes.
Newest to oldest for find the root (at this stage kdesvn modifies the  
origin log-entries that way, that in a specific entry is set where the  
item in next revision is copied to. how tortoise does it i don't know  
exactly but should be same), assigning the copies to the logentries of  
the copy-from-revision is 	necessary 'cause otherwise it isn't able to  
detect the branches.

example: /trunk/a at rev 1 is copied to /branches/dev/a on rev 20
so log for rev 20 has copied from "/trunk/a" at rev 1.
Now both clients enhance the logentry for rev 1 internal so: "copy-to  
/branches/dev/a in rev 20". you'll understand that this is only  
possible when going from newest to oldest log, eg, to addition of  
item. eg, newest to oldest must first stage.

oldest to newest is the converting/painting stuff itself (kdesvn  
produces a dot-file, tortoise paints direct)
A lot details more but them are real offtopic here I think. but the  
cost for such revision tree isn't the smalles one.


>
>> When only getting a specific subset of log....
>
> Does this sentence end: "it would be much faster"?  Of have I missed
> your point?
Yes... much faster, less memory usage, less cpu glowing. Sorry.

I'm not sure if this is possible, if this makes sense for subversion  
itself or ever should be a client side problem nor if this makes  
sense. 'Cause - traffic is the one, the server load the other side.  
And I think on a public repository like kde such a feature will kill  
the server if a lot clients tries to fetch such a history. Or  
subversion tracks this copy-to in a second database on serverside so  
the history must not parsed.


so. just my long thougts. hope, i could make a bit clear whats the  
problem behind such history feature.

but of course, if I think complete wrong, tell me ;)

Rajko

Re: Revision graph functionality?

[Kylo Ginsberg <ky...@gmail.com>]

On 3/23/07, Rajko Albrecht <ra...@alwins-world.de> wrote:
> Am Freitag, 23. März 2007 schrieb Ryan Schmidt:
> > I don't know if any of the existing tools do this, but it sounds to
> > me like it should be possible for a post-commit hook to record
> > relevant information about the commit in a database, and for a
> > revision graph maker to query that database more quickly than
> > querying the repository directly.
> >
> > I don't have experience with revision graphs... what does one look
> > like? Can you send me a sample?
>
>
> http://www.alwins-world.de/wiki/programs/kdesvn/ScreenShots, the last image is
> such an example, too.
>
> The problem behind is: you must fetch it. With my tests with kdesvn it shows,
> the most time costs the fetch. Parsing that bunch of revisions isn't a
> problem. I'd somettimes tried for testing generating a graph of a item in dke
> repository with at this time more then 500k entries, this is impossible. You
> have wait hours getting all logs.

Yes, this was my understanding -- the log retrieval time is where most
of the time is spent.

> So, what would help: a log-call where only log entries belonging to an item
> existing at a specific revision would send.

Can you elaborate?  Do you mean an svn log command which would show
not just ancestry, but descent and "cousins" for a given file?  That
is quite literally what one wants, but I'm not sure if that's what you
mean or something else?

>TortoiseSvn and kdesvn must get
> the whole log, parsing it twice (from newest to oldest and back) to detect
> copies/moves and so on.

Why parsing it twice?  Once, newest to oldest to find the root parent,
then back from the root to find all descendents?

> When only getting a specific subset of log....

Does this sentence end: "it would be much faster"?  Of have I missed your point?

> Hooks into a database isn't real usable for both clients, 'cause them have to

Agreed, not a solution that would support multiple clients (unless it
exposed its own api, but this could be prohibitively elaborate).

Thanks,
Kylo

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Revision graph functionality?

[Rajko Albrecht <ra...@alwins-world.de>]

Am Freitag, 23. März 2007 schrieb Ryan Schmidt:
> I don't know if any of the existing tools do this, but it sounds to
> me like it should be possible for a post-commit hook to record
> relevant information about the commit in a database, and for a
> revision graph maker to query that database more quickly than
> querying the repository directly.
>
> I don't have experience with revision graphs... what does one look
> like? Can you send me a sample?


http://www.alwins-world.de/wiki/programs/kdesvn/ScreenShots, the last image is 
such an example, too.

The problem behind is: you must fetch it. With my tests with kdesvn it shows, 
the most time costs the fetch. Parsing that bunch of revisions isn't a 
problem. I'd somettimes tried for testing generating a graph of a item in dke 
repository with at this time more then 500k entries, this is impossible. You 
have wait hours getting all logs.

So, what would help: a log-call where only log entries belonging to an item 
existing at a specific revision would send. TortoiseSvn and kdesvn must get 
the whole log, parsing it twice (from newest to oldest and back) to detect 
copies/moves and so on. When only getting a specific subset of log....

Hooks into a database isn't real usable for both clients, 'cause them have to 
know about. (but for the users, of course, it is, thats why the answer to 
this post and not to the answer of Kylo)

Hope, its now a little bit clear whats the problem behind, but I think its 
a "nice to have" not a "must be".

bye

Rajko

Re: Revision graph functionality?

[Ryan Schmidt <su...@ryandesign.com>]

On Mar 22, 2007, at 23:18, Kylo Ginsberg wrote:

> We've just converted a cvs repository to svn, and the resultant
> repository is 5 GB with about 79k revs.  A number of cvs users were
> accustomed to using TortoiseCVS to view revision graphs of salient
> files.  Of course, this is dead easy and lightning fast with cvs since
> every file's genealogy is represented in a single file in the
> repository.  TortoiseSVN will produce revision graphs, but since that
> requires parsing the log history for the entire repository, it is
> impractically slow to put it mildly.
>
> Although I reference the Tortoise guis as examples, my real questions
> are wrt subversion proper:
> * Are changes that could enable rapid construction of revision graphs
> on the svn future feature list?  I imagine this would be quite an
> undertaking, but it can't hurt to ask.
> * Are there 3rd party scripts that will cook up revision graphs, or
> perhaps, a database which could be queried dynamically to make
> revision graphs? I'm imagining a tool that could be run nightly to
> build the database indices.
> * We're on fsfs now, but would the performance be dramatically
> different with bdb?

I don't know if any of the existing tools do this, but it sounds to  
me like it should be possible for a post-commit hook to record  
relevant information about the commit in a database, and for a  
revision graph maker to query that database more quickly than  
querying the repository directly.

I don't have experience with revision graphs... what does one look  
like? Can you send me a sample?


-- 

To reply to the mailing list, please use your mailer's Reply To All  
function


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org