You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Greg Stein <gs...@gmail.com> on 2009/07/21 19:25:25 UTC
State of the WC-NG World
Hey all,
Over the past couple months, I haven't had a chance to work on wc-ng
due to travel and getting married and whatnot. Hyrum has been pulling
all the weight of wc-ng lately. But I'm trying to dig back in and
continue pushing it forward.
Right now, two of the big goals in the work are:
1) remove all usage of svn_wc_entry_t
2) remove all usage of svn_wc_adm_access_t
The entry_t structure needs to disappear because it doesn't model the
working copy very well. We've been using APIs from wc_db.h internally,
and we do not (yet) have any kind of analogue to entry_t in the svn_wc
API. We may end up making a very limited structure available, or
create APIs like wc_db where there are lots of OUT parameters rather
than a structure (which allows for optional data requests). We've been
pushing along on the assumption of "no replacement" and will see where
that goes.
The access_t structure is disappearing because we no longer require
per-directory locks/state/cache to operate on a working copy. It is
being replaced with svn_wc_context_t which is constructed once and
used for all operations across all directories and even across
distinct working copies. Essentially, a process can create one at
startup and use it for all calls.
Once we get rid of the entry_t and access_t usage, then we'll have
removed the primary vestiges of the old working copy, and will be move
things around "underneath" to have a single .svn subdirectory at the
root of a working copy (rather than a .svn per directory).
Another big change is removing the old "loggy" mechanism in favor of a
sqlite-based queue of operations to perform (essentially where we need
to "transact" changes across sqlite and the filesystem).
libsvn_wc will always have some references to entry_t and access_t for
backwards compatibility purposes (to support the old APIs). However,
as a rough measure of how much additional work is needed, the # uses
of each data type in libsvn_wc are:
svn_wc_entry_t 246
svn_wc_adm_access_t 475
And they should be removed entirely in libsvn_client, but the counts are:
svn_wc_entry_t 110
svn_wc_adm_access_t 218
The overall point here is that a lot of work is still ahead of us. I
also have to contend with a move from Brussels back to the USA over
the next two months. I believe that we're going to be busy on this
until at least the end of October.
For 1.6, we started to hold back new feature development and
stabilized the codbase starting in early November. There was a lot of
tree conflict work to finish, and so 1.6 took four months to
stabilize, wrap, and ship (in early March). I don't think that we're
going to need that much time to wrap since the wc-ng work "should be"
done before we begin the wrap process. I would guess that we could
ship 1.7 right before Christmas, or maybe right after New Year's.
Part of the reason that I'm writing this is to discuss whether that
timeframe works. We had a sorta goal of "6 month release cycle", but
this would be about 9 months.
Thoughts? Concerns?
Cheers,
-g
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2373285
Re: State of the WC-NG World
Posted by "Hyrum K. Wright" <hy...@hyrumwright.org>.
On Jul 21, 2009, at 7:11 PM, Gavin 'Beau' Baumanis wrote:
>
> While ultimately related, I apologise for "kind of / sot of"
> hijacking the thread.
>
>
> On 22/07/2009, at 06:19 , Hyrum K. Wright wrote:
>
>> On Jul 21, 2009, at 12:25 PM, Greg Stein wrote:
>>
>>> Hey all,
>>>
>>> Over the past couple months, I haven't had a chance to work on wc-ng
>>> due to travel and getting married and whatnot. Hyrum has been
>>> pulling
>>> all the weight of wc-ng lately. But I'm trying to dig back in and
>>> continue pushing it forward.
>>>
>>> Right now, two of the big goals in the work are:
>>> 1) remove all usage of svn_wc_entry_t
>>> 2) remove all usage of svn_wc_adm_access_t
>>>
>>
>> Getting rid of the adm_access batons and entry_t usages is fairly
>> straightforward at this point, and lends itself to a hight degree of
>> parallelization. I'd like to see as many people as possible get
>> involved. It would help get people aware of the wc_db APIs, and also
>> help us get to 1.7 sooner.
>
> Is any of this work "relatively" simple?
> Ultimately, I'm asking if there is a low enough barrier to allow non-
> committers (well perhaps its more along the lines of non-
> technically aware people - as opposed to the "rights" of commit or
> otherwise) the ability to help here at any level?
Some of the transformations are pretty straight-forward, but others
are can be more complex. The trouble is the difference isn't really
that apparent at a glance. The simple cases are pretty straight-
forward, and non-technical people (with appropriate review) would be
able to handle them. What I'm scared is somebody getting mired down
in one of the complex cases.
That being said, there appears to be little response to this thread
from people willing to help, so any additional work would be
welcomed. (I'm tempted to ping users@ and troll for folks, but I
don't have the time to review the plethora of patches that may create.)
>>> Part of the reason that I'm writing this is to discuss whether that
>>> timeframe works. We had a sorta goal of "6 month release cycle", but
>>> this would be about 9 months.
>>>
>>> Thoughts? Concerns?
>
>
> I realise that my opinion doesn't necessarily carry a lot of weight
> in the overall scheme of things (being relatively new and ignorant
> to the history of the project and certainly ignorant technically) -
> but none the less;
> As an end-user, I'd much rather see point releases that address
> known show-stopper bugs and allow wc-ng the appropriate time for
> release - certainly more so than anyone feeling "pressured" into
> releasing it in any set timeframe.
> I'm not saying we shouldn't have goals / or a plan... but since the
> wc-ng is an architectural change surely, (rational-thinking) people
> aren't going to get annoyed that the community's self-imposed
> release cycle isn't strictly followed.
>
> 1.5 with its merge / re-integration changes was another such large
> project whereby the release was longer than expected. But I most
> certainly appreciate the functionality it has brought to SVN for me
> to use.
Hopefully we learned our lesson from the 1.5 release (see the paper
linked to from HACKING). The WC-NG bits that are going into 1.7 are
actually a small subset of what *could* happen. Greg and I have had
several discussions about what could be, after we get the WC-NG
architectural changes in place. The current work is designed to be
"stable" from a backward compat perspective, since we are going to
have to support it for a long while to come.
> And I don't presume that there is any such "scope" - but if there
> are simple changes that could be done by "anyone" - how many people
> are there lurking on the mailing lists, thinking,
> "I wish I could so something to help. Almost anything.
> But the time requirements to reach the appropriate technical
> knowledge to place me in a position to be genuinely helpful as
> opposed to consuming more time way from the project that I produce
> for it - is simply just too great."
>
> I don't have an answer - and there may well not even be one - but
> I'd certainly volunteer for any task that doesn't have a massive
> technical pre-requisite.
Go for it. I'm happy to review wc-ng-related patches that come in.
-Hyrum
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2376656
Re: State of the WC-NG World
Posted by Gavin Baumanis <ga...@thespidernet.com>.
While ultimately related, I apologise for "kind of / sot of"
hijacking the thread.
On 22/07/2009, at 06:19 , Hyrum K. Wright wrote:
> On Jul 21, 2009, at 12:25 PM, Greg Stein wrote:
>
>> Hey all,
>>
>> Over the past couple months, I haven't had a chance to work on wc-ng
>> due to travel and getting married and whatnot. Hyrum has been pulling
>> all the weight of wc-ng lately. But I'm trying to dig back in and
>> continue pushing it forward.
>>
>> Right now, two of the big goals in the work are:
>> 1) remove all usage of svn_wc_entry_t
>> 2) remove all usage of svn_wc_adm_access_t
>>
>
> Getting rid of the adm_access batons and entry_t usages is fairly
> straightforward at this point, and lends itself to a hight degree of
> parallelization. I'd like to see as many people as possible get
> involved. It would help get people aware of the wc_db APIs, and also
> help us get to 1.7 sooner.
Is any of this work "relatively" simple?
Ultimately, I'm asking if there is a low enough barrier to allow non-
committers (well perhaps its more along the lines of non-technically
aware people - as opposed to the "rights" of commit or otherwise) the
ability to help here at any level?
>> Part of the reason that I'm writing this is to discuss whether that
>> timeframe works. We had a sorta goal of "6 month release cycle", but
>> this would be about 9 months.
>>
>> Thoughts? Concerns?
I realise that my opinion doesn't necessarily carry a lot of weight in
the overall scheme of things (being relatively new and ignorant to the
history of the project and certainly ignorant technically) - but none
the less;
As an end-user, I'd much rather see point releases that address known
show-stopper bugs and allow wc-ng the appropriate time for release -
certainly more so than anyone feeling "pressured" into releasing it in
any set timeframe.
I'm not saying we shouldn't have goals / or a plan... but since the
wc-ng is an architectural change surely, (rational-thinking) people
aren't going to get annoyed that the community's self-imposed release
cycle isn't strictly followed.
1.5 with its merge / re-integration changes was another such large
project whereby the release was longer than expected. But I most
certainly appreciate the functionality it has brought to SVN for me to
use.
And I don't presume that there is any such "scope" - but if there are
simple changes that could be done by "anyone" - how many people are
there lurking on the mailing lists, thinking,
"I wish I could so something to help. Almost anything.
But the time requirements to reach the appropriate technical knowledge
to place me in a position to be genuinely helpful as opposed to
consuming more time way from the project that I produce for it - is
simply just too great."
I don't have an answer - and there may well not even be one - but I'd
certainly volunteer for any task that doesn't have a massive technical
pre-requisite.
Gavin.
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2373979
Re: State of the WC-NG World
Posted by "Hyrum K. Wright" <hy...@hyrumwright.org>.
On Jul 21, 2009, at 12:25 PM, Greg Stein wrote:
> Hey all,
>
> Over the past couple months, I haven't had a chance to work on wc-ng
> due to travel and getting married and whatnot. Hyrum has been pulling
> all the weight of wc-ng lately. But I'm trying to dig back in and
> continue pushing it forward.
>
> Right now, two of the big goals in the work are:
> 1) remove all usage of svn_wc_entry_t
> 2) remove all usage of svn_wc_adm_access_t
>
> The entry_t structure needs to disappear because it doesn't model the
> working copy very well. We've been using APIs from wc_db.h internally,
> and we do not (yet) have any kind of analogue to entry_t in the svn_wc
> API. We may end up making a very limited structure available, or
> create APIs like wc_db where there are lots of OUT parameters rather
> than a structure (which allows for optional data requests). We've been
> pushing along on the assumption of "no replacement" and will see where
> that goes.
>
> The access_t structure is disappearing because we no longer require
> per-directory locks/state/cache to operate on a working copy. It is
> being replaced with svn_wc_context_t which is constructed once and
> used for all operations across all directories and even across
> distinct working copies. Essentially, a process can create one at
> startup and use it for all calls.
>
> Once we get rid of the entry_t and access_t usage, then we'll have
> removed the primary vestiges of the old working copy, and will be move
> things around "underneath" to have a single .svn subdirectory at the
> root of a working copy (rather than a .svn per directory).
Getting rid of the adm_access batons and entry_t usages is fairly
straightforward at this point, and lends itself to a hight degree of
parallelization. I'd like to see as many people as possible get
involved. It would help get people aware of the wc_db APIs, and also
help us get to 1.7 sooner.
> Another big change is removing the old "loggy" mechanism in favor of a
> sqlite-based queue of operations to perform (essentially where we need
> to "transact" changes across sqlite and the filesystem).
>
> libsvn_wc will always have some references to entry_t and access_t for
> backwards compatibility purposes (to support the old APIs). However,
> as a rough measure of how much additional work is needed, the # uses
> of each data type in libsvn_wc are:
> svn_wc_entry_t 246
> svn_wc_adm_access_t 475
>
> And they should be removed entirely in libsvn_client, but the counts
> are:
> svn_wc_entry_t 110
> svn_wc_adm_access_t 218
>
>
> The overall point here is that a lot of work is still ahead of us. I
> also have to contend with a move from Brussels back to the USA over
> the next two months. I believe that we're going to be busy on this
> until at least the end of October.
>
> For 1.6, we started to hold back new feature development and
> stabilized the codbase starting in early November. There was a lot of
> tree conflict work to finish, and so 1.6 took four months to
> stabilize, wrap, and ship (in early March). I don't think that we're
> going to need that much time to wrap since the wc-ng work "should be"
> done before we begin the wrap process. I would guess that we could
> ship 1.7 right before Christmas, or maybe right after New Year's.
>
> Part of the reason that I'm writing this is to discuss whether that
> timeframe works. We had a sorta goal of "6 month release cycle", but
> this would be about 9 months.
>
> Thoughts? Concerns?
We also need to handle moving pristine storage around, and a few under-
the-hood things like moving properties into the Sqlite DB and making
the format upgrade code a bit more robust.
-Hyrum
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2373372
Re: State of the WC-NG World
Posted by Stefan Sperling <st...@elego.de>.
On Tue, Jul 21, 2009 at 09:25:25PM +0200, Greg Stein wrote:
> Thoughts? Concerns?
No concerns here (I knew we were not gonna get it done within
6 months :P), just a request.
I'd like you wc-ng guys to help HuiHuang a little.
I know this may sound like extra work on top of what you're
already going through but it's really needed.
The work he has been doing so far will be obsoleted by wc-ng.
I've asked him in #svn-dev today whether he'd rather continue
his task in wc-ng, and he said yes.
He has built up familiarity with the code base over the last
few weeks. But he will still need assistance with figuring
out how his problem space (generate a single commit from multiple
working copies) applies to wc-ng.
I bet you'll say "this is a solved problem" since the design
of the db is aimed towards dealing with multiple disjoint
working copies anyway. But that's not the point.
The point is that huihaung could shift his focus from trying
to get this to work with access batons to starting to fill
in the bits which are still missing until we can actually run
svn commit wc1/ wc2/
with wc-ng. That is, I'd like to see him working on the
wc-ng transition of the commit code.
As his gsoc mentor, I should be the one advising him on this,
but I'm afraid I don't have the knowledge right now to help him.
Since hwright keeps bugging me about doing something with tree
conflicts and wc-ng (whatever that means), I'll probably build
up at least a bit of domain knowledge in the coming weeks, so
I'll try to make this as little of an extra burden for Hyrum and
you as I can.
I'd expect huihuang to ask a few questions about wc-ng soon, and
you answering them on this list would be highly appreciated.
It might also help others who don't know much about wc-ng yet.
Stefan
Re: State of the WC-NG World
Posted by Mark Phippard <ma...@gmail.com>.
On Tue, Jul 21, 2009 at 3:25 PM, Greg Stein<gs...@gmail.com> wrote:
> The overall point here is that a lot of work is still ahead of us. I
> also have to contend with a move from Brussels back to the USA over
> the next two months. I believe that we're going to be busy on this
> until at least the end of October.
>
> For 1.6, we started to hold back new feature development and
> stabilized the codbase starting in early November. There was a lot of
> tree conflict work to finish, and so 1.6 took four months to
> stabilize, wrap, and ship (in early March). I don't think that we're
> going to need that much time to wrap since the wc-ng work "should be"
> done before we begin the wrap process. I would guess that we could
> ship 1.7 right before Christmas, or maybe right after New Year's.
>
> Part of the reason that I'm writing this is to discuss whether that
> timeframe works. We had a sorta goal of "6 month release cycle", but
> this would be about 9 months.
>
> Thoughts? Concerns?
I do not think doing a release from the current code is really an
option because of the performance regression, so all I can see us
doing is continue to move forward as best we can.
I see that Hyrum has sort of addressed what I was going to ask, but I
was just going to say what can be done to get more people involved? I
am sure I will be asked to get Paul and Mike involved in this to help
speed things along. Besides being busy fixing merge issues (Paul) and
working on the HTTPv2 protocol (Mike), I think neither one of them
really knows how to get started on helping. I am sure there are other
committers who feel the same. So maybe we need to make a more
concerted effort to bring in additional help and guide them on how to
get started?
--
Thanks
Mark Phippard
http://markphip.blogspot.com/
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=462&dsMessageId=2373388