Subversion Vision and Roadmap Proposal

["C. Michael Pilato" <cm...@collab.net>]

Last week, five of Subversion's developers -- myself, Hyrum Wright (who
helped me author this lengthy mail), Greg Stein, Stefan Sperling, and Karl
Fogel ("we", henceforth) -- met in New York City to evaluate Subversion's
current trajectory as a project and to jointly develop suggestions for any
course correction deemed necessary.  The unfortunate impetus for the meeting
was feedback from representatives of some very large installations of
Subversion that, from where they sat, the Subversion project was stagnating.
 This is a somewhat fair assertion.  It's not that the community is or has
ever been in danger of doing nothing.  But even when we march in step
towards a goal, we don't do a good job of communicating outwardly about that
fact, leading to the appearance of inactivity.  And lately, our ongoing
efforts have been less like a team of folks working in concert towards a
common vision, and more like the individual piddlings common to mature open
source software.

Make no mistake:  Subversion is mature open source software.  It works well
for open source projects and -- in one of the biggest software coups of the
last decade -- we've found that it's also good enough to supplant other
well-established version control systems, open-source or otherwise, inside
the highly structured walls of the enterprise.  Unfortunately, "good enough"
still leaves the enterprise user base frustrated at the most inopportune
times (like, say, when managing branches near releases).  And if you judge
success by mindshare, it's now clear the DVCS tools have rendered our "good
enough" somewhat obsolete when it comes to serving many open source
projects, even.

VISION

Subversion has no future as a DVCS tool.  Let's just get that out there.  At
least two very successful such tools exist already, and to squeeze another
horse into that race would be a poor investment of energy and talent.
What's more, huge classes of users remain categorically opposed to the very
tenets on which the DVCS systems are based.  They need centralization.  They
need control.  They need meaningful path-based authorization.  They need
simplicity.  In short, they desperately need Subversion.  It's this class of
user -- the corporate developer -- that stands to benefit hugely from what
Subversion brings to the party.  And it's that class of user which we
believe Subversion should cater to, ideally without ostracizing the open
source volunteers who remain our largest source of development investment.

Corporate developers come in all shapes and sizes, from self-administering
10-person teams to geographically-distributed organizations of thousands of
developers on hundreds of teams serviced by dedicated IT departments.
Subversion can't be all things to all people, but it can be a well-defined
subset of things to most people.  It shouldn't sacrifice its trademark
simplicity when growing the features most likely to be used in large-scale
deployments; it shouldn't optimize for the large enterprise in a way that
will alienate the long tail composed of much smaller deployments.  By
defining the subset of things Subversion is and those it is not, we
recognize our own boundaries and prevent years of wandering through the
proverbial wilderness of feature creep.

Someone wiser than I once said, "Where there is no vision, the people
perish."  So recognizing the benefits that Subversion already offers, and
projecting a bit into the future what we'd like to see Subversion become, we
offer the following vision statement for your review:

   Subversion exists to be universally recognized and adopted as an
   open-source, centralized version control system characterized by its
   reliability as a safe haven for valuable data; the simplicity of its
   model and usage; and its ability to support the needs of a wide variety
   of users and projects, from individuals to large-scale enterprise
   operations.

A shorter, business-card-sized motto (offered as a replacement to the
obsolete "A compelling replacement for CVS") might be:  "Enterprise-class
centralized version control for the masses".

ROADMAP

With that vision in mind, we identified a number of high-value improvements
which Subversion should gain in coming releases.  Then we took a casual pass
at assigning some technical "plumbing" dependencies for each of these.
Here's what we came up with, in the form "FEATURE: [DEPENDENCY ...]", where
"FS-NG" = major FS backend overhaul, "WC-NG" = WC-NG, and "Ev2" = svn_editor_t:

    * Obliterate:  FS-NG
    * Shelve/Checkpoint:  WC-NG
    * Repository-dictated Configuration:  WC-NG (?)
    * Rename Tracking:  WC-NG, Ev2, FS-NG (?)
    * Improved Merging:  WC-NG
    * Improved Tree Conflict Handling:  WC-NG, Ev2, Rename Tracking
    * Enterprise Authentication Mechanisms:
    * Forward History Searching:  FS-NG (?)
    * Log Message Templates:  Repository-dictated Configuration

By examining this dependency chain, factoring in the development momentum
which will exist around WC-NG, and admitting that some of the major plumbing
overhauls not currently underway may prove to be just as costly as the WC-NG
work (if not more so), we believe that the following feature roadmap is one
which will serve Subversion users well:

   1.7: WC-NG; HTTPv2; 'svn patch'; typical slew of various bug fixes

   1.8: repository-dictated configuration; tree conflicts improvements;
        WC-NG-enabled stuff (rename tracking, compressed pristines,
        shelving/checkpointing, ...)

   1.9: Editor v2 (for server->client rename handling improvements)

   [...]

   2.0: FS-NG (ideally a parallelized task), with some (to-be-identified)
        FS-NG enabled features.

That last item is likely to be contentious, so it bears explaining.  We
believe that our current two filesystem offerings are stifling innovation.
On the one hand we have the BDB backend which is a breeze to develop for but
is only occasional used; on the other is the far-more-popular FSFS backend
whose fundamental principles so constrain the system as to destroy much hope
of serious design overhaul; and in the middle lies the feature parity
requirement we've been living under thus far which binds Subversion to the
union of the two backends' weaknesses.  We confidently assert that to break
system-wide compatibility with a so-called 2.0 release will be Subversion's
great overall detriment, both from the perspective of efficient use of
development energy and in user adoption.  So we propose that an
as-yet-fictional Subversion 2.0 be allowed to break compatibility with the
1.x line only in ways which can be mitigated using the RA layer as a
compatibility shim.  Additionally, we believe that merely releasing a 2.0
with a new FS backend but without new user-visible features enabled by that
overhaul will be ill-received.

COMMUNITY

We also discussed matters of communication and community.  Subversion has
historically had a very tight-knit community of developers, and we've
provided a resource for communicating with users through the various mailing
lists.  With the increase in corporate involvement, and the changing roles
(and employers!) of committers, the Subversion ecosystem can at times seems
a bit fractured.  To an enterprise manager responsible for thousands of
users, and millions of dollars of investment, and billions of bytes of data,
this fracturing appears as a liability when considering an investment in
Subversion.  Most corporations understand the open source nature of
Subversion's development method (indeed, this very thing has driven
Subversion adoption), but they still want a unified face when it comes to
communication, planning, and project feedback.

One proposed solution is a Subversion "planet", to be hosted at a
re-purposed subversion.org.  The planet would aggregate feeds from
individual contributors, as well as the various corporate entities
interested in Subversion development.  While various commercial interests
(CollabNet, WANdisco, elego, etc.) may compete in some areas, they are all
committed to improving Subversion as a whole.  Enterprise users need to see
this unity across Subversion's corporate sponsors, and a communication
stream which interleaves these corporate voices works toward that end.

Whatever the solution, we need a defined plan which we then communicate to
the end users and customers who are deploying Subversion installations.

But communication alone isn't enough.  We need to find ways to grow our
developer community itself.  Attendance at the recent Subversion Corporation
Annual Members Meeting was low (by design and recommendation, of course),
but a startling realization was made there.  When the agenda item for voting
new full committers into membership was on the table, there were no
candidates.  Think about that:  no new full committers for Subversion in the
past year.  This is a bad thing.  We need to find a way to embrace and
empower would-be Subversion contributors.  Telling them to troll through the
issue tracker looking for bite-sized issues is not the way to do that -- it
communicates "we can't be bothered to mentor you".  When somebody wants to
start making contributions, our community must recognize the value of that
contributor and mentor him or her toward full committership, for the good of
that contributor and of the community.  Further, our public roadmap needs to
highlight the items that we really wish we could be working on but lack the
manpower to handle.  This would provide those looking for ways to accelerate
Subversion's roadmap with some cues for doing that in harmony with the Big
Picture.

SUMMARY

I've covered a lot of ground here, but decisions in this community have
always been and will be made on this mailing list, and they deserve to be
made with as much information as possible.  You now know where a small
contingent of developers stand on these issues.  I'd like to publicize on
our project website a *community-endorsed* vision and roadmap ASAP, and then
get to the business of moving Subversion forward along those lines.

So what say you?

-- C-Mike, for the NYC conference attendees

-- 
C. Michael Pilato <cm...@collab.net>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand

Re: Subversion Vision and Roadmap Proposal

[Vadim Chekan <ko...@gmail.com>]

Yes, I do use commit hooks and permissions. But it is fixing a
problem. And why should we fix a problem if it could be avoided?
The notion of a "snapshot" is clearly understood by everybody, so why
not to introduce it as a first class feature?

On Fri, Apr 9, 2010 at 9:18 PM, Alexey Neyman <st...@att.net> wrote:
> Have you looked at pre-commit hooks that may serve that purpose? I think
> svnperms.py may be what you're looking for - just disallow all operations
> except additions on 'tags/*', and disallow all operations on 'tags/*/*'.
>
> Hope that helps,
> Alexey.
>
> On Friday 09 April 2010 05:56:44 pm Vadim Chekan wrote:
>> On Fri, Apr 9, 2010 at 11:31 AM, B Smith-Mannschott
>>
>> <bs...@gmail.com> wrote:
>> > On Fri, Apr 9, 2010 at 10:13, Vadim Chekan <ko...@gmail.com> wrote:
>> >> P.S. Pleeeease, introduce true tags, no more "lets pretend this copy is
>> >> a tag".
>> >
>> > What's a "true" tag? What's it good for? How would it behave?
>>
>> Tag is named snapshot. When I do release, I want to tag the code I've
>> just released so that nobody can modify it.
>> I want to be sure that in any moment in the future, I can checkout
>> "tags/product-1.0" and I will get exactly what has been released and I
>> want a guarantee that no genius "fixed" something in tags/
>>
>> Vadim.
>>
>



-- 
>From RFC 2631: In ASN.1, EXPLICIT tagging is implicit unless IMPLICIT
is explicitly specified

Re: Subversion Vision and Roadmap Proposal

[Vadim Chekan <ko...@gmail.com>]

Yes, I do use commit hooks and permissions. But it is fixing a
problem. And why should we fix a problem if it could be avoided?
The notion of a "snapshot" is clearly understood by everybody, so why
not to introduce it as a first class feature?

On Fri, Apr 9, 2010 at 9:18 PM, Alexey Neyman <st...@att.net> wrote:
> Have you looked at pre-commit hooks that may serve that purpose? I think
> svnperms.py may be what you're looking for - just disallow all operations
> except additions on 'tags/*', and disallow all operations on 'tags/*/*'.
>
> Hope that helps,
> Alexey.
>
> On Friday 09 April 2010 05:56:44 pm Vadim Chekan wrote:
>> On Fri, Apr 9, 2010 at 11:31 AM, B Smith-Mannschott
>>
>> <bs...@gmail.com> wrote:
>> > On Fri, Apr 9, 2010 at 10:13, Vadim Chekan <ko...@gmail.com> wrote:
>> >> P.S. Pleeeease, introduce true tags, no more "lets pretend this copy is
>> >> a tag".
>> >
>> > What's a "true" tag? What's it good for? How would it behave?
>>
>> Tag is named snapshot. When I do release, I want to tag the code I've
>> just released so that nobody can modify it.
>> I want to be sure that in any moment in the future, I can checkout
>> "tags/product-1.0" and I will get exactly what has been released and I
>> want a guarantee that no genius "fixed" something in tags/
>>
>> Vadim.
>>
>



-- 

Re: Subversion Vision and Roadmap Proposal

[Greg Stein <gs...@gmail.com>]

On Mon, Apr 12, 2010 at 13:14, Mike Dixon
<mi...@denovosoftware.com> wrote:
>...
> And so we have identified one of the problems.
>
> If you're serious about getting new people involved in development, getting
> a patch reviewed needs to be *easy*. If someone sends in a patch and it gets
> ignored (not maliciously, of course), it's possible that the submitter is so
> dedicated that they'll keep sending emails until someone pays attention.
> This certainly does happen sometimes. However, it's much more likely that
> they'll just decide that nobody's interested, and leave.
>
> Every patch that rots is a potential new developer that you've turned away
> at the door. If it seems like there's no time to do patch review, think how
> much less time there will be if Subversion continues to grow, but no new
> developers join.

I strongly agree, and that was the one, primary thing that I raised at
our meeting. The attempt to try a "mentor" is a way/attempt to be more
active against leaving these developers/patches lying around.

Cheers,
-g

Re: Subversion Vision and Roadmap Proposal

[Mike Dixon <mi...@denovosoftware.com>]

C. Michael Pilato wrote:
> When the agenda item for voting new full committers into membership
> was on the table, there were no candidates.  Think about that:  no
> new full committers for Subversion in the past year.  This is a bad
> thing.  We need to find a way to embrace and empower would-be
> Subversion contributors.

Stefan Sperling wrote:
> If you have a strong interest in getting your patches reviewed, they
> will eventually be reviewed. If they pass review they will be
> committed (committing isn't the hard part -- review is the hard
> part). It takes some degree of patience and persistence on the
> submitter's behalf. Maybe a bit much of it, which is certainly not
> ideal, because it makes drive-by bugfixing quite hard. But if the
> submitter simply gives up then the patch is very likely to get
> ignored.

And so we have identified one of the problems.

If you're serious about getting new people involved in development, 
getting a patch reviewed needs to be *easy*. If someone sends in a patch 
and it gets ignored (not maliciously, of course), it's possible that the 
submitter is so dedicated that they'll keep sending emails until someone 
pays attention. This certainly does happen sometimes. However, it's much 
more likely that they'll just decide that nobody's interested, and leave.

Every patch that rots is a potential new developer that you've turned 
away at the door. If it seems like there's no time to do patch review, 
think how much less time there will be if Subversion continues to grow, 
but no new developers join.

-Mike

Re: Subversion Vision and Roadmap Proposal

[Stefan Sperling <st...@elego.de>]

On Sat, Apr 10, 2010 at 11:38:22PM -0700, Alexey Neyman wrote:
> 2. In the same issue, there is a reference to keyword substitution 
> implementation used by FreeBSD. Again, no further description as to why this 
> implementation was not acceptable, etc.

There was, in this thread:
http://svn.haxx.se/dev/archive-2008-07/0113.shtml
If that thread isn't linked from the issue, it should be.
Could you add the link if it's not yet present there? Thanks!

> Given these two examples, I am somewhat afraid to invest my time into learning 
> Subversion internals and coming up with such changes only to have them rot in 
> the issue tracker indefinitely.

There's no reason to be afraid.
If you have a strong interest in getting your patches reviewed, they will
eventually be reviewed. If they pass review they will be committed
(committing isn't the hard part -- review is the hard part).
It takes some degree of patience and persistence on the submitter's behalf.
Maybe a bit much of it, which is certainly not ideal, because it makes
drive-by bugfixing quite hard. But if the submitter simply gives up
then the patch is very likely to get ignored. And if patches do get
ignored, the reason is not malicious intent of Subversion's developers. 
The reason is that the patch failed to stay above the threshold of
developer attention, amidst the flood of other things developers need
to (and are expected to) spend their time on. There are more things
to do than people actively working on the code can reasonably handle
concurrently, so some sort of selection is bound to happen. But if you
are willing to invest time and energy into making sure your contributions
do not get lost, you will be rewarded for it (I know this first-hand
because I've gone through the same process before I became a developer).
The closer you get involved with the developer community the easier
submitting patches will be. E.g. it's a good idea to hang out in
the #svn-dev channel on Freenode to get an idea of who the developers
are, who is working on what, and who might have time or be interested
in reviewing your submissions.

Taking the FreeBSD custom-keywords example, this rots in the issue tracker
because the submitter himself said it wasn't ready for submission yet
(see thread I linked above). The patch is used locally at FreeBSD,
but they themselves consider it a hack. However, they seem to be happy
enough with it as is, and aren't interested in (or capable of, e.g. due
to lack of time) doing any additional work to get it into upstream Subversion.
It seems they'd rather hack their patch into the subversion 1.7.x FreeBSD
port when 1.7.x is released, rather than getting it into Subversion trunk
now. And that's unfortunate, but if they are happy with maintaining their
patch this way rather than pushing it upstream, we're not gonna try to
force them to push it upstream.

To get this work integrated upstream, someone (either an existing
Subversion developer, or a contributer) needs to take the patch,
port it to trunk, make sure it's sane and fit for general purpose use,
and commit/submit the result of that work. And that takes time.
If you have the time, please do so.

Thanks,
Stefan

Re: Subversion Vision and Roadmap Proposal

[Alexey Neyman <st...@att.net>]

Hyrum,

On Saturday 10 April 2010 03:24:27 pm Hyrum K. Wright wrote:
> > Even CVS had the ability to (a) customize keyword expansion and (b)
> > provide custom log message templates. These features, as far as I
> > remember, were first
> > targeted as "pre-1.0" and are postponed ever since.
> 
> Well, this feature has only been postponed because nobody has written it
> yet.  Many of us would welcome patches which implement this, would you like
> to work on it?  (Or, alternatively, provide development resources within
> your organization to help make this feature a reality?)

Have you looked at issue #890? 

1. It has patches for the first step in implementing this feature (well, 
against Subversion 1.2.0). There is no response as to whether these patches 
have been rejected (and the reason why), or they are plainly ignored.

2. In the same issue, there is a reference to keyword substitution 
implementation used by FreeBSD. Again, no further description as to why this 
implementation was not acceptable, etc.

Given these two examples, I am somewhat afraid to invest my time into learning 
Subversion internals and coming up with such changes only to have them rot in 
the issue tracker indefinitely.

Best regards,
Alexey.

Re: Subversion Vision and Roadmap Proposal

[Greg Stein <gs...@gmail.com>]

On Sat, Apr 10, 2010 at 18:04, Alexey Neyman <st...@att.net> wrote:
>...
> I am not against standardizing most common approach. I just think it would be
> wasteful to focus on the issues that could be easily dealt with by configuring
> the repository.
>
> Here's an idea: have 'svnadmin create' populate the hooks that will perform
> enforcement of some of the desired models. E.g.:
>
> $ svnadmin create --model=none <PATH>
> [default] Creates an empty hooks area, allows for the flexible configuration
>
> $ svnadmin create --model=ttb <PATH>
> Creates /{trunk,tags,branches} directories in the repo, adds hook scripts that
> enforce tag immutability.
>
> $ svnadmin create --model=project_ttb <PATH
> Same as above, for the /{PROJECT1,PROJECT2}/{trunk,tags,branches} layout.
>
> These should provide good starting point for most projects, then the
> repository admin could fine-tune the settings as he likes it.

hahaha... it is very funny that you spec'd the above. I *implemented*
something almost exactly like the above, way WAY back in the first
year of svn development. It provided templates for both hooks and an
initial repository [layout] commit. We ripped it out soon thereafter
because it was just too easy for an external script to do this kind of
work, rather than building it into Subversion itself. :-P

IOW, yes I agree with the concept. I'm probably still in agreement
with the others who asked me to pull the feature: a simple script can
manage the above.

>...

Cheers,
-g

Re: Subversion Vision and Roadmap Proposal

["Hyrum K. Wright" <hy...@mail.utexas.edu>]

On Sat, Apr 10, 2010 at 11:04 PM, Alexey Neyman <st...@att.net> wrote:

> Yet, one the core features for a centralized VCS is postponed once again:
>
> =====================
>  1.8: repository-dictated configuration
> =====================
>
> Even CVS had the ability to (a) customize keyword expansion and (b) provide
> custom log message templates. These features, as far as I remember, were
> first
> targeted as "pre-1.0" and are postponed ever since.
>

Well, this feature has only been postponed because nobody has written it
yet.  Many of us would welcome patches which implement this, would you like
to work on it?  (Or, alternatively, provide development resources within
your organization to help make this feature a reality?)

The more practical reason why is is being postponed is two-fold: firstly, we
need a better data storage model to handle the information the repository
sends down, beyond just log message templates and such.  This requires
WC-NG, which itself is a major undertaking and is currently the prime
feature for 1.7.  The second reason is related to the first: if we included
this (or any other wc-ng-dependent feature) in 1.7, it would only delay it
farther, which wouldn't do *anybody* any good.  So, repos-dictated config is
probably going to be a little while longer, but you can still help speed up
the process.

-Hyrum

Re: Subversion Vision and Roadmap Proposal

[Alexey Neyman <st...@att.net>]

On Saturday 10 April 2010 01:57:32 pm Vadim Chekan wrote:
> On Fri, Apr 9, 2010 at 9:37 PM, Greg Stein <gs...@gmail.com> wrote:
> > On Sat, Apr 10, 2010 at 00:18, Alexey Neyman <st...@att.net> wrote:
> >> Have you looked at pre-commit hooks that may serve that purpose? I think
> >> svnperms.py may be what you're looking for - just disallow all
> >> operations except additions on 'tags/*', and disallow all operations on
> >> 'tags/*/*'.
> >
> > I think people are asking about formalizing the trunks/tags/branches
> > ("TTB") concepts (or some appropriate variant), and adding some
> > built-in technical enforcement of the concepts. Then some further
> > layering of merge handling within that restricted environment.
> >
> > These kinds of changes would be quite welcome, I believe. "If you want
> > solid merge-tracking, both performant and easy, then do XYZ. There are
> > more open/flexible approaches, but those approaches come with issues."
> 
> Thanks Greg, exactly what I mean.
> I *do* like flexibility: life is crazy and you never know what
> configuration you'll need in your next project. So it would be good to
> preserver the current svn ability to work with any folder as with TTB.
> If possible.
> BUT: I want svn to reflect the way 99% of programmers work 99% of their
>  time. If explicit branching can help to simplify merging algorithms,
>  great. Tagging can prevent newbies (or absent minded pros) from modifying
>  released code - great!

I am not against standardizing most common approach. I just think it would be 
wasteful to focus on the issues that could be easily dealt with by configuring 
the repository.

Here's an idea: have 'svnadmin create' populate the hooks that will perform 
enforcement of some of the desired models. E.g.:

$ svnadmin create --model=none <PATH>
[default] Creates an empty hooks area, allows for the flexible configuration

$ svnadmin create --model=ttb <PATH>
Creates /{trunk,tags,branches} directories in the repo, adds hook scripts that 
enforce tag immutability.

$ svnadmin create --model=project_ttb <PATH
Same as above, for the /{PROJECT1,PROJECT2}/{trunk,tags,branches} layout.

These should provide good starting point for most projects, then the 
repository admin could fine-tune the settings as he likes it.

Why I don't see these issues as a high priority? Let me quote from the first 
email in this thread:

=====================
.... What's more, huge classes of users remain categorically opposed to the 
very tenets on which the DVCS systems are based.  They need centralization.  
They need control.  They need meaningful path-based authorization....
=====================

Yet, one the core features for a centralized VCS is postponed once again:

=====================
 1.8: repository-dictated configuration
=====================

Even CVS had the ability to (a) customize keyword expansion and (b) provide 
custom log message templates. These features, as far as I remember, were first 
targeted as "pre-1.0" and are postponed ever since.

Best regards,
Alexey.

Re: Subversion Vision and Roadmap Proposal

[Vadim Chekan <ko...@gmail.com>]

On Fri, Apr 9, 2010 at 9:37 PM, Greg Stein <gs...@gmail.com> wrote:
> On Sat, Apr 10, 2010 at 00:18, Alexey Neyman <st...@att.net> wrote:
>> Have you looked at pre-commit hooks that may serve that purpose? I think
>> svnperms.py may be what you're looking for - just disallow all operations
>> except additions on 'tags/*', and disallow all operations on 'tags/*/*'.
>
> I think people are asking about formalizing the trunks/tags/branches
> ("TTB") concepts (or some appropriate variant), and adding some
> built-in technical enforcement of the concepts. Then some further
> layering of merge handling within that restricted environment.
>
> These kinds of changes would be quite welcome, I believe. "If you want
> solid merge-tracking, both performant and easy, then do XYZ. There are
> more open/flexible approaches, but those approaches come with issues."

Thanks Greg, exactly what I mean.
I *do* like flexibility: life is crazy and you never know what
configuration you'll need in your next project. So it would be good to
preserver the current svn ability to work with any folder as with TTB.
If possible.
BUT: I want svn to reflect the way 99% of programmers work 99% of their time.
If explicit branching can help to simplify merging algorithms, great.
Tagging can prevent newbies (or absent minded pros) from modifying
released code - great!

Vadim.

Re: Subversion Vision and Roadmap Proposal

[Greg Stein <gs...@gmail.com>]

On Sat, Apr 10, 2010 at 00:18, Alexey Neyman <st...@att.net> wrote:
> Have you looked at pre-commit hooks that may serve that purpose? I think
> svnperms.py may be what you're looking for - just disallow all operations
> except additions on 'tags/*', and disallow all operations on 'tags/*/*'.

I think people are asking about formalizing the trunks/tags/branches
("TTB") concepts (or some appropriate variant), and adding some
built-in technical enforcement of the concepts. Then some further
layering of merge handling within that restricted environment.

These kinds of changes would be quite welcome, I believe. "If you want
solid merge-tracking, both performant and easy, then do XYZ. There are
more open/flexible approaches, but those approaches come with issues."

I would like to add a very large nugget of history to this conversation:

Lest anybody say "damn. you guys fucked up with that design. should
have been more restricted from the start, so we wouldn't be in this
mess." Please recall that in the year 2000, when we started, the
version control of the day was CVS. NONE of these issues were anywhere
on the horizon. We set out to create a completely open and flexible
system, devoid of specific policy. We didn't know what would work
best. We didn't know what people needed or how they wanted to mold svn
to their particular development workflow. So we left it open, on
purpose. Even TTB was regarded as just a helpful layout.

Ten years later? Yup. We have a ton of input. We can fix a lot of
these things, and let people opt into these new solutions.

Cheers,
-g

Re: Subversion Vision and Roadmap Proposal

[Alexey Neyman <st...@att.net>]

Have you looked at pre-commit hooks that may serve that purpose? I think 
svnperms.py may be what you're looking for - just disallow all operations 
except additions on 'tags/*', and disallow all operations on 'tags/*/*'.

Hope that helps,
Alexey.

On Friday 09 April 2010 05:56:44 pm Vadim Chekan wrote:
> On Fri, Apr 9, 2010 at 11:31 AM, B Smith-Mannschott
> 
> <bs...@gmail.com> wrote:
> > On Fri, Apr 9, 2010 at 10:13, Vadim Chekan <ko...@gmail.com> wrote:
> >> P.S. Pleeeease, introduce true tags, no more "lets pretend this copy is
> >> a tag".
> >
> > What's a "true" tag? What's it good for? How would it behave?
> 
> Tag is named snapshot. When I do release, I want to tag the code I've
> just released so that nobody can modify it.
> I want to be sure that in any moment in the future, I can checkout
> "tags/product-1.0" and I will get exactly what has been released and I
> want a guarantee that no genius "fixed" something in tags/
> 
> Vadim.
> 

RE: Subversion Vision and Roadmap Proposal

[Bob Archer <Bo...@amsi.com>]

> On Fri, Apr 9, 2010 at 10:13, Vadim Chekan <ko...@gmail.com> wrote:
> > P.S. Pleeeease, introduce true tags, no more "lets pretend this copy is
> a tag".
> 
> What's a "true" tag? What's it good for? How would it behave?

I'm not Vadim but...

Basically an attribute added to a path at a certain revisions. I would think of it sort of like an alias to a revision number. Rather than be a copied path that people can check stuff into. Also, you should be able to float the tag so it stays at head... or move if from one revision to another. 


> CVS, on the other hand implemented tags in a fashion that was very
> different. Tags were effectively like labels that you could apply to
> particular versions of particular files. You can  tag incrementally,
> marking files as they became "ready". This approach is file-centric
> and doesn't mesh well with the atomic, whole-tree model of versioning
> presented by Svn, Hg, Git, Bzr.

well, this is like I described above, but instead of attaching to a file it is attached to a revision. 

Having this would allow me to get a "tag" for example...

svn co ^/MyProject tag="Release 1.0.0" or
svn export ^/MyProject tag="Release 1.0.0"

svn would simply figure out that this was /MyProject/trunk@1234 for example. 

This would also mean if someone tried to commit a co from a tag they would get a "Your not up to date dummy" message.

BOb

Re: Subversion Vision and Roadmap Proposal

[Vadim Chekan <ko...@gmail.com>]

On Fri, Apr 9, 2010 at 11:31 AM, B Smith-Mannschott
<bs...@gmail.com> wrote:
> On Fri, Apr 9, 2010 at 10:13, Vadim Chekan <ko...@gmail.com> wrote:
>> P.S. Pleeeease, introduce true tags, no more "lets pretend this copy is a tag".
>
> What's a "true" tag? What's it good for? How would it behave?

Tag is named snapshot. When I do release, I want to tag the code I've
just released so that nobody can modify it.
I want to be sure that in any moment in the future, I can checkout
"tags/product-1.0" and I will get exactly what has been released and I
want a guarantee that no genius "fixed" something in tags/

Vadim.

Re: Subversion Vision and Roadmap Proposal

[Stefan Sperling <st...@elego.de>]

On Fri, Apr 09, 2010 at 10:29:30PM +0100, Stefan Sperling wrote:
> On Fri, Apr 09, 2010 at 04:39:08PM -0400, Mark Mielke wrote:
> > Branch is similar. If I want to set to the point on trunk at which
> > branches/2.0 was branched, how do I do this?
> > In GIT, it's just "git checkout master", "git reset --hard branches/2.0".
> 
> Find the revision at which the branch was created using 
> "svn log" as above. Then copy the old version of the branch
> on top of the current version:
> 
> svn rm ^/branch
> svn copy ^/trunk@12345 ^/branch

Sorry, it seems the above example was wrong.
Correction:

Assuming you want to reset trunk to the state it had at r12345
(because this is the revision your hypothetical 2.0 branch was
created in), do:

 svn rm ^/trunk
 svn copy ^/trunk@12345 ^/trunk

RE: Subversion Vision and Roadmap Proposal

[Bob Archer <Bo...@amsi.com>]

> The trick is to use the --stop-on-copy (to stop at branch point)
> -v (to show patsh) options of the svn log command:
> 
> $ svn log --stop-on-copy -v .
> ------------------------------------------------------------------------
> r3 | stsp | 2010-04-09 22:25:24 +0100 (Fri, 09 Apr 2010) | 1 line
> Changed paths:
>    M /branch/beta
> 
> a commit on the branch
> ------------------------------------------------------------------------
> r2 | stsp | 2010-04-09 22:24:57 +0100 (Fri, 09 Apr 2010) | 1 line
> Changed paths:
>    A /branch (from /trunk:1)
> 
> creating branch
> ------------------------------------------------------------------------
> $
> 
> So /branch was copied from /trunk@1.
> 
> > Branch is similar. If I want to set to the point on trunk at which
> > branches/2.0 was branched, how do I do this?
> > In GIT, it's just "git checkout master", "git reset --hard
> branches/2.0".
> 
> Find the revision at which the branch was created using
> "svn log" as above. Then copy the old version of the branch
> on top of the current version:
> 
> svn rm ^/branch
> svn copy ^/trunk@12345 ^/branch

Wouldn't this cause a folder/path already exists error?

BOb

Re: Subversion Vision and Roadmap Proposal

[Stefan Sperling <st...@elego.de>]

On Fri, Apr 09, 2010 at 04:39:08PM -0400, Mark Mielke wrote:
> In fact, I've found it a bit difficult to reliable say "what
> revision of trunk was this tag created from?" I think the
> information is in the copy-from attribute in the underlying FSFS
> commit, but the last time I tried to get access to it, I was unable
> to find it.

The trick is to use the --stop-on-copy (to stop at branch point)
-v (to show patsh) options of the svn log command:

$ svn log --stop-on-copy -v .        
------------------------------------------------------------------------
r3 | stsp | 2010-04-09 22:25:24 +0100 (Fri, 09 Apr 2010) | 1 line
Changed paths:
   M /branch/beta

a commit on the branch
------------------------------------------------------------------------
r2 | stsp | 2010-04-09 22:24:57 +0100 (Fri, 09 Apr 2010) | 1 line
Changed paths:
   A /branch (from /trunk:1)

creating branch
------------------------------------------------------------------------
$ 

So /branch was copied from /trunk@1.

> Branch is similar. If I want to set to the point on trunk at which
> branches/2.0 was branched, how do I do this?
> In GIT, it's just "git checkout master", "git reset --hard branches/2.0".

Find the revision at which the branch was created using 
"svn log" as above. Then copy the old version of the branch
on top of the current version:

svn rm ^/branch
svn copy ^/trunk@12345 ^/branch

See also http://svnbook.red-bean.com/nightly/en/svn.branchmerge.basicmerging.html#svn.branchmerge.basicmerging.undo

Stefan

Re: Subversion Vision and Roadmap Proposal

[Mark Mielke <ma...@mark.mielke.cc>]

On 04/09/2010 02:31 PM, B Smith-Mannschott wrote:
> What I still don't know is this: how would "true tags" behave, if
> Subversion had them? What problems would that enable me to solve more
> conveniently than I can now?
>    

I don't know what the original poster meant - but something to consider:

DVCS systems like GIT let one use the "branch" or "tag" as a version 
identifier. For example, I can say "set me to the version of 
component/trunk that was tagged with 2.0.1". I don't think you can do 
this in Subversion - or not as one step anyways.

In fact, I've found it a bit difficult to reliable say "what revision of 
trunk was this tag created from?" I think the information is in the 
copy-from attribute in the underlying FSFS commit, but the last time I 
tried to get access to it, I was unable to find it.

Branch is similar. If I want to set to the point on trunk at which 
branches/2.0 was branched, how do I do this?

In GIT, it's just "git checkout master", "git reset --hard branches/2.0".

Is there some part of Subversion that would give this capability that I 
have missed?

Cheers,
mark

Re: Subversion Vision and Roadmap Proposal

[B Smith-Mannschott <bs...@gmail.com>]

On Fri, Apr 9, 2010 at 10:13, Vadim Chekan <ko...@gmail.com> wrote:
> P.S. Pleeeease, introduce true tags, no more "lets pretend this copy is a tag".

What's a "true" tag? What's it good for? How would it behave?

Subversion and the DVCs mean more or less the same thing when they say
"tag", modulo the fact that an Subversion repository can hold more
than one project. In all of these systems a tag is effectively 'very
much like to a branch, but never changed after creation'.

Subversion implements these as cheap copies, but they could just as
well be implemented as a file of path+revision. (Akin to how Mercurial
implements tags).

CVS, on the other hand implemented tags in a fashion that was very
different. Tags were effectively like labels that you could apply to
particular versions of particular files. You can  tag incrementally,
marking files as they became "ready". This approach is file-centric
and doesn't mesh well with the atomic, whole-tree model of versioning
presented by Svn, Hg, Git, Bzr.

I suppose you could simulate some of the tags-as-labels use cases with
properties or by copying files from one place in the repository to
another as they become ready. Neither of these approaches fully
simulates what CVS means by "tag", however.

This isn't the first time this topic has come up on this list. The
discussions always seem to end in exhortations to model the solution
to your issue in a fashion compatible with what Subversion offers,
rather than fighting against it.

What I still don't know is this: how would "true tags" behave, if
Subversion had them? What problems would that enable me to solve more
conveniently than I can now?

// Ben

RE: Subversion Vision and Roadmap Proposal

[Bob Archer <Bo...@amsi.com>]

> > P.S. Pleeeease, introduce true tags, no more "lets pretend this copy
> is
> > a tag".
> 
> I agree here - branches are great, but sometimes you want a 'snapshot'
> or a 'baseline' that doesn't require a whole new directory in the repo
> (especially as the number of branches you have increases, things start
> to become less manageable - just the nature of having lots of
> things)(for example, we have 300 projects in our repo and we release
> very regularly. Creating a branch for each release would be a bit messy
> after just a few - not to mention the problems of deciding how to
> structure the repo to account for all those projects, do we put branches
> at the root and have a huge list of releases that require special naming
> conventions, or branch at the project level and miss the
> interdependencies between them. All tricky stuff for the branching
> mechanism).

I for one totally agree with this. I can see the copy command being broken up into three commands, branch, fork, tag. I think the huge win would be branch. Sure, internally it would be a copy. But... a property on the branch folder to indicated it is a branch. Also, the merge command should be modified to ensure that all merges were treated as if they are done and the root folder of the branch. 

While there is a lot of flexibility with the current scheme... with flexibility adds complexity. And, I agree with what someone else said here or on another thread. People aren't using Git/Hg because it is distributed they are using it because it is FAST and Branches and Tags are first class citizens. I am guessing most teams using Git/Hg do have a central repository that is the "one true copy" of the project which build servers and code reviews, etc all go through. I man, the popularity of this is evidenced by the popularity of GitHub.

BOb

RE: Subversion Vision and Roadmap Proposal

["Bolstridge, Andrew" <an...@intergraph.com>]

> -----Original Message-----
> From: Vadim Chekan [mailto:kot.begemot@gmail.com]
> Sent: Friday, April 09, 2010 9:14 AM
> To: Subversion Development
> Subject: Re: Subversion Vision and Roadmap Proposal
> 
> P.S. Pleeeease, introduce true tags, no more "lets pretend this copy
is
> a tag".

I agree here - branches are great, but sometimes you want a 'snapshot'
or a 'baseline' that doesn't require a whole new directory in the repo
(especially as the number of branches you have increases, things start
to become less manageable - just the nature of having lots of
things)(for example, we have 300 projects in our repo and we release
very regularly. Creating a branch for each release would be a bit messy
after just a few - not to mention the problems of deciding how to
structure the repo to account for all those projects, do we put branches
at the root and have a huge list of releases that require special naming
conventions, or branch at the project level and miss the
interdependencies between them. All tricky stuff for the branching
mechanism).

And yes, I know this has been gone over many times in the past.

Re: Subversion Vision and Roadmap Proposal

[Vadim Chekan <ko...@gmail.com>]

Thanks for great overview Michael,

I am playing with dvcs and "d" is not the point. Even if we will adopt
dvcs, it will be configured in centralized fashion. What attracts me
is:

1.  ability to create a branch locally in <5sec (big project) and
never expose it. I could do branches even for code reviews with dcvs.
And when done, I switch back to where I was. With application which
requires a lot of configuration it is a life saver to keep all
branches in the same location.

2. Merging. Still do not have too much experience with dvcs but svn
merges reached my pain tolerance threshold and I'm looking for better
option. Directory rename is a disaster when it comes to merging :(

So once again, it is not a fight between distributed and centralized.
Dcvs do allow centralized configuration and I suspect that 90% of
commercial usages will be centralized. Perhaps the main advantage is
to have local copy of the repository which opens your "wc" for super
fast local branching and let you experiment freely.

Vadim.

P.S. Pleeeease, introduce true tags, no more "lets pretend this copy is a tag".

RE: Pushmi support

["Bolstridge, Andrew" <an...@intergraph.com>]

> -----Original Message-----
> From: hyrum@hyrumwright.org [mailto:hyrum@hyrumwright.org] On Behalf
Of
> Hyrum K. Wright
> Sent: Friday, April 09, 2010 9:58 PM
> To: Stefan Sperling
> Cc: Philip Martin; Tim Starling; Greg Stein; Subversion Development
> Subject: Re: Pushmi support
> 
> The last thing I want to do is add more process (someones earlier
> comments
> about HACKING are quick illuminating), but it might be worth creating
a
> template for such a document and posting it somewhere.  I don't think
we
> need to go the full route of PEPs, but having some way to track design
> proposals with more finesse than the issue tracker would be nice.

Collabnet's TeamForge?

Re: Pushmi support

[Greg Stein <gs...@gmail.com>]

On Fri, Apr 9, 2010 at 16:58, Hyrum K. Wright
<hy...@mail.utexas.edu> wrote:
>
> On Fri, Apr 9, 2010 at 9:51 PM, Stefan Sperling <st...@elego.de> wrote:
>>
>> Or writing a design proposal, that clearly explains the intent
>> of the change, the reason why it's needed, the security implications,
>> how exactly the change will be implemented (e.g. where will the config
>> setting live that allows the new behaviour?), etc.
>> Something that helps people with loading the entire problem into
>> their brain quickly, so they can reason about it
>
> The last thing I want to do is add more process (someones earlier comments
> about HACKING are quick illuminating), but it might be worth creating a
> template for such a document and posting it somewhere.  I don't think we
> need to go the full route of PEPs, but having some way to track design
> proposals with more finesse than the issue tracker would be nice.

This is why patches are nice. Already have the format!


But yah... if it is going to be a lot of work, then discussion is at
least warranted. That's the best way to get a feeling for whether the
effort will be worth it.

In this case, I think the question "should I bother to work on this?"
was probably left hanging because the problem/solution was too vague
for people to definitively answer.

Cheers,
-g

Re: Pushmi support

["Hyrum K. Wright" <hy...@mail.utexas.edu>]

On Fri, Apr 9, 2010 at 9:51 PM, Stefan Sperling <st...@elego.de> wrote:

> Or writing a design proposal, that clearly explains the intent
> of the change, the reason why it's needed, the security implications,
> how exactly the change will be implemented (e.g. where will the config
> setting live that allows the new behaviour?), etc.
> Something that helps people with loading the entire problem into
> their brain quickly, so they can reason about it
>

The last thing I want to do is add more process (someones earlier comments
about HACKING are quick illuminating), but it might be worth creating a
template for such a document and posting it somewhere.  I don't think we
need to go the full route of PEPs, but having some way to track design
proposals with more finesse than the issue tracker would be nice.

-Hyrum

Re: Pushmi support

[Stefan Sperling <st...@elego.de>]

On Fri, Apr 09, 2010 at 03:36:10PM +0100, Philip Martin wrote:
> Tim Starling <ts...@wikimedia.org> writes:
> 
> > That was my fourth post to this list, I was referring to the other
> > three, particularly:
> >
> > <http://mail-archives.apache.org/mod_mbox/subversion-dev/201003.mbox/%3C4BAA79DF.9040506@wikimedia.org%3E>
> >
> > I could see that after being ignored on that issue, my only chance of
> > being heard would be to get to know the developers and then to take a
> > more personal approach. That's why I stayed on this list. Unfortunately
> > I haven't had much time to read it.
> 
> You were not exactly ignored, but asking for in-principle approval
> before submitting a patch is not the way the project usually works,

We do encourage people to write design proposals before writing
patches.

> particularly for something that could be a security issue.

Well, since we're well aware of the security implications
(Greg Hudons explains them well in
http://mail-archives.apache.org/mod_mbox/subversion-dev/201003.mbox/%3C1269433056.7493.527.camel@ray%3E
) I'd say this isn't a problem. We just need to make sure to
make it secure by default.
I don't see a problem in principle with providing an optional
mechanism for users like Tim to pass credential information
via selected environment variables (like SSH_AUTH_SOCK) to
tools called by hook scripts.

> > could be corrected with a patch to Subversion, possibly as short as a
> > few lines of code. I'm offerring to write the patch.
> 
> Writing the patch would be a way forward.

Or writing a design proposal, that clearly explains the intent
of the change, the reason why it's needed, the security implications,
how exactly the change will be implemented (e.g. where will the config
setting live that allows the new behaviour?), etc.
Something that helps people with loading the entire problem into
their brain quickly, so they can reason about it.

Stefan

Re: Pushmi support

[Philip Martin <ph...@wandisco.com>]

Tim Starling <ts...@wikimedia.org> writes:

> That was my fourth post to this list, I was referring to the other
> three, particularly:
>
> <http://mail-archives.apache.org/mod_mbox/subversion-dev/201003.mbox/%3C4BAA79DF.9040506@wikimedia.org%3E>
>
> I could see that after being ignored on that issue, my only chance of
> being heard would be to get to know the developers and then to take a
> more personal approach. That's why I stayed on this list. Unfortunately
> I haven't had much time to read it.

You were not exactly ignored, but asking for in-principle approval
before submitting a patch is not the way the project usually works,
particularly for something that could be a security issue.

[...]
> could be corrected with a patch to Subversion, possibly as short as a
> few lines of code. I'm offerring to write the patch.

Writing the patch would be a way forward.

-- 
Philip

Re: Pushmi support (was Re: Subversion Vision and Roadmap Proposal)

[Tim Starling <ts...@wikimedia.org>]

Alec Kloss wrote:
> On 2010-04-09 21:54, Tim Starling wrote:
> [chop]
>   
>> My problem is that Subversion is so slow, in terms of network
>> round-trips, that's it's virtually unusable in our application for tasks
>> like merging and annotating. Chia-liang Kao has done an awesome job of
>>     
> [chop]
>
> I assume you're talking about https mechanism.  Subversion is much
> faster with svn:// or svn+ssh://.
>   

We use svn+ssh. It's very slow. I can believe that https would be slower
still.

Obviously it depends on:
* The number of files in your working copy
* The number of revisions you're trying to work with
* The distance between you and the server

We have plenty of all three.

-- Tim Starling

Re: Pushmi support (was Re: Subversion Vision and Roadmap Proposal)

[Alec Kloss <al...@oracle.com>]

On 2010-04-09 21:54, Tim Starling wrote:
[chop]
> My problem is that Subversion is so slow, in terms of network
> round-trips, that's it's virtually unusable in our application for tasks
> like merging and annotating. Chia-liang Kao has done an awesome job of
[chop]

I assume you're talking about https mechanism.  Subversion is much
faster with svn:// or svn+ssh://.

-- 
Alec.Kloss@oracle.com			Oracle Middleware
PGP key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xEBD1FF14

Pushmi support (was Re: Subversion Vision and Roadmap Proposal)

[Tim Starling <ts...@wikimedia.org>]

Stefan Sperling wrote:
> On Fri, Apr 09, 2010 at 12:13:56AM -0400, Greg Stein wrote:
>   
>> On Thu, Apr 8, 2010 at 22:55, Tim Starling <ts...@wikimedia.org> wrote:
>>     
>>> C. Michael Pilato wrote:
>>>       
>>>> We need to find a way to embrace and
>>>> empower would-be Subversion contributors.
>>>>         
>>> You could start by replying to their mailing list posts. Just a thought.
>>>       
>
> Tim, could you provide a link to whichever post(s) you had in mind?
> The lists have lots of traffic, and not everyone reads every post.
> It's possible that your posts were simply overlooked or arrived at
> a time where none of those reading your posts were able to reply.
> Thanks.
>
> (In general, when referring to other posts made to the list, providing
> a link to the archives is a very good idea...)
>   

That was my fourth post to this list, I was referring to the other
three, particularly:

<http://mail-archives.apache.org/mod_mbox/subversion-dev/201003.mbox/%3C4BAA79DF.9040506@wikimedia.org%3E>

I could see that after being ignored on that issue, my only chance of
being heard would be to get to know the developers and then to take a
more personal approach. That's why I stayed on this list. Unfortunately
I haven't had much time to read it.

My problem is that Subversion is so slow, in terms of network
round-trips, that's it's virtually unusable in our application for tasks
like merging and annotating. Chia-liang Kao has done an awesome job of
providing a local cache layer for Subversion in first SVK and now
Pushmi. This would provide a good response to the many developers in our
community who are agitating for a switch to Git, except that due to a
minor deficit in Subversion itself, Pushmi is not usable for us. This
could be corrected with a patch to Subversion, possibly as short as a
few lines of code. I'm offerring to write the patch.

The Pushmi solution could then be documented on our wiki page on this topic:

http://www.mediawiki.org/wiki/Making_Subversion_faster

-- Tim Starling

Re: Subversion Vision and Roadmap Proposal

[Julian Foad <ju...@wandisco.com>]

Stefan Sperling wrote:
> On Fri, Apr 09, 2010 at 12:13:56AM -0400, Greg Stein wrote:
> > On Thu, Apr 8, 2010 at 22:55, Tim Starling <ts...@wikimedia.org> wrote:
> > > C. Michael Pilato wrote:
> > >> We need to find a way to embrace and
> > >> empower would-be Subversion contributors.
> > >
> > > You could start by replying to their mailing list posts. Just a thought.
> 
> Tim, could you provide a link to whichever post(s) you had in mind?
> The lists have lots of traffic, and not everyone reads every post.
> It's possible that your posts were simply overlooked or arrived at
> a time where none of those reading your posts were able to reply.

I read Tim's comment as a general observation, and it strikes a chord
with me.

I tend to leave in my mailbox the messages that might turn out to be of
some interest to me.  Occasionally I'll look back at the ones that
received no reply and post a follow-up or ping if nobody else has, but
all too often I don't.  Here are the most recent ones still sitting in
my in-box:

                              From: 
Alexander Kitaev
<Al...@svnkit.com>
                          Reply-to: 
Alexander.Kitaev@svnkit.com
                                To: 
dev@subversion.apache.org
                                Cc: 
Syncro SVN Client support
<su...@syncrosvnclient.com>
                           Subject: 
Subversion 1.6.9: update and status
fails on versioned obstruction.
                              Date: 
02/04/10 23:52:52 (Sat, 03 Apr 2010
00:52:52 +0200)

                              From: 
Tobias Bading <tb...@web.de>
                                To: 
dev@subversion.apache.org
                           Subject: 
Bug: Properties are lost in merges
between repositories
                              Date: 
27/03/10 14:11:13 (Sat, 27 Mar 2010
15:11:13 +0100)

                              From: 
Stefan Küng <to...@gmail.com>
                                To: 
dev@subversion.apache.org
                           Subject: 
segfault in 1.6.9 when updating
                              Date: 
02/02/10 18:36:15 (Tue, 02 Feb 2010
19:36:15 +0100

(No archive links - too much hassle.)

There were probably also some unanswered messages on subjects that I
don't even try to monitor, which I will have just deleted.

So I guess I could help by posting pings when I see this.  Of course we
encourage original posters to follow up their own messages, but that
doesn't mean we shouldn't try to follow up too.

- Julian

Re: Subversion Vision and Roadmap Proposal

[Stefan Sperling <st...@elego.de>]

On Fri, Apr 09, 2010 at 12:13:56AM -0400, Greg Stein wrote:
> On Thu, Apr 8, 2010 at 22:55, Tim Starling <ts...@wikimedia.org> wrote:
> > C. Michael Pilato wrote:
> >> We need to find a way to embrace and
> >> empower would-be Subversion contributors.
> >
> > You could start by replying to their mailing list posts. Just a thought.

Tim, could you provide a link to whichever post(s) you had in mind?
The lists have lots of traffic, and not everyone reads every post.
It's possible that your posts were simply overlooked or arrived at
a time where none of those reading your posts were able to reply.
Thanks.

(In general, when referring to other posts made to the list, providing
a link to the archives is a very good idea...)

> Well aware of that, and snarky behavior won't really endear you to anybody.
> 
> In particular, there are people who have patches for svn, and we don't
> handle them very well. In some cases, there is a high bar of
> review/change/resubmit that turns people off. In other cases, people
> are focused on whatever task they're working on, and the patches are
> left by the wayside.

One particular example is the various SASL-related patches for svnserve
which have been submitted during the last year.
None of them got it in, because the developer doing the original SASL
support is not active anymore. And it seems that none of the others
(myself definitely included) know enough about SASL to digest the patches.
I have now started to (slowly) learn about SASL with the goal to finally
handle these patches.

Stefan

RE: Subversion Vision and Roadmap Proposal

[Bob Archer <Bo...@amsi.com>]

> In particular, there are people who have patches for svn, and we don't
> handle them very well. In some cases, there is a high bar of
> review/change/resubmit that turns people off. In other cases, people
> are focused on whatever task they're working on, and the patches are
> left by the wayside. We have a Patch Manager that attempts to ensure
> we don't completely lose patches, but that is a far cry from
> *engaging* those patch developers. In other scenarios, we have several
> people all providing "input" and "direction" to the would-be
> patch-provider. That incoherency and pulls in different directions can
> also turn off potential contributors.

I wonder if the maillist method of patches and reviews is just to brittle. Is there any consideration to using a code review tool... perhaps review board or something else. This way the submitting and management of patches is more centralized than the disjointed threads on a mail list.

> We talked about these issues, and how we could provide "mentors" to
> work with people that arrive with patches. A person would be volunteer
> to work with the patcher, and provide a single point of
> contact/review.

That sounds like a good idea... but it also disrupts the distributed nature of an open source team. That mentor may get hung up with real life work or be unavailable, or in a different time zone...etc.


> Of course, do we have enough people willing and able to fill that
> role? (if that is even the best way!)

Oh yea... what I just said... you said too.

BOb

Re: Subversion Vision and Roadmap Proposal

[Greg Stein <gs...@gmail.com>]

On Thu, Apr 8, 2010 at 22:55, Tim Starling <ts...@wikimedia.org> wrote:
> C. Michael Pilato wrote:
>> We need to find a way to embrace and
>> empower would-be Subversion contributors.
>
> You could start by replying to their mailing list posts. Just a thought.

Well aware of that, and snarky behavior won't really endear you to anybody.

In particular, there are people who have patches for svn, and we don't
handle them very well. In some cases, there is a high bar of
review/change/resubmit that turns people off. In other cases, people
are focused on whatever task they're working on, and the patches are
left by the wayside. We have a Patch Manager that attempts to ensure
we don't completely lose patches, but that is a far cry from
*engaging* those patch developers. In other scenarios, we have several
people all providing "input" and "direction" to the would-be
patch-provider. That incoherency and pulls in different directions can
also turn off potential contributors.

We talked about these issues, and how we could provide "mentors" to
work with people that arrive with patches. A person would be volunteer
to work with the patcher, and provide a single point of
contact/review.

Of course, do we have enough people willing and able to fill that
role? (if that is even the best way!)

Cheers,
-g

Re: Subversion Vision and Roadmap Proposal

[Tim Starling <ts...@wikimedia.org>]

C. Michael Pilato wrote:
> We need to find a way to embrace and
> empower would-be Subversion contributors. 

You could start by replying to their mailing list posts. Just a thought.

-- Tim Starling

Re: Subversion Vision and Roadmap Proposal

["C. Michael Pilato" <cm...@collab.net>]

Hyrum K. Wright wrote:
> While there has been lots of discussion about aspects of the following,
> there really hasn't been any dissent about the core vision and other
> ideas presented below.  Given that, I propose that we massage this into
> a public, web-facing statement, and put it on the website somewhere.  I
> can start working on that next week, if nobody beats me to it.
> 
> -Hyrum

The vision statements (short and long) belong on the homepage.  I've just
added them there.

We have a roadmap page already that should accept those bits of the proposal
(including the stuff we mentioned at the conference that we'd like to do but
didn't find a roadmap slot for yet).

-- 
C. Michael Pilato <cm...@collab.net>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand

Re: Subversion Vision and Roadmap Proposal

["Hyrum K. Wright" <hy...@mail.utexas.edu>]

While there has been lots of discussion about aspects of the following,
there really hasn't been any dissent about the core vision and other ideas
presented below.  Given that, I propose that we massage this into a public,
web-facing statement, and put it on the website somewhere.  I can start
working on that next week, if nobody beats me to it.

-Hyrum


On Fri, Apr 2, 2010 at 4:28 PM, C. Michael Pilato <cm...@collab.net>wrote:

> Last week, five of Subversion's developers -- myself, Hyrum Wright (who
> helped me author this lengthy mail), Greg Stein, Stefan Sperling, and Karl
> Fogel ("we", henceforth) -- met in New York City to evaluate Subversion's
> current trajectory as a project and to jointly develop suggestions for any
> course correction deemed necessary.  The unfortunate impetus for the
> meeting
> was feedback from representatives of some very large installations of
> Subversion that, from where they sat, the Subversion project was
> stagnating.
>  This is a somewhat fair assertion.  It's not that the community is or has
> ever been in danger of doing nothing.  But even when we march in step
> towards a goal, we don't do a good job of communicating outwardly about
> that
> fact, leading to the appearance of inactivity.  And lately, our ongoing
> efforts have been less like a team of folks working in concert towards a
> common vision, and more like the individual piddlings common to mature open
> source software.
>
> Make no mistake:  Subversion is mature open source software.  It works well
> for open source projects and -- in one of the biggest software coups of the
> last decade -- we've found that it's also good enough to supplant other
> well-established version control systems, open-source or otherwise, inside
> the highly structured walls of the enterprise.  Unfortunately, "good
> enough"
> still leaves the enterprise user base frustrated at the most inopportune
> times (like, say, when managing branches near releases).  And if you judge
> success by mindshare, it's now clear the DVCS tools have rendered our "good
> enough" somewhat obsolete when it comes to serving many open source
> projects, even.
>
> VISION
>
> Subversion has no future as a DVCS tool.  Let's just get that out there.
>  At
> least two very successful such tools exist already, and to squeeze another
> horse into that race would be a poor investment of energy and talent.
> What's more, huge classes of users remain categorically opposed to the very
> tenets on which the DVCS systems are based.  They need centralization.
>  They
> need control.  They need meaningful path-based authorization.  They need
> simplicity.  In short, they desperately need Subversion.  It's this class
> of
> user -- the corporate developer -- that stands to benefit hugely from what
> Subversion brings to the party.  And it's that class of user which we
> believe Subversion should cater to, ideally without ostracizing the open
> source volunteers who remain our largest source of development investment.
>
> Corporate developers come in all shapes and sizes, from self-administering
> 10-person teams to geographically-distributed organizations of thousands of
> developers on hundreds of teams serviced by dedicated IT departments.
> Subversion can't be all things to all people, but it can be a well-defined
> subset of things to most people.  It shouldn't sacrifice its trademark
> simplicity when growing the features most likely to be used in large-scale
> deployments; it shouldn't optimize for the large enterprise in a way that
> will alienate the long tail composed of much smaller deployments.  By
> defining the subset of things Subversion is and those it is not, we
> recognize our own boundaries and prevent years of wandering through the
> proverbial wilderness of feature creep.
>
> Someone wiser than I once said, "Where there is no vision, the people
> perish."  So recognizing the benefits that Subversion already offers, and
> projecting a bit into the future what we'd like to see Subversion become,
> we
> offer the following vision statement for your review:
>
>   Subversion exists to be universally recognized and adopted as an
>   open-source, centralized version control system characterized by its
>   reliability as a safe haven for valuable data; the simplicity of its
>   model and usage; and its ability to support the needs of a wide variety
>   of users and projects, from individuals to large-scale enterprise
>   operations.
>
> A shorter, business-card-sized motto (offered as a replacement to the
> obsolete "A compelling replacement for CVS") might be:  "Enterprise-class
> centralized version control for the masses".
>
> ROADMAP
>
> With that vision in mind, we identified a number of high-value improvements
> which Subversion should gain in coming releases.  Then we took a casual
> pass
> at assigning some technical "plumbing" dependencies for each of these.
> Here's what we came up with, in the form "FEATURE: [DEPENDENCY ...]", where
> "FS-NG" = major FS backend overhaul, "WC-NG" = WC-NG, and "Ev2" =
> svn_editor_t:
>
>    * Obliterate:  FS-NG
>    * Shelve/Checkpoint:  WC-NG
>    * Repository-dictated Configuration:  WC-NG (?)
>    * Rename Tracking:  WC-NG, Ev2, FS-NG (?)
>    * Improved Merging:  WC-NG
>    * Improved Tree Conflict Handling:  WC-NG, Ev2, Rename Tracking
>    * Enterprise Authentication Mechanisms:
>    * Forward History Searching:  FS-NG (?)
>    * Log Message Templates:  Repository-dictated Configuration
>
> By examining this dependency chain, factoring in the development momentum
> which will exist around WC-NG, and admitting that some of the major
> plumbing
> overhauls not currently underway may prove to be just as costly as the
> WC-NG
> work (if not more so), we believe that the following feature roadmap is one
> which will serve Subversion users well:
>
>   1.7: WC-NG; HTTPv2; 'svn patch'; typical slew of various bug fixes
>
>   1.8: repository-dictated configuration; tree conflicts improvements;
>        WC-NG-enabled stuff (rename tracking, compressed pristines,
>        shelving/checkpointing, ...)
>
>   1.9: Editor v2 (for server->client rename handling improvements)
>
>   [...]
>
>   2.0: FS-NG (ideally a parallelized task), with some (to-be-identified)
>        FS-NG enabled features.
>
> That last item is likely to be contentious, so it bears explaining.  We
> believe that our current two filesystem offerings are stifling innovation.
> On the one hand we have the BDB backend which is a breeze to develop for
> but
> is only occasional used; on the other is the far-more-popular FSFS backend
> whose fundamental principles so constrain the system as to destroy much
> hope
> of serious design overhaul; and in the middle lies the feature parity
> requirement we've been living under thus far which binds Subversion to the
> union of the two backends' weaknesses.  We confidently assert that to break
> system-wide compatibility with a so-called 2.0 release will be Subversion's
> great overall detriment, both from the perspective of efficient use of
> development energy and in user adoption.  So we propose that an
> as-yet-fictional Subversion 2.0 be allowed to break compatibility with the
> 1.x line only in ways which can be mitigated using the RA layer as a
> compatibility shim.  Additionally, we believe that merely releasing a 2.0
> with a new FS backend but without new user-visible features enabled by that
> overhaul will be ill-received.
>
> COMMUNITY
>
> We also discussed matters of communication and community.  Subversion has
> historically had a very tight-knit community of developers, and we've
> provided a resource for communicating with users through the various
> mailing
> lists.  With the increase in corporate involvement, and the changing roles
> (and employers!) of committers, the Subversion ecosystem can at times seems
> a bit fractured.  To an enterprise manager responsible for thousands of
> users, and millions of dollars of investment, and billions of bytes of
> data,
> this fracturing appears as a liability when considering an investment in
> Subversion.  Most corporations understand the open source nature of
> Subversion's development method (indeed, this very thing has driven
> Subversion adoption), but they still want a unified face when it comes to
> communication, planning, and project feedback.
>
> One proposed solution is a Subversion "planet", to be hosted at a
> re-purposed subversion.org.  The planet would aggregate feeds from
> individual contributors, as well as the various corporate entities
> interested in Subversion development.  While various commercial interests
> (CollabNet, WANdisco, elego, etc.) may compete in some areas, they are all
> committed to improving Subversion as a whole.  Enterprise users need to see
> this unity across Subversion's corporate sponsors, and a communication
> stream which interleaves these corporate voices works toward that end.
>
> Whatever the solution, we need a defined plan which we then communicate to
> the end users and customers who are deploying Subversion installations.
>
> But communication alone isn't enough.  We need to find ways to grow our
> developer community itself.  Attendance at the recent Subversion
> Corporation
> Annual Members Meeting was low (by design and recommendation, of course),
> but a startling realization was made there.  When the agenda item for
> voting
> new full committers into membership was on the table, there were no
> candidates.  Think about that:  no new full committers for Subversion in
> the
> past year.  This is a bad thing.  We need to find a way to embrace and
> empower would-be Subversion contributors.  Telling them to troll through
> the
> issue tracker looking for bite-sized issues is not the way to do that -- it
> communicates "we can't be bothered to mentor you".  When somebody wants to
> start making contributions, our community must recognize the value of that
> contributor and mentor him or her toward full committership, for the good
> of
> that contributor and of the community.  Further, our public roadmap needs
> to
> highlight the items that we really wish we could be working on but lack the
> manpower to handle.  This would provide those looking for ways to
> accelerate
> Subversion's roadmap with some cues for doing that in harmony with the Big
> Picture.
>
> SUMMARY
>
> I've covered a lot of ground here, but decisions in this community have
> always been and will be made on this mailing list, and they deserve to be
> made with as much information as possible.  You now know where a small
> contingent of developers stand on these issues.  I'd like to publicize on
> our project website a *community-endorsed* vision and roadmap ASAP, and
> then
> get to the business of moving Subversion forward along those lines.
>
> So what say you?
>
> -- C-Mike, for the NYC conference attendees
>
> --
> C. Michael Pilato <cm...@collab.net>
> CollabNet   <>   www.collab.net   <>   Distributed Development On Demand
>
>

Re: Subversion Vision and Roadmap Proposal

["Hyrum K. Wright" <hy...@mail.utexas.edu>]

On Sat, Apr 3, 2010 at 8:59 PM, Karl Fogel <kf...@red-bean.com> wrote:

> ...

Btw, I looked for an issue filed in Apache Infrastructure about getting
> the domain pointed to a box where we can install the planet software,
> but didn't see anything.  Have any steps been taken on this yet, or is
> it "patches welcome" right now?
>

 https://issues.apache.org/jira/browse/INFRA-2571

This is about transferring the domains to the ASF, but remaining under our
control.  Once that happens, we can (presumably) put the Planet software on
the box and run amok.

-Hyrum

Re: Subversion Vision and Roadmap Proposal

[Karl Fogel <kf...@red-bean.com>]

"C. Michael Pilato" <cm...@collab.net> writes:
>projecting a bit into the future what we'd like to see Subversion become, we
>offer the following vision statement for your review:
>
>   Subversion exists to be universally recognized and adopted as an
>   open-source, centralized version control system characterized by its
>   reliability as a safe haven for valuable data; the simplicity of its
>   model and usage; and its ability to support the needs of a wide variety
>   of users and projects, from individuals to large-scale enterprise
>   operations.
>
>A shorter, business-card-sized motto (offered as a replacement to the
>obsolete "A compelling replacement for CVS") might be:  "Enterprise-class
>centralized version control for the masses".

A big +1 to the above, Mike, and thanks for writing up this summary.

Obviously, what a few people in a room find compelling may or may not
persuade the entire community, so I'm not taking any of this as written
in stone until it's discussed here.  But reading it now, a few days
after the discussions, it's still very convincing to me.  Subversion can
be *the* centralized version control system, and what centralized
version control provides is exactly what many organizations need.

>ROADMAP
>
> [... WC-NG WC-NG WC-NG WC-NG "WC-NG" WC-NG WC-NG ...]

(Sorry, I might be quoting inaccurately, but I wanted to save space...)

I think WC-NG is going to open up room for a lot of good stuff on the
client side.  I agree with the proposed roadmap, but I'm really glad
that the WC-NG stuff comes first and that WC-NG coding is so far along.
That plus the repository-sent config stuff in the first two proposed
minor releases...

>   1.7: WC-NG; HTTPv2; 'svn patch'; typical slew of various bug fixes
>
>   1.8: repository-dictated configuration; tree conflicts improvements;
>        WC-NG-enabled stuff (rename tracking, compressed pristines,
>        shelving/checkpointing, ...)

...would already be a big step in the right direction.  I agree with the
analysis of the problems caused by having two FS backends; FS-NG will be
hard, but it may be necessary.

>One proposed solution is a Subversion "planet", to be hosted at a
>re-purposed subversion.org.  The planet would aggregate feeds from
>individual contributors, as well as the various corporate entities
>interested in Subversion development.  While various commercial interests
>(CollabNet, WANdisco, elego, etc.) may compete in some areas, they are all
>committed to improving Subversion as a whole.  Enterprise users need to see
>this unity across Subversion's corporate sponsors, and a communication
>stream which interleaves these corporate voices works toward that end.

Btw, I looked for an issue filed in Apache Infrastructure about getting
the domain pointed to a box where we can install the planet software,
but didn't see anything.  Have any steps been taken on this yet, or is
it "patches welcome" right now?

>SUMMARY
>
>I've covered a lot of ground here, but decisions in this community have
>always been and will be made on this mailing list, and they deserve to be
>made with as much information as possible.  You now know where a small
>contingent of developers stand on these issues.  I'd like to publicize on
>our project website a *community-endorsed* vision and roadmap ASAP, and then
>get to the business of moving Subversion forward along those lines.
>
>So what say you?

+1, in case it wasn't obvious.

-Karl

RE: Properties (was Re: Subversion Vision and Roadmap Proposal)

[Bert Huijben <be...@qqmail.nl>]

> -----Original Message-----
> From: Greg Stein [mailto:gstein@gmail.com]
> Sent: donderdag 8 april 2010 22:41
> To: C. Michael Pilato
> Cc: dev@subversion.apache.org
> Subject: Re: Properties (was Re: Subversion Vision and Roadmap
> Proposal)
> 
> On Thu, Apr 8, 2010 at 15:35, C. Michael Pilato <cm...@collab.net>
> wrote:
> > Martin Hauner wrote:
> >> Hi,
> >>
> >> On 06.04.10 00:01, Stefan Sperling wrote:
> >>> On Mon, Apr 05, 2010 at 07:28:57PM +0200, Martin Hauner wrote:
> >>>> [..]
> >>>> With svn:mergeinfo I have to update after each commit because its
> on
> >>>> my root folder and always is out of date on the next commit.
> >>>
> >>> The out-of-dateness really comes from the mixed-revision working
> >>> copy concept, not from mergeinfo.
> >>>
> >>> The root of your working copy is not out of date right after
> >>> the commit. It is at the HEAD revision after commit:
> >>>
> >>> $ ls
> >>> alpha    beta     epsilon/ gamma/
> >>> $ svn merge ^/trunk
> >>> --- Merging r2 through r4 into '.':
> >>> U    alpha
> >>> --- Recording mergeinfo for merge of r2 through r4 into '.':
> >>>   U   .
> >>> $ svn ci -m merge
> >>> Sending        .
> >>> Sending        alpha
> >>> Transmitting file data .
> >>> Committed revision 5.
> >>> $ svn info . | grep ^Rev
> >>> Revision: 5
> >>>
> >>> Maybe you do not commit the mergeinfo which is set on the root of
> your
> >>> working copy? If so, why not?
> >>
> >> I always commit merges on the root.
> >>
> >> I have tried to create an example, but it works in my test
> repository. I
> >> can't reproduce my "at work" behavior.
> >>
> >> It still complains that '.' is out of date most os the at work. And
> I'm
> >> 100% sure that it started when subversion went merge-tracking.
> >
> > Subversion has always disallowed commits of propchanges on a
> directory
> > that's not fully up to date.  Why?  For the simple reason that, after
> the
> > commit happens, that directory cannot be considered "at the new
> revision X"
> > because we never got the changes we were missing from the server.
> >
> > What happened with merge tracking is that propchanges on directories
> > suddenly became quite a bit more common.  Like ... after every merge.
> 
> If there are no propchanges on the directory in the interim, then why
> not allow the commit?
> 
> IOW, did we disallow because we were lazy, or is there a fundamental
> problem with allowing the commit to proceed?

We can only do this if there are no property changes on the directory *and*
when there are no children added or removed from the directory.

(This is not necessary an issue on committing, but when  we upgrade the
revision number of the directory, without updating the list of children this
list of children is out of date in a way that can't be fixed by updating).

	Bert

Re: Properties (was Re: Subversion Vision and Roadmap Proposal)

[Greg Stein <gs...@gmail.com>]

On Thu, Apr 8, 2010 at 15:35, C. Michael Pilato <cm...@collab.net> wrote:
> Martin Hauner wrote:
>> Hi,
>>
>> On 06.04.10 00:01, Stefan Sperling wrote:
>>> On Mon, Apr 05, 2010 at 07:28:57PM +0200, Martin Hauner wrote:
>>>> [..]
>>>> With svn:mergeinfo I have to update after each commit because its on
>>>> my root folder and always is out of date on the next commit.
>>>
>>> The out-of-dateness really comes from the mixed-revision working
>>> copy concept, not from mergeinfo.
>>>
>>> The root of your working copy is not out of date right after
>>> the commit. It is at the HEAD revision after commit:
>>>
>>> $ ls
>>> alpha    beta     epsilon/ gamma/
>>> $ svn merge ^/trunk
>>> --- Merging r2 through r4 into '.':
>>> U    alpha
>>> --- Recording mergeinfo for merge of r2 through r4 into '.':
>>>   U   .
>>> $ svn ci -m merge
>>> Sending        .
>>> Sending        alpha
>>> Transmitting file data .
>>> Committed revision 5.
>>> $ svn info . | grep ^Rev
>>> Revision: 5
>>>
>>> Maybe you do not commit the mergeinfo which is set on the root of your
>>> working copy? If so, why not?
>>
>> I always commit merges on the root.
>>
>> I have tried to create an example, but it works in my test repository. I
>> can't reproduce my "at work" behavior.
>>
>> It still complains that '.' is out of date most os the at work. And I'm
>> 100% sure that it started when subversion went merge-tracking.
>
> Subversion has always disallowed commits of propchanges on a directory
> that's not fully up to date.  Why?  For the simple reason that, after the
> commit happens, that directory cannot be considered "at the new revision X"
> because we never got the changes we were missing from the server.
>
> What happened with merge tracking is that propchanges on directories
> suddenly became quite a bit more common.  Like ... after every merge.

If there are no propchanges on the directory in the interim, then why
not allow the commit?

IOW, did we disallow because we were lazy, or is there a fundamental
problem with allowing the commit to proceed?

Cheers,
-g

Re: Properties (was Re: Subversion Vision and Roadmap Proposal)

["C. Michael Pilato" <cm...@collab.net>]

Martin Hauner wrote:
> Hi,
> 
> On 06.04.10 00:01, Stefan Sperling wrote:
>> On Mon, Apr 05, 2010 at 07:28:57PM +0200, Martin Hauner wrote:
>>> [..]
>>> With svn:mergeinfo I have to update after each commit because its on
>>> my root folder and always is out of date on the next commit.
>>
>> The out-of-dateness really comes from the mixed-revision working
>> copy concept, not from mergeinfo.
>>
>> The root of your working copy is not out of date right after
>> the commit. It is at the HEAD revision after commit:
>>
>> $ ls
>> alpha    beta     epsilon/ gamma/
>> $ svn merge ^/trunk
>> --- Merging r2 through r4 into '.':
>> U    alpha
>> --- Recording mergeinfo for merge of r2 through r4 into '.':
>>   U   .
>> $ svn ci -m merge
>> Sending        .
>> Sending        alpha
>> Transmitting file data .
>> Committed revision 5.
>> $ svn info . | grep ^Rev
>> Revision: 5
>>
>> Maybe you do not commit the mergeinfo which is set on the root of your
>> working copy? If so, why not?
> 
> I always commit merges on the root.
> 
> I have tried to create an example, but it works in my test repository. I
> can't reproduce my "at work" behavior.
> 
> It still complains that '.' is out of date most os the at work. And I'm
> 100% sure that it started when subversion went merge-tracking.

Subversion has always disallowed commits of propchanges on a directory
that's not fully up to date.  Why?  For the simple reason that, after the
commit happens, that directory cannot be considered "at the new revision X"
because we never got the changes we were missing from the server.

What happened with merge tracking is that propchanges on directories
suddenly became quite a bit more common.  Like ... after every merge.

-- 
C. Michael Pilato <cm...@collab.net>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand

Re: Properties (was Re: Subversion Vision and Roadmap Proposal)

[Martin Hauner <ma...@gmx.net>]

Hi,

On 06.04.10 00:01, Stefan Sperling wrote:
> On Mon, Apr 05, 2010 at 07:28:57PM +0200, Martin Hauner wrote:
>> [..]
>> With svn:mergeinfo I have to update after each commit because its on
>> my root folder and always is out of date on the next commit.
>
> The out-of-dateness really comes from the mixed-revision working
> copy concept, not from mergeinfo.
>
> The root of your working copy is not out of date right after
> the commit. It is at the HEAD revision after commit:
>
> $ ls
> alpha    beta     epsilon/ gamma/
> $ svn merge ^/trunk
> --- Merging r2 through r4 into '.':
> U    alpha
> --- Recording mergeinfo for merge of r2 through r4 into '.':
>   U   .
> $ svn ci -m merge
> Sending        .
> Sending        alpha
> Transmitting file data .
> Committed revision 5.
> $ svn info . | grep ^Rev
> Revision: 5
>
> Maybe you do not commit the mergeinfo which is set on the root of your
> working copy? If so, why not?

I always commit merges on the root.

I have tried to create an example, but it works in my test repository. I can't 
reproduce my "at work" behavior.

It still complains that '.' is out of date most os the at work. And I'm 100% 
sure that it started when subversion went merge-tracking.

I'm heavily confused now. ;-)

 > [..]
> We can however improve performance (and we're already working on that).
> So let's assume for a moment that update took 1 nanosecond.
> Would that increase your productivity in an acceptable way, or would
> you still be put off by the fact that you need to run update after commit?
> And is this a big enough problem for your work flow that would make you
> switch to something else for version control?

If it runs fast enough I personally can live with it. No system is perfect :-)

I will not change the system as long as there is nothing that's a lot better 
than subversion. Moving from sourcs safe to cvs and from cvs to svn were big 
improvements.

But I don't think switching from svn to something else pays off at the moment. I 
don't see anything that is way better for what we do at work.


> Thanks,
> Stefan

-- 
Martin

Subcommander 2.0.0 Beta 5 - http://subcommander.tigris.org
a Win32/Unix/MacOSX subversion GUI client & diff/merge tool.

Re: Properties (was Re: Subversion Vision and Roadmap Proposal)

[Stefan Sperling <st...@elego.de>]

On Mon, Apr 05, 2010 at 07:28:57PM +0200, Martin Hauner wrote:
> Hi,
> 
> On 05.04.10 17:06, Stefan Sperling wrote:
> >On Mon, Apr 05, 2010 at 04:14:21PM +0200, Martin Hauner wrote:
> >[..]
> >>In case of merging (mostly cherry picking from trunk to live and
> >>next release branches, merge tracking is nice BUT svn:merginfo on
> >>the root folder kills it again. After each merge I have to update
> >>again (ie. wait another few minutes) before I'm able to commit
> >>again.
> >
> >So mergeinfo kills it because it takes a long time to update?
> >How is this any different from the performance problem you mentioned,
> >and what has it got to do with mergeinfo?
> >Or do you generally oppose the idea of storing merge information
> >in svn: properties? If so, were else would you store it?
> 
> Äh, performance yes. ;-)
> 
> The performance problem makes merge tracking more a pain than a gain.
> 
> Before merge tracking I could do this:
> 
> merge (cherry pick) from trunk to branch
> commit, add merged revision to log message
> merge (cherry pick) from trunk to branch
> commit, add merged revision to log message
> 
> and so on...
>
> In most cases this didn't touch common folders so while updating my
> branch I could mostly skip the update step.
> 
> With svn:mergeinfo I have to update after each commit because its on
> my root folder and always is out of date on the next commit.

The out-of-dateness really comes from the mixed-revision working
copy concept, not from mergeinfo.

The root of your working copy is not out of date right after
the commit. It is at the HEAD revision after commit:

$ ls
alpha    beta     epsilon/ gamma/
$ svn merge ^/trunk
--- Merging r2 through r4 into '.':
U    alpha
--- Recording mergeinfo for merge of r2 through r4 into '.':
 U   .
$ svn ci -m merge
Sending        .
Sending        alpha
Transmitting file data .
Committed revision 5.
$ svn info . | grep ^Rev
Revision: 5

Maybe you do not commit the mergeinfo which is set on the root of your
working copy? If so, why not?

Or maybe you don't realise that you always have a mixed-revision working
copy after a commit?
Even if the root folder is at HEAD, it may be "incomplete" because some
of its children aren't at HEAD yet. So Subversion may not allow you
to commit some of the children (because they really are out of date
and it has to check the repository for new changes and maybe get them,
which is what update is for). So, in general, you need to run svn update
to bring mixed-revision working copies to a single revision.

Of course, if update is so slow that you avoid using it, you will end
up with many of the files and folders in your working copy at different
revisions, and you'll often run into "out of date" errors.
But that still seems to point to the performance problem with update.

So I see several factors which cause the problems for the work flow
you would like to use:

 - subversion supports mixed revision working copies, which has
   implications on the working copy state after commit, and which
   means you should update your entire working copy after every
   commit if you want single-revision working copy behavior
 - mergeinfo is stored as a property, so it is handled like any other
   versioned data, and you need to commit it, causing mixed-revision
   working copies
 - update is too slow

We can't really toss the mixed-revision working copy concept.
Doing that would force users of busy repositories to update even more often.
And we don't want to let 'svn commit' do an auto-update of the working
copy either. That's the job of 'svn update'.

Changing the way mergeinfo is stored now would be a lot of work,
and we might even use more properties in the future to give users
more control over what Subversion is doing. So properties to manage
meta-data will likely not go away.

We can however improve performance (and we're already working on that).
So let's assume for a moment that update took 1 nanosecond.
Would that increase your productivity in an acceptable way, or would
you still be put off by the fact that you need to run update after commit?
And is this a big enough problem for your work flow that would make you
switch to something else for version control?

Thanks,
Stefan

Re: Properties (was Re: Subversion Vision and Roadmap Proposal)

[Stefan Sperling <st...@elego.de>]

On Mon, Apr 05, 2010 at 07:28:57PM +0200, Martin Hauner wrote:
> Hi,
> 
> On 05.04.10 17:06, Stefan Sperling wrote:
> >On Mon, Apr 05, 2010 at 04:14:21PM +0200, Martin Hauner wrote:
> >[..]
> >>In case of merging (mostly cherry picking from trunk to live and
> >>next release branches, merge tracking is nice BUT svn:merginfo on
> >>the root folder kills it again. After each merge I have to update
> >>again (ie. wait another few minutes) before I'm able to commit
> >>again.
> >
> >So mergeinfo kills it because it takes a long time to update?
> >How is this any different from the performance problem you mentioned,
> >and what has it got to do with mergeinfo?
> >Or do you generally oppose the idea of storing merge information
> >in svn: properties? If so, were else would you store it?
> 
> Äh, performance yes. ;-)
> 
> The performance problem makes merge tracking more a pain than a gain.
> 
> Before merge tracking I could do this:
> 
> merge (cherry pick) from trunk to branch
> commit, add merged revision to log message
> merge (cherry pick) from trunk to branch
> commit, add merged revision to log message
> 
> and so on...
>
> In most cases this didn't touch common folders so while updating my
> branch I could mostly skip the update step.
> 
> With svn:mergeinfo I have to update after each commit because its on
> my root folder and always is out of date on the next commit.

The out-of-dateness really comes from the mixed-revision working
copy concept, not from mergeinfo.

The root of your working copy is not out of date right after
the commit. It is at the HEAD revision after commit:

$ ls
alpha    beta     epsilon/ gamma/
$ svn merge ^/trunk
--- Merging r2 through r4 into '.':
U    alpha
--- Recording mergeinfo for merge of r2 through r4 into '.':
 U   .
$ svn ci -m merge
Sending        .
Sending        alpha
Transmitting file data .
Committed revision 5.
$ svn info . | grep ^Rev
Revision: 5

Maybe you do not commit the mergeinfo which is set on the root of your
working copy? If so, why not?

Or maybe you don't realise that you always have a mixed-revision working
copy after a commit?
Even if the root folder is at HEAD, it may be "incomplete" because some
of its children aren't at HEAD yet. So Subversion may not allow you
to commit some of the children (because they really are out of date
and it has to check the repository for new changes and maybe get them,
which is what update is for). So, in general, you need to run svn update
to bring mixed-revision working copies to a single revision.

Of course, if update is so slow that you avoid using it, you will end
up with many of the files and folders in your working copy at different
revisions, and you'll often run into "out of date" errors.
But that still seems to point to the performance problem with update.

So I see several factors which cause the problems for the work flow
you would like to use:

 - subversion supports mixed revision working copies, which has
   implications on the working copy state after commit, and which
   means you should update your entire working copy after every
   commit if you want single-revision working copy behavior
 - mergeinfo is stored as a property, so it is handled like any other
   versioned data, and you need to commit it, causing mixed-revision
   working copies
 - update is too slow

We can't really toss the mixed-revision working copy concept.
Doing that would force users of busy repositories to update even more often.
And we don't want to let 'svn commit' do an auto-update of the working
copy either. That's the job of 'svn update'.

Changing the way mergeinfo is stored now would be a lot of work,
and we might even use more properties in the future to give users
more control over what Subversion is doing. So properties to manage
meta-data will likely not go away.

We can however improve performance (and we're already working on that).
So let's assume for a moment that update took 1 nanosecond.
Would that increase your productivity in an acceptable way, or would
you still be put off by the fact that you need to run update after commit?
And is this a big enough problem for your work flow that would make you
switch to something else for version control?

Thanks,
Stefan

Properties (was Re: Subversion Vision and Roadmap Proposal)

[Martin Hauner <ma...@gmx.net>]

Hi,

On 05.04.10 17:06, Stefan Sperling wrote:
> On Mon, Apr 05, 2010 at 04:14:21PM +0200, Martin Hauner wrote:
> [..]
>> In case of merging (mostly cherry picking from trunk to live and
>> next release branches, merge tracking is nice BUT svn:merginfo on
>> the root folder kills it again. After each merge I have to update
>> again (ie. wait another few minutes) before I'm able to commit
>> again.
>
> So mergeinfo kills it because it takes a long time to update?
> How is this any different from the performance problem you mentioned,
> and what has it got to do with mergeinfo?
> Or do you generally oppose the idea of storing merge information
> in svn: properties? If so, were else would you store it?

Äh, performance yes. ;-)

The performance problem makes merge tracking more a pain than a gain.

Before merge tracking I could do this:

merge (cherry pick) from trunk to branch
commit, add merged revision to log message
merge (cherry pick) from trunk to branch
commit, add merged revision to log message

and so on...

In most cases this didn't touch common folders so while updating my branch I 
could mostly skip the update step.

With svn:mergeinfo I have to update after each commit because its on my root 
folder and always is out of date on the next commit. So the extra step takes 
away another few minutes because of the performance issue.

merge (cherry pick) from trunk to branch
commit
update
merge (cherry pick) from trunk to branch
commit
update

This is annoying.. :)

So yes, apart from the performance issue I don't like that svn:merginfo is a 
property on the root folder. Using a property for this kind of information feels 
wrong to me.

You mention that there is the idea of using another svn: property to configure 
branching/merging strategy. I don't like that either.

I see that looking from the "how do we get this into subversion" point of view 
properties are an option because we don't have a better place. But from a user 
point of view this is completely arcane.

This is "high level" info or meta information and not interesting for single 
files & folders. Having them user visible at "special" places (i.e. the root 
folder, which it is only by convention) or "normal" files & folders is confusing.

In case of svn:mergeinfo on the root folder it even interferes with the normal 
work flow. On /trunk I can commit without update for a while. On cherry picking 
& merging I always have to commit & update.

So as a user I ask: why is this two steps then..? What a strange system ;-)))


I can't offer a better place to store such information, but in my opinion it 
shows that subversion is missing something, a basic concept, here.


Some random thoughts:

Maybe such configuration could be stored in global properties? Properties not 
set on files/folders or revisions but on the repository.


 From an svn client view, I would like to have api calls to get all interesting 
info (for example the repository layout) for any given repository location.

svn_client_get_root 
("http://svn.apache.org/repos/asf/subversion/trunk/subversion/svn_client");

would give me "http://svn.apache.org/repos/asf/subversion/trunk"

svn_client_get_root 
("http://svn.apache.org/repos/asf/subversion/branches/1.6.x/subversion/svn_client");

would give me "http://svn.apache.org/repos/asf/subversion/branches/1.6.x"


Then I could ask

svn_client_layout ("http://svn.apache.org/repos/asf/subversion/trunk");
or
svn_client_layout ("http://svn.apache.org/repos/asf/subversion/branches/1.6.x");

which would tell me

trunk:    "http://svn.apache.org/repos/asf/subversion/trunk"
branches: "http://svn.apache.org/repos/asf/subversion/branches"
tags:     "http://svn.apache.org/repos/asf/subversion/tags"

Then an "svn branch" could check that the target is indeed in branches and not tags.


> Thanks,
> Stefan

-- 
Martin

Subcommander 2.0.0 Beta 5 - http://subcommander.tigris.org
a Win32/Unix/MacOSX subversion GUI client & diff/merge tool.

Properties (was Re: Subversion Vision and Roadmap Proposal)

[Martin Hauner <ma...@gmx.net>]

Hi,

On 05.04.10 17:06, Stefan Sperling wrote:
> On Mon, Apr 05, 2010 at 04:14:21PM +0200, Martin Hauner wrote:
> [..]
>> In case of merging (mostly cherry picking from trunk to live and
>> next release branches, merge tracking is nice BUT svn:merginfo on
>> the root folder kills it again. After each merge I have to update
>> again (ie. wait another few minutes) before I'm able to commit
>> again.
>
> So mergeinfo kills it because it takes a long time to update?
> How is this any different from the performance problem you mentioned,
> and what has it got to do with mergeinfo?
> Or do you generally oppose the idea of storing merge information
> in svn: properties? If so, were else would you store it?

Äh, performance yes. ;-)

The performance problem makes merge tracking more a pain than a gain.

Before merge tracking I could do this:

merge (cherry pick) from trunk to branch
commit, add merged revision to log message
merge (cherry pick) from trunk to branch
commit, add merged revision to log message

and so on...

In most cases this didn't touch common folders so while updating my branch I 
could mostly skip the update step.

With svn:mergeinfo I have to update after each commit because its on my root 
folder and always is out of date on the next commit. So the extra step takes 
away another few minutes because of the performance issue.

merge (cherry pick) from trunk to branch
commit
update
merge (cherry pick) from trunk to branch
commit
update

This is annoying.. :)

So yes, apart from the performance issue I don't like that svn:merginfo is a 
property on the root folder. Using a property for this kind of information feels 
wrong to me.

You mention that there is the idea of using another svn: property to configure 
branching/merging strategy. I don't like that either.

I see that looking from the "how do we get this into subversion" point of view 
properties are an option because we don't have a better place. But from a user 
point of view this is completely arcane.

This is "high level" info or meta information and not interesting for single 
files & folders. Having them user visible at "special" places (i.e. the root 
folder, which it is only by convention) or "normal" files & folders is confusing.

In case of svn:mergeinfo on the root folder it even interferes with the normal 
work flow. On /trunk I can commit without update for a while. On cherry picking 
& merging I always have to commit & update.

So as a user I ask: why is this two steps then..? What a strange system ;-)))


I can't offer a better place to store such information, but in my opinion it 
shows that subversion is missing something, a basic concept, here.


Some random thoughts:

Maybe such configuration could be stored in global properties? Properties not 
set on files/folders or revisions but on the repository.


 From an svn client view, I would like to have api calls to get all interesting 
info (for example the repository layout) for any given repository location.

svn_client_get_root 
("http://svn.apache.org/repos/asf/subversion/trunk/subversion/svn_client");

would give me "http://svn.apache.org/repos/asf/subversion/trunk"

svn_client_get_root 
("http://svn.apache.org/repos/asf/subversion/branches/1.6.x/subversion/svn_client");

would give me "http://svn.apache.org/repos/asf/subversion/branches/1.6.x"


Then I could ask

svn_client_layout ("http://svn.apache.org/repos/asf/subversion/trunk");
or
svn_client_layout ("http://svn.apache.org/repos/asf/subversion/branches/1.6.x");

which would tell me

trunk:    "http://svn.apache.org/repos/asf/subversion/trunk"
branches: "http://svn.apache.org/repos/asf/subversion/branches"
tags:     "http://svn.apache.org/repos/asf/subversion/tags"

Then an "svn branch" could check that the target is indeed in branches and not tags.


> Thanks,
> Stefan

-- 
Martin

Subcommander 2.0.0 Beta 5 - http://subcommander.tigris.org
a Win32/Unix/MacOSX subversion GUI client & diff/merge tool.

Re: Subversion Vision and Roadmap Proposal

[Miha Vitorovic <mi...@gmail.com>]

On 13.4.2010 19:32, Johan Corveleyn wrote:
> On Tue, Apr 13, 2010 at 12:29 PM, Julian Foad<ju...@wandisco.com>  wrote:
>    
>> Bolstridge, Andrew wrote:
>>      
>>> To get new developers, I think the first thing that needs to be done
>>> is to make entry easier. That means providing a setup ready-to-debug
>>> for people. The initial hurdle on any software project is getting set
>>> up (I find) so if you could release a VM image with everything set, or
>>> a visual studio project with the code and dependencies in there ready
>>> for someone to press 'compile', then you've made a massive headway to
>>> helping new devs get going. A pre-built Windows solution would be the
>>> ideal - if anyone has one already... let us know :) The documentation
>>> could be better (as always :) ), but getting the code running in a
>>> debugger is probably documentation enough, especially for a new
>>> developer who wants to modify things.
>>>        
>> That's a very good idea.  I wonder if any of our Windows gurus can come
>> up with a practical way to make a ready-to-build package.
>>      
> Yes, that's probably one of the best investments right now to lower
> the bar for new developers (it's also one of the things I mentioned in
> the other thread reacting to the vision/roadmap, but that thread seems
> to have been discarded ... oh well, no problem ...).
>
> Getting the build up and running on Windows (and getting it to run
> from the debugger of Visual C Express 2008) was a huge waste of
> time...
>
> Also, don't forget to make it easy to setup an environment to run the
> tests (I haven't focused on running the tests on my setup yet, but I
> expect that I will lose a lot of time with that again ...).
>
>    
Here's just a thought on how to achieve this - I don't know if it is 
doable because of the licensing, but - how about creating a Vmware (or 
some other VM) image that has everything already set up. This way a 
working Windows environment would be just a download away. I just don't 
know if updating SVN and dependencies to the latest build would be 
simple or hard...

Br, Miha

Re: Subversion Vision and Roadmap Proposal

[Johan Corveleyn <jc...@gmail.com>]

On Tue, Apr 13, 2010 at 12:29 PM, Julian Foad <ju...@wandisco.com> wrote:
> Bolstridge, Andrew wrote:
>> To get new developers, I think the first thing that needs to be done
>> is to make entry easier. That means providing a setup ready-to-debug
>> for people. The initial hurdle on any software project is getting set
>> up (I find) so if you could release a VM image with everything set, or
>> a visual studio project with the code and dependencies in there ready
>> for someone to press 'compile', then you've made a massive headway to
>> helping new devs get going. A pre-built Windows solution would be the
>> ideal - if anyone has one already... let us know :) The documentation
>> could be better (as always :) ), but getting the code running in a
>> debugger is probably documentation enough, especially for a new
>> developer who wants to modify things.
>
> That's a very good idea.  I wonder if any of our Windows gurus can come
> up with a practical way to make a ready-to-build package.

Yes, that's probably one of the best investments right now to lower
the bar for new developers (it's also one of the things I mentioned in
the other thread reacting to the vision/roadmap, but that thread seems
to have been discarded ... oh well, no problem ...).

Getting the build up and running on Windows (and getting it to run
from the debugger of Visual C Express 2008) was a huge waste of
time...

Also, don't forget to make it easy to setup an environment to run the
tests (I haven't focused on running the tests on my setup yet, but I
expect that I will lose a lot of time with that again ...).

-- 
Johan

RE: Subversion Vision and Roadmap Proposal

[Julian Foad <ju...@wandisco.com>]

Bolstridge, Andrew wrote:
> > ROADMAP
[...]
> > COMMUNITY
[...]
> > SUMMARY
[...]
> > So what say you?
> 
> Well, I (as an outsider to the svn dev community) say great!
> 
> My thoughts on this: firstly, to attract more people to the community,
> you need to go where they are. This dev mailing list is all well and
> good for subversion developers, but that's exactly the audience you've
> already got. I think this roadmap/vision/invitation needs to be posted
> elsewhere, even if watered down. If you want suggestions for things to
> add to the roadmap then post to stackoverflow.com and serverfault.com
> and see what the people there come up with (I'll happily create such
> posts if you want me to)

Posting to sites such as those could result in useful feedback.  Of
course it also tends to elicit lots of spammy off-topic comments.  If
you are prepared to both make the post and summarize the useful bits of
feedback for us, that would be fantastic.


> I think other main issues are:
[...]


> To get new developers, I think the first thing that needs to be done
> is to make entry easier. That means providing a setup ready-to-debug
> for people. The initial hurdle on any software project is getting set
> up (I find) so if you could release a VM image with everything set, or
> a visual studio project with the code and dependencies in there ready
> for someone to press 'compile', then you've made a massive headway to
> helping new devs get going. A pre-built Windows solution would be the
> ideal - if anyone has one already... let us know :) The documentation
> could be better (as always :) ), but getting the code running in a
> debugger is probably documentation enough, especially for a new
> developer who wants to modify things.

That's a very good idea.  I wonder if any of our Windows gurus can come
up with a practical way to make a ready-to-build package.


> Good news on this email though, I love subversion and just look
> forward to making it better.

Woderful.  What would you like to work on first?  :-)  (In a new thread
please.)

- Julian

RE: Subversion Vision and Roadmap Proposal

["Bolstridge, Andrew" <an...@intergraph.com>]

> ROADMAP

> With that vision in mind, we identified a number of high-value improvements
> which Subversion should gain in coming releases.  Then we took a casual pass
> at assigning some technical "plumbing" dependencies for each of these.
> Here's what we came up with, in the form "FEATURE: [DEPENDENCY ...]", where
> "FS-NG" = major FS backend overhaul, "WC-NG" = WC-NG, and "Ev2" = svn_editor_t:
>
>     * Obliterate:  FS-NG
>     * Shelve/Checkpoint:  WC-NG
>     * Repository-dictated Configuration:  WC-NG (?)
>     * Rename Tracking:  WC-NG, Ev2, FS-NG (?)
>     * Improved Merging:  WC-NG
>     * Improved Tree Conflict Handling:  WC-NG, Ev2, Rename Tracking
>     * Enterprise Authentication Mechanisms:
>     * Forward History Searching:  FS-NG (?)
>     * Log Message Templates:  Repository-dictated Configuration



> COMMUNITY
>
>
> But communication alone isn't enough.  We need to find ways to grow our
> developer community itself.  Attendance at the recent Subversion Corporation
> Annual Members Meeting was low (by design and recommendation, of course),
> but a startling realization was made there.  When the agenda item for voting
> new full committers into membership was on the table, there were no
> candidates.  Think about that:  no new full committers for Subversion in the
> past year.  This is a bad thing.  We need to find a way to embrace and
> empower would-be Subversion contributors.  
>

> SUMMARY
>
> I've covered a lot of ground here, but decisions in this community have
> always been and will be made on this mailing list, and they deserve to be
> made with as much information as possible.  You now know where a small
> contingent of developers stand on these issues.  I'd like to publicize on
> our project website a *community-endorsed* vision and roadmap ASAP, and then
> get to the business of moving Subversion forward along those lines.
>
> So what say you?
>

Well, I (as an outsider to the svn dev community) say great!

My thoughts on this: firstly, to attract more people to the community, you need to go where they are. This dev mailing list is all well and good for subversion developers, but that's exactly the audience you've already got. I think this roadmap/vision/invitation needs to be posted elsewhere, even if watered down. If you want suggestions for things to add to the roadmap then post to stackoverflow.com and serverfault.com and see what the people there come up with (I'll happily create such posts if you want me to)


I think other main issues are:
	Repo-dictated configuration (worth saying again, this is more important than you think)
	General performance, especially with lots and lots of files (I need to work with 20,000 for one project - it doesn’t work, I have to commit individual directories instead. The failure when trying looked really bad when my boss watched it once. If I was a manager and saw that, I wouldn’t allow subversion to be used for my team).
	Baselining (branches everywhere doesn't really meet many corporate requirements as we have a lot of projects in our repos, while I love the visibility of branching in svn compared to many lesser SCMs, there is still a problem with managing multiple branches - it's too easy to create a huge 'directory structure' of branches. If you have 300 projects in your repo, you can see how it quickly gets out of hand. This is probably why there's so much heat in past discussions of tagging), 
	Searching in the repo - it's not that we need a new FS backend, but an index on that. After all, the repo files don’t often get touched unless you want to see old versions but the history associated with them does. So really, the revprops need a different format. That'd probably be sufficient for a perf boost for most tasks.


To get new developers, I think the first thing that needs to be done is to make entry easier. That means providing a setup ready-to-debug for people. The initial hurdle on any software project is getting set up (I find) so if you could release a VM image with everything set, or a visual studio project with the code and dependencies in there ready for someone to press 'compile', then you've made a massive headway to helping new devs get going. A pre-built Windows solution would be the ideal - if anyone has one already... let us know :) The documentation could be better (as always :) ), but getting the code running in a debugger is probably documentation enough, especially for a new developer who wants to modify things.

Good news on this email though, I love subversion and just look forward to making it better.

Andy

Re: Subversion Vision and Roadmap Proposal

[Johan Corveleyn <jc...@gmail.com>]

A bit late perhaps, but also from me a big +1 to this vision and
roadmap proposal.

There are some minor points which can be discussed of course (as
always), but overall I think it's a very good plan. I especially
applaud the initiative itself to spend some time for this, taking a
step back, looking a bit further down the road than just the next
minor release or the next fire that needs to be put out, trying to get
the community more focused and more active, ...

This is all very important, so thanks to all who participate(d). I
hope you guys can repeat this kind of gettogether once in a while, to
toss ideas around, to provide continuing direction, ...

Some detailed responses to the proposal and to the other replies below.

On Fri, Apr 2, 2010 at 5:28 PM, C. Michael Pilato <cm...@collab.net> wrote:
> VISION
> [...]
>   Subversion exists to be universally recognized and adopted as an
>   open-source, centralized version control system characterized by its
>   reliability as a safe haven for valuable data; the simplicity of its
>   model and usage; and its ability to support the needs of a wide variety
>   of users and projects, from individuals to large-scale enterprise
>   operations.
>
> A shorter, business-card-sized motto (offered as a replacement to the
> obsolete "A compelling replacement for CVS") might be:  "Enterprise-class
> centralized version control for the masses".

Absolutely spot on. Subversion needs to be the best out there for
centralized version control (which is what enterprises usually
need/want). It has no place in the DVCS world.

Like it probably also happens in other companies, in our company there
are some grassroots experiments of using DVCS's. But it's not their
distributedness that attracts our developers, it's simply because they
are faster to work with, have better merging capabilities, better
handling of renames, ... things that AFAICS do not depend on the
distributed nature of the VCS. If Subversion would be able to catch up
with the DVCS's in these areas (both feature-wise and especially
performance-wise), I see no reason to prefer a DVCS (except of course
if you need the D because you require your VCS to be uh ...
distributed).

> ROADMAP
> [...]
>   1.7: WC-NG; HTTPv2; 'svn patch'; typical slew of various bug fixes

Eagerly awaiting WC-NG here. We need *fast* working copies (that know
how to preserve all the information of arbitrary series of
adds/moves/copies/renames/modifications/deletes correctly), and we
need them now ;-). Cheering you guys on for 1.7-ness ...

>   1.8: repository-dictated configuration; tree conflicts improvements;
>        WC-NG-enabled stuff (rename tracking, compressed pristines,
>        shelving/checkpointing, ...)

Yes, tree conflicts improvements and rename tracking are absolutely
critical. I know they have to wait for WC-NG right now, but boy do we
need them... It's good to see them proposed for 1.8.

Great also to see repository-dictated configuration come into the
picture. In an enterprise environment the lack of this feature has
always been kind of embarrassing (having to catch things in a
pre-commit hook, and telling your users they have to install the
correct config in their client, ...), and also a huge waste of time.

>   1.9: Editor v2 (for server->client rename handling improvements)
>
>   [...]
>
>   2.0: FS-NG (ideally a parallelized task), with some (to-be-identified)
>        FS-NG enabled features.
>
> That last item is likely to be contentious, so it bears explaining.  We
> believe that our current two filesystem offerings are stifling innovation.
> On the one hand we have the BDB backend which is a breeze to develop for but
> is only occasional used; on the other is the far-more-popular FSFS backend
> whose fundamental principles so constrain the system as to destroy much hope
> of serious design overhaul; and in the middle lies the feature parity
> requirement we've been living under thus far which binds Subversion to the
> union of the two backends' weaknesses.  We confidently assert that to break
> system-wide compatibility with a so-called 2.0 release will be Subversion's
> great overall detriment, both from the perspective of efficient use of
> development energy and in user adoption.  So we propose that an
> as-yet-fictional Subversion 2.0 be allowed to break compatibility with the
> 1.x line only in ways which can be mitigated using the RA layer as a
> compatibility shim.  Additionally, we believe that merely releasing a 2.0
> with a new FS backend but without new user-visible features enabled by that
> overhaul will be ill-received.

This is the first I hear of FS-NG, but I'm quite enthousiastic. I
completely understand the rationale that you give above.

I especially hope that FS-NG will be *fast*. We use FSFS and I've
always felt that it's not very much designed for performance
(especially for big read operations like log, blame, checkout, ...
which have to do way too much I/O to get the job done). Thinking back
right now to a quote of Jon Trowbridge in a recent thread
(http://svn.haxx.se/dev/archive-2010-03/0735.shtml): "Subversion
assumes that I/O is essentially free.  But in a cloud-based FS, that's
not true." Well I can assure you, cloud-based FS or not, this is
*never true* (unless your back-end is on a local SSD, and even then
...).

Of course you guys are probably thinking much further/deeper than just
performance, but for me that's one of the most important things right
now. If FS-NG would be an order of magnitude faster for some of those
massive read operations, that would be a compelling reason for me to
upgrade as quickly as possible :).

> COMMUNITY
> [...]
> But communication alone isn't enough.  We need to find ways to grow our
> developer community itself.

Very true. I have some observations about this myself ("from the other
side"), but will put them in a separate thread...

---------
Some more responses to the intermediate replies below...

On Mon, Apr 5, 2010 at 5:06 PM, Stefan Sperling <st...@elego.de> wrote:
> The same goes for merging. E.g. DCVS tools simply don't have the concept
> of doing tracked subtree merges (so merging is always damn easy and even
> ruby on rails programmers get it ;), nor do they have the concept of mixed
> revision working copies. But svn does support these concepts and they
> add a lot of complexity. E.g. users end up doing subtree merges by mistake,
> creating mergeinfo all over the place. They don't realise that the normal
> use case is to have mergeinfo only on the branch root, and not doing any
> subtree merges.

I think it's also often done simply to merge quicker. If you merge
only the specific subtree which you know is affected by your
cherrypick merge, you can merge orders of magnitude faster than if you
merge the same cherrypick merge to the root of the branch. Most svn
end users who don't fully grok how merge works will not understand
that it makes a difference (why should it? I *know* this revision only
affects this subtree, so I can just merge it here directly, no?). They
think they have found a good workaround for merging faster.

So ultimately this is again a performance issue... (maybe WC-NG will
help a lot for this?)

On Tue, Apr 6, 2010 at 10:22 PM, Branko Čibej <br...@xbc.nu> wrote:
> On 06.04.2010 12:18, Stefan Sperling wrote:
>> On Tue, Apr 06, 2010 at 07:04:42AM +0200, Branko Čibej wrote:
>>
>>> On 05.04.2010 17:06, Stefan Sperling wrote:
>>>
>>>> An idea we're playing with to mitigate this problem is having designated
>>>> properties in the svn: namespace which allow users to tell svn what their
>>>> branching/merging strategy is.
>>>>
>>> (Thereby making things even more flexible, complex, and error-prone.)
>>> In the light of previous discussions
>>>
>> I'd like to read them. Can you provide pointers?
>>
>
> I was referring to previous posts to this thread.
>
>>> this is a less than optimal way of
>>> making branching and merging easier for users. This is another feature
>>> that would probably only be used by perhaps 1% of all users ...
>>>
>> Do you mean "used" or "configured" (as in setting the props)?
>>
>
> I mean "configured" but "used" works just as well because that's where
> the complexity and maintenance burden comes in. Such properties are
> pretty much equivalent to mergeinfo in the sense that they control the
> behaviour of the client and server, and would have the same tendency
> towards complex edge cases and unexpected usage scenarios.
>
>>> whilst
>>> most people just want to say, "create a branch" and "merge from that
>>> branch" and be done with it.
>>>
>> Yes, they do.
>>
>> But how else would you make it easier to use before going 2.0
>> (in the "2.0" sense of writing a new centralised SCM from scratch)?
>> The flexibility^Wcomplexity is there now, and we have set for
>> ourselves the goal to continue supporting it (for better or worse).
>>
>
> I've proposed before that we introduce proper branches, and gradually
> phase out support for the copy-is-branch paradigm, for example, by not
> supporting smart merging between directory copies but only between
> branches. There is no good reason to continue to support every feature
> we've ever had, assuming we provide a migration path *and* publish a
> roadmap so that users can be made aware of such upcoming changes.
>
> -- Brane
>

I agree with Brane here. I think the only way to really tackle this
area is to make "branches" and "tags" first class citizens
(disregarding for a moment how difficult this might be to
implement/add and what consequences this will have). Not working
towards this goal will likely be patchwork, adding even more
complexity both for developers and for users.

I have no idea how this could be integrated with svn's
model/architecture, but it might be a good idea to start thinking
about this for 2.0 ... it just might be the right time to say to
people: "look, you have to use this new proper branching/tagging
system if you want quality merge support etc. Feel free to continue
using the old copy-is-branch/tag system, but know that its
functionality is limited (i.e. svn cannot be as smart with them as it
can be with proper branches)"

But as I said, I can offer zero contribution on how to accomplish this
technically ...

-- 
Johan

Re: Subversion Vision and Roadmap Proposal

[Branko Čibej <br...@xbc.nu>]

On 06.04.2010 12:18, Stefan Sperling wrote:
> On Tue, Apr 06, 2010 at 07:04:42AM +0200, Branko Čibej wrote:
>   
>> On 05.04.2010 17:06, Stefan Sperling wrote:
>>     
>>> An idea we're playing with to mitigate this problem is having designated
>>> properties in the svn: namespace which allow users to tell svn what their
>>> branching/merging strategy is.
>>>       
>> (Thereby making things even more flexible, complex, and error-prone.)
>> In the light of previous discussions
>>     
> I'd like to read them. Can you provide pointers?
>   

I was referring to previous posts to this thread.

>> this is a less than optimal way of
>> making branching and merging easier for users. This is another feature
>> that would probably only be used by perhaps 1% of all users ...
>>     
> Do you mean "used" or "configured" (as in setting the props)?
>   

I mean "configured" but "used" works just as well because that's where
the complexity and maintenance burden comes in. Such properties are
pretty much equivalent to mergeinfo in the sense that they control the
behaviour of the client and server, and would have the same tendency
towards complex edge cases and unexpected usage scenarios.

>> whilst
>> most people just want to say, "create a branch" and "merge from that
>> branch" and be done with it.
>>     
> Yes, they do.
>
> But how else would you make it easier to use before going 2.0
> (in the "2.0" sense of writing a new centralised SCM from scratch)?
> The flexibility^Wcomplexity is there now, and we have set for
> ourselves the goal to continue supporting it (for better or worse).
>   

I've proposed before that we introduce proper branches, and gradually
phase out support for the copy-is-branch paradigm, for example, by not
supporting smart merging between directory copies but only between
branches. There is no good reason to continue to support every feature
we've ever had, assuming we provide a migration path *and* publish a
roadmap so that users can be made aware of such upcoming changes.

-- Brane

Re: Subversion Vision and Roadmap Proposal

[Stefan Sperling <st...@elego.de>]

On Tue, Apr 06, 2010 at 10:31:56AM -0400, Bob Archer wrote:
> > On Tue, Apr 06, 2010 at 07:04:42AM +0200, Branko Čibej wrote:
> > > On 05.04.2010 17:06, Stefan Sperling wrote:
> > > > An idea we're playing with to mitigate this problem is having
> > designated
> > > > properties in the svn: namespace which allow users to tell svn what
> > their
> > > > branching/merging strategy is.
> > >
> > > (Thereby making things even more flexible, complex, and error-prone.)
> > > In the light of previous discussions
> > 
> > I'd like to read them. Can you provide pointers?
> > 
> > > this is a less than optimal way of
> > > making branching and merging easier for users. This is another feature
> > > that would probably only be used by perhaps 1% of all users ...
> > 
> > Do you mean "used" or "configured" (as in setting the props)?
> 
> Have you guys considered rather than storing all this info in the
> config file rather than versioned properties. Especially since it is
> config info. Then provide a WC level config sorted in a separate files
> such as .svnconfig ? This has the advantage of being hidden away...
> but also has the advantage of being easily managed by users. The
> .svnconfig file could even be a way to have WC level config that would
> override the global config file that is currently stored outside of
> the WC in the users folder. It would also be easy for a user to copy
> this file from one project to another.

If you throw in server-side configuration (another feature on the
roadmap), I kind of like the idea.  

However, note that using properties to carry information about paths
(which is what branches are in Subversion) has the advantage that
when the path changes, the property follows the change. Maintaining
this information in a separate file would require users (or svn) to
keep this file in sync with changes made to the tree structure.

Stefan

RE: Subversion Vision and Roadmap Proposal

[Bob Archer <Bo...@amsi.com>]

> On Tue, Apr 06, 2010 at 07:04:42AM +0200, Branko Čibej wrote:
> > On 05.04.2010 17:06, Stefan Sperling wrote:
> > > An idea we're playing with to mitigate this problem is having
> designated
> > > properties in the svn: namespace which allow users to tell svn what
> their
> > > branching/merging strategy is.
> >
> > (Thereby making things even more flexible, complex, and error-prone.)
> > In the light of previous discussions
> 
> I'd like to read them. Can you provide pointers?
> 
> > this is a less than optimal way of
> > making branching and merging easier for users. This is another feature
> > that would probably only be used by perhaps 1% of all users ...
> 
> Do you mean "used" or "configured" (as in setting the props)?

Have you guys considered rather than storing all this info in the config file rather than versioned properties. Especially since it is config info. Then provide a WC level config sorted in a separate files such as .svnconfig ? This has the advantage of being hidden away... but also has the advantage of being easily managed by users. The .svnconfig file could even be a way to have WC level config that would override the global config file that is currently stored outside of the WC in the users folder. It would also be easy for a user to copy this file from one project to another.

BOb
 

Re: Subversion Vision and Roadmap Proposal

[Stefan Sperling <st...@elego.de>]

On Tue, Apr 06, 2010 at 07:04:42AM +0200, Branko Čibej wrote:
> On 05.04.2010 17:06, Stefan Sperling wrote:
> > An idea we're playing with to mitigate this problem is having designated
> > properties in the svn: namespace which allow users to tell svn what their
> > branching/merging strategy is.
> 
> (Thereby making things even more flexible, complex, and error-prone.)
> In the light of previous discussions

I'd like to read them. Can you provide pointers?

> this is a less than optimal way of
> making branching and merging easier for users. This is another feature
> that would probably only be used by perhaps 1% of all users ...

Do you mean "used" or "configured" (as in setting the props)?

> whilst
> most people just want to say, "create a branch" and "merge from that
> branch" and be done with it.

Yes, they do.

But how else would you make it easier to use before going 2.0
(in the "2.0" sense of writing a new centralised SCM from scratch)?
The flexibility^Wcomplexity is there now, and we have set for
ourselves the goal to continue supporting it (for better or worse).

Stefan

Re: Subversion Vision and Roadmap Proposal

[Branko Čibej <br...@xbc.nu>]

On 05.04.2010 17:06, Stefan Sperling wrote:
> An idea we're playing with to mitigate this problem is having designated
> properties in the svn: namespace which allow users to tell svn what their
> branching/merging strategy is.

(Thereby making things even more flexible, complex, and error-prone.)
In the light of previous discussions this is a less than optimal way of
making branching and merging easier for users. This is another feature
that would probably only be used by perhaps 1% of all users ... whilst
most people just want to say, "create a branch" and "merge from that
branch" and be done with it.

-- Brane

Re: Subversion Vision and Roadmap Proposal

[Stefan Sperling <st...@elego.de>]

On Mon, Apr 05, 2010 at 04:14:21PM +0200, Martin Hauner wrote:
> Simplicity... actually I think subversion could or should be even
> simpler. It may be easy from our point of view around here but when
> I see how people use it, it still seems to be too complicated. One
> has to draw a line at some level of knowledge, sure, but think we
> could improve here.
>
> Referring to my merging example below, It is astonishing difficult
> to explain the svn:merginfo property and why you have to update
> after each commit. From a user perspective (which are still
> programmers but don't know very much of subversion) this is a "what
> the heck" issue.
> 
> The /trunk/branches/tags convention works but it happens from time
> to time that people get the copy target wrong and copy folders into
> itself or something like this. Would be nice if subversion could
> catch this.
> 
> Just an idea here, what about a convention enforcing wrapper around
> the svn client api which would prevent wrong copies based on the
> convention rules.(however one decides that)?

Your concerns are very valid.

At the meeting we also briefly discussed that svn makes it too easy
for users to shoot themselves in the foot. This is the price of high
flexibility. People can do things that don't make any sense (e.g.
svn copy ^/branches/trunk ^/branches/someotherbranch/mybranch), and
svn will not warn or complain.

The same goes for merging. E.g. DCVS tools simply don't have the concept
of doing tracked subtree merges (so merging is always damn easy and even
ruby on rails programmers get it ;), nor do they have the concept of mixed
revision working copies. But svn does support these concepts and they
add a lot of complexity. E.g. users end up doing subtree merges by mistake,
creating mergeinfo all over the place. They don't realise that the normal
use case is to have mergeinfo only on the branch root, and not doing any
subtree merges. They notice all the mergeinfo and do not understand
where it is coming from. They end up treating svn mergeinfo as black magic,
something they cannot control, just because they don't realise they're
making abundant use of a feature which was meant to be used in very
specific cases only.

An idea we're playing with to mitigate this problem is having designated
properties in the svn: namespace which allow users to tell svn what their
branching/merging strategy is. E.g. by setting a special property on the
branch root you could tell svn things like "my branch should only do sync
merges from trunk, cherry picking is not allowed from trunk. Cherry
picking is allowed from my bugfix branch, however. Also do not allow
subtree merges into this branch.", and svn would then try to enforce this.
So once the merge gurus have configured the policy, users who don't
fully grok the concepts cannot make a mess of it.

> I think subversion needs to improve on performance. I know that
> WC-NG will hopefully help here but I think it should be listed
> explicitly.

It will definitely help, not just hopefully.
People have done benchmarks already:
http://svn.haxx.se/dev/archive-2010-02/0457.shtml
http://svn.haxx.se/dev/archive-2010-02/0239.shtml
http://svn.haxx.se/dev/archive-2010-02/0241.shtml

> In case of merging (mostly cherry picking from trunk to live and
> next release branches, merge tracking is nice BUT svn:merginfo on
> the root folder kills it again. After each merge I have to update
> again (ie. wait another few minutes) before I'm able to commit
> again.

So mergeinfo kills it because it takes a long time to update?
How is this any different from the performance problem you mentioned,
and what has it got to do with mergeinfo?
Or do you generally oppose the idea of storing merge information
in svn: properties? If so, were else would you store it?

Thanks,
Stefan

Re: Subversion Vision and Roadmap Proposal

["C. Michael Pilato" <cm...@collab.net>]

Martin Hauner wrote:
> Performance
> 
> I think subversion needs to improve on performance. I know that WC-NG
> will hopefully help here but I think it should be listed explicitly.

Agreed.  We said many times over those 2 1/2 days "Performance is a feature."

We had performance in mind as we listed things like "improved merging" and
of course are interested in seeing how WC-NG and HTTPv2 will help out.
Also, a better FS design could encourage more server-side bearing of the
calculative burden without resource exhaustion, which ultimately helps
performance, too.  Our general thought was, "We should be improving
performance at every opportunity", but we admit that we're really in need of
some good profiling and clear recommendations for improvement in this space.

-- 
C. Michael Pilato <cm...@collab.net>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand

Re: Subversion Vision and Roadmap Proposal

[Martin Hauner <ma...@gmx.net>]

Hi,

On 02.04.10 17:28, C. Michael Pilato wrote:
> [..]

 > VISION
 >
 > Subversion has no future as a DVCS tool.  Let's just get that out there.
 > [..]
 > Someone wiser than I once said, "Where there is no vision, the people
 > perish."  So recognizing the benefits that Subversion already offers, and
 > projecting a bit into the future what we'd like to see Subversion become, we
 > offer the following vision statement for your review:
 >
 >     Subversion exists to be universally recognized and adopted as an
 >     open-source, centralized version control system characterized by its
 >     reliability as a safe haven for valuable data; the simplicity of its
 >     model and usage; and its ability to support the needs of a wide variety
 >     of users and projects, from individuals to large-scale enterprise
 >     operations.

Simplicity... actually I think subversion could or should be even simpler. It 
may be easy from our point of view around here but when I see how people use it, 
it still seems to be too complicated. One has to draw a line at some level of 
knowledge, sure, but think we could improve here.

Referring to my merging example below, It is astonishing difficult to explain 
the svn:merginfo property and why you have to update after each commit. From a 
user perspective (which are still programmers but don't know very much of 
subversion) this is a "what the heck" issue.

The /trunk/branches/tags convention works but it happens from time to time that 
people get the copy target wrong and copy folders into itself or something like 
this. Would be nice if subversion could catch this.

Just an idea here, what about a convention enforcing wrapper around the svn 
client api which would prevent wrong copies based on the convention 
rules.(however one decides that)?

I guess there is more stuff where we could improve usability.


 > [..]
> ROADMAP
>
> With that vision in mind, we identified a number of high-value improvements
> which Subversion should gain in coming releases.  Then we took a casual pass
> at assigning some technical "plumbing" dependencies for each of these.
> Here's what we came up with, in the form "FEATURE: [DEPENDENCY ...]", where
> "FS-NG" = major FS backend overhaul, "WC-NG" = WC-NG, and "Ev2" = svn_editor_t:
>
>      * Obliterate:  FS-NG
>      * Shelve/Checkpoint:  WC-NG
>      * Repository-dictated Configuration:  WC-NG (?)
>      * Rename Tracking:  WC-NG, Ev2, FS-NG (?)
>      * Improved Merging:  WC-NG
>      * Improved Tree Conflict Handling:  WC-NG, Ev2, Rename Tracking
>      * Enterprise Authentication Mechanisms:
>      * Forward History Searching:  FS-NG (?)
>      * Log Message Templates:  Repository-dictated Configuration


Performance

I think subversion needs to improve on performance. I know that WC-NG will 
hopefully help here but I think it should be listed explicitly.

At work (Window Vista) it takes me minutes to update into, commit from or merge 
into my working copies. I don't have numbers available but in our third party 
folder we have boost and ACE/TAO with a lot of files which I guess is the root 
cause (apart from using Windows ;-) for the slow performance.

In case of merging (mostly cherry picking from trunk to live and next release 
branches, merge tracking is nice BUT svn:merginfo on the root folder kills it 
again. After each merge I have to update again (ie. wait another few minutes) 
before I'm able to commit again.

Working with a moderately large working copy on Windows is just pain.


Just my 2 euro-cent ;-)

>
> So what say you?
>
> -- C-Mike, for the NYC conference attendees

thanks for this write up :-)

-- 
Martin

Subcommander 2.0.0 Beta 5 - http://subcommander.tigris.org
a Win32/Unix/MacOSX subversion GUI client & diff/merge tool.