You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Paul Breen <gr...@yahoo.com> on 2010/04/27 06:07:28 UTC

proposed export option

Hello,

I'd like to add a new option to the export command that allows the user
to export files only if the file size has changed--something like

 svn export --skipfilesmatchingsize

The idea is to do a quick-and-easy, traffic-minimizing deployment of
files that are unlikely to remain the same size when modified.  I'm
doing this as a project for a client who would like the change to be
incorporated into a release version on Collabnet's website.  I'm new
to subversion development, and I'm looking for any advice or
suggestions the developers here may have regarding the proposition.

Thanks,

Paul Breen

Re: proposed export option

Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
(+1 to what Ryan and Stefan said already)

Paul Breen wrote on Tue, 27 Apr 2010 at 01:57 -0700:
> My client is aware of rsync utility, but would prefer to use svn.  I'll
> have to correspond with her to find out why she doesn't want to use
> rsync.  As far as making the remote deployment a working folder and
> using svn update, she doesn't want to have the clutter of the hidden
> .svn folders on the deployed computers.

$ svn update $local_wc
$ rsync --exclude=.svn $local_wc/ $deployment_site/

Re: proposed export option

Posted by Stefan Sperling <st...@elego.de>.
On Tue, Apr 27, 2010 at 01:57:28AM -0700, Paul Breen wrote:
> For anyone who's interested, the details of the project I'm working on
> can be found at
> 
> www.rentacoder.com/RentACoder/misc/BidRequests/ShowBidRequest.asp?lngBidRequestId=1395905


Hi Paul,

On the above linked page, your client says:
"Following the coding guidelines for the project should be sufficient
to allow your contribution to be accepted."

This statement is not true and very misleading.
There a large number of other factors at play when contributing to
an open source project such as Subversion.

Of course, anyone can add specific features to Subversion for a client
and provide them with a custom Subversion distribution.
But if getting your work integrated into the official Subversion
distribution is part of the deal, you should be aware of the
implications of this.

Karl Fogel's book "Producing Open Source Software"
explains the implications well, especially this section:
http://producingoss.com/en/money.html

Quote of a relevant paragraph:
"""
Although money needs to be used carefully, that doesn't mean it can't
buy influence. It most certainly can. The trick is that it can't buy
influence directly. In a straightforward commercial transaction, you
trade money for what you want. If you need a feature added, you sign a
contract, pay for it, and it gets done. In an open source project, it's
not so simple. You may sign a contract with some developers, but they'd
be fooling themselves — and you — if they guaranteed that the work you paid
for would be accepted by the development community simply because you
paid for it. The work can only be accepted on its own merits and on how
it fits into the community's vision for the software. You may have some
say in that vision, but you won't be the only voice.
"""

So, the straightforward commercial transaction exists only between
the consultant and the client. If the client is happy with a custom
Subversion distribution provided by the consultant, all is fine and easy.

But when it gets to getting your work integrated into the project,
I think the second to last sentence sums up the situation much better
than what your client wrote:
"The work can only be accepted on its own merits and on how
it fits into the community's vision for the software."

So, with all that said, here is my own opinion:

The feature being suggested only applies to one particular use case.
>From the point of view of your client it may well be worth adding because
it fixes a problem for them (it is unclear to me what the problem is exactly,
but that doesn't matter for the purpose of this discussion).
But from the point of view of the community, it may not be worth adding
because the majority of users won't benefit.

To see why, imagine requests for other options such as
  svn export --skip-files-bigger-than-10MB
  svn export --skip-files-containing-java-code
  svn export --skip-empty-files
Why should a general-purpose version control system address such
needs with an endless amount of command line switches, each geared to
the needs of one particular small subset of users?

In my opinion a new feature should:
 1) fix a well understood general problem
 2) fix it without getting in the way of other features
 3) fix it in a way that is general enough to be useful to all users
    who face the problem

You may get a different list of items if you ask other developers,
but in general you will need to justify the necessity of your
new feature based on the above and/or similar criteria.

Stefan

Re: proposed export option

Posted by Stefan Sperling <st...@elego.de>.
On Tue, Apr 27, 2010 at 01:57:28AM -0700, Paul Breen wrote:
> For anyone who's interested, the details of the project I'm working on
> can be found at
> 
> www.rentacoder.com/RentACoder/misc/BidRequests/ShowBidRequest.asp?lngBidRequestId=1395905


Hi Paul,

On the above linked page, your client says:
"Following the coding guidelines for the project should be sufficient
to allow your contribution to be accepted."

This statement is not true and very misleading.
There a large number of other factors at play when contributing to
an open source project such as Subversion.

Of course, anyone can add specific features to Subversion for a client
and provide them with a custom Subversion distribution.
But if getting your work integrated into the official Subversion
distribution is part of the deal, you should be aware of the
implications of this.

Karl Fogel's book "Producing Open Source Software"
explains the implications well, especially this section:
http://producingoss.com/en/money.html

Quote of a relevant paragraph:
"""
Although money needs to be used carefully, that doesn't mean it can't
buy influence. It most certainly can. The trick is that it can't buy
influence directly. In a straightforward commercial transaction, you
trade money for what you want. If you need a feature added, you sign a
contract, pay for it, and it gets done. In an open source project, it's
not so simple. You may sign a contract with some developers, but they'd
be fooling themselves — and you — if they guaranteed that the work you paid
for would be accepted by the development community simply because you
paid for it. The work can only be accepted on its own merits and on how
it fits into the community's vision for the software. You may have some
say in that vision, but you won't be the only voice.
"""

So, the straightforward commercial transaction exists only between
the consultant and the client. If the client is happy with a custom
Subversion distribution provided by the consultant, all is fine and easy.

But when it gets to getting your work integrated into the project,
I think the second to last sentence sums up the situation much better
than what your client wrote:
"The work can only be accepted on its own merits and on how
it fits into the community's vision for the software."

So, with all that said, here is my own opinion:

The feature being suggested only applies to one particular use case.

Re: proposed export option

Posted by Ryan Schmidt <su...@ryandesign.com>.
On Apr 27, 2010, at 03:57, Paul Breen wrote:

> On Mon, 4/26/10, Ryan Schmidt wrote:
>> Changed since when?
> Changed since the last deployment.  The routine would send the file
> sizes of the files on the deployed computer to the repository server,
> and if the file sizes differed from the file sizes of the given revision
> (the current revision by default), the server would send the updated
> file to the deployed computer.

Ah, ok, so unlike with a regular export, the proposed "svn export --skipfilesmatchingsize" would operate on an existing directory. It would be like a more-efficient version of "svn export --force", then.

The danger of missing a file that needed to be updated, because the old and new filesizes were the same, seems like a concern that would keep the developers from accepting this proposal. Even if your client can guarantee that her files will always differ in filesize, that's not a statement that can be made about all users everywhere, and even if the option is named "skipfilesmatchingsize", some users may come to know the option as "export faster" and not think about what the option name means (or might not be native English speakers), might therefore use the option in cases where filesizes were the same, might therefore not receive updates to some files that did differ, and might experience problems as a result and/or complain here. If checksums were sent, instead of sizes, that should alleviate that concern. In that case one might even argue for naming the option "svn export --update".

The developers might still reject this feature on the grounds that it essentially duplicates the functionality of "svn update".


>> I think
>> you'll be better off using existing solutions to this
>> problem, such as rsync from a local svn export to your
>> remote deployment, or even simply making your remote
>> deployment a working copy and using svn update there.
> My client is aware of rsync utility, but would prefer to use svn.  I'll
> have to correspond with her to find out why she doesn't want to use
> rsync.  As far as making the remote deployment a working folder and
> using svn update, she doesn't want to have the clutter of the hidden
> .svn folders on the deployed computers.
> 
> For anyone who's interested, the details of the project I'm working on
> can be found at
> 
> www.rentacoder.com/RentACoder/misc/BidRequests/ShowBidRequest.asp?lngBidRequestId=1395905
> 
> Here my client gives the use cases for the feature she's looking for.
> I'll speak with her about the suggestions I got here and relay her
> responses on this mailing list.

It seems to me that before a bid request for a coder is advertised, the proposed changes should have already been accepted by the developers. But perhaps the process of getting the idea approved by the developers is part of what your client is paying you for. Note that I am not a Subversion developer, so it's not me you ultimately have to convince, but getting a bunch of Subversion users to agree that the idea is a good one is a good first step before approaching the developer list.

If part of your client's concern about using a working copy is the proliferation of .svn directories, she should know that Subversion 1.7 will feature a redesigned working copy model which will consolidate the .svn directories into a single location at the root of the working copy. Perhaps this will already help.

Re: proposed export option

Posted by Les Mikesell <le...@gmail.com>.
Paul Breen wrote:
> Thanks for your reply, Ryan.
> 
> On Mon, 4/26/10, Ryan Schmidt <su...@ryandesign.com> wrote:
>> Changed since when?
> Changed since the last deployment.  The routine would send the file
> sizes of the files on the deployed computer to the repository server,
> and if the file sizes differed from the file sizes of the given revision
> (the current revision by default), the server would send the updated
> file to the deployed computer.
> 
>> I think
>> you'll be better off using existing solutions to this
>> problem, such as rsync from a local svn export to your
>> remote deployment, or even simply making your remote
>> deployment a working copy and using svn update there.
> My client is aware of rsync utility, but would prefer to use svn.  I'll
> have to correspond with her to find out why she doesn't want to use
> rsync.  As far as making the remote deployment a working folder and
> using svn update, she doesn't want to have the clutter of the hidden
> .svn folders on the deployed computers.

This is as simple as using a working copy as a staging point, doing an update or 
switch to a release tag there, then running rsync with the -C option to copy to 
the production location(s).  Rsync also knows enough to copy only the changed 
parts of a large file with small changes, and it builds the new copy under a 
different name, renaming only when complete, where an svn export will probably 
break running programs that have those files open.

-- 
   Les Mikesell
    lesmikesell@gmail.com

Re: proposed export option

Posted by Paul Breen <gr...@yahoo.com>.
Thanks for your reply, Ryan.

On Mon, 4/26/10, Ryan Schmidt <su...@ryandesign.com> wrote:
> Changed since when?
Changed since the last deployment.  The routine would send the file
sizes of the files on the deployed computer to the repository server,
and if the file sizes differed from the file sizes of the given revision
(the current revision by default), the server would send the updated
file to the deployed computer.

> I think
> you'll be better off using existing solutions to this
> problem, such as rsync from a local svn export to your
> remote deployment, or even simply making your remote
> deployment a working copy and using svn update there.
My client is aware of rsync utility, but would prefer to use svn.  I'll
have to correspond with her to find out why she doesn't want to use
rsync.  As far as making the remote deployment a working folder and
using svn update, she doesn't want to have the clutter of the hidden
.svn folders on the deployed computers.

For anyone who's interested, the details of the project I'm working on
can be found at

www.rentacoder.com/RentACoder/misc/BidRequests/ShowBidRequest.asp?lngBidRequestId=1395905

Here my client gives the use cases for the feature she's looking for.
I'll speak with her about the suggestions I got here and relay her
responses on this mailing list.

Thanks,

--Paul Breen

Re: proposed export option

Posted by Ryan Schmidt <su...@ryandesign.com>.
On Apr 27, 2010, at 01:35, Ryan Schmidt wrote:

> or even simply making your remote deployment a working copy and using svn update there

And with this option, consider using SVN::Notify::Mirror:

http://search.cpan.org/perldoc?SVN::Notify::Mirror

Re: proposed export option

Posted by Ryan Schmidt <su...@ryandesign.com>.
On Apr 27, 2010, at 01:07, Paul Breen wrote:

> I'd like to add a new option to the export command that allows the user
> to export files only if the file size has changed

Changed since when?

I don't imagine this proposal will be accepted. I think you'll be better off using existing solutions to this problem, such as rsync from a local svn export to your remote deployment, or even simply making your remote deployment a working copy and using svn update there.


Re: proposed export option

Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Ryan Schmidt wrote on Wed, 2 Jun 2010 at 17:35 -0500:
> On Jun 2, 2010, at 13:48, Paul Breen wrote:
> 
> > I previously inquired about adding a –skipfilesmatchingsize option
> > to the export subcommand.  I have implemented that feature, and
> > I was wondering if it has any chance of making it into the trunk
> > directory for svn.  It currently has performance issues, and I don't
> > want to spend a lot of time optimizing the code if it's never going
> > to be incorporated into the main development line.
> > 
> > If the feature is not acceptable as described in my previous
> > message, could it be made acceptable if it were expanded somehow to
> > make it of more general use?  Maybe by using checksums instead of
> > file size or, more ambitiously, I was thinking something like
> > a rsync subcommand that allows a person to synchronize an
> > unversioned folder with an svn repository.
> > 

svn checkout --force

(will convert a .svn-less tree into a working copy)

> > Any comments or suggestions would be greatly appreciated.
> 
> You recall that there was already a discussion in April when you
> initially suggested this feature?
> 
> http://svn.haxx.se/users/archive-2010-04/index.shtml#405
> 
> There were objections by myself and others that what you proposed was
> already fairly easily accomplished using Subversion's existing
> commands, and you were going to inquire with your client as to why
> those existing methods weren't suitable.
> 
> 
> 

Re: proposed export option

Posted by Ryan Schmidt <su...@ryandesign.com>.
On Jun 2, 2010, at 13:48, Paul Breen wrote:

> I previously inquired about adding a –skipfilesmatchingsize option to the export subcommand.  I have implemented that feature, and I was wondering if it has any chance of making it into the trunk directory for svn.  It currently has performance issues, and I don't want to spend a lot of time optimizing the code if it's never going to be incorporated into the main development line.
> 
> If the feature is not acceptable as described in my previous message, could it be made acceptable if it were expanded somehow to make it of more general use?  Maybe by using checksums instead of file size or, more ambitiously, I was thinking something like a rsync subcommand that allows a person to synchronize an unversioned folder with an svn repository.
> 
> Any comments or suggestions would be greatly appreciated.

You recall that there was already a discussion in April when you initially suggested this feature?

http://svn.haxx.se/users/archive-2010-04/index.shtml#405

There were objections by myself and others that what you proposed was already fairly easily accomplished using Subversion's existing commands, and you were going to inquire with your client as to why those existing methods weren't suitable.

Re: proposed export option

Posted by Paul Breen <gr...@yahoo.com>.
Hello,

I previously inquired about adding a –skipfilesmatchingsize option to the export subcommand.  I have implemented that feature, and I was wondering if it has any chance of making it into the trunk directory for svn.  It currently has performance issues, and I don't want to spend a lot of time optimizing the code if it's never going to be incorporated into the main development line.

If the feature is not acceptable as described in my previous message, could it be made acceptable if it were expanded somehow to make it of more general use?  Maybe by using checksums instead of file size or, more ambitiously, I was thinking something like a rsync subcommand that allows a person to synchronize an unversioned folder with an svn repository.

Any comments or suggestions would be greatly appreciated.

--Paul Breen

--- On Mon, 4/26/10, Paul Breen <gr...@yahoo.com> wrote:

> From: Paul Breen <gr...@yahoo.com>
> Subject: proposed export option
> To: users@subversion.apache.org
> Date: Monday, April 26, 2010, 11:07 PM
> Hello,
> 
> I'd like to add a new option to the export command that
> allows the user
> to export files only if the file size has
> changed--something like
> 
>  svn export --skipfilesmatchingsize
> 
> The idea is to do a quick-and-easy, traffic-minimizing
> deployment of
> files that are unlikely to remain the same size when
> modified.  I'm
> doing this as a project for a client who would like the
> change to be
> incorporated into a release version on Collabnet's
> website.  I'm new
> to subversion development, and I'm looking for any advice
> or
> suggestions the developers here may have regarding the
> proposition.
> 
> Thanks,
> 
> Paul Breen
>