You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Ben Reser <be...@reser.org> on 2005/04/01 21:30:52 UTC
Re: svn commit: r13838 - branches/1.1.x
On Fri, Apr 01, 2005 at 02:24:15PM -0600, jerenkrantz@tigris.org wrote:
> # USAGE: ./dist.sh -v VERSION -r REVISION [-rs REVISION-SVN] [-pr REPOS-PATH]
> -# [-apr PATH-TO-APR ] [-apru PATH-TO-APR-UTIL]
> -# [-apri PATH-TO-APR-ICONV] [-neon PATH-TO-NEON]
> -# [-zip] [-alpha ALPHA_NUM|-beta BETA_NUM|-rc RC_NUM]
> +# [-apr APR-PATH] [-apu APR-UTIL-PATH]
> +# [-api APR-ICONV-PATH] [-neon NEON-PATH]
> +# [-apr-tag APR-TAG] [-apu-tag APR-UTIL-TAG]
> +# [-api-tag APR-ICONV-TAG] [-neon-tag NEON-TAG]
> +# [-zip] [-sign] [-fetch]
> +# [-alpha ALPHA_NUM|-beta BETA_NUM|-rc RC_NUM]
The option names -apri and -apru were discussed at length when I added
them. Karl felt that -api was ambiguous and shouldn't be used. Please
change this back.
> -echo "Copying $APR_PATH into sandbox, making extraclean..."
> -cp -r "$APR_PATH" "$DISTPATH/apr"
> -(cd "$DISTPATH/apr" && [ -f Makefile ] && make extraclean)
> -echo "Removing all CVS/ and .cvsignore files from apr..."
> -find "$DISTPATH/apr" -name CVS -type d -print | xargs rm -fr
> -find "$DISTPATH/apr" -name .cvsignore -print | xargs rm -f
> -
> -echo "Copying $APRU_PATH into sandbox, making extraclean..."
> -cp -r "$APRU_PATH" "$DISTPATH/apr-util"
> -(cd "$DISTPATH/apr-util" && [ -f Makefile ] && make extraclean)
> -echo "Removing all CVS/ and .cvsignore files from apr-util..."
> -find "$DISTPATH/apr-util" -name CVS -type d -print | xargs rm -fr
> -find "$DISTPATH/apr-util" -name .cvsignore -print | xargs rm -f
I really think we should leave the cleanup code here just for paranoia
purposes. Your change completely wipes it out.
> +install_dependency()
> +{
> + DEP_NAME=$1
> + DEP_TAG=$2
> + if [ -z $3 ]; then
> + DEP_PATH=/dev/null
> + else
> + DEP_PATH=$3
> + fi
> + if [ -d $DEP_PATH ]; then
> + if [ -d $DEP_PATH/.svn ]; then
> + echo "Exporting local $DEP_NAME into sandbox"
> + ${SVN:-svn} export -q $EXTRA_EXPORT_OPTIONS "$DEP_PATH" "$DISTPATH/$DEP_NAME"
> + else
> + echo "Copying local $DEP_NAME into sandbox"
> + cp -r "$DEP_PATH" "$DISTPATH/$DEP_NAME"
> + (cd "$DISTPATH/$DEP_NAME" && [ -f Makefile ] && make extraclean)
> + echo "Removing all CVS/ and .cvsignore files from $DEP_NAME..."
> + find "$DISTPATH/$DEP_NAME" -name CVS -type d -print | xargs rm -fr
> + find "$DISTPATH/$DEP_NAME" -name .cvsignore -print | xargs rm -f
> + fi
> + else
> + echo "Exporting $DEP_NAME into sandbox ($DEP_TAG)"
> + ${SVN:-svn} export -q $EXTRA_EXPORT_OPTIONS $DEP_TAG "$DISTPATH/$DEP_NAME"
> + fi
> +}
> +
> +install_dependency apr http://svn.apache.org/repos/asf/apr/apr/tags/${APR_TAG} "$APR_PATH"
> +install_dependency apr-util http://svn.apache.org/repos/asf/apr/apr-util/tags/${APU_TAG} "$APU_PATH"
I reiterate the veto I made to introducing code to use svn export to
fetch dependencies in the past. The dependencies must be verified as
valid releases. I don't see anyway to do that with export. Tarballs
have PGP/GPG signatures that can be validated. I leave it up to the
individual projects to be sure that however they got their code they put
in their tarball is valid. If they want to trust their repo that's up
to them. But I really think we should only be using released tarballs
for dependencies.
> if [ -n "$ZIP" ]; then
> - echo "Copying $APRI_PATH into sandbox, making extraclean..."
> - cp -r "$APRI_PATH" "$DISTPATH/apr-iconv"
> - (cd "$DISTPATH/apr-iconv" && [ -f Makefile ] && make extraclean)
> - echo "Removing all CVS/ and .cvsignore files from apr-iconv..."
> - find "$DISTPATH/apr-iconv" -name CVS -type d -print | xargs rm -fr
> - find "$DISTPATH/apr-iconv" -name .cvsignore -print | xargs rm -f
> + install_dependency apr-iconv http://svn.apache.org/repos/asf/apr/apr-iconv/tags/${API_TAG} "$API_PATH"
> fi
>
> -echo "Coping neon into sandbox, making clean..."
> -cp -r "$NEON_PATH" "$DISTPATH/neon"
> -(cd "$DISTPATH/neon" && [ -f Makefile ] && make distclean)
> -echo "Cleaning *.o in neon..."
> -find "$DISTPATH/neon/src" -name '*.o' -print | xargs rm -f
> +install_dependency neon http://svn.webdav.org/repos/projects/neon/tags/${NEON_TAG} "$NEON_PATH"
Again more cleanup code ripped out.
--
Ben Reser <be...@reser.org>
http://ben.reser.org
"Conscience is the inner voice which warns us somebody may be looking."
- H.L. Mencken
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Purpose of dist.sh was Re: svn commit: r13838 - branches/1.1.x
Posted by kf...@collab.net.
Justin Erenkrantz <ju...@erenkrantz.com> writes:
> I feel that dist.sh is a 'canonical' way for *anyone* to roll a
> Subversion tarball. dist.sh should encapsulate any knowledge that we
> have to create a 'proper' Subversion tarball.
>
> The only thing that makes our 'public' tarball special is that we
> decide to call it 'Subversion' - anyone else can still roll a tarball,
> but they just can't call it Subversion. What they do (or don't do)
> with it is their concern.
> Therefore, we should be willing to accept patches (and/or commits)
> that don't compromise or conflict with dist.sh for our internal
> purposes. This is what I would expect for any file under our purview.
> ("You want to add some new functionality? Submit your patches!") --
I don't see what the last part has to do with anything. We reject
patches all the time, on the grounds that we don't want to maintain
the extra complexity in Subversion. In fact, I believe rejection (of
a patch or of a proposed patch) is far more common than acceptance.
I don't agree with this proposed mission for dist.sh. I feel it is
unclear and gives us no guidance about how to maintain the script. I
already said why in detail in my last mail, so I won't repeat that
here.
So: two people disagree somewhat about what dist.sh should do. That's
fine. Let's hear what others have to say. It may be that a clear
consensus emerges. If not, well, we can cross that bridge when we
come to it.
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Purpose of dist.sh was Re: svn commit: r13838 - branches/1.1.x
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
--On Saturday, April 2, 2005 10:46 PM -0600 kfogel@collab.net wrote:
> Okay, it sounds like we're homing in on the central question here:
>
> "What and who is dist.sh for?"
>
> If the answer is "It's for us to produce our releases with", then the
> export changes probably don't belong in it.
>
> If the answer is "It's for anyone who wants to roll some sort of
> Subversion release" (or something like that), then the export features
> start to make sense, and possibly so do a lot of other things. In
> fact, it becomes difficult to know what features we can leave *out*!
> After all, we know what it means for us to roll our own releases --
> that's a well-understood task. But we don't really know what other
> groups might be up to in rolling specialized releases of their own.
> How could we ever judge dist.sh's feature set or interface with any
> confidence, if go down this road?
I feel that dist.sh is a 'canonical' way for *anyone* to roll a Subversion
tarball. dist.sh should encapsulate any knowledge that we have to create a
'proper' Subversion tarball.
The only thing that makes our 'public' tarball special is that we decide to
call it 'Subversion' - anyone else can still roll a tarball, but they just
can't call it Subversion. What they do (or don't do) with it is their concern.
Therefore, we should be willing to accept patches (and/or commits) that don't
compromise or conflict with dist.sh for our internal purposes. This is what I
would expect for any file under our purview. ("You want to add some new
functionality? Submit your patches!") -- justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r13838 - branches/1.1.x
Posted by kf...@collab.net.
Okay, it sounds like we're homing in on the central question here:
"What and who is dist.sh for?"
If the answer is "It's for us to produce our releases with", then the
export changes probably don't belong in it.
If the answer is "It's for anyone who wants to roll some sort of
Subversion release" (or something like that), then the export features
start to make sense, and possibly so do a lot of other things. In
fact, it becomes difficult to know what features we can leave *out*!
After all, we know what it means for us to roll our own releases --
that's a well-understood task. But we don't really know what other
groups might be up to in rolling specialized releases of their own.
How could we ever judge dist.sh's feature set or interface with any
confidence, if go down this road?
Therefore, I prefer the first answer. I think we should consider
dist.sh to be the tool we use to roll our own releases, nothing more.
When we originally wrote it, we never envisioned it taking on any more
burden than that. Why would we try to expand its mission now? I feel
like the compelling use cases and examples just aren't there. Sure, a
few people have done it from time to time, but people do lots of
things that the Subversion project doesn't provide direct support for.
In the long run, it'll be much easier to maintain dist.sh if it has a
narrow, clear mission.
That's my $0.02. Justin, if I'm missing any important parts of the
argument you're making, sorry & please correct.
Thanks,
-Karl
Justin Erenkrantz <ju...@erenkrantz.com> writes:
> --On Saturday, April 2, 2005 4:26 PM -0800 Ben Reser <be...@reser.org> wrote:
>
> > a) This is a feature... When we add features we have to way the
> > benefits, costs, and if it's even really useful. We have tried to shy
> > away from implementing features that are highly specialized to a
> > particlar group of people or that nobody has really ever asked for or
> > can come up with a use case for.
> >
> > Granted, I know of some use cases for this, but they are rare. I can
> > think of all of 4 people that I'm aware of that have ever run dist.sh
> > recently. And none of them have ever had a problem getting
> > dependencies.
>
> Might not their jobs be easier if there was another way? I'd would
> like to reduce the complexity required by the person running the
> dist.sh script to make it easier to roll a tarball (be it for random
> use or for public distribution).
>
> > b) We also have to look at the alternatives to the feature. What could
> > we do that might be better.
>
> The 'svn export' is a solution that allows the dependencies to bundle
> the dependencies without having them downloaded previously. To be
> clear, in IRC on Friday, I had suggested that dist.sh have the option
> to fetch the release tarballs from the site and allow the RM to
> confirm the dependency signatures manually before proceeding. But,
> you said that you would veto that as well even if I wrote that
> functionality.
>
> > I still think if we want to make it easier to produce a quick snapshot
> > tarball we make dist.sh capable of simply ignoring the depencies. Our
> > build system has no problem with this and it would be insanely simple to
> > do in dist.sh
>
> Yes, I agree that could be goodness as well. (i.e. -nodeps flag to dist.sh?)
>
> > c) We have to look at the drawbacks. In this particular case the vast
> > majority of the use of it is to produce our own tarball. If this
> > feature is used to produce the tarball it could be really bad. I'm not
> > going to harp on why again.
>
> And, as I've said, I don't think that's a major issue.
>
> > Wouldn't it lessen the overhead even more just by removing the need to
> > include the dependencies? We include them to lessen the burden on our
> > users downloading the tarball. But as far as I know most of the use
> > cases of dist.sh outside of our uses don't care about these dependencies
> > and only deal with them because dist.sh requires them.
>
> In the cases where I was creating snapshots, I was creating them for
> folks who didn't have the dependencies either (i.e. an OS with just a
> compiler that needed some SVN patches that weren't in a SVN release
> just yet). -- justin
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r13838 - branches/1.1.x
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
--On Saturday, April 2, 2005 4:26 PM -0800 Ben Reser <be...@reser.org> wrote:
> a) This is a feature... When we add features we have to way the
> benefits, costs, and if it's even really useful. We have tried to shy
> away from implementing features that are highly specialized to a
> particlar group of people or that nobody has really ever asked for or
> can come up with a use case for.
>
> Granted, I know of some use cases for this, but they are rare. I can
> think of all of 4 people that I'm aware of that have ever run dist.sh
> recently. And none of them have ever had a problem getting
> dependencies.
Might not their jobs be easier if there was another way? I'd would like to
reduce the complexity required by the person running the dist.sh script to
make it easier to roll a tarball (be it for random use or for public
distribution).
> b) We also have to look at the alternatives to the feature. What could
> we do that might be better.
The 'svn export' is a solution that allows the dependencies to bundle the
dependencies without having them downloaded previously. To be clear, in IRC
on Friday, I had suggested that dist.sh have the option to fetch the release
tarballs from the site and allow the RM to confirm the dependency signatures
manually before proceeding. But, you said that you would veto that as well
even if I wrote that functionality.
> I still think if we want to make it easier to produce a quick snapshot
> tarball we make dist.sh capable of simply ignoring the depencies. Our
> build system has no problem with this and it would be insanely simple to
> do in dist.sh
Yes, I agree that could be goodness as well. (i.e. -nodeps flag to dist.sh?)
> c) We have to look at the drawbacks. In this particular case the vast
> majority of the use of it is to produce our own tarball. If this
> feature is used to produce the tarball it could be really bad. I'm not
> going to harp on why again.
And, as I've said, I don't think that's a major issue.
> Wouldn't it lessen the overhead even more just by removing the need to
> include the dependencies? We include them to lessen the burden on our
> users downloading the tarball. But as far as I know most of the use
> cases of dist.sh outside of our uses don't care about these dependencies
> and only deal with them because dist.sh requires them.
In the cases where I was creating snapshots, I was creating them for folks who
didn't have the dependencies either (i.e. an OS with just a compiler that
needed some SVN patches that weren't in a SVN release just yet). -- justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r13838 - branches/1.1.x
Posted by Ben Reser <be...@reser.org>.
On Sat, Apr 02, 2005 at 03:44:31PM -0800, Justin Erenkrantz wrote:
> The issue here is that creating a tarball for your own purposes would be,
> IMHO, overly complicated if you had to find and extract release tarballs of
> all the dependencies. Creating a 'private distribution' (or
> limited-release distribution; which I used to do back in the day to package
> releases to some clients and users) would be streamlined by just exporting
> from the upstream repositories.
I guess I'm looking at this from two points:
a) This is a feature... When we add features we have to way the
benefits, costs, and if it's even really useful. We have tried to shy
away from implementing features that are highly specialized to a
particlar group of people or that nobody has really ever asked for or
can come up with a use case for.
Granted, I know of some use cases for this, but they are rare. I can
think of all of 4 people that I'm aware of that have ever run dist.sh
recently. And none of them have ever had a problem getting
dependencies.
b) We also have to look at the alternatives to the feature. What could
we do that might be better.
I suspect that the few people that are producing such tarballs probably
don't care about including dependencies in them anyway. They're
producing them for a specialized audience that they've probably already
have the dependencies installed.
I still think if we want to make it easier to produce a quick snapshot
tarball we make dist.sh capable of simply ignoring the depencies. Our
build system has no problem with this and it would be insanely simple to
do in dist.sh
c) We have to look at the drawbacks. In this particular case the vast
majority of the use of it is to produce our own tarball. If this
feature is used to produce the tarball it could be really bad. I'm not
going to harp on why again.
> I think having dist.sh capable of running the svn export lessens the
> overhead required to get a tarball out the door. For public releases, we
> can state that the RM must use extracted tarballs.
Wouldn't it lessen the overhead even more just by removing the need to
include the dependencies? We include them to lessen the burden on our
users downloading the tarball. But as far as I know most of the use
cases of dist.sh outside of our uses don't care about these dependencies
and only deal with them because dist.sh requires them.
--
Ben Reser <be...@reser.org>
http://ben.reser.org
"Conscience is the inner voice which warns us somebody may be looking."
- H.L. Mencken
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r13838 - branches/1.1.x
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
--On Saturday, April 2, 2005 2:46 PM -0600 kfogel@collab.net wrote:
> [I realize this may be asking you to repeat stuff you've already said
> in IRC.]
>
> Are such releases made often, and for purposes that we want to
> support? In general, I agree that we should be using released
> tarballs only for our own releases (sounds like no one disagrees on
> that point).
>
> I don't feel like having the export code in there is a huge deal
> either way, but I'd like to understand the justification better.
The issue here is that creating a tarball for your own purposes would be,
IMHO, overly complicated if you had to find and extract release tarballs of
all the dependencies. Creating a 'private distribution' (or limited-release
distribution; which I used to do back in the day to package releases to some
clients and users) would be streamlined by just exporting from the upstream
repositories.
I think having dist.sh capable of running the svn export lessens the overhead
required to get a tarball out the door. For public releases, we can state
that the RM must use extracted tarballs. -- justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r13838 - branches/1.1.x
Posted by kf...@collab.net.
Justin Erenkrantz <ju...@erenkrantz.com> writes:
> > I reiterate the veto I made to introducing code to use svn export to
> > fetch dependencies in the past. The dependencies must be verified as
> > valid releases. I don't see anyway to do that with export. Tarballs
> > have PGP/GPG signatures that can be validated. I leave it up to the
> > individual projects to be sure that however they got their code they put
> > in their tarball is valid. If they want to trust their repo that's up
> > to them. But I really think we should only be using released tarballs
> > for dependencies.
>
> I'm concerned that this adds an extra step that may not always be
> needed. I'd be fine with saying that official releases must use
> extracted tarballs, but I think that there are legitimate times
> (i.e. for non-official releases) where fetching the dependencies via
> svn export is fine. Therefore, to make it easier to produce those
> types of releases, I think the export calls should stay. Note that
> using 'svn export' isn't the default - using the local files is. --
> justin
[I realize this may be asking you to repeat stuff you've already said
in IRC.]
Are such releases made often, and for purposes that we want to
support? In general, I agree that we should be using released
tarballs only for our own releases (sounds like no one disagrees on
that point).
I don't feel like having the export code in there is a huge deal
either way, but I'd like to understand the justification better.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: svn commit: r13838 - branches/1.1.x
Posted by Justin Erenkrantz <ju...@erenkrantz.com>.
--On Friday, April 1, 2005 1:30 PM -0800 Ben Reser <be...@reser.org> wrote:
> The option names -apri and -apru were discussed at length when I added
> them. Karl felt that -api was ambiguous and shouldn't be used. Please
> change this back.
apr, apu, and api are the official acronyms used by APR (i.e. apu-config).
Therefore, for consistency, I believe we need to follow their acronyms not
come up with our own acronyms.
> I really think we should leave the cleanup code here just for paranoia
> purposes. Your change completely wipes it out.
No, it doesn't. It just moved inside the 'install_dependency' function.
As we discussed in #svn, we should switch it to 'distclean' instead of
'extraclean' so that it works with neon. The extra steps included in
'extraclean' are not really required for APR-* because we're re-running
buildconf which overwrites configure, aclocal.m4, and
apr_private_config.h.in, etc. anyway.
So, switching to 'distclean' should be sufficient for our purposes. Agreed?
(This only matters for folks who use 'dirty' tarball extracts rather than
'clean' tarball extracts.)
> I reiterate the veto I made to introducing code to use svn export to
> fetch dependencies in the past. The dependencies must be verified as
> valid releases. I don't see anyway to do that with export. Tarballs
> have PGP/GPG signatures that can be validated. I leave it up to the
> individual projects to be sure that however they got their code they put
> in their tarball is valid. If they want to trust their repo that's up
> to them. But I really think we should only be using released tarballs
> for dependencies.
I'm concerned that this adds an extra step that may not always be needed.
I'd be fine with saying that official releases must use extracted tarballs,
but I think that there are legitimate times (i.e. for non-official
releases) where fetching the dependencies via svn export is fine.
Therefore, to make it easier to produce those types of releases, I think
the export calls should stay. Note that using 'svn export' isn't the
default - using the local files is. -- justin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org