You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by "Malte S. Stretz" <ms...@gmx.net> on 2004/09/11 21:29:21 UTC
Our repository structure
Moin,
I noticed that we started started to mix the "classical" SVN structure with
{trunk,branches,tags} with other top-level dirs. Especially since
"updates" was added I have the feeling that this is the best way into
chaos ;~)
Which adds in is, that we (as an Apache Top-Level project) might (probably
will as I'm thinking about a few ones) offer official plugins for which we
either had get additional repos or throw into our own structure. I'd like
to avoid having to contact infra for a new repo each time I'd want to test
out some ideas I have (I like to use some revision management if I try out
things even if they will get completely dumped later on -- and if I use a
public one, nothing will get lost in some private dummy one I set up
somewhere). Another thing might if SARE ever wanted to use our repo, too.
So I'd suggest that we create something like sub-repos below spamassassin,
calling the main one "core" os something like that. The first attachment
in this mail is how I think it how it could look like if we used the common
SVN directories. (That would also be a a good time to get rid of the
obsolete tags we have).
Actually would I prefer something more flat like the second attachment
shows. In that case I split core up into two sub-repos where one holds the
pre-ASF code/tags/branches. I chose the naming scheme because that way
versions will be alphabetically grouped together.
Some things for the latter case I'm still thinking about about:
* The name of the branches. "3.0-branch" or maybe "3.0.x"?
* If we ever needs some tags/branches which aren't releases (like some
parallel code rework), we could maybe still use the SVN directory
structure beneath the release tags for that.
* Maybe we should split the sub-repos by the major version, like "core-1.x",
"core-2.x", "core-3.x"?
As always: Comments?
Cheers,
Malte
P.S.: Arguments like "existing software like the ASF script which produces
nightly builds" don't count as I volunteer to fix those scripts :)
--
[SGT] Simon G. Tatham: "How to Report Bugs Effectively"
<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>
[ESR] Eric S. Raymond: "How To Ask Questions The Smart Way"
<http://www.catb.org/~esr/faqs/smart-questions.html>
Re: Our repository structure
Posted by "Malte S. Stretz" <ms...@gmx.net>.
On Tuesday 14 September 2004 21:29 CET Daniel Quinlan wrote:
> "Malte S. Stretz" <ms...@gmx.net> writes:
> > It's the trust-stuff I already talked about. (Did I? Or did I write
> > in the Wiki?) What I want to try is:
> > * Query PGP servers for cross-signed keys and take that as a trust
> > path. * See what Web-O-Trust might give us. After it got revitalized
> > maybe ;-) * Have a look how FOAF fits into that scheme. Maybe create a
> > Web-O-Trust scheme to merge that information with FOAF.
> >
> > I don't want to throw that into trunk as it will be probably just some
> > kind of braindump in the beginning.
>
> This is definitely experimental, so I agree that you could probably
> start it elsewhere. The code in the tree should generally work well (as
> always), even with plugins in the mix. :-)
So are you fine if I then create a subdir plugin-trustnet (or something
along the lines) when I start hacking?
> > Another pluging which does IMO not belong into trunk is the PostgreSQL-
> > optimized SQL backend talked about.
>
> I completely disagree. I think we need an optimized SQL backend that is
> more compatible with the Apache License 2.0 and that backend is
> PostgreSQL. It's also cleanly modularized out.
>
> > Also the MSExec plugin Theo just committed should not be there.
>
> I think this is fine. I have plans to rename that plugin and generalize
> it, though, to just be a "binary" module.
The reason why I don't like to have them in trunk is that it just bloats the
codebase. The SQL functionality itself is used by very few people and I
guess a PostgreSQL-optimized one won't be used by even fewer people. The
MSExec/Binary Plugin will probably used by quite some people who think its
better than a Virus scanner but a whole lot of other people don't need it.
The same is true for the LDAP functionality and some other special stuff.
Maybe even Bayes, who knows.
So what point is it if we offer the possibility to modularize our codebase
but all we do with it is to enable or disable some parts of the
functionality while everybody still has to download and build that code and
run the regression tests (which already take ages)?
Or do you plan to pull the whole stuff into parts after it got downloaded,
maybe have Makefile.PL ask if I want to build the PostgreSQL Plugin and not
build, install, and test it it if the user choses "no"? That will give a
real unmaintainable mess in Makefile.PL and friends.
Why not split everything which is a plugin into its own module/subproject?
That would make things also much easier from the POV of the packagers. I
more or less maintain the ebuild for Gentoo and it would be much easier if
the user could install, say, 'spamassassin-plugin-msexec' and get that
plugin on top of his installed SA without the need to pull the whole
tarball and me to write the stuff to pull that one apart.
> > And there was IMO some discussion about splitting out the Pyzor,
> > Razor, etc. stuff.
>
> Please read my recent post about 3.1 goals. :-)
I did but I missed the discussion. Which doesn't mean that I don't think
it's a good idea.
> > And it seems like RC3 will be released tomorrow, the final version
> > should follow a week after :)
>
> RC5.
I meant subversion's RC3 :~)
> > I catually count "site" and "updates" as subprojects :) To make it
> > possible for every subproject to branch and tag if that might ever be
> > needed (even though we can't imageine any reason *now* doesn't mean
> > that there won't be one in the future -- like a major site redesign or
> > something), IIRC we should start with a consistent structure from the
> > first day. And if some subprojects will just contain a "trunk" for
> > all the time being, so be it, but lets call the spade a spade.
>
> They're *not* subprojects, though, just different "modules" in the SVN
> tree. I don't see *any* reason to create branches until (a) it is
> needed and (b) SVN 1.1 is out.
Bah, call it "subprojects" or call it "modules", I don't care about the
terminology. Just to make it clear:
(a) I don't want to create branches. I want to create the *possibility* to
create branches, ie move the site data below site/trunk so one day we
could branch if we wanted.
(b) To me the subdirs below "updates" are actually branches which are
updated nightly -- if I understood your proposal correctly.
(c) I do *not* want to change anything *now*. I'm fine with waiting for SVN
1.1. I'd just like to lay out a clean repository structure.
Cheers,
Malte
--
[SGT] Simon G. Tatham: "How to Report Bugs Effectively"
<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>
[ESR] Eric S. Raymond: "How To Ask Questions The Smart Way"
<http://www.catb.org/~esr/faqs/smart-questions.html>
Re: Our repository structure
Posted by Daniel Quinlan <qu...@pathname.com>.
"Malte S. Stretz" <ms...@gmx.net> writes:
> It's the trust-stuff I already talked about. (Did I? Or did I write in the
> Wiki?) What I want to try is:
> * Query PGP servers for cross-signed keys and take that as a trust path.
> * See what Web-O-Trust might give us. After it got revitalized maybe ;-)
> * Have a look how FOAF fits into that scheme. Maybe create a Web-O-Trust
> scheme to merge that information with FOAF.
>
> I don't want to throw that into trunk as it will be probably just some kind
> of braindump in the beginning.
This is definitely experimental, so I agree that you could probably
start it elsewhere. The code in the tree should generally work well (as
always), even with plugins in the mix. :-)
> Another pluging which does IMO not belong into trunk is the PostgreSQL-
> optimized SQL backend talked about.
I completely disagree. I think we need an optimized SQL backend that is
more compatible with the Apache License 2.0 and that backend is
PostgreSQL. It's also cleanly modularized out.
> Also the MSExec plugin Theo just committed should not be there.
I think this is fine. I have plans to rename that plugin and generalize
it, though, to just be a "binary" module.
> And there was IMO some discussion about splitting out the Pyzor,
> Razor, etc. stuff.
Please read my recent post about 3.1 goals. :-)
> And it seems like RC3 will be released tomorrow, the final version should
> follow a week after :)
RC5.
> I catually count "site" and "updates" as subprojects :) To make it
> possible for every subproject to branch and tag if that might ever be
> needed (even though we can't imageine any reason *now* doesn't mean
> that there won't be one in the future -- like a major site redesign or
> something), IIRC we should start with a consistent structure from the
> first day. And if some subprojects will just contain a "trunk" for
> all the time being, so be it, but lets call the spade a spade.
They're *not* subprojects, though, just different "modules" in the SVN
tree. I don't see *any* reason to create branches until (a) it is
needed and (b) SVN 1.1 is out.
Daniel
--
Daniel Quinlan
http://www.pathname.com/~quinlan/
Re: Our repository structure
Posted by Theo Van Dinter <fe...@kluge.net>.
On Tue, Sep 14, 2004 at 09:19:13PM +0200, Malte S. Stretz wrote:
> Another pluging which does IMO not belong into trunk is the PostgreSQL-
> optimized SQL backend talked about. Also the MSExec plugin Theo just
Sure it should.
> committed should not be there. And there was IMO some discussion about
Why not?
> splitting out the Pyzor, Razor, etc. stuff.
3.1 goal, yeah.
--
Randomly Generated Tagline:
#define NULL 0 /* silly thing is, we don't even use this */
-- Larry Wall in perl.c from the perl source code
Re: Our repository structure
Posted by "Malte S. Stretz" <ms...@gmx.net>.
On Saturday 11 September 2004 23:23 CET Daniel Quinlan wrote:
> "Malte S. Stretz" <ms...@gmx.net> writes:
> > Updates consists of branches per se, it probably won't ever have a
> > "trunk" (at least so it looks to me, I actually have no clue what that
> > directory is intended for -- can somebody who knoes please put a
> > README in there?). This is also not about branching and tagging but
> > about keeping stuff separate.
>
> I've posted about the daily updates stuff several times on this list,
> including the directory hierarchy I would use. I'll add a README,
> though. :-)
Yeah, I was pretty sure that I read something about it but my search query
didn't qield anything. On a second try it looks like I'm simply too stupid
to use my mailer's search function ;-)
> > I plan to start working on a plugin the next few weeks -- where shall
> > I put it?
>
> It depends on the plug-in. Maybe in trunk, maybe elsewhere.
It's the trust-stuff I already talked about. (Did I? Or did I write in the
Wiki?) What I want to try is:
* Query PGP servers for cross-signed keys and take that as a trust path.
* See what Web-O-Trust might give us. After it got revitalized maybe ;-)
* Have a look how FOAF fits into that scheme. Maybe create a Web-O-Trust
scheme to merge that information with FOAF.
I don't want to throw that into trunk as it will be probably just some kind
of braindump in the beginning.
Another pluging which does IMO not belong into trunk is the PostgreSQL-
optimized SQL backend talked about. Also the MSExec plugin Theo just
committed should not be there. And there was IMO some discussion about
splitting out the Pyzor, Razor, etc. stuff.
> > This will be fixed in 1.1 [1] which is scheduled for mid-September, an
> > RC2 is currently available (though I don't know how stable it is).
> > This sounds near enough to discuss this now, doesn't it?
And it seems like RC3 will be released tomorrow, the final version should
follow a week after :)
> I'm not in a hurry. ;-)
Me neither, once we came to a conclusion that doesn't mean that this should
happen yesterday :)
> I don't really mind having trunk and branches in the root of the tree be
> for the core project, I think that's fine.
I don't like that at all, it's messy all over. It's not really a technical
reason, but something which will annoy me each time I browse the
repository. And once the diff issue is solved with subversion 1.1 there's
no reason not to move the code into a sane place (once and for all).
> For other subprojects (none yet) or trees (site and updates), each should
> be able to not be branched or to be branched.
I catually count "site" and "updates" as subprojects :) To make it possible
for every subproject to branch and tag if that might ever be needed (even
though we can't imageine any reason *now* doesn't mean that there won't be
one in the future -- like a major site redesign or something), IIRC we
should start with a consistent structure from the first day. And if some
subprojects will just contain a "trunk" for all the time being, so be it,
but lets call the spade a spade.
> I think the hierarchy is mostly okay now, but maybe when SVN is updated we
> agree on some small changes.
Can't we agree now and apply later? :)
> I think it's important to note that the tree structure has never caused
> a single significant problem that this would solve.
That's true, till now (see the site-branch-stuff above). OTOH do I really
like to keep stuff like this clean -- call it German Perfectionism if you
want ;~) And currently we are at a point where it's still easy to agree on
a unified repository structure and pin it down for the future.
> > We're all pretty quick giving -1s lately. Can't we discuss stuff first
> > before we throw vetoes into the ring?
>
> +0 maybe we can wait longer before veto, but I really meant it and I
> believe I had good technical reasons. :-)
A veto always ends a discussion in a very harsh way, maybe before it even
started. Even with good technical reasons these might get reconsidered.
For example did I not like the idea of including a debian/ directory in the
trunk, for several reasons. But instead of vetoing it, I slept a night
over it and came to the conclusion that I could live with it and didn't
even comment on it :)
Cheers,
Malte
--
[SGT] Simon G. Tatham: "How to Report Bugs Effectively"
<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>
[ESR] Eric S. Raymond: "How To Ask Questions The Smart Way"
<http://www.catb.org/~esr/faqs/smart-questions.html>
Re: Our repository structure
Posted by Daniel Quinlan <qu...@pathname.com>.
"Malte S. Stretz" <ms...@gmx.net> writes:
> Updates consists of branches per se, it probably won't ever have a
> "trunk" (at least so it looks to me, I actually have no clue what that
> directory is intended for -- can somebody who knoes please put a
> README in there?). This is also not about branching and tagging but
> about keeping stuff separate.
I've posted about the daily updates stuff several times on this list,
including the directory hierarchy I would use. I'll add a README,
though. :-)
> I plan to start working on a plugin the next few weeks -- where shall
> I put it?
It depends on the plug-in. Maybe in trunk, maybe elsewhere.
> This will be fixed in 1.1 [1] which is scheduled for mid-September, an
> RC2 is currently available (though I don't know how stable it is).
> This sounds near enough to discuss this now, doesn't it?
I'm not in a hurry. ;-)
I don't really mind having trunk and branches in the root of the tree be
for the core project, I think that's fine. For other subprojects (none
yet) or trees (site and updates), each should be able to not be branched
or to be branched. I think the hierarchy is mostly okay now, but maybe
when SVN is updated we agree on some small changes.
I think it's important to note that the tree structure has never caused
a single significant problem that this would solve.
> We're all pretty quick giving -1s lately. Can't we discuss stuff first
> before we throw vetoes into the ring?
+0 maybe we can wait longer before veto, but I really meant it and I
believe I had good technical reasons. :-)
Daniel
--
Daniel Quinlan
http://www.pathname.com/~quinlan/
Re: Our repository structure
Posted by "Malte S. Stretz" <ms...@gmx.net>.
On Saturday 11 September 2004 22:14 CET Daniel Quinlan wrote:
> "Malte S. Stretz" <ms...@gmx.net> writes:
> > I noticed that we started started to mix the "classical" SVN structure
> > with {trunk,branches,tags} with other top-level dirs. Especially since
> > "updates" was added I have the feeling that this is the best way into
> > chaos ;~)
>
> Most of those trees (both "updates" and "site") are probably never going
> to fork. I think keeping this simple is probably the right way to do
> this. Also, it's still pretty simple, so I think an overhaul is
> premature.
Updates consists of branches per se, it probably won't ever have a
"trunk" (at least so it looks to me, I actually have no clue what that
directory is intended for -- can somebody who knoes please put a README in
there?). This is also not about branching and tagging but about keeping
stuff separate. I plan to start working on a plugin the next few weeks --
where shall I put it?
> Finally, I don't want to even think about moving stuff around until the
> latest stable SVN version supports following a file automatically across
> moves. I understand that feature is being added, but right now, you
> have to specify the full path (including the repository name, location,
> and version) to diff across a filename move.
This will be fixed in 1.1 [1] which is scheduled for mid-September, an RC2
is currently available (though I don't know how stable it is). This sounds
near enough to discuss this now, doesn't it?
> We can think about creating a "core" directory or whatever later if the
> tree gets too chaotic, but definitely not before SVN handles moves much
> more gracefully.
I actually like to plan such things in advance, not when its too late. In a
few months the argument argument against might be be "every expects the
trunk to be where its now, we can't change it".
> -1 for the above reasons
We're all pretty quick giving -1s lately. Can't we discuss stuff first
before we throw vetoes into the ring?
Cheers,
Malte
[1] http://subversion.tigris.org/issues/show_bug.cgi?id=1093
--
[SGT] Simon G. Tatham: "How to Report Bugs Effectively"
<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>
[ESR] Eric S. Raymond: "How To Ask Questions The Smart Way"
<http://www.catb.org/~esr/faqs/smart-questions.html>
Re: Our repository structure
Posted by Daniel Quinlan <qu...@pathname.com>.
"Malte S. Stretz" <ms...@gmx.net> writes:
> I noticed that we started started to mix the "classical" SVN structure with
> {trunk,branches,tags} with other top-level dirs. Especially since
> "updates" was added I have the feeling that this is the best way into
> chaos ;~)
Most of those trees (both "updates" and "site") are probably never going
to fork. I think keeping this simple is probably the right way to do
this. Also, it's still pretty simple, so I think an overhaul is
premature.
Finally, I don't want to even think about moving stuff around until the
latest stable SVN version supports following a file automatically across
moves. I understand that feature is being added, but right now, you
have to specify the full path (including the repository name, location,
and version) to diff across a filename move.
We can think about creating a "core" directory or whatever later if the
tree gets too chaotic, but definitely not before SVN handles moves much
more gracefully.
-1 for the above reasons
Daniel
--
Daniel Quinlan
http://www.pathname.com/~quinlan/