You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by hw...@hotmail.com on 2008/12/09 03:03:19 UTC
Recursive externals problem
I've seem similar questions in the mail list but haven't found a convincing solutions, so I would like to start a discussion on following problem again.
Example:
projectB -depend-on-> projectA
projectC -depend-on-> projectA
projectD -depend-on-> projectA, projectB, projectC
When I checkout projectB, projectA will be checkout through externals property, when I checkout projectC, projectA will be checkout through externals property as well. For projectD, it is configured with 3 externals properties to projectA, projectB and projectC, so when I checkout projectD, projectA, projectB and projectC will be checkout as well, but projectA will be checkout thrice as projectB and projectC also configured with externals properties to it.
As my project grow larger, the extra checkout causes by the recursive externals checkout became a major problem as a lot of time and space are wasted. In the example above, I only need 3 projects but svn checkout 5 of them.
I read a bit on the new "--depth" option but not quite sure about whether it is the solution to this problem or not. Anyone can share some ideas on how to deal with this kind of problem?
IMHO, I'm thinking of adding a new "recursive level" option in the externals property might help, for the example above, in externals properties of projectD we specify the "recursive level" of projectA, projectB, projectC to be "0", thus svn will only checkout projectA, projectB and projectC, and will not checkout externals properties of projectB and projectC, thus we will have only 1 copy of projectA. I would like to gather more opinions on this before I decide to try to work on patch for this option.
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=981444
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Re: Re: Recursive externals problem
Posted by Hilco Wijbenga <hi...@gmail.com>.
2008/12/12 Koh Hwee Miin <hw...@hotmail.com>:
> By the way, can you evaluate more about "think twice before I'd commit build results into Subversion"? I'm still a newbie when managing projects on svn, hope you won't mind sharing your experience on svn.
If you commit your build results you'll never be sure how those build
results were generated. Imagine being in a hurry so you quickly fix
some bug (locally) and commit only the build results. Now the source
and binaries no longer match. It can happen without any bad intent,
simply the result of a mistake or omission.
The build should be reproducible and should be the only way to create
the binaries. After that, an official release process can make those
binaries available to everybody else. Any changes should happen in
code first, followed by a release (with a new unique version number).
> Also, do you know any good articles/books on managing c++ projects on svn?
No, but I encourage you to have a look at Maven
(http://maven.apache.org/). This is mostly for Java projects so it
will not help you directly but the ideas behind it are universally
applicable. Note, especially, the notion of a central repository for
binaries (so you don't have to rebuild all your dependencies every
time).
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=983863
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Re: Re: Recursive externals problem
Posted by Hilco Wijbenga <hi...@gmail.com>.
2008/12/12 Koh Hwee Miin <hw...@hotmail.com>:
> By the way, can you evaluate more about "think twice before I'd commit build results into Subversion"? I'm still a newbie when managing projects on svn, hope you won't mind sharing your experience on svn.
If you commit your build results you'll never be sure how those build
results were generated. Imagine being in a hurry so you quickly fix
some bug (locally) and commit only the build results. Now the source
and binaries no longer match. It can happen without any bad intent,
simply the result of a mistake or omission.
The build should be reproducible and should be the only way to create
the binaries. After that, an official release process can make those
binaries available to everybody else. Any changes should happen in
code first, followed by a release (with a new unique version number).
> Also, do you know any good articles/books on managing c++ projects on svn?
No, but I encourage you to have a look at Maven
(http://maven.apache.org/). This is mostly for Java projects so it
will not help you directly but the ideas behind it are universally
applicable. Note, especially, the notion of a central repository for
binaries (so you don't have to rebuild all your dependencies every
time).
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=983863
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=988222
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
RE: Re: Re: Recursive externals problem
Posted by Koh Hwee Miin <hw...@hotmail.com>.
Thanks Hilco, that's a cool idea.
By the way, can you evaluate more about "think twice before I'd commit build results into Subversion"? I'm still a newbie when managing projects on svn, hope you won't mind sharing your experience on svn.
Also, do you know any good articles/books on managing c++ projects on svn?
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=983301
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Re: Recursive externals problem
Posted by Hilco Wijbenga <hi...@gmail.com>.
2008/12/10 Koh Hwee Miin <hw...@hotmail.com>:
> Following is the layout of our project directory:
> projectA\
> |-bin\
> |-lib\
> |-include\
> |-src\
> |-depend\
> So externals will put all depending projects in the "depend" folder, after we do a development build, the outputs will be put into the "bin", "lib" and "include" folders, then these outputs will be committed to the svn, so other projects which depend on this project can reuse the development build outputs, thus no need to build the depending projects again. This is to save build time during development.
I would think twice before I'd commit build results into Subversion
... but that's up to you.
> Back to the projectA, projectB, projectC and projectD example. In this case when we try to build projectD, the only project involves in the build is the projectD itself, as projectA, projectB and projectC already have their build outputs ready when they are checkout from svn, projectD will reuse those build outputs during it's build.
If I were you I'd change your setup a little. Create projectA without
/depend as codeProjectA, the same for codeProjectB, etcetera. Then
create umbrella projects projectA, projectB, projectC, and projectD
that simply have an svn:externals link to their dependencies. You'd
get
codeProjectA
bin, lib, include, src
codeProjectB
bin, lib, include, src
codeProjectC
bin, lib, include, src
codeProjectD
bin, lib, include, src
projectA
link_to_codeProjectA
projectB
link_to_codeProjectA
link_to_codeProjectB
projectC
link_to_codeProjectA
link_to_codeProjectC
projectD
link_to_codeProjectA
link_to_codeProjectB
link_to_codeProjectC
link_to_codeProjectD
All projects now know exactly where to find everything and every
project is checked out only once.
Cheers,
Hilco
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=983274
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
RE: Re: Recursive externals problem
Posted by Koh Hwee Miin <hw...@hotmail.com>.
Yes, I should have explained our svn directory structure first...
Following is the layout of our project directory:
projectA\
|-bin\
|-lib\
|-include\
|-src\
|-depend\
So externals will put all depending projects in the "depend" folder, after we do a development build, the outputs will be put into the "bin", "lib" and "include" folders, then these outputs will be committed to the svn, so other projects which depend on this project can reuse the development build outputs, thus no need to build the depending projects again. This is to save build time during development.
Back to the projectA, projectB, projectC and projectD example. In this case when we try to build projectD, the only project involves in the build is the projectD itself, as projectA, projectB and projectC already have their build outputs ready when they are checkout from svn, projectD will reuse those build outputs during it's build.
This is my way of managing development build, may not be the best way of course. Anyone who like the discuss on the "managing development build" related issues are welcome also.
We have different strategy on the build server/production build, but those are no longer related to this "recursive externals" problem I raised.
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=982647
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Recursive externals problem
Posted by Hilco Wijbenga <hi...@gmail.com>.
2008/12/8 <hw...@hotmail.com>:
> Example:
> projectB -depend-on-> projectA
> projectC -depend-on-> projectA
> projectD -depend-on-> projectA, projectB, projectC
<snip/>
> IMHO, I'm thinking of adding a new "recursive level" option in the externals property might help, for
> the example above, in externals properties of projectD we specify the "recursive level" of projectA,
> projectB, projectC to be "0", thus svn will only checkout projectA, projectB and projectC, and will
> not checkout externals properties of projectB and projectC, thus we will have only 1 copy of
> projectA. I would like to gather more opinions on this before I decide to try to work on patch for this
> option.
I don't really understand how Subversion would be able to help you. If
it simply ignores the svn:externals to projectA when checking out
projectB (e.g.) then how does projectB's build (or whatever is
appropriate in your environment) know where to find projectA? It would
be somewhere under projectC and projectD but how would projectB's
build know this?
Each project would expect its dependencies in specific places, you
can't just remove those, can you?
Cheers,
Hilco
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=981935
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Recursive externals problem
Posted by Hilco Wijbenga <hi...@gmail.com>.
2008/12/8 <hw...@hotmail.com>:
> Example:
> projectB -depend-on-> projectA
> projectC -depend-on-> projectA
> projectD -depend-on-> projectA, projectB, projectC
<snip/>
> IMHO, I'm thinking of adding a new "recursive level" option in the externals property might help, for
> the example above, in externals properties of projectD we specify the "recursive level" of projectA,
> projectB, projectC to be "0", thus svn will only checkout projectA, projectB and projectC, and will
> not checkout externals properties of projectB and projectC, thus we will have only 1 copy of
> projectA. I would like to gather more opinions on this before I decide to try to work on patch for this
> option.
I don't really understand how Subversion would be able to help you. If
it simply ignores the svn:externals to projectA when checking out
projectB (e.g.) then how does projectB's build (or whatever is
appropriate in your environment) know where to find projectA? It would
be somewhere under projectC and projectD but how would projectB's
build know this?
Each project would expect its dependencies in specific places, you
can't just remove those, can you?
Cheers,
Hilco
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=981935
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=982511
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Recursive externals problem
Posted by Hilco Wijbenga <hi...@gmail.com>.
2008/12/8 <hw...@hotmail.com>:
> Example:
> projectB -depend-on-> projectA
> projectC -depend-on-> projectA
> projectD -depend-on-> projectA, projectB, projectC
<snip/>
> IMHO, I'm thinking of adding a new "recursive level" option in the externals property might help, for
> the example above, in externals properties of projectD we specify the "recursive level" of projectA,
> projectB, projectC to be "0", thus svn will only checkout projectA, projectB and projectC, and will
> not checkout externals properties of projectB and projectC, thus we will have only 1 copy of
> projectA. I would like to gather more opinions on this before I decide to try to work on patch for this
> option.
I don't really understand how Subversion would be able to help you. If
it simply ignores the svn:externals to projectA when checking out
projectB (e.g.) then how does projectB's build (or whatever is
appropriate in your environment) know where to find projectA? It would
be somewhere under projectC and projectD but how would projectB's
build know this?
Each project would expect its dependencies in specific places, you
can't just remove those, can you?
Cheers,
Hilco
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=981938
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
Re: Recursive externals problem
Posted by Hilco Wijbenga <hi...@gmail.com>.
2008/12/8 <hw...@hotmail.com>:
> Example:
> projectB -depend-on-> projectA
> projectC -depend-on-> projectA
> projectD -depend-on-> projectA, projectB, projectC
<snip/>
> IMHO, I'm thinking of adding a new "recursive level" option in the externals property might help, for
> the example above, in externals properties of projectD we specify the "recursive level" of projectA,
> projectB, projectC to be "0", thus svn will only checkout projectA, projectB and projectC, and will
> not checkout externals properties of projectB and projectC, thus we will have only 1 copy of
> projectA. I would like to gather more opinions on this before I decide to try to work on patch for this
> option.
I don't really understand how Subversion would be able to help you. If
it simply ignores the svn:externals to projectA when checking out
projectB (e.g.) then how does projectB's build (or whatever is
appropriate in your environment) know where to find projectA? It would
be somewhere under projectC and projectD but how would projectB's
build know this?
Each project would expect its dependencies in specific places, you
can't just remove those, can you?
Cheers,
Hilco
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=981938
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=982497
To unsubscribe from this discussion, e-mail: [users-unsubscribe@subversion.tigris.org].