You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@corinthia.apache.org by "Dennis E. Hamilton" <de...@acm.org> on 2015/01/01 20:31:13 UTC
Visual Studio Solutions (was RE: platform with minzip, commit or not commit.)
Do we know of any ASF Project that includes Visual Studio artifacts in its source-code tree? If someone has managed it, we need to find out what they do and how they satisfy the strict rules about what Apache Project Releases are.
- Dennis
-- responding below to --
From: jan i [mailto:jani@apache.org]
Sent: Thursday, January 1, 2015 02:15
To: dev@corinthia.incubator.apache.org; Dennis Hamilton
Subject: Re: platform with minzip, commit or not commit.
[ ... ]
I still believe we should have visual studio solution checked in, so that
windows developers are up an running in no time.
[ ... ]
BACKGROUND OBSERVATIONS
The TL;DR of it: I can do it, but not without putting binary artifacts in the tree.
DETAILS
I have been looking, in a side project, at how to include Visual Studio Solutions in the source tree without making the code hard to work with if you don't want to (or can't) use Visual Studio. What I have been doing there, but can't do here, is put the VS project into a Zip that is extracted into a directory that is not under source control and that Visual Studio can be launched on. In that context I don't need a command-line way to unzip the project, I can just use the Windows GUI to extract the zip into the directory whenever I want to use the Visual Studio Project. These are all tiny little projects.
What may be useful to know for Corinthia is this:
* A Visual Studio solution .sln file is just a text file.
It appears to be rather stable, in that launching VS from it does
not cause it to be altered.
* A Visual Studio VC project .vcxproj file is an XML (MSBuild) Document.
It is not altered by a build. Configuration and setting changes might
impact it.
* The Visual Studio VC Project Filter .vcxproj.filters file is another
MSBuild XML Document. This is essential - it changes if anything is
added to the code, whether includes to use, more source files, any
resources, or locations of those. (I have the source code external
to the project, not in the same directory with the project.)
* The Visual Studio VC Project User .vcxproj.user file is another
MSBuild XML Document. I don't know what has this change. Mine
don't have much in them.
Then there are some binary artifacts
* The .sdf file is a cache that Visual Studio builds, which is quite
large. This does not have to be preserved. It can be gigantic.
It will be rebuilt on the first Build or Compile.
* The VS User Options .suo file (one for each version of Visual Studio
that has been used) is a *binary* file. And it is needed in order
to preserve the compilation configuration. For example, the target
platform and whether Debug or Release. The extension is .v12.suo
for Visual Studio 2013. It is .v14.suo for Visual Studio 2015 and
2015 will upgrade the .v12.suo to a .v14.suo but leave the .v12
one.
Here are the problems I see with including these directly in a source tree:
* They include one binary artifact, the User Options file.
* They are not actually source code, even when in text or XML, in that
producing or understanding them manually, while possible, is not
normal and they are useless to non-Visual Studio developers.
* They can change through incidental use that need not mean the source
tree actually needs to be synchronized (and we'd get collisions on this
stuff that would be bizarre to resolve).
Bundling these into Zip files so they are stable and in the code tree but
their extracted form is not works just fine *except* it puts a binary
artifact in the source-code tree and that is quite toxic on ASF projects.
I see no point in fighting City Hall on this one.
In a way, such Zips are a form of convenience binary, just not the usual
kind. But they aren't derived from anything in the source tree, so they
are really external artifacts. And they need to be injected into a
working copy of the source tree to be useful, since they operate along-
side the source code.
I don't like where this is leading. It makes no sense to treat Visual
Studio projects as externals.
An alternative would be places in the source code tree are sort of "Your
Visual Studio Project Here" with instructions on what project to create.
That's even more unpleasant in my thinking.
Maybe there's a hybrid case. There can be placeholders in the source
tree. That makes sense. There are probably alternative build arrangements
nearby. One could have a convenience binary that is a Zip for Visual
Studio Developers. If downloaded to the root of the working copy of the
Source and extracted right there, it could interleave the Visual Studio
Projects in just the right places (along with all the .gitignores that
keep its material out of the source tree).
This seems totally bizarre.
RE: Visual Studio Solutions (was RE: platform with minzip, commit or not commit.)
Posted by "Dennis E. Hamilton" <de...@acm.org>.
-- replying below to --
From: jan i [mailto:jani@apache.org]
Sent: Thursday, January 1, 2015 13:06
To: dev@corinthia.incubator.apache.org; Dennis Hamilton
Subject: Re: Visual Studio Solutions (was RE: platform with minzip, commit or not commit.)
On 1 January 2015 at 20:31, Dennis E. Hamilton <de...@acm.org>
wrote:
> Do we know of any ASF Project that includes Visual Studio artifacts in its
> source-code tree? If someone has managed it, we need to find out what they
> do and how they satisfy the strict rules about what Apache Project Releases
> are.
>
At the moment I dont know other projects, except that in AOO we have run
experiments how to do this (capstone branch).
But since it is XML files, there are no problem seen from the bylaws of
Apache, that is something I checked last year before starting the capstone
project in AOO.
<orcmid>
As long as the binary files that are automatically placed in
solutions/project directories .sdf and .suo are in .gitignore,
that could work.
I also think it is messy if the source code is mingled in the
solution/project directories. As well as I can tell from the
student work, there was a lot of copying back and forth. I
have not broken the code on that. I do know the solutions
don't work if opened where they are on the capstone branch.
Also, because I opened one solution in a later version of
Visual Studio than it was created with, the .vcxproj file
was changed, so I had to revert it to prevent my checkout
of the SVN being marked as having commits to be made.
Oh my, and cygWin .sh scripts. Ah well.
</orcmid>
[ ... ]
Re: Visual Studio Solutions (was RE: platform with minzip, commit or not commit.)
Posted by Peter Kelly <pm...@apache.org>.
> On 2 Jan 2015, at 5:31 am, Dennis E. Hamilton <de...@acm.org> wrote:
>
> So when a VS user clones the repo and opens the solution, the
> VC Project files will be broken because there is little chance
> that the absolute path is preserved (except by happy accident).
> So this user has to run CMake to get new VC Project files to use?
>
> Is that it?
> </orcmid>
>
> [ ... ]
>
> Please remember that the origin is CMakeFile.txt, so whenever someone
> changes in CMakeFile.txt a cmake -G must be run to update the VC files.
> This was if I remember right why peter did not like my proposal, the fear
> of those not being in sync.
Correct - If we have visual studio projects in the repository (and one could make a similar argument for Xcode project and Unix makefiles) then it becomes quite problematic when someone wants to add a new file. With CMake, all they have to do is add a single line to the CMakeLists.txt, then all platforms will get the file. If we maintain IDE-specific project files, then the person who adds a new file in say VC has to also update the Xcode project file as well, and vice-versa. CMake gives us “one source of truth”, so to speak, from which all others are derived.
> <orcmid>
> OK, now I am flummoxed by the need to handle a CMake dependency.
> We are expecting the Visual Studio user to use CMake to build the
> system? And to make the vc project files?
>
> What about when a Visual Studio developer *starts* some code using
> Visual Studio. He is adding some source to the tree, he is putting
> his project and solution in some prescribed place so he can work
> on the code in Visual Studio, or he simply wants to add a project
> To the solution that is already there.
>
> And he expects to commit his source and the new solution/project.
>
> I am having difficulty imagining the instructions that the VS-skilled
> developer is given to have this work.
I agree it adds complexity to the act of creating a new file - but it’s a necessary evil if we want the project to continue building on all platforms. If I were working off an Xcode project in the repository and broke the build for windows and Linux, it would be very inconvenient for others, who would then have to go and debug the problem to figure out what had changed.
We’ll need to write clear instructions about how to use CMake. It’s not that complicated to do once you know the steps, and means we can maintain cross-platform interoperability.
> <orcmid>
> Well, I would love to say that, but I don't think it is accurate.
> The CMakeFile.txt is not generated is it? I thought it was
> inserted by the developer to help CMake do the right thing.
>
> Am I mistaken?
> </orcmid>
That’s correct - it instructs the build system in use what to compile and how to compile it (in a way that works with any supported build system).
> I am also not coping with the constraints of a build-generation system
> like CMake. I would love to never get to that. These are for my
> personal projects that I want beginners to be able to use [;<).
Again, it’s necessary if we want something that builds on multiple platforms. For a Windows-only project, I would go with VC project files for sure.
—
Dr Peter M. Kelly
pmkelly@apache.org
PGP key: http://www.kellypmk.net/pgp-key <http://www.kellypmk.net/pgp-key>
(fingerprint 5435 6718 59F0 DD1F BFA0 5E46 2523 BAA1 44AE 2966)
RE: Visual Studio Solutions (was RE: platform with minzip, commit or not commit.)
Posted by "Dennis E. Hamilton" <de...@acm.org>.
-- replying below to --
From: jan i [mailto:jani@apache.org]
Sent: Thursday, January 1, 2015 13:06
To: dev@corinthia.incubator.apache.org; Dennis Hamilton
Subject: Re: Visual Studio Solutions (was RE: platform with minzip, commit or not commit.)
On 1 January 2015 at 20:31, Dennis E. Hamilton <de...@acm.org>
wrote:
[ ... ]
> * A Visual Studio solution .sln file is just a text file.
> It appears to be rather stable, in that launching VS from it does
> not cause it to be altered.
>
That is correct, this is the first file we need to commit.
There is a slight challenge here, cmake generates full path, and we need
relative path (to enable windows programmers to relocate the sources).
Microsoft has no problem with relative path.
* A Visual Studio VC project .vcxproj file is an XML (MSBuild) Document.
> It is not altered by a build. Configuration and setting changes might
> impact it.
>
These files are changed by cmake, and should not be changed manually. One
should not change settings within visual studio.
There is a slight challenge here, cmake generates full path, and we need
relative path (to enable windows programmers to relocate the sources).
Microsoft has no problem with relative path.
<orcmid>
Checking my understanding.
When I create a VC Project myself, using existing source files, it
will provide relative links to those files in the VC Project file.
So, it does not matter where the root of the code tree is cloned,
since its absolute location does not matter, so long as the
relative pathes from the VC Project to the existing source files in
the repository are preserved.
When CMake produces VC Project files, those will have absolute
paths to the existing source files.
So when a VS user clones the repo and opens the solution, the
VC Project files will be broken because there is little chance
that the absolute path is preserved (except by happy accident).
So this user has to run CMake to get new VC Project files to use?
Is that it?
</orcmid>
[ ... ]
Please remember that the origin is CMakeFile.txt, so whenever someone
changes in CMakeFile.txt a cmake -G must be run to update the VC files.
This was if I remember right why peter did not like my proposal, the fear
of those not being in sync.
<orcmid>
OK, now I am flummoxed by the need to handle a CMake dependency.
We are expecting the Visual Studio user to use CMake to build the
system? And to make the vc project files?
What about when a Visual Studio developer *starts* some code using
Visual Studio. He is adding some source to the tree, he is putting
his project and solution in some prescribed place so he can work
on the code in Visual Studio, or he simply wants to add a project
To the solution that is already there.
And he expects to commit his source and the new solution/project.
I am having difficulty imagining the instructions that the VS-skilled
developer is given to have this work.
... and we're worrying about unzipping. Heh.
</orcmid>
[ ... ]
> * The VS User Options .suo file (one for each version of Visual Studio
> that has been used) is a *binary* file. And it is needed in order
> to preserve the compilation configuration. For example, the target
> platform and whether Debug or Release. The extension is .v12.suo
> for Visual Studio 2013. It is .v14.suo for Visual Studio 2015 and
> 2015 will upgrade the .v12.suo to a .v14.suo but leave the .v12
> one.
>
No it is not needed, it will be generated if not there. The compilation
configuration, is stored as XML in the vcxproj file.
<orcmid>
I just double-checked. I have a configuration set to build x86 and
Release. If I delete the .suo file, the solution opens with the
default x86 and Debug configuration.
</orcmid>
>
> Here are the problems I see with including these directly in a source tree:
[ ... ]
> * They are not actually source code, even when in text or XML, in that
> producing or understanding them manually, while possible, is not
> normal and they are useless to non-Visual Studio developers
You could say the same about our CMakeFile.txt
<orcmid>
Well, I would love to say that, but I don't think it is accurate.
The CMakeFile.txt is not generated is it? I thought it was
inserted by the developer to help CMake do the right thing.
Am I mistaken?
</orcmid>
[ ... ]
> Bundling these into Zip files so they are stable and in the code tree but
> their extracted form is not works just fine *except* it puts a binary
> artifact in the source-code tree and that is quite toxic on ASF projects.
> I see no point in fighting City Hall on this one.
>
NO binary artifacts, that is illegal, but also not needed. In the capstone
project (AOO), we managed to generate all that was needed with ONLY XML
files.
I would also not make a zip file. I would make a directory build.windows
and commit it normally.
<orcmid>
My use of zip is to keep the Visual Studio parts out of the direct
source tree. (The zip is there, but it is the zip that is committed,
not the individual Visual Studio components.)
I can do that on my projects, because it is not at the ASF.
It also bypasses problems like the one I mentioned where Visual Studio
for any reason modifies one of those artifacts when VS is used to
work on the code.
I do see that there is a lot of copying around done in the capstone
project. I am managing just to extract the build materials and leave
the source code in its common location.
I am also not coping with the constraints of a build-generation system
like CMake. I would love to never get to that. These are for my
personal projects that I want beginners to be able to use [;<).
</orcmid>
[ ... ]
Re: Visual Studio Solutions (was RE: platform with minzip, commit or
not commit.)
Posted by jan i <ja...@apache.org>.
On 1 January 2015 at 20:31, Dennis E. Hamilton <de...@acm.org>
wrote:
> Do we know of any ASF Project that includes Visual Studio artifacts in its
> source-code tree? If someone has managed it, we need to find out what they
> do and how they satisfy the strict rules about what Apache Project Releases
> are.
>
At the moment I dont know other projects, except that in AOO we have run
experiments how to do this (capstone branch).
But since it is XML files, there are no problem seen from the bylaws of
Apache, that is something I checked last year before starting the capstone
project in AOO.
>
> - Dennis
>
> -- responding below to --
> From: jan i [mailto:jani@apache.org]
> Sent: Thursday, January 1, 2015 02:15
> To: dev@corinthia.incubator.apache.org; Dennis Hamilton
> Subject: Re: platform with minzip, commit or not commit.
>
> [ ... ]
>
> I still believe we should have visual studio solution checked in, so that
> windows developers are up an running in no time.
>
> [ ... ]
>
> BACKGROUND OBSERVATIONS
>
> The TL;DR of it: I can do it, but not without putting binary artifacts in
> the tree.
>
> DETAILS
>
> I have been looking, in a side project, at how to include Visual Studio
> Solutions in the source tree without making the code hard to work with if
> you don't want to (or can't) use Visual Studio. What I have been doing
> there, but can't do here, is put the VS project into a Zip that is
> extracted into a directory that is not under source control and that Visual
> Studio can be launched on. In that context I don't need a command-line way
> to unzip the project, I can just use the Windows GUI to extract the zip
> into the directory whenever I want to use the Visual Studio Project. These
> are all tiny little projects.
>
A Zip file, would be a solution, but not one I prefer.
>
> What may be useful to know for Corinthia is this:
>
> * A Visual Studio solution .sln file is just a text file.
> It appears to be rather stable, in that launching VS from it does
> not cause it to be altered.
>
That is correct, this is the first file we need to commit.
There is a slight challenge here, cmake generates full path, and we need
relative path (to enable windows programmers to relocate the sources).
Microsoft has no problem with relative path.
* A Visual Studio VC project .vcxproj file is an XML (MSBuild) Document.
> It is not altered by a build. Configuration and setting changes might
> impact it.
>
These files are changed by cmake, and should not be changed manually. One
should not change settings within visual studio.
There is a slight challenge here, cmake generates full path, and we need
relative path (to enable windows programmers to relocate the sources).
Microsoft has no problem with relative path.
Again these are files we need to commit.
>
> * The Visual Studio VC Project Filter .vcxproj.filters file is another
> MSBuild XML Document. This is essential - it changes if anything is
> added to the code, whether includes to use, more source files, any
> resources, or locations of those. (I have the source code external
> to the project, not in the same directory with the project.)
>
Everything changes (at least vcxproj and filter file both change when a
file is added/removed).
Please remember that the origin is CMakeFile.txt, so whenever someone
changes in CMakeFile.txt a cmake -G must be run to update the VC files.
This was if I remember right why peter did not like my proposal, the fear
of those not being in sync.
These 3 files are the only ones you need to start visual studio. Try for
the fun of it, to delete all other visual studio files, and you will see
the others get recreated (I do it very often, in another project).
* The Visual Studio VC Project User .vcxproj.user file is another
> MSBuild XML Document. I don't know what has this change. Mine
> don't have much in them.
>
This is not needed, but VC generates it. In a multiuser environment it
allows you to have local personal settings.
>
> Then there are some binary artifacts
>
> * The .sdf file is a cache that Visual Studio builds, which is quite
> large. This does not have to be preserved. It can be gigantic.
> It will be rebuilt on the first Build or Compile.
>
yes
>
> * The VS User Options .suo file (one for each version of Visual Studio
> that has been used) is a *binary* file. And it is needed in order
> to preserve the compilation configuration. For example, the target
> platform and whether Debug or Release. The extension is .v12.suo
> for Visual Studio 2013. It is .v14.suo for Visual Studio 2015 and
> 2015 will upgrade the .v12.suo to a .v14.suo but leave the .v12
> one.
>
No it is not needed, it will be generated if not there. The compilation
configuration, is stored as XML in the vcxproj file.
>
> Here are the problems I see with including these directly in a source tree:
>
> * They include one binary artifact, the User Options file.
>
not needed
> * They are not actually source code, even when in text or XML, in that
> producing or understanding them manually, while possible, is not
> normal and they are useless to non-Visual Studio developers
You could say the same about our CMakeFile.txt
> .
> * They can change through incidental use that need not mean the sourceYou
> tree actually needs to be synchronized (and we'd get collisions on this
> stuff that would be bizarre to resolve).
>
You could say the same about our CMakeFile.txt
>
> Bundling these into Zip files so they are stable and in the code tree but
> their extracted form is not works just fine *except* it puts a binary
> artifact in the source-code tree and that is quite toxic on ASF projects.
> I see no point in fighting City Hall on this one.
>
NO binary artifacts, that is illegal, but also not needed. In the capstone
project (AOO), we managed to generate all that was needed with ONLY XML
files.
I would also not make a zip file. I would make a directory build.windows
and commit it normally.
> In a way, such Zips are a form of convenience binary, just not the usual
> kind. But they aren't derived from anything in the source tree, so they
> are really external artifacts. And they need to be injected into a
> working copy of the source tree to be useful, since they operate along-
> side the source code.
>
no zip.
>
> I don't like where this is leading. It makes no sense to treat Visual
> Studio projects as externals.
>
No we should not, and not as zip, we should treat it as a normal directory
(build.windows) with xml files (.sln .vcxproj, .vcxproj.filter)
>
> An alternative would be places in the source code tree are sort of "Your
> Visual Studio Project Here" with instructions on what project to create.
> That's even more unpleasant in my thinking.
>
I agree that would be unpleasant.
>
> Maybe there's a hybrid case. There can be placeholders in the source
> tree. That makes sense. There are probably alternative build arrangements
> nearby. One could have a convenience binary that is a Zip for Visual
> Studio Developers. If downloaded to the root of the working copy of the
> Source and extracted right there, it could interleave the Visual Studio
> Projects in just the right places (along with all the .gitignores that
> keep its material out of the source tree).
>
Why not use the simple way I suggest.
>
> This seems totally bizarre.
>
> rgds
jan i