You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Matt Broadstone <mb...@gmail.com> on 2016/09/02 02:14:07 UTC
gitattributes
Hey all,
Would it be possible to fix up the .gitattributes for the qpid-cpp git
repository? I'm having an incredibly hard time updating the sources for the
debian packaging because there are many mixed in CRLF files throughout the
repository (seemingly without rhyme or reason either, i.e. not all .ps1 or
.bat files have CRLF line endings). Anyone know how to clean this up easily?
Thanks,
Matt
Re: gitattributes
Posted by Alan Conway <ac...@redhat.com>.
On Fri, 2016-09-02 at 12:39 -0400, Andrew Stitcher wrote:
> On Fri, 2016-09-02 at 12:11 -0400, Matt Broadstone wrote:
> >
> > ...
> > >
> > > Can you explain exactly what the problem you are seeing actually
> > > is?
> > >
> > >
> > Sure.
> >
> > The problem comes down to debian's sanity checks to ensure that
> > files
> > have
> > not been modified locally before building and packaging the
> > resultant
> > deb
> > file.
> > ...
>
> Thanks for the explanation. I think the issue may be that changing
> the
> repository has changed the line endings. Svn line ending handling and
> git line ending handling are different.
>
> >
> >
> > Generally speaking though, I think this is a problem you'll want to
> > solve
> > anyway given that you have mixed OS developers accessing the code
> > on
> > a
> > somewhat regular basis.
>
> Well, I did put some effort into solving this issue many years ago
> for
> Svn and things quieted down! I think there may actually be an
> attributes file for svn which does this stuff.
>
> For git typically you don't want to set up any attributes for this at
> all and just let git check out exactly what ever is checked in. I
> suspect this is the issue - svn was changing the line endings to suit
> the platform but git doesn't.
>
> As far as I can tell, Visual Studio (and all competant Windows
> editors
> - just don't use notepad!) cope fine with \n line endings nowadays so
> I'd be in favor of fixing the line endings in all text files to be \n
> rather than adding a whole slew of attributes.
>
> I'd rather enforce this on commits, than change what's checked out
> with
> attributes.
>�
+1 although git seems to have a more flexible story than SVN. It can
automatically adjust line endings between your local clone repo and
your workspace, not just on push or pull. So a .gitattributes file
could enforce normalization at everybody's clone, not just on push.
People who like CRLF could set up local attributes to get them added
back in their own workspace, even though they never appear in any repo.
Haven't played with it before so: Warning! May contain nonsense.
Prepared in a facility that uses nonsense.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: gitattributes
Posted by Andrew Stitcher <as...@redhat.com>.
On Fri, 2016-09-02 at 12:11 -0400, Matt Broadstone wrote:
> ...
> > Can you explain exactly what the problem you are seeing actually
> > is?
> >
> >
> Sure.
>
> The problem comes down to debian's sanity checks to ensure that files
> have
> not been modified locally before building and packaging the resultant
> deb
> file.
> ...
Thanks for the explanation. I think the issue may be that changing the
repository has changed the line endings. Svn line ending handling and
git line ending handling are different.
>
> Generally speaking though, I think this is a problem you'll want to
> solve
> anyway given that you have mixed OS developers accessing the code on
> a
> somewhat regular basis.
Well, I did put some effort into solving this issue many years ago for
Svn and things quieted down! I think there may actually be an
attributes file for svn which does this stuff.
For git typically you don't want to set up any attributes for this at
all and just let git check out exactly what ever is checked in. I
suspect this is the issue - svn was changing the line endings to suit
the platform but git doesn't.
As far as I can tell, Visual Studio (and all competant Windows editors
- just don't use notepad!) cope fine with \n line endings nowadays so
I'd be in favor of fixing the line endings in all text files to be \n
rather than adding a whole slew of attributes.
I'd rather enforce this on commits, than change what's checked out with
attributes.
Andrew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: gitattributes
Posted by Matt Broadstone <mb...@gmail.com>.
On Fri, Sep 2, 2016 at 11:51 AM, Andrew Stitcher <as...@redhat.com>
wrote:
> On Thu, 2016-09-01 at 22:14 -0400, Matt Broadstone wrote:
> > Hey all,
> >
> > Would it be possible to fix up the .gitattributes for the qpid-cpp
> > git
> > repository? I'm having an incredibly hard time updating the sources
> > for the
> > debian packaging because there are many mixed in CRLF files
> > throughout the
> > repository
>
> Can you explain exactly what the problem you are seeing actually is?
>
>
Sure.
The problem comes down to debian's sanity checks to ensure that files have
not been modified locally before building and packaging the resultant deb
file.
When you guys make a new release, I import those tarballs using
`git-buildpackage` to a new `upstream/<rel>` tag, which is to be used later
to generate the `orig` (pristine) tarball for the sanity check. What seems
to be the problem (though I'm really guessing at best here, since I'm not a
debian packager by trade) is that line endings have been changed on a few
files between the releases, and so we now have differences in newly
imported files - as well as line ending changes - the latter of which are
not just "handled" for me during the merge. What happens then is that
debian's sanity checks when recreating the `orig` tarball indicate that
I've made a bunch of changes and need to resolve that.
Generally speaking though, I think this is a problem you'll want to solve
anyway given that you have mixed OS developers accessing the code on a
somewhat regular basis. If anyone here is far stronger in
debian-packaging-foo than I, please let me know - I'm sure I could use help
;)
Matt
I have had no problem checking out the code from the repo on both Linux
> and Windows and building there.
>
> What is different about creating debian packages?
>
> Andrew
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: gitattributes
Posted by Andrew Stitcher <as...@redhat.com>.
On Thu, 2016-09-01 at 22:14 -0400, Matt Broadstone wrote:
> Hey all,
>
> Would it be possible to fix up the .gitattributes for the qpid-cpp
> git
> repository? I'm having an incredibly hard time updating the sources
> for the
> debian packaging because there are many mixed in CRLF files
> throughout the
> repository�
Can you explain exactly what the problem you are seeing actually is?
I have had no problem checking out the code from the repo on both Linux
and Windows and building there.
What is different about creating debian packages?
Andrew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: gitattributes
Posted by Chuck Rolke <cr...@redhat.com>.
----- Original Message -----
> From: "Alan Conway" <ac...@redhat.com>
> To: users@qpid.apache.org
> Sent: Friday, September 2, 2016 10:44:35 AM
> Subject: Re: gitattributes
>
> On Fri, 2016-09-02 at 08:49 +0200, Volker Diels-Grabsch wrote:
> > Matt Broadstone schrieb:
> > >
> > > Would it be possible to fix up the .gitattributes for the qpid-cpp
> > > git
> > > repository? I'm having an incredibly hard time updating the sources
> > > for the
> > > debian packaging because there are many mixed in CRLF files
> > > throughout the
> > > repository (seemingly without rhyme or reason either, i.e. not all
> > > .ps1 or
> > > .bat files have CRLF line endings). Anyone know how to clean this
> > > up easily?
> >
> > This sounds like it should be really cleaned up in the repository and
> > not just on packaging. Maybe the following helps. This is my
> > general
> > way to cleanup sources, i.e. not specific for qpid-cpp.
> >
> > You can cleanup e.g. all "*.cpp" and "*.in" files with a single
> > command:
> >
> > cd qpid-cpp-0.34
> > find * -type f -name '*.cpp' -exec sed 's/\r//g' -i {} \;
> > find * -type f -name '*.in' -exec sed 's/\r//g' -i {} \;
>
> That will only work until folks start re-editing the files with their
> random win/unix editors and change the endings again. If you want it to
> stay pure forever I think we need .gitattributes to define file types
> by extension as "text" so that git normalizes them, and then let users
> set their core.eol to whatever they prefer for text files in their
> working tree. Its probably more complicated than that but you see what
> I'm getting at...
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Setting this stuff up in .gitattributes will override user core.autocrlf.
This is what we want. It keeps the repo in good shape but also lets
users have different LF/CRLF in their work areas. And it will 'correct'
a user (like me) who naively tries to switch line endings here or there
on a commit. Some of the line ending issues were present before the
migration from SVN.
There will be an interesting flag day when the correction pass happens.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: gitattributes
Posted by Alan Conway <ac...@redhat.com>.
On Fri, 2016-09-02 at 08:49 +0200, Volker Diels-Grabsch wrote:
> Matt Broadstone schrieb:
> >
> > Would it be possible to fix up the .gitattributes for the qpid-cpp
> > git
> > repository? I'm having an incredibly hard time updating the sources
> > for the
> > debian packaging because there are many mixed in CRLF files
> > throughout the
> > repository (seemingly without rhyme or reason either, i.e. not all
> > .ps1 or
> > .bat files have CRLF line endings). Anyone know how to clean this
> > up easily?
>
> This sounds like it should be really cleaned up in the repository and
> not just on packaging.��Maybe the following helps.��This is my
> general
> way to cleanup sources, i.e. not specific for qpid-cpp.
>
> You can cleanup e.g. all "*.cpp" and "*.in" files with a single
> command:
>
> ����cd qpid-cpp-0.34
> ����find * -type f -name '*.cpp' -exec sed 's/\r//g' -i {} \;
> ����find * -type f -name '*.in' -exec sed 's/\r//g' -i {} \;
That will only work until folks start re-editing the files with their
random win/unix editors and change the endings again. If you want it to
stay pure forever I think we need .gitattributes to define file types
by extension as "text" so that git normalizes them, and then let users
set their core.eol to whatever they prefer for text files in their
working tree. Its probably more complicated than that but you see what
I'm getting at...
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: gitattributes
Posted by Matt Broadstone <mb...@gmail.com>.
On Fri, Sep 2, 2016 at 2:49 AM, Volker Diels-Grabsch <v...@njh.eu> wrote:
> Matt Broadstone schrieb:
> > Would it be possible to fix up the .gitattributes for the qpid-cpp git
> > repository? I'm having an incredibly hard time updating the sources for
> the
> > debian packaging because there are many mixed in CRLF files throughout
> the
> > repository (seemingly without rhyme or reason either, i.e. not all .ps1
> or
> > .bat files have CRLF line endings). Anyone know how to clean this up
> easily?
>
> This sounds like it should be really cleaned up in the repository and
> not just on packaging. Maybe the following helps. This is my general
> way to cleanup sources, i.e. not specific for qpid-cpp.
>
>
Yes this was my proposal, by the time we get to the packaging for the
release the damage has been done afaict.
For reference I had to use this sort of frankenstein gitattributes to get
to a place where I could move forward with debian builds:
*.ico binary
*.bmp binary
*.sln eol=crlf
*.vcproj eol=crlf
bindings/qpid/dotnet/**/* eol=crlf
packaging/winsdk/**/* eol=crlf
packaging/windows/**/* eol=crlf
src/qpid/store/ms-sql/**/* eol=crlf
src/windows/**/* eol=crlf
src/CMakeWinVersions.cmake eol=crlf
INSTALL-WINDOWS.txt eol=crlf
docs/design/windows_clfs_store_design.txt eol=crlf
managementgen/qmfgen/templates/CMakeLists.cmake eol=crlf
src/tests/background.ps1 eol=crlf
src/tests/find_prog.ps1 eol=crlf
bindings/qpid/dotnet/src/Logger.h eol=crlf
packaging/winsdk/bld-winsdk.ps1 eol=crlf
bindings/qpid/dotnet/CMakeLists.txt eol=lf
bindings/qpid/dotnet/examples/csharp.example.helloworld/csharp.example.helloworld.cs
eol=lf
packaging/windows/LICENSE.rtf eol=lf
not ideal
> You can cleanup e.g. all "*.cpp" and "*.in" files with a single command:
>
> cd qpid-cpp-0.34
> find * -type f -name '*.cpp' -exec sed 's/\r//g' -i {} \;
> find * -type f -name '*.in' -exec sed 's/\r//g' -i {} \;
>
> These command could be run once for the whole code base and committed.
> Also, an appropriate .gitattributes settings file would help to
> prevent this is the future. Still, it may make sense to put this into
> a dev script or into the Makefile as separate target, so it could be
> re-run in case these issues are accidentally re-introduced.
>
> Variants:
>
> You could pack these commands into a single command via
> "... -name '*.cpp' -o -name '*.in' ...",
> but I prefer multiple commands for better readability.
>
> You could also run this without "-name '*.cpp'" to convert _all_
> files, but that would break binary files such as "qpid-icon.png" which
> you would have to revert manually after the command.
>
> You could use "find ." instead of "find *", but then it would modify
> files under '.git', '.svn' or similar toplevel subdirectories, thus
> killing your version control information. So you would have to backup
> these directories before running the command. On the other hand,
> "find ." would catch toplevel dot-files such as ".gitignore", which
> "find *" can't catch.
>
> HTH,
> Volker
>
> --
> Volker Diels-Grabsch
> ----<<<((()))>>>----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
>
>
Re: gitattributes
Posted by Volker Diels-Grabsch <v...@njh.eu>.
Matt Broadstone schrieb:
> Would it be possible to fix up the .gitattributes for the qpid-cpp git
> repository? I'm having an incredibly hard time updating the sources for the
> debian packaging because there are many mixed in CRLF files throughout the
> repository (seemingly without rhyme or reason either, i.e. not all .ps1 or
> .bat files have CRLF line endings). Anyone know how to clean this up easily?
This sounds like it should be really cleaned up in the repository and
not just on packaging. Maybe the following helps. This is my general
way to cleanup sources, i.e. not specific for qpid-cpp.
You can cleanup e.g. all "*.cpp" and "*.in" files with a single command:
cd qpid-cpp-0.34
find * -type f -name '*.cpp' -exec sed 's/\r//g' -i {} \;
find * -type f -name '*.in' -exec sed 's/\r//g' -i {} \;
These command could be run once for the whole code base and committed.
Also, an appropriate .gitattributes settings file would help to
prevent this is the future. Still, it may make sense to put this into
a dev script or into the Makefile as separate target, so it could be
re-run in case these issues are accidentally re-introduced.
Variants:
You could pack these commands into a single command via
"... -name '*.cpp' -o -name '*.in' ...",
but I prefer multiple commands for better readability.
You could also run this without "-name '*.cpp'" to convert _all_
files, but that would break binary files such as "qpid-icon.png" which
you would have to revert manually after the command.
You could use "find ." instead of "find *", but then it would modify
files under '.git', '.svn' or similar toplevel subdirectories, thus
killing your version control information. So you would have to backup
these directories before running the command. On the other hand,
"find ." would catch toplevel dot-files such as ".gitignore", which
"find *" can't catch.
HTH,
Volker
--
Volker Diels-Grabsch
----<<<((()))>>>----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org
Re: gitattributes
Posted by Alan Conway <ac...@redhat.com>.
On Thu, 2016-09-01 at 22:14 -0400, Matt Broadstone wrote:
> Hey all,
>
> Would it be possible to fix up the .gitattributes for the qpid-cpp
> git
> repository? I'm having an incredibly hard time updating the sources
> for the
> debian packaging because there are many mixed in CRLF files
> throughout the
> repository (seemingly without rhyme or reason either, i.e. not all
> .ps1 or
> .bat files have CRLF line endings). Anyone know how to clean this up
> easily?
>
> Thanks,
> Matt
I'd say open a JIRA and make a proposal of how it should be, once the
ensuing Windows/Unix flamewar dies down and there is consensus we can
make it so. We have a long history of mixed win/unix developers and the
current state probably just reflects who's editor touched the file last
without any attempt at normalization. (Actually you can probably
estimate the % of win/unix based devs by the % of CRLF/LF files in the
repo ;)
Cheers,
Alan.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org