You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Shawn Heisey <ap...@elyograg.org> on 2018/03/02 19:45:34 UTC
"ant jar-checksums" has problems on Windows
I notice that when "ant jar-checksums" is run on Windows, it modifies
every single hash file for dependent jars. The actual hashes aren't
modified unless a dependency has changed, but the line endings are.
I get lots of lines similar to these (on stderr) if I do "git diff"
after the ant action:
warning: LF will be replaced by CRLF in
lucene/licenses/Tagger-2.3.1.jar.sha1.
The file will have its original line endings in your working directory.
If I run "git commit" then it informs me that all the .jar.sha1 files
have changed.
My %USERPROFILE%\.gitconfig file does have the following config:
[core]
autocrlf = true
I'm using "git for windows" also known as msysgit, which should be
paying attention to the .gitconfig file.
The difference happens because in the checkout, those .jar.sha1 files
have CRLF line endings, but when the jar-checksums target finishes, the
line endings are LF instead.
If I remove the "fixcrlf" tag from the jar-checksum-macro target in
lucene/common-build.xml, it seems to fix the issue on Windows without
breaking it on Linux.
An alternate fix would be to set up an attribute for git so that it
doesn't consider *.sha1 files to be text, but that might have unintended
consequences.
Should I go ahead and open an issue?
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: "ant jar-checksums" has problems on Windows
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,
I agree with Dawid. By default e.g. Jenkins Slave and also TortoiseGit on Windows don't set windows file endings by default. You have to enable it in the config or the installation program.
Currently the Lucene build system has some SVN relics in it. If you recreate the autogenerated files (like javacc,...) We currently apply local line endings. On the other hand for the checksums we changed to Linux endings.
My proposal ist to review all code that regenerates committable files and change the crlf ant task to use Unix line endings. Currently regenerating files is a mess for me.
If some committees want windows line endings we can add a ant property that defaults to Unix and is used by all fixcrlf tasks for regenerating stuff. People that have a git installed that uses Windows endings can then put a lot in their home for to configure it. In the same way like they configure git. It must just be in line.
Should I open an issue, I started to do that in some other branch already after the last Desaster with autogenerated files.
Uwe
Am March 3, 2018 4:51:47 PM UTC schrieb Dawid Weiss <da...@gmail.com>:
>> Lots of people on Windows use programs like Notepad for "serious"
>work.
>
>This is not an argument for me. Using broken tools is not an excuse.
>There
> are many excellent text editors for Windows (in fact, I haven't been
>able to find an editor
>seriously competing with EmEditor in either Linux or Mac world when
>handling very
>large files). EmEditor is commercial, but there are lots of excellent
>open source alternatives
>(JEdit, notepad++).
>
>If you're using notepad for development you can't be seriously a
>Windows user... you wouldn't get any
>work done at all.
>
>> If the source control system didn't convert line endings, we would
>have
>> many complaints from Windows users who double-click on files like
>> CHANGES.txt and can't easily read it because the formatting's all
>> screwed up, and from people who want to use the text editor built
>into
>> their OS for editing source code.
>
>I disagree. You need to know and respect your audience, but you also
>have the right to say what you expect.
>Source code-level access is for developers. If they're able to check
>out the source code using git and compile it then
>we must be able to assume they have enough technical knowledge to
>figure out opening a text file with notepad is a
>bad idea.
>
>Otherwise, by similar argument, we should distribute two binary
>releases -- one for \n systems and
>the other for \r\n just so that people can read the txt files with
>installation instructions. This isn't sane.
>
>I don't know why git even has this "facility" of converting line
>endings, this is just asking for trouble. Marking a perfectly valid
>text file as binary is a bad idea (for example, changes to binary
>files are not listed in diffs).
>
>Dawid
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>For additional commands, e-mail: dev-help@lucene.apache.org
--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de
Re: "ant jar-checksums" has problems on Windows
Posted by Dawid Weiss <da...@gmail.com>.
> Lots of people on Windows use programs like Notepad for "serious" work.
This is not an argument for me. Using broken tools is not an excuse. There
are many excellent text editors for Windows (in fact, I haven't been
able to find an editor
seriously competing with EmEditor in either Linux or Mac world when
handling very
large files). EmEditor is commercial, but there are lots of excellent
open source alternatives
(JEdit, notepad++).
If you're using notepad for development you can't be seriously a
Windows user... you wouldn't get any
work done at all.
> If the source control system didn't convert line endings, we would have
> many complaints from Windows users who double-click on files like
> CHANGES.txt and can't easily read it because the formatting's all
> screwed up, and from people who want to use the text editor built into
> their OS for editing source code.
I disagree. You need to know and respect your audience, but you also
have the right to say what you expect.
Source code-level access is for developers. If they're able to check
out the source code using git and compile it then
we must be able to assume they have enough technical knowledge to
figure out opening a text file with notepad is a
bad idea.
Otherwise, by similar argument, we should distribute two binary
releases -- one for \n systems and
the other for \r\n just so that people can read the txt files with
installation instructions. This isn't sane.
I don't know why git even has this "facility" of converting line
endings, this is just asking for trouble. Marking a perfectly valid
text file as binary is a bad idea (for example, changes to binary
files are not listed in diffs).
Dawid
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: "ant jar-checksums" has problems on Windows
Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/2/2018 1:58 PM, Dawid Weiss wrote:
> I typically set my windows git to NOT alter anything with regard to
> line endings. (checkout as-is, commit as-is).
> This really helps in avoiding weird errors.
When using sane tools that deal with line ending discrepancies properly,
this is a perfectly valid way to work. But the tools built into Windows
are not sane. Microsoft has had *decades* to fix the most basic
line-ending problems with Notepad, their ubiquitous text editor. They
appear to have no interest in acknowledging that their way is not the
only way, so opening a unix-generated text file still looks
spectacularly bad.
Lots of people on Windows use programs like Notepad for "serious" work.
If the source control system didn't convert line endings, we would have
many complaints from Windows users who double-click on files like
CHANGES.txt and can't easily read it because the formatting's all
screwed up, and from people who want to use the text editor built into
their OS for editing source code. Telling people the basic truth, that
Notepad is the problem, will increase irritation toward the project, not
Microsoft.
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: "ant jar-checksums" has problems on Windows
Posted by Dawid Weiss <da...@gmail.com>.
I typically set my windows git to NOT alter anything with regard to
line endings. (checkout as-is, commit as-is).
This really helps in avoiding weird errors.
Dawid
On Fri, Mar 2, 2018 at 9:43 PM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 3/2/2018 12:57 PM, Uwe Schindler wrote:
>> This change would break my checkout' because my git is configured for
>> Linux line endings. So the crlf fix is needed.
>>
>> Windows Jenkins on 👮 ist also using Linux line endings.
>>
>> IMHO we should do it like in svn: mark those files as binary. Not sure
>> how.
>>
>> BTW. The same issue appears with other autogenerated files.
>
> There's usually something I haven't thought of, glad I asked! The
> following change (on my own github repo, not the official repo) seems to
> work, but I had to clone it again after the change to make it actually
> effective:
>
> https://github.com/elyograg/jar-checksums/commit/70da99819f6401b9dd5bb23c86d59cff0ec4ddc4
>
> Thanks,
> Shawn
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: "ant jar-checksums" has problems on Windows
Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/2/2018 12:57 PM, Uwe Schindler wrote:
> This change would break my checkout' because my git is configured for
> Linux line endings. So the crlf fix is needed.
>
> Windows Jenkins on 👮 ist also using Linux line endings.
>
> IMHO we should do it like in svn: mark those files as binary. Not sure
> how.
>
> BTW. The same issue appears with other autogenerated files.
There's usually something I haven't thought of, glad I asked! The
following change (on my own github repo, not the official repo) seems to
work, but I had to clone it again after the change to make it actually
effective:
https://github.com/elyograg/jar-checksums/commit/70da99819f6401b9dd5bb23c86d59cff0ec4ddc4
Thanks,
Shawn
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: "ant jar-checksums" has problems on Windows
Posted by Uwe Schindler <uw...@thetaphi.de>.
This change would break my checkout' because my git is configured for Linux line endings. So the crlf fix is needed.
Windows Jenkins on 👮 ist also using Linux line endings.
IMHO we should do it like in svn: mark those files as binary. Not sure how.
BTW. The same issue appears with other autogenerated files.
Am March 2, 2018 7:45:34 PM UTC schrieb Shawn Heisey <ap...@elyograg.org>:
>I notice that when "ant jar-checksums" is run on Windows, it modifies
>every single hash file for dependent jars. The actual hashes aren't
>modified unless a dependency has changed, but the line endings are.
>
>I get lots of lines similar to these (on stderr) if I do "git diff"
>after the ant action:
>
>warning: LF will be replaced by CRLF in
>lucene/licenses/Tagger-2.3.1.jar.sha1.
>The file will have its original line endings in your working directory.
>
>If I run "git commit" then it informs me that all the .jar.sha1 files
>have changed.
>
>My %USERPROFILE%\.gitconfig file does have the following config:
>
>[core]
> autocrlf = true
>
>I'm using "git for windows" also known as msysgit, which should be
>paying attention to the .gitconfig file.
>
>The difference happens because in the checkout, those .jar.sha1 files
>have CRLF line endings, but when the jar-checksums target finishes, the
>line endings are LF instead.
>
>If I remove the "fixcrlf" tag from the jar-checksum-macro target in
>lucene/common-build.xml, it seems to fix the issue on Windows without
>breaking it on Linux.
>
>An alternate fix would be to set up an attribute for git so that it
>doesn't consider *.sha1 files to be text, but that might have
>unintended
>consequences.
>
>Should I go ahead and open an issue?
>
>Thanks,
>Shawn
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>For additional commands, e-mail: dev-help@lucene.apache.org
--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de