You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Francis Chuang <fr...@apache.org> on 2019/12/12 09:03:19 UTC
Line endings for source files on Windows
In this commit, Vladimir brought to my attention that editors on Windows
will complain about line endings if there isn't a zip source release
with Windows line endings:
https://github.com/apache/calcite-avatica/commit/34bbcb63f9216d3a5bc29dae1981a55e335d30df#commitcomment-36393594
I don't really work on the Java source directly, do not have personal
experience with this. In CALCITE-2333[1], we stopped releasing zip
archives and have stuck to only producing a tar.gz source release.
Should we re-introduce a Zip archive with all files converted to Windows
line endings?
I use Windows as my main operating system and Goland (IntelliJ) and
Notepad++ as my editors and I exclusively used Linux line endings for my
source files. For Go source files, I've not had any issues with the
compilation and for the Java source files I open in IntelliJ or
Notepad++, I've not run into any issues.
[1] https://issues.apache.org/jira/browse/CALCITE-2333
Re: Line endings for source files on Windows
Posted by Julian Hyde <jh...@apache.org>.
No, I’m going to ignore this trolling.
> On Dec 12, 2019, at 11:57 AM, Vladimir Sitnikov <si...@gmail.com> wrote:
>
> Julian>For all other purpose, just use git.
>
> "use git" contradicts with
> http://www.apache.org/legal/release-policy.html#publication
>
> legal/release-policy> Projects MUST direct outsiders towards official
> releases rather than raw source repositories, nightly builds, snapshots,
> release candidates, or any other similar packages. The only people who are
> supposed to know about such developer resources are individuals actively
> participating in development or following the dev list and thus aware of
> the conditions placed on unreleased materials.
>
> Julian>for no reason
>
> Ok. You have chosen to ignore Windows users.
>
> Vladimir
Re: Line endings for source files on Windows
Posted by Vladimir Sitnikov <si...@gmail.com>.
Julian>For all other purpose, just use git.
"use git" contradicts with
http://www.apache.org/legal/release-policy.html#publication
legal/release-policy> Projects MUST direct outsiders towards official
releases rather than raw source repositories, nightly builds, snapshots,
release candidates, or any other similar packages. The only people who are
supposed to know about such developer resources are individuals actively
participating in development or following the dev list and thus aware of
the conditions placed on unreleased materials.
Julian>for no reason
Ok. You have chosen to ignore Windows users.
Vladimir
Re: Line endings for source files on Windows
Posted by Julian Hyde <jh...@apache.org>.
The main purpose of the source distribution is to have legal record of the release, and something from which people could re-create the release if GitHub and all mirrors thereof were to disappear.
For all other purpose, just use git.
So, I see no reason to create different source distributions for different platforms. It is complication for no reason.
Julian
> On Dec 12, 2019, at 11:00 AM, Vladimir Sitnikov <si...@gmail.com> wrote:
>
> Julian>The git repo, at a particular commit, has an objective contents
>
> That is true for binary blobs.
> However, text files are converted on checkout as per core.autocrlf and
> core.eol settings.
>
> Julian>I suspect that when you use ‘git checkout’ with particular options
>
> I suspect you are not very well aware of typical recommendations for Git
> for Windows.
>
> Here's what GitHub recommends:
> https://help.github.com/en/github/using-git/configuring-git-to-handle-line-endings#global-settings-for-line-endings
>
> "core.autocrlf true" means Git will convert text files to CRLF line endings
> when "simple" "git clone https://..." is used.
>
> Vladimir
Re: Line endings for source files on Windows
Posted by Vladimir Sitnikov <si...@gmail.com>.
Julian>The git repo, at a particular commit, has an objective contents
That is true for binary blobs.
However, text files are converted on checkout as per core.autocrlf and
core.eol settings.
Julian>I suspect that when you use ‘git checkout’ with particular options
I suspect you are not very well aware of typical recommendations for Git
for Windows.
Here's what GitHub recommends:
https://help.github.com/en/github/using-git/configuring-git-to-handle-line-endings#global-settings-for-line-endings
"core.autocrlf true" means Git will convert text files to CRLF line endings
when "simple" "git clone https://..." is used.
Vladimir
Re: Line endings for source files on Windows
Posted by Julian Hyde <jh...@apache.org>.
The git repo, at a particular commit, has an objective contents. I suspect that when you use ‘git checkout’ with particular options, you don’t get those objective contents, you get something customized for your line-ending preference.
> On Dec 12, 2019, at 7:46 AM, Vladimir Sitnikov <si...@gmail.com> wrote:
>
>> I think a source distribution should contain the raw, unprocessed source
> files.
>
> What do you mean by "raw"?
>
> If I check out the same repository on Windows and macOS, I would get
> **different** file contents for *.java files.
> Windows machine would check out files as CRLF, and macOS would checkout the
> same files as LF.
>
> It is something that is controlled with
> https://github.com/apache/calcite-avatica/blob/512bbee4aa24ef9fb8106d0286d1243679dce2d0/.gitattributes#L2
>
> Vladimir
Re: Line endings for source files on Windows
Posted by Vladimir Sitnikov <si...@gmail.com>.
Michael>The source code should be what unprocessed,
Michael>but since Windows
I do not get what you mean by "unprocessed".
Even Maven build had lots of exclude-include patterns, so it did "process"
the sources.
Then, the current source release contains LICENSE file that contains some
processing (because it needs to gather third-party licenses).
Michael>I would be
Michael>interested in exploring producing reproducible builds of
Michael>Calcite/Avatica
You must have missed that, but it is already implemented.
Feel free to try.
Note: one of the point of using `-Prelease` flag in the "RC-VOTE" mail is
to verify build reproducibility.
In other words, if people build from the same Git commit (or from source
release), they should end up with exactly the same SHAs provided they build
the same version (release vs snapshot) and they use similar javac (because
different javac versions might produce slightly different bytecode).
Vladimir
Re: Line endings for source files on Windows
Posted by Michael Mior <mm...@apache.org>.
I agree with both points. The source code should be what unprocessed,
but since Windows and Linux/macOS users get different source code
anyway, I'm not opposed to having two archives with different line
endings. This is mostly unrelated, but at some point, I would be
interested in exploring producing reproducible builds of
Calcite/Avatica. If we did this, hopefully we could have identical
JARs produced regardless of source line endings.
--
Michael Mior
mmior@apache.org
Le jeu. 12 déc. 2019 à 10:47, Vladimir Sitnikov
<si...@gmail.com> a écrit :
>
> >I think a source distribution should contain the raw, unprocessed source
> files.
>
> What do you mean by "raw"?
>
> If I check out the same repository on Windows and macOS, I would get
> **different** file contents for *.java files.
> Windows machine would check out files as CRLF, and macOS would checkout the
> same files as LF.
>
> It is something that is controlled with
> https://github.com/apache/calcite-avatica/blob/512bbee4aa24ef9fb8106d0286d1243679dce2d0/.gitattributes#L2
>
> Vladimir
Re: Line endings for source files on Windows
Posted by Vladimir Sitnikov <si...@gmail.com>.
>I think a source distribution should contain the raw, unprocessed source
files.
What do you mean by "raw"?
If I check out the same repository on Windows and macOS, I would get
**different** file contents for *.java files.
Windows machine would check out files as CRLF, and macOS would checkout the
same files as LF.
It is something that is controlled with
https://github.com/apache/calcite-avatica/blob/512bbee4aa24ef9fb8106d0286d1243679dce2d0/.gitattributes#L2
Vladimir
Re: Line endings for source files on Windows
Posted by Julian Hyde <jh...@gmail.com>.
I think a source distribution should contain the raw, unprocessed source files. The contents of the .zip and .tar.gz should be identical.
If people want to change line endings they can do it for themselves. Or use an appropriate git setting.
> On Dec 12, 2019, at 1:03 AM, Francis Chuang <fr...@apache.org> wrote:
>
> In this commit, Vladimir brought to my attention that editors on Windows will complain about line endings if there isn't a zip source release with Windows line endings: https://github.com/apache/calcite-avatica/commit/34bbcb63f9216d3a5bc29dae1981a55e335d30df#commitcomment-36393594
>
> I don't really work on the Java source directly, do not have personal experience with this. In CALCITE-2333[1], we stopped releasing zip archives and have stuck to only producing a tar.gz source release.
>
> Should we re-introduce a Zip archive with all files converted to Windows line endings?
>
> I use Windows as my main operating system and Goland (IntelliJ) and Notepad++ as my editors and I exclusively used Linux line endings for my source files. For Go source files, I've not had any issues with the compilation and for the Java source files I open in IntelliJ or Notepad++, I've not run into any issues.
>
> [1] https://issues.apache.org/jira/browse/CALCITE-2333