You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Vladimir Sitnikov <si...@gmail.com> on 2019/07/17 11:19:36 UTC
Re: Problems with checkstyle tests on Windows (Was [VOTE] Release
Apache Maven Checkstyle Plugin version 3.1.0)
Robert>A clone from Git succeeds, but the sources.zip fails.
Robert>The files in the zip are generated on a unix system, so all EOLs in
text files are LF
Robert>...
Robert>The fix: add a setup.groovy to the IT and rewrite the java files
with OS specific EOLs
Alternative approaches:
A) Provide both Linux (LF) and Windows (CRLF) source distributions (e.g.
*.zip and *.tgz).
B) Specify "lineSeparator" explicitly. Then you could have both CRLF and LF
files at the same time and verify if those work
C) Generate file at the build stage. If you generate it into target/
directory, then you could generate the file with appropriate for the
platform enconding
D) Ensure the file is always in LF or CRLF by adding a relevant
.gitattributes entry
Vladimir
Re: Problems with checkstyle tests on Windows (Was [VOTE] Release
Apache Maven Checkstyle Plugin version 3.1.0)
Posted by Vladimir Sitnikov <si...@gmail.com>.
Robert> According to the ASF the -sources.zip are the official release, not
the git tag
That's right
Robert>In this case you don't control the EOL, it is based on the OS of the
release manager.
Is it? I would disagree here.
For instance, Apache JMeter produces both -src.zip (CRLF), and -src.tar.gz
with LF-defaults for text files.
Of course, one might take -src.tar.gz and try building it on Windows,
however that is not like RM's machine controls EOL.
On top of that, https://reproducible-builds.org/ is a thing, so it makes
sense to be able to reproduce release files independent of the release
manager (=producezips with the same sha512 on different OS)
So both Git and the release script control the way the release is produced.
Input data could contain both variations of the EOL styles by the way,
which might be handy for EOL-targeted tests. That is Git repo could just
contain a file with CRLF, and a file with LF. So would have the release
zip.
However I would agree some sort of the treatment should be baked into the
test.
Vladimir
Re: Problems with checkstyle tests on Windows (Was [VOTE] Release
Apache Maven Checkstyle Plugin version 3.1.0)
Posted by Robert Scholte <rf...@apache.org>.
Keep in mind there's another important situation that you can't control it like this.
According to the ASF the -sources.zip are the official release, not the git tag. This means that you should unpack the zip and be able to verify it on all OSes.
In this case you don't control the EOL, it is based on the OS of the release manager.
The only way to solve this is to generate files on the fly, or adjust them before testing.
You won't see this issue with the CI server, only after a release and when verified with different OSes.
thanks,
Robert
On 18-7-2019 14:49:02, Eric Lilja <mi...@gmail.com> wrote:
Ah, thanks Vladimir, that's even better. I was not 100% sure it would be
possible to retain complete control over resulting line endings for those
files, regardless of user git settings of stuff like autocrlf = true and
whatnot, but it seems there is, that's great news. Thanks!
- Eric L
On Thu, Jul 18, 2019 at 2:00 PM Vladimir Sitnikov <>
sitnikov.vladimir@gmail.com> wrote:
> Eric>In that case, we should generate the test files (to
> Eric>avoid git interfering), one with linux-style EOLs and one with
> Eric>Windows-style EOLs and test with both.
>
> You'd better have those files under Git control, and you could just specify
> .gitattributes so the LF file is always LF, and CRLF file is always CRLF.
>
> That is way simpler than generation of the file(s), and it is way easier to
> understand by humans
>
> Vladimir
>
Re: Problems with checkstyle tests on Windows (Was [VOTE] Release
Apache Maven Checkstyle Plugin version 3.1.0)
Posted by Eric Lilja <mi...@gmail.com>.
Ah, thanks Vladimir, that's even better. I was not 100% sure it would be
possible to retain complete control over resulting line endings for those
files, regardless of user git settings of stuff like autocrlf = true and
whatnot, but it seems there is, that's great news. Thanks!
- Eric L
On Thu, Jul 18, 2019 at 2:00 PM Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:
> Eric>In that case, we should generate the test files (to
> Eric>avoid git interfering), one with linux-style EOLs and one with
> Eric>Windows-style EOLs and test with both.
>
> You'd better have those files under Git control, and you could just specify
> .gitattributes so the LF file is always LF, and CRLF file is always CRLF.
>
> That is way simpler than generation of the file(s), and it is way easier to
> understand by humans
>
> Vladimir
>
Re: Problems with checkstyle tests on Windows (Was [VOTE] Release
Apache Maven Checkstyle Plugin version 3.1.0)
Posted by Vladimir Sitnikov <si...@gmail.com>.
Eric>In that case, we should generate the test files (to
Eric>avoid git interfering), one with linux-style EOLs and one with
Eric>Windows-style EOLs and test with both.
You'd better have those files under Git control, and you could just specify
.gitattributes so the LF file is always LF, and CRLF file is always CRLF.
That is way simpler than generation of the file(s), and it is way easier to
understand by humans
Vladimir
Re: Problems with checkstyle tests on Windows (Was [VOTE] Release
Apache Maven Checkstyle Plugin version 3.1.0)
Posted by Eric Lilja <mi...@gmail.com>.
Hi Enrico!
If I remember correctly, a number of weeks ago I tried using Checkstyle
8.21 with maven-checkstyle-plugin (I had simply cloned the
maven-checkstyle-plugin repository) and it did solve the problem with the
test case discussed in this thread. However, as I recall there was another,
unrelated, test that started failing with Checkstyle 8.21, which I did not
investigate. I haven't tried Checkstyle 8.22, which has been released since.
Additionally, we could be a bit more ambitious in the
file-should-end-with-newline-test, if we also want to verify it's indeed
platform-agnostic now. In that case, we should generate the test files (to
avoid git interfering), one with linux-style EOLs and one with
Windows-style EOLs and test with both.
I might give it a go to uplift Checkstyle to 8.22 and possibly expand the
test as outlined above if you think it sounds like a reasonable proposition
- Eric L
On Wed, Jul 17, 2019 at 1:32 PM Enrico Olivelli <eo...@gmail.com> wrote:
> Il mer 17 lug 2019, 13:28 Eric Lilja <mi...@gmail.com> ha scritto:
>
> > I believe, if we instead upgrade to Checkstyle 8.21 or later, we don't
> need
> > to do any of those alternative approaches.
> >
> > https://github.com/checkstyle/checkstyle/issues/4073
>
>
> Eric
> I think it is the best idea
> Do you have cycles to give it a try?
>
> Enrico
>
>
>
> >
> > - Eric L
> >
> >
> > On Wed, Jul 17, 2019 at 1:20 PM Vladimir Sitnikov <
> > sitnikov.vladimir@gmail.com> wrote:
> >
> > > Robert>A clone from Git succeeds, but the sources.zip fails.
> > > Robert>The files in the zip are generated on a unix system, so all EOLs
> > in
> > > text files are LF
> > > Robert>...
> > > Robert>The fix: add a setup.groovy to the IT and rewrite the java files
> > > with OS specific EOLs
> > >
> > > Alternative approaches:
> > > A) Provide both Linux (LF) and Windows (CRLF) source distributions
> (e.g.
> > > *.zip and *.tgz).
> > > B) Specify "lineSeparator" explicitly. Then you could have both CRLF
> and
> > LF
> > > files at the same time and verify if those work
> > > C) Generate file at the build stage. If you generate it into target/
> > > directory, then you could generate the file with appropriate for the
> > > platform enconding
> > > D) Ensure the file is always in LF or CRLF by adding a relevant
> > > .gitattributes entry
> > >
> > > Vladimir
> > >
> >
>
Re: Problems with checkstyle tests on Windows (Was [VOTE] Release
Apache Maven Checkstyle Plugin version 3.1.0)
Posted by Vladimir Sitnikov <si...@gmail.com>.
Just in case: I see you seem to be battling against CRLF
For instance: https://github.com/apache/maven-checkstyle-plugin/pull/16
Did you consider to add the relevant .gitattributes file to ensure Git
converts the files automatically?
With a proper .gitattributes file, it is just impossible to commit a file
with "a wrong EOL".
PS. I've came across this thread when Apache JMeter project noticed that
Window-based builds were broken by
https://issues.apache.org/jira/browse/INFRA-18383
Vladimir
Re: Problems with checkstyle tests on Windows (Was [VOTE] Release
Apache Maven Checkstyle Plugin version 3.1.0)
Posted by Enrico Olivelli <eo...@gmail.com>.
Il mer 17 lug 2019, 13:28 Eric Lilja <mi...@gmail.com> ha scritto:
> I believe, if we instead upgrade to Checkstyle 8.21 or later, we don't need
> to do any of those alternative approaches.
>
> https://github.com/checkstyle/checkstyle/issues/4073
Eric
I think it is the best idea
Do you have cycles to give it a try?
Enrico
>
> - Eric L
>
>
> On Wed, Jul 17, 2019 at 1:20 PM Vladimir Sitnikov <
> sitnikov.vladimir@gmail.com> wrote:
>
> > Robert>A clone from Git succeeds, but the sources.zip fails.
> > Robert>The files in the zip are generated on a unix system, so all EOLs
> in
> > text files are LF
> > Robert>...
> > Robert>The fix: add a setup.groovy to the IT and rewrite the java files
> > with OS specific EOLs
> >
> > Alternative approaches:
> > A) Provide both Linux (LF) and Windows (CRLF) source distributions (e.g.
> > *.zip and *.tgz).
> > B) Specify "lineSeparator" explicitly. Then you could have both CRLF and
> LF
> > files at the same time and verify if those work
> > C) Generate file at the build stage. If you generate it into target/
> > directory, then you could generate the file with appropriate for the
> > platform enconding
> > D) Ensure the file is always in LF or CRLF by adding a relevant
> > .gitattributes entry
> >
> > Vladimir
> >
>
Re: Problems with checkstyle tests on Windows (Was [VOTE] Release
Apache Maven Checkstyle Plugin version 3.1.0)
Posted by Eric Lilja <mi...@gmail.com>.
I believe, if we instead upgrade to Checkstyle 8.21 or later, we don't need
to do any of those alternative approaches.
https://github.com/checkstyle/checkstyle/issues/4073
- Eric L
On Wed, Jul 17, 2019 at 1:20 PM Vladimir Sitnikov <
sitnikov.vladimir@gmail.com> wrote:
> Robert>A clone from Git succeeds, but the sources.zip fails.
> Robert>The files in the zip are generated on a unix system, so all EOLs in
> text files are LF
> Robert>...
> Robert>The fix: add a setup.groovy to the IT and rewrite the java files
> with OS specific EOLs
>
> Alternative approaches:
> A) Provide both Linux (LF) and Windows (CRLF) source distributions (e.g.
> *.zip and *.tgz).
> B) Specify "lineSeparator" explicitly. Then you could have both CRLF and LF
> files at the same time and verify if those work
> C) Generate file at the build stage. If you generate it into target/
> directory, then you could generate the file with appropriate for the
> platform enconding
> D) Ensure the file is always in LF or CRLF by adding a relevant
> .gitattributes entry
>
> Vladimir
>