You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@plc4x.apache.org by Christofer Dutz <ch...@c-ware.de> on 2018/02/08 09:12:10 UTC

Re: Is Junit5 the way to go?

Ok ... today Justin pointed out my changes from yesterday caused tests to fail
I tried and had to agree with him ... the code was simply wrong. But I wrote tests for it and when manually executing the tests in IntelliJ they failed. 
As I did a full maven build with tests before committing and Jenkins also was fine with my changes it seems that the current Junit 5 support is simply not reliable. 
I did see that there are now also options for running parametrized test without some insane hackery, so I would also join your opinion that I think it's best to switch to Junit 4

So I'll whip up a vote thread on this. Just that we get to vote a little more here __

Chris



Am 26.01.18, 10:47 schrieb "Justin Mclean" <ju...@classsoftware.com>:

    Hi,
    
    > Well the one thing I like with TestNG is the support for parametrized tests.
    > With JUnit 4 this sucks greatly and it very easy with TestNG. I do think that we need parametrized tests as we do have a lot of options to test.
    
    
    There are some existing parametrised tests and I've used it in Junit4 (ages ago) and don’t remember it being that bad, but I’ve not used that in TestNG so perhaps it’s better there?
    
    Thanks,
    Justin

Re: Is Junit5 the way to go?

Posted by Justin Mclean <ju...@classsoftware.com>.

…but it does make for a good test passing rate :-)

Re: Is Junit5 the way to go?

Posted by Christofer Dutz <ch...@c-ware.de>.

Hi Niclas,

thanks for that ... this way I deffiitely can get a better feeling for the hamcrest style.
I think we should simply take this to another discussion thread and hear the others oppinions.
I know hamcrest for quite some time as the original author of the flexmojos plugin liked hamcrest too. 
Haven't used it that much in the past though.

Chris

Am 12.02.18, 06:56 schrieb "hedhman@gmail.com im Auftrag von Niclas Hedhman" <hedhman@gmail.com im Auftrag von niclas@hedhman.org>:

    Personally, I find the AssertJ (and practically all other attempts at this)
    less powerful and harder to work with, as the DSL is not necessarily
    intuitive, well it isn't for me when writing it, although one can guess
    what it means when reading it, but perhaps that is the case for Hamcrest
    too for other people. Additionally, if you ONLY know JUnit4, then a subset
    of Hamcrest is already included inside the JUnit jar file. In essence, both
    AssertJ and Hamcrest was a "solution" to the older JUnit assertion
    mechanism.
    
    In AssertJ, there seems to be a "bias" towards creating your own Unit Test
    Assertion classes, so that the tests reads more nicely. And in Hamcrest,
    the "bias" is towards creating a ever-growing library of generic assertions
    independent of the unit tests at hand. IMHO, the former can still be done
    in Hamcrest, but I have not seen that, and I have also seldom seen anyone
    creating their own AssertJ assertion classes.
    
    Across the Java community, more people are comfortable with Hamcrest than
    AssertJ, probably due to Junit's inclusion of some of it.
    
    My recommendation is to use JUnit4 + Hamcrest library + creating PLC4X
    specific Matchers (if needed).
    
    
    I spent this morning changing the AssertJ to Hamcrest across all tests. You
    want a PR?
    https://github.com/niclash/incubator-plc4x/commit/2080c40c02249db5f5ec0cf0b37b0ea0524de922
    
    
    Cheers
    Niclas
    
    
    
    On Sun, Feb 11, 2018 at 5:31 PM, Christofer Dutz <ch...@c-ware.de>
    wrote:
    
    > Hi Nicolas,
    >
    > While I was at it, I press everything to junit 4 and migrated all junit
    > Assert statements with assertj assertions. Do you have any opinion to these
    > two options? Hamcrest vs. AssertJ?
    >
    > Chris
    >
    > Outlook for Android<https://aka.ms/ghei36> herunterladen
    >
    >
    >
    > Von: Niclas Hedhman
    > Gesendet: Sonntag, 11. Februar, 05:17
    > Betreff: Re: Is Junit5 the way to go?
    > An: dev@plc4x.apache.org
    >
    >
    > The difference between Junit4 and Junit5 are relatively minor from a
    > user's perspective. The real difference is when creating Runners for
    > special need systems, such as Spring So, going with JUnit4 for now is not a
    > bad thing... But, it seems that there is some legacy that should be
    > scrapped, because it actually makes a difference. Avoid the
    > Assert.isEqual(), Assert.isTrue() and so on, and only use the Hamcrest
    > library for conditions, i.e. assertThat. The benefit is that one can use
    > (or make) much more complex tests, which are otherwise ugly to write or
    > report back what is expected. assertThat( myInteger, equalTo( 15 ) );
    > assertThat( myCollection, hasItem( "Niclas" )); assertThat( myCollection,
    > hasItems( "Niclas", "Justin", "Chris" )); Arbitrarily complex matchers can
    > be made, and give a reasonable error message when failing. If I can find
    > some time in the next couple of days, I will take a look and propose
    > changes... On Feb 9, 2018 16:35, "Justin Mclean" wrote: > Hi, > > > thanks
    > for finding that ... guess when porting all these thousands of >
    > statements, I must have missed one or two "replace: ' == ' with >
    > ').isEqualTo('” __ > > Yep I can imaging my brain zoning out when doing
    > that :-) > > > Well the main reason was probably, that I wanted to replace
    > the > "assertTrue(A==B)" with something like "assertEqual(A, B)" as this
    > outputs > the "expected" and the actual "value" and hereby provides a
    > little more > information than a simple "was false". So I had the option of
    > converting it > to JUint "Assert.assertEquals" or update it to AssertJ's >
    > "Assertions.assertThat().isEqualTo()" which I think is a little more >
    > readable > > You know that Junit4 also has a assertThat? It’s has a
    > slightly different > signature which what threw me when I first looked at
    > the changes. > > Thanks, > Justin
    >
    >
    
    
    -- 
    Niclas Hedhman, Software Developer
    http://polygene.apache.org - New Energy for Java

Re: Is Junit5 the way to go?

Posted by Niclas Hedhman <ni...@hedhman.org>.

Personally, I find the AssertJ (and practically all other attempts at this)
less powerful and harder to work with, as the DSL is not necessarily
intuitive, well it isn't for me when writing it, although one can guess
what it means when reading it, but perhaps that is the case for Hamcrest
too for other people. Additionally, if you ONLY know JUnit4, then a subset
of Hamcrest is already included inside the JUnit jar file. In essence, both
AssertJ and Hamcrest was a "solution" to the older JUnit assertion
mechanism.

In AssertJ, there seems to be a "bias" towards creating your own Unit Test
Assertion classes, so that the tests reads more nicely. And in Hamcrest,
the "bias" is towards creating a ever-growing library of generic assertions
independent of the unit tests at hand. IMHO, the former can still be done
in Hamcrest, but I have not seen that, and I have also seldom seen anyone
creating their own AssertJ assertion classes.

Across the Java community, more people are comfortable with Hamcrest than
AssertJ, probably due to Junit's inclusion of some of it.

My recommendation is to use JUnit4 + Hamcrest library + creating PLC4X
specific Matchers (if needed).

I spent this morning changing the AssertJ to Hamcrest across all tests. You
want a PR?
https://github.com/niclash/incubator-plc4x/commit/2080c40c02249db5f5ec0cf0b37b0ea0524de922

Cheers
Niclas

On Sun, Feb 11, 2018 at 5:31 PM, Christofer Dutz <ch...@c-ware.de>
wrote:

> Hi Nicolas,
>
> While I was at it, I press everything to junit 4 and migrated all junit
> Assert statements with assertj assertions. Do you have any opinion to these
> two options? Hamcrest vs. AssertJ?
>
> Chris
>
> Outlook for Android<https://aka.ms/ghei36> herunterladen
>
>
>
> Von: Niclas Hedhman
> Gesendet: Sonntag, 11. Februar, 05:17
> Betreff: Re: Is Junit5 the way to go?
> An: dev@plc4x.apache.org
>
>
> The difference between Junit4 and Junit5 are relatively minor from a
> user's perspective. The real difference is when creating Runners for
> special need systems, such as Spring So, going with JUnit4 for now is not a
> bad thing... But, it seems that there is some legacy that should be
> scrapped, because it actually makes a difference. Avoid the
> Assert.isEqual(), Assert.isTrue() and so on, and only use the Hamcrest
> library for conditions, i.e. assertThat. The benefit is that one can use
> (or make) much more complex tests, which are otherwise ugly to write or
> report back what is expected. assertThat( myInteger, equalTo( 15 ) );
> assertThat( myCollection, hasItem( "Niclas" )); assertThat( myCollection,
> hasItems( "Niclas", "Justin", "Chris" )); Arbitrarily complex matchers can
> be made, and give a reasonable error message when failing. If I can find
> some time in the next couple of days, I will take a look and propose
> changes... On Feb 9, 2018 16:35, "Justin Mclean" wrote: > Hi, > > > thanks
> for finding that ... guess when porting all these thousands of >
> statements, I must have missed one or two "replace: ' == ' with >
> ').isEqualTo('” __ > > Yep I can imaging my brain zoning out when doing
> that :-) > > > Well the main reason was probably, that I wanted to replace
> the > "assertTrue(A==B)" with something like "assertEqual(A, B)" as this
> outputs > the "expected" and the actual "value" and hereby provides a
> little more > information than a simple "was false". So I had the option of
> converting it > to JUint "Assert.assertEquals" or update it to AssertJ's >
> "Assertions.assertThat().isEqualTo()" which I think is a little more >
> readable > > You know that Junit4 also has a assertThat? It’s has a
> slightly different > signature which what threw me when I first looked at
> the changes. > > Thanks, > Justin
>
>

-- 
Niclas Hedhman, Software Developer
http://polygene.apache.org - New Energy for Java

Re: Is Junit5 the way to go?

Posted by Christofer Dutz <ch...@c-ware.de>.

Hi Nicolas,

While I was at it, I press everything to junit 4 and migrated all junit Assert statements with assertj assertions. Do you have any opinion to these two options? Hamcrest vs. AssertJ?

Chris

Outlook for Android<https://aka.ms/ghei36> herunterladen



Von: Niclas Hedhman
Gesendet: Sonntag, 11. Februar, 05:17
Betreff: Re: Is Junit5 the way to go?
An: dev@plc4x.apache.org


The difference between Junit4 and Junit5 are relatively minor from a user's perspective. The real difference is when creating Runners for special need systems, such as Spring So, going with JUnit4 for now is not a bad thing... But, it seems that there is some legacy that should be scrapped, because it actually makes a difference. Avoid the Assert.isEqual(), Assert.isTrue() and so on, and only use the Hamcrest library for conditions, i.e. assertThat. The benefit is that one can use (or make) much more complex tests, which are otherwise ugly to write or report back what is expected. assertThat( myInteger, equalTo( 15 ) ); assertThat( myCollection, hasItem( "Niclas" )); assertThat( myCollection, hasItems( "Niclas", "Justin", "Chris" )); Arbitrarily complex matchers can be made, and give a reasonable error message when failing. If I can find some time in the next couple of days, I will take a look and propose changes... On Feb 9, 2018 16:35, "Justin Mclean" wrote: > Hi, > > > thanks for finding that ... guess when porting all these thousands of > statements, I must have missed one or two "replace: ' == ' with > ').isEqualTo('” __ > > Yep I can imaging my brain zoning out when doing that :-) > > > Well the main reason was probably, that I wanted to replace the > "assertTrue(A==B)" with something like "assertEqual(A, B)" as this outputs > the "expected" and the actual "value" and hereby provides a little more > information than a simple "was false". So I had the option of converting it > to JUint "Assert.assertEquals" or update it to AssertJ's > "Assertions.assertThat().isEqualTo()" which I think is a little more > readable > > You know that Junit4 also has a assertThat? It’s has a slightly different > signature which what threw me when I first looked at the changes. > > Thanks, > Justin

Re: Is Junit5 the way to go?

Posted by Niclas Hedhman <he...@gmail.com>.

The difference between Junit4 and Junit5 are relatively minor from a user's
perspective. The real difference is when creating Runners for special need
systems, such as Spring

So, going with JUnit4 for now is not a bad thing...

But, it seems that there is some legacy that should be scrapped, because it
actually makes a difference.

Avoid the Assert.isEqual(), Assert.isTrue() and so on, and only use the
Hamcrest library for conditions, i.e. assertThat.
The benefit is that one can use (or make) much more complex tests, which
are otherwise ugly to write or report back what is expected.

assertThat( myInteger, equalTo( 15 ) );

assertThat( myCollection, hasItem( "Niclas" ));

assertThat( myCollection, hasItems( "Niclas", "Justin", "Chris" ));

Arbitrarily complex matchers can be made, and give a reasonable error
message when failing.

If I can find some time in the next couple of days, I will take a look and
propose changes...

On Feb 9, 2018 16:35, "Justin Mclean" <ju...@me.com> wrote:

> Hi,
>
> > thanks for finding that ... guess when porting all these thousands of
> statements, I must have missed one or two "replace: ' == ' with
> ').isEqualTo('” __
>
> Yep I can imaging my brain zoning out when doing that :-)
>
> > Well the main reason was probably, that I wanted to replace the
> "assertTrue(A==B)" with something like "assertEqual(A, B)" as this outputs
> the "expected" and the actual "value" and hereby provides a little more
> information than a simple "was false". So I had the option of converting it
> to JUint "Assert.assertEquals" or update it to AssertJ's
> "Assertions.assertThat().isEqualTo()" which I think is a little more
> readable
>
> You know that Junit4 also has a assertThat? It’s has a slightly different
> signature which what threw me when I first looked at the changes.
>
> Thanks,
> Justin

Re: Is Junit5 the way to go?

Posted by Justin Mclean <ju...@me.com>.

Hi,

> thanks for finding that ... guess when porting all these thousands of statements, I must have missed one or two "replace: ' == ' with ').isEqualTo('” __

Yep I can imaging my brain zoning out when doing that :-)

> Well the main reason was probably, that I wanted to replace the "assertTrue(A==B)" with something like "assertEqual(A, B)" as this outputs the "expected" and the actual "value" and hereby provides a little more information than a simple "was false". So I had the option of converting it to JUint "Assert.assertEquals" or update it to AssertJ's "Assertions.assertThat().isEqualTo()" which I think is a little more readable

You know that Junit4 also has a assertThat? It’s has a slightly different signature which what threw me when I first looked at the changes.

Thanks,
Justin

Re: Is Junit5 the way to go?

Posted by Christofer Dutz <ch...@c-ware.de>.

Hi Justin,

thanks for finding that ... guess when porting all these thousands of statements, I must have missed one or two "replace: ' == ' with ').isEqualTo('" __

Well the main reason was probably, that I wanted to replace the "assertTrue(A==B)" with something like "assertEqual(A, B)" as this outputs the "expected" and the actual "value" and hereby provides a little more information than a simple "was false". So I had the option of converting it to JUint "Assert.assertEquals" or update it to AssertJ's "Assertions.assertThat().isEqualTo()" which I think is a little more readable (Depending on the framework the order of "expected" and "actual" do tend to vary and with this notation it's clear) ... and I think we had an Issue and corresponding discussion to using AssertJ which allows much more detailed checks.

Chris



Am 09.02.18, 02:53 schrieb "Justin Mclean" <ju...@classsoftware.com>:

    Hi,
    
    BTW any reason you use assertThat form AssertJ rather than the one built into Junit 4? Just curious no strong views either way.
    
    You might want to fix a couple of your assertThat’s for instance this:
    
    assertThat(tpdu.getTpduCode() == TpduCode.CONNECTION_REQUEST);
    
    Isn’t probably doing what you think it is. :-)
    
    Thanks,
    Justin

Re: Is Junit5 the way to go?

Posted by Justin Mclean <ju...@classsoftware.com>.

Hi,

BTW any reason you use assertThat form AssertJ rather than the one built into Junit 4? Just curious no strong views either way.

You might want to fix a couple of your assertThat’s for instance this:

assertThat(tpdu.getTpduCode() == TpduCode.CONNECTION_REQUEST);

Isn’t probably doing what you think it is. :-)

Thanks,
Justin

Re: Is Junit5 the way to go?

Posted by Christofer Dutz <ch...@c-ware.de>.

I just compared the surefire report generated by master with Junit5 with the one in the feature branch for java4 and

Junit5: Lists 76 Tests
Junit4: Lists 138 Tests

So all the time we have been relying on a test-tool that doesn't really seem to do its job ... I strongly think we should get rid of Junit5.

Chris

Am 08.02.18, 14:07 schrieb "Christofer Dutz" <ch...@c-ware.de>:

Hi all,

so I just finished porting PLC4X back to JUnit 4 ... it was a quite straight forward thing. While at it I also ported all to use AssertJ a little more.
So far I didn't see any problems. Parametrized tests are a little ugly from my point of view (See Plc4XS7ProtocolTest)
What I did notice, was that when I ported a lot of the tests, I had to fine tune them as:

assertTrue(X == Y, "some text")

We're comparing X and Y with different types, which should not work ... I had to explicitly cast the reference type to make the tests pass.
Also I did notice, that I couldn't just click on a test-package with IntellIJ and simply run the tests with Junit 5 after porting back to 4 I can now do this correctly.

I committed my changes to the "feature/junit4" branch, so you can have a look before voting.

Chris

Am 08.02.18, 10:12 schrieb "Christofer Dutz" <ch...@c-ware.de>:

Ok ... today Justin pointed out my changes from yesterday caused tests to fail
I tried and had to agree with him ... the code was simply wrong. But I wrote tests for it and when manually executing the tests in IntelliJ they failed.
As I did a full maven build with tests before committing and Jenkins also was fine with my changes it seems that the current Junit 5 support is simply not reliable.
I did see that there are now also options for running parametrized test without some insane hackery, so I would also join your opinion that I think it's best to switch to Junit 4

So I'll whip up a vote thread on this. Just that we get to vote a little more here __

Chris

Am 26.01.18, 10:47 schrieb "Justin Mclean" <ju...@classsoftware.com>:

Hi,

> Well the one thing I like with TestNG is the support for parametrized tests.
> With JUnit 4 this sucks greatly and it very easy with TestNG. I do think that we need parametrized tests as we do have a lot of options to test.

There are some existing parametrised tests and I've used it in Junit4 (ages ago) and don’t remember it being that bad, but I’ve not used that in TestNG so perhaps it’s better there?

Thanks,
Justin

Re: Is Junit5 the way to go?

Posted by Christofer Dutz <ch...@c-ware.de>.

Hi all,

assertTrue(X == Y, "some text")

I committed my changes to the "feature/junit4" branch, so you can have a look before voting.

Chris

Am 08.02.18, 10:12 schrieb "Christofer Dutz" <ch...@c-ware.de>:

So I'll whip up a vote thread on this. Just that we get to vote a little more here __

Chris

Am 26.01.18, 10:47 schrieb "Justin Mclean" <ju...@classsoftware.com>:

Hi,

There are some existing parametrised tests and I've used it in Junit4 (ages ago) and don’t remember it being that bad, but I’ve not used that in TestNG so perhaps it’s better there?

Thanks,
Justin