You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Wojtek <wo...@unir.se> on 2023/03/20 16:03:03 UTC
Tests (packages, output, execution times)
Hi,
followup to
https://github.com/apache/james-project/pull/1476#issuecomment-1475140112
>> And the second thing - why some testing code is in the main directoryinstead oftest`?
>
> You have a specific example in mind?
For example in module
`server/protocols/jmap-rfc-8621-integration-tests/jmap-rfc-8621-integration-tests-common`
(server/protocols/jmap-rfc-8621-integration-tests/jmap-rfc-8621-integration-tests-common/src/main/scala/org/apache/james/jmap/rfc8621/contract)
or module `mpt/impl/imap-mailbox/core`
(https://github.com/apache/james-project/blob/master/mpt/impl/imap-mailbox/core/src/main/java/org/apache/james/mpt/imapmailbox/suite/Condstore.java)
Searching for `@Test` in IDEA gives ~1,5k results in production (i.e.
"main")
~~
Regarding test execution time and touching on
https://issues.apache.org/jira/browse/JAMES-3879 - would you be OK with
enabling one of maven extensions that measure execution time
(https://github.com/khmarbaise/maven-buildtime-profiler or
https://github.com/timgifford/maven-buildtime-extension)?
~~
And for exceptions, for example running tests in
`/mpt/impl/imap-mailbox` gives:
```
12:27:47.892 [ERROR] c.r.c.i.ForgivingExceptionHandler - Consumer
com.rabbitmq.client.impl.recovery.AutorecoveringChannel$2@289514d6
(amq.ctag-UxIwBfXKU00QZN9cT-Wd4w) method handleCancel for channel
AMQChannel(amqp://guest@127.0.0.1:58238/,6) threw an exception for
channel AMQChannel(amqp://guest@127.0.0.1:58238/,6)
com.rabbitmq.client.ShutdownSignalException: clean connection shutdown;
protocol method: #method<connection.close>(reply-code=200,
reply-text=OK, class-id=0, method-id=0)
at
com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66)
at
com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36)
at
com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:502)
at com.rabbitmq.client.impl.ChannelN.close(ChannelN.java:617)
```
It's exception but looks like just information that shutdown happens and
it went ok?
And the full build
(https://ci-builds.apache.org/job/james/job/ApacheJames/job/PR-1476/9/console)
has other issues, for example lots of NoSuchMethodException from
`MailetdocsReport.buildDescriptors (MailetdocsReport.java:49)` (those
are quite probably caused by switch to injection-by-constructor and lack
of default constructor) or ClassNotFoundException (not sure about those,
classpath/lack of dependency) but then the whole Mailetdocs Report fails
in a lot of cases (~1,6k exceptions from total of ~8k)
I'm not sure what's the best course of action is here: IMHO if the test
passes then there should only be single line in the output: "Tests run:
x, Failures: 0, Errors: 0, Skipped: 0". If there are failures then only
ST related to that failure should be logged?
--
Wojtek
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Tests (packages, output, execution times)
Posted by Wojtek <wo...@unir.se>.
Hi,
On 22/03/2023 03:54, Benoit TELLIER wrote:
>> I get that those are abstract classes and are not intended to be run
>> directly (or can't be for that matter) - while testing I leaned on
>> that to simply run "in memory" flavour which was simply the
>> fastest/most covenient.
>>
>> I get that you can use it as maven dependency though I think it should
>> be possible to do it as "test" scope as well - maven should still
>> allow to resolve transitive dependencies here?
>
> There's no transitive dependencies for test-jar qualifier.
>
> exemple: all dependencies declared in mpt-imapmailbox-core would need to
> be re-declared in mpt-imapmailbox-memory, mpt-imapmailbox-cassandra, etc...
>
> That's the main reason behind moving contract test suites for maven
> projects dedicated to tests into main.
> Best regards,
> Benoit
TIL!
Also, not a new issue it seems:
https://issues.apache.org/jira/browse/MNG-1378 (Created: 31/Oct/05) :-)
w.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Tests (packages, output, execution times)
Posted by Benoit TELLIER <bt...@apache.org>.
Hello,
On 21/03/2023 20:55, Wojtek wrote:
> Hi
>
> On 20/03/2023 19:40, Benoit TELLIER wrote:
>> On 20/03/2023 23:03, Wojtek wrote:
>>> For example in module
>>> `server/protocols/jmap-rfc-8621-integration-tests/jmap-rfc-8621-integration-tests-common`
>>> (server/protocols/jmap-rfc-8621-integration-tests/jmap-rfc-8621-integration-tests-common/src/main/scala/org/apache/james/jmap/rfc8621/contract)
>>>
>>> or module `mpt/impl/imap-mailbox/core`
>>> (https://github.com/apache/james-project/blob/master/mpt/impl/imap-mailbox/core/src/main/java/org/apache/james/mpt/imapmailbox/suite/Condstore.java)
>>>
>>> Searching for `@Test` in IDEA gives ~1,5k results in production
>>> (i.e. "main")
>>
>> That is because those modules defines contracts tests: they are only
>> testing interfaces, not made to be run directly.
>> James implementation would derive those tests inside a test folder,
>> and run them against the implementation.
>> The reason to locate those contracts into mail folder is to benefit
>> from maven transitive dependencies.
>
> I get that those are abstract classes and are not intended to be run
> directly (or can't be for that matter) - while testing I leaned on
> that to simply run "in memory" flavour which was simply the
> fastest/most covenient.
>
> I get that you can use it as maven dependency though I think it should
> be possible to do it as "test" scope as well - maven should still
> allow to resolve transitive dependencies here?
There's no transitive dependencies for test-jar qualifier.
exemple: all dependencies declared in mpt-imapmailbox-core would need to
be re-declared in mpt-imapmailbox-memory, mpt-imapmailbox-cassandra, etc...
That's the main reason behind moving contract test suites for maven
projects dedicated to tests into main.
[...]
Best regards,
Benoit
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Tests (packages, output, execution times)
Posted by Wojtek <wo...@unir.se>.
Hi
On 20/03/2023 19:40, Benoit TELLIER wrote:
> On 20/03/2023 23:03, Wojtek wrote:
>> For example in module
>> `server/protocols/jmap-rfc-8621-integration-tests/jmap-rfc-8621-integration-tests-common` (server/protocols/jmap-rfc-8621-integration-tests/jmap-rfc-8621-integration-tests-common/src/main/scala/org/apache/james/jmap/rfc8621/contract)
>> or module `mpt/impl/imap-mailbox/core`
>> (https://github.com/apache/james-project/blob/master/mpt/impl/imap-mailbox/core/src/main/java/org/apache/james/mpt/imapmailbox/suite/Condstore.java)
>>
>> Searching for `@Test` in IDEA gives ~1,5k results in production (i.e.
>> "main")
>
> That is because those modules defines contracts tests: they are only
> testing interfaces, not made to be run directly.
> James implementation would derive those tests inside a test folder, and
> run them against the implementation.
> The reason to locate those contracts into mail folder is to benefit from
> maven transitive dependencies.
I get that those are abstract classes and are not intended to be run
directly (or can't be for that matter) - while testing I leaned on that
to simply run "in memory" flavour which was simply the fastest/most
covenient.
I get that you can use it as maven dependency though I think it should
be possible to do it as "test" scope as well - maven should still allow
to resolve transitive dependencies here?
>> ~~
>>
>> Regarding test execution time and touching on
>> https://issues.apache.org/jira/browse/JAMES-3879 - would you be OK
>> with enabling one of maven extensions that measure execution time
>> (https://github.com/khmarbaise/maven-buildtime-profiler or
>> https://github.com/timgifford/maven-buildtime-extension)?
> I bet it would!
I created PR: https://github.com/apache/james-project/pull/1498
>> ~~
>>
>> And for exceptions, for example running tests in
>> `/mpt/impl/imap-mailbox` gives:
>>
>> ```
>> 12:27:47.892 [ERROR] c.r.c.i.ForgivingExceptionHandler - Consumer
>> com.rabbitmq.client.impl.recovery.AutorecoveringChannel$2@289514d6
>> (amq.ctag-UxIwBfXKU00QZN9cT-Wd4w) method handleCancel for channel
>> AMQChannel(amqp://guest@127.0.0.1:58238/,6) threw an exception for
>> channel AMQChannel(amqp://guest@127.0.0.1:58238/,6)
>> com.rabbitmq.client.ShutdownSignalException: clean connection
>> shutdown; protocol method: #method<connection.close>(reply-code=200,
>> reply-text=OK, class-id=0, method-id=0)
>> at
>> com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66)
>> at
>> com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36)
>> at
>> com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:502)
>> at com.rabbitmq.client.impl.ChannelN.close(ChannelN.java:617)
>>
>> ```
>>
>> It's exception but looks like just information that shutdown happens
>> and it went ok?
>
> I imagine this could either be that, or an AMQP consumer that is not
> shut down before the connection?
I'm not sure, not much experience with rabbitmq/AMQP unfrotunatelly.
>> And the full build
>> (https://ci-builds.apache.org/job/james/job/ApacheJames/job/PR-1476/9/console) has other issues, for example lots of NoSuchMethodException from `MailetdocsReport.buildDescriptors (MailetdocsReport.java:49)` (those are quite probably caused by switch to injection-by-constructor and lack of default constructor) or ClassNotFoundException (not sure about those, classpath/lack of dependency) but then the whole Mailetdocs Report fails in a lot of cases (~1,6k exceptions from total of ~8k)
> +1 to open a ticket and address that one.
>
> Likely a easy fix inside the mailetdoc plugin....
Done: https://issues.apache.org/jira/browse/JAMES-3896
--
Wojtek
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Tests (packages, output, execution times)
Posted by Benoit TELLIER <bt...@linagora.com>.
Hello Wojtek,
Answers inlined.
Best regards
Benoit
On 20/03/2023 23:03, Wojtek wrote:
> Hi,
> followup to
> https://github.com/apache/james-project/pull/1476#issuecomment-1475140112
>
>
>>> And the second thing - why some testing code is in the main
>>> directoryinstead oftest`?
>>
>> You have a specific example in mind?
>
>
> For example in module
> `server/protocols/jmap-rfc-8621-integration-tests/jmap-rfc-8621-integration-tests-common`
> (server/protocols/jmap-rfc-8621-integration-tests/jmap-rfc-8621-integration-tests-common/src/main/scala/org/apache/james/jmap/rfc8621/contract)
>
> or module `mpt/impl/imap-mailbox/core`
> (https://github.com/apache/james-project/blob/master/mpt/impl/imap-mailbox/core/src/main/java/org/apache/james/mpt/imapmailbox/suite/Condstore.java)
>
> Searching for `@Test` in IDEA gives ~1,5k results in production (i.e.
> "main")
That is because those modules defines contracts tests: they are only
testing interfaces, not made to be run directly.
James implementation would derive those tests inside a test folder, and
run them against the implementation.
The reason to locate those contracts into mail folder is to benefit from
maven transitive dependencies.
>
> ~~
>
> Regarding test execution time and touching on
> https://issues.apache.org/jira/browse/JAMES-3879 - would you be OK
> with enabling one of maven extensions that measure execution time
> (https://github.com/khmarbaise/maven-buildtime-profiler or
> https://github.com/timgifford/maven-buildtime-extension)?
I bet it would!
>
> ~~
>
> And for exceptions, for example running tests in
> `/mpt/impl/imap-mailbox` gives:
>
> ```
> 12:27:47.892 [ERROR] c.r.c.i.ForgivingExceptionHandler - Consumer
> com.rabbitmq.client.impl.recovery.AutorecoveringChannel$2@289514d6
> (amq.ctag-UxIwBfXKU00QZN9cT-Wd4w) method handleCancel for channel
> AMQChannel(amqp://guest@127.0.0.1:58238/,6) threw an exception for
> channel AMQChannel(amqp://guest@127.0.0.1:58238/,6)
> com.rabbitmq.client.ShutdownSignalException: clean connection
> shutdown; protocol method: #method<connection.close>(reply-code=200,
> reply-text=OK, class-id=0, method-id=0)
> at
> com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66)
> at
> com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36)
> at
> com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:502)
> at com.rabbitmq.client.impl.ChannelN.close(ChannelN.java:617)
>
> ```
>
> It's exception but looks like just information that shutdown happens
> and it went ok?
I imagine this could either be that, or an AMQP consumer that is not
shut down before the connection?
>
> And the full build
> (https://ci-builds.apache.org/job/james/job/ApacheJames/job/PR-1476/9/console)
> has other issues, for example lots of NoSuchMethodException from
> `MailetdocsReport.buildDescriptors (MailetdocsReport.java:49)` (those
> are quite probably caused by switch to injection-by-constructor and
> lack of default constructor) or ClassNotFoundException (not sure about
> those, classpath/lack of dependency) but then the whole Mailetdocs
> Report fails in a lot of cases (~1,6k exceptions from total of ~8k)
+1 to open a ticket and address that one.
Likely a easy fix inside the mailetdoc plugin....
>
> I'm not sure what's the best course of action is here: IMHO if the
> test passes then there should only be single line in the output:
> "Tests run: x, Failures: 0, Errors: 0, Skipped: 0". If there are
> failures then only ST related to that failure should be logged?
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org