You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@james.apache.org by GitBox <gi...@apache.org> on 2020/11/24 09:51:33 UTC
[GitHub] [james-project] jeantil opened a new pull request #264: JAMES-3225 configures CI for builds.apache.org
jeantil opened a new pull request #264:
URL: https://github.com/apache/james-project/pull/264
As an individual contributor I prefer having slow CI builds than no CI
builds at all. This commit introduces a minimal JenkinsFile to be
further refined.
I was shamelessly adapted from the activemq project's
[JenkinsFile](https://raw.githubusercontent.com/apache/activemq/master/Jenkinsfile)
and the [JenkinsFile](https://raw.githubusercontent.com/apache/james-project/3017d55642ec48171e6190aad06a7bfbfd74b400/Jenkinsfile)
proposed in the gradle branch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] aduprat commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
aduprat commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r529440705
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,170 @@
+#!groovy
+
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def AGENT_LABEL = env.AGENT_LABEL ?: 'ubuntu'
+def JDK_NAME = env.JDK_NAME ?: 'JDK 11 (latest)'
+
+pipeline {
+
+ agent {
+ node {
+ label AGENT_LABEL
+ }
+ }
+
+ environment {
+ // ... setup any environment variables ...
+ MVN_LOCAL_REPO_OPT = '-Dmaven.repo.local=.repository'
+ MVN_TEST_FAIL_IGNORE = '-Dmaven.test.failure.ignore=true'
+ CI=true
+ }
+
+ tools {
+ // ... tell Jenkins what java version, maven version or other tools are required ...
+ maven 'maven_3_latest'
+ jdk JDK_NAME
+ }
+
+ options {
+ // Configure an overall timeout for the build of 4 hours.
+ timeout(time: 4, unit: 'HOURS')
+ // When we have test-fails e.g. we don't need to run the remaining steps
+ skipStagesAfterUnstable()
+ buildDiscarder(
+ logRotator(artifactNumToKeepStr: '5', numToKeepStr: '10')
+ )
+ disableConcurrentBuilds()
+ }
+
+ stages {
+ stage('Initialization') {
+ steps {
+ echo 'Building branch ' + env.BRANCH_NAME
+ echo 'Using PATH ' + env.PATH
+ }
+ }
+
+ stage('Cleanup') {
+ steps {
+ echo 'Cleaning up the workspace'
+ deleteDir()
+ }
+ }
+
+ stage('Checkout') {
+ steps {
+ echo 'Checking out branch ' + env.BRANCH_NAME
+ checkout scm
+ }
+ }
+
+ stage('Build') {
+ steps {
+ echo 'Building'
+ sh 'mvn -U -B -e clean install -DskipTests'
+ }
+ }
+
+ stage('Tests') {
+ steps {
+ echo 'Running tests'
+ // all tests is very very long (10 hours on Apache Jenkins)
+ sh 'mvn -B -e test'
+ }
+ post {
+ always {
+ junit(testResults: '**/surefire-reports/*.xml', allowEmptyResults: true)
+ junit(testResults: '**/failsafe-reports/*.xml', allowEmptyResults: true)
+ }
+ }
+ }
+
+ stage('Deploy') {
+ when {
Review comment:
I never tried.
According to the documentation, it's available only on `stage`, so it should not work on a `post` step.
```
The when directive allows the Pipeline to determine whether the stage should be executed depending on the given condition.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-752896739
According to https://issues.apache.org/jira/browse/INFRA-20673 and https://plugins.jenkins.io/pipeline-github/#issuecommenttrigger : test this please should work now
sorry to the participants for the extra notification
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil edited a comment on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil edited a comment on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-774474432
We finally got a green run : https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/51/pipeline
to get there we had to drop all attempts at parallelisation: no surefire-forks, no running jobs in parallel and be lucky not to be hit by the [copy on write issue](https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/49/tests) (see the detailed logs for org.apache.james.smtp.SmtpRandomStoringTest and you will find the signature null pointer exception ```
15:53:44.340 [ERROR] o.a.j.t.m.d.MailDispatcher - Error While storing mail.
java.lang.NullPointerException: null
at org.apache.james.server.core.MimeMessageInputStream.<init>(MimeMessageInputStream.java:63)
```
or by timeouts as in https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/50/tests
build 52 where I attempted to run test suites in parallel again failed with a linshare container initialization failure which had been [encountered before](https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/41/tests):
```
java.lang.ExceptionInInitializerError
at org.apache.james.webadmin.integration.memory.vault.MemoryLinshareBlobExportMechanismIntegrationTest.<clinit>(MemoryLinshareBlobExportMechanismIntegrationTest.java:34)
Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed
at org.apache.james.webadmin.integration.memory.vault.MemoryLinshareBlobExportMechanismIntegrationTest.<clinit>(MemoryLinshareBlobExportMechanismIntegrationTest.java:34)
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
at org.apache.james.webadmin.integration.memory.vault.MemoryLinshareBlobExportMechanismIntegrationTest.<clinit>(MemoryLinshareBlobExportMechanismIntegrationTest.java:34)
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
```
I started archiving the test-run.log on failure and logging as much test container related stuff as possible to have more details on such failures: it looks like the container fails to coordinate internally or maybe is killed during initialization by a timeout that is a bit too optimistic for the CI plateform ... (the database receives a fast-shutdown command while it is apply migrations and the backend which is starting fails to connect)
```
03:59:52.056 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> COMMIT
03:59:52.105 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.155 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.222 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.271 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.345 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.397 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.455 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.456 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database>
03:59:52.462 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> LOG: received fast shutdown request
03:59:52.462 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> waiting for server to shut down...LOG: aborting any active transactions
03:59:52.462 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> .LOG: autovacuum launcher shutting down
03:59:52.466 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> LOG: shutting down
03:59:52.706 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> [WARN]:localhost-startStop-1:20210206.035952:org.hibernate.orm.cache:createCache:HHH90001006: Missing cache[default-update-timestamps-region] was created on-the-fly. The created cache will use a provider-specific default configuration: make sure you defined one. You can disable this warning by setting 'hibernate.cache.ehcache.missing_cache_strategy' to 'create'.
03:59:52.706 [DEBUG] o.t.c.output.WaitingConsumer - STDOUT: [WARN]:localhost-startStop-1:20210206.035952:org.hibernate.orm.cache:createCache:HHH90001006: Missing cache[default-update-timestamps-region] was created on-the-fly. The created cache will use a provider-specific default configuration: make sure you defined one. You can disable this warning by setting 'hibernate.cache.ehcache.missing_cache_strategy' to 'create'.efault-query-results-region] was created on-the-fly. The created cache will use a provider-specific default configuration: make sure you defined one. You can disable this warning by setting 'hibernate.cache.ehcache.missing_cache_strategy' to 'create'.
03:59:52.737 [DEBUG] o.t.c.output.WaitingConsumer - STDOUT: [WARN]:localhost-startStop-1:20210206.035952:org.hibernate.orm.cache:createCache:HHH90001006: Missing cache[default-query-results-region] was created on-the-fly. The created cache will use a provider-specific default configuration: make sure you defined one. You can disable this warning by setting 'hibernate.cache.ehcache.missing_cache_strategy' to 'create'..ehcache.missing_cache_strategy' to 'create'.
03:59:56.296 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> 06-Feb-2021 03:59:56.291 WARNING [localhost-startStop-1] org.postgresql.core.v3.ConnectionFactoryImpl.log ConnectException occurred while connecting to linshare_database:5432
03:59:56.296 [DEBUG] o.t.c.output.WaitingConsumer - STDERR: java.net.ConnectException: Connection refused (Connection refused)
03:59:56.296 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> java.net.ConnectException: Connection refused (Connection refused)
03:59:56.296 [DEBUG] o.t.c.output.WaitingConsumer - STDERR: at org.hibernate.tool.schema.internal.exec.ImprovedExtractionContextImpl.getJdbcConnection(ImprovedExtractionContextImpl.java:60)
03:59:56.296 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> at java.net.PlainSocketImpl.socketConnect(Native Method)
03:59:56.296 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-775729448
new successful build : https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/58/pipeline/ 7h10
build 57 failed because docker cassandra failed to start within 1 min, analysing the logs it actually took 1m10s to start
on my laptop (in an isolated case) the container started in 22 seconds.
curious I tried adding test steps to measure node performance, sysbench was an attempt with low probability but cpuinfo and free worked.
The compute node for 59 advertises 24 6-core Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz CPUs (`cat /proc/cpu`) and 94GB of RAM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] mbaechler commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
mbaechler commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-775791198
MERGED :tada:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r529442842
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,170 @@
+#!groovy
+
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def AGENT_LABEL = env.AGENT_LABEL ?: 'ubuntu'
+def JDK_NAME = env.JDK_NAME ?: 'JDK 11 (latest)'
+
+pipeline {
+
+ agent {
+ node {
+ label AGENT_LABEL
+ }
+ }
+
+ environment {
+ // ... setup any environment variables ...
+ MVN_LOCAL_REPO_OPT = '-Dmaven.repo.local=.repository'
+ MVN_TEST_FAIL_IGNORE = '-Dmaven.test.failure.ignore=true'
+ CI=true
+ }
+
+ tools {
+ // ... tell Jenkins what java version, maven version or other tools are required ...
+ maven 'maven_3_latest'
+ jdk JDK_NAME
+ }
+
+ options {
+ // Configure an overall timeout for the build of 4 hours.
+ timeout(time: 4, unit: 'HOURS')
+ // When we have test-fails e.g. we don't need to run the remaining steps
+ skipStagesAfterUnstable()
+ buildDiscarder(
+ logRotator(artifactNumToKeepStr: '5', numToKeepStr: '10')
+ )
+ disableConcurrentBuilds()
+ }
+
+ stages {
+ stage('Initialization') {
+ steps {
+ echo 'Building branch ' + env.BRANCH_NAME
+ echo 'Using PATH ' + env.PATH
+ }
+ }
+
+ stage('Cleanup') {
+ steps {
+ echo 'Cleaning up the workspace'
+ deleteDir()
+ }
+ }
+
+ stage('Checkout') {
+ steps {
+ echo 'Checking out branch ' + env.BRANCH_NAME
+ checkout scm
+ }
+ }
+
+ stage('Build') {
+ steps {
+ echo 'Building'
+ sh 'mvn -U -B -e clean install -DskipTests'
+ }
+ }
+
+ stage('Tests') {
+ steps {
+ echo 'Running tests'
+ // all tests is very very long (10 hours on Apache Jenkins)
+ sh 'mvn -B -e test'
+ }
+ post {
+ always {
+ junit(testResults: '**/surefire-reports/*.xml', allowEmptyResults: true)
+ junit(testResults: '**/failsafe-reports/*.xml', allowEmptyResults: true)
+ }
+ }
+ }
+
+ stage('Deploy') {
+ when {
Review comment:
thanks I'll leave it like this for now
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571486075
##########
File path: Jenkinsfile
##########
@@ -47,7 +47,7 @@ pipeline {
options {
// Configure an overall timeout for the build of 4 hours.
- timeout(time: 4, unit: 'HOURS')
Review comment:
Build 51 (the only one green at this stage ) took over 7 hours to complete.
4 hours was appropriate with surefire forkReuse=true
instead of dropping the commit I fixed it up to the commit which introduces the Jenkinsfile
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-732811406
I intend to make a followup PR adding `-T1C` to the maven commands to try and speedup the build but I prefer to start with a simpler build :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r529410152
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,170 @@
+#!groovy
+
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def AGENT_LABEL = env.AGENT_LABEL ?: 'ubuntu'
+def JDK_NAME = env.JDK_NAME ?: 'JDK 11 (latest)'
+
+pipeline {
+
+ agent {
+ node {
+ label AGENT_LABEL
+ }
+ }
+
+ environment {
+ // ... setup any environment variables ...
+ MVN_LOCAL_REPO_OPT = '-Dmaven.repo.local=.repository'
+ MVN_TEST_FAIL_IGNORE = '-Dmaven.test.failure.ignore=true'
+ CI=true
+ }
+
+ tools {
+ // ... tell Jenkins what java version, maven version or other tools are required ...
+ maven 'maven_3_latest'
+ jdk JDK_NAME
+ }
+
+ options {
+ // Configure an overall timeout for the build of 4 hours.
+ timeout(time: 4, unit: 'HOURS')
+ // When we have test-fails e.g. we don't need to run the remaining steps
+ skipStagesAfterUnstable()
+ buildDiscarder(
+ logRotator(artifactNumToKeepStr: '5', numToKeepStr: '10')
+ )
+ disableConcurrentBuilds()
+ }
+
+ stages {
+ stage('Initialization') {
+ steps {
+ echo 'Building branch ' + env.BRANCH_NAME
+ echo 'Using PATH ' + env.PATH
+ }
+ }
+
+ stage('Cleanup') {
+ steps {
+ echo 'Cleaning up the workspace'
+ deleteDir()
+ }
+ }
+
+ stage('Checkout') {
+ steps {
+ echo 'Checking out branch ' + env.BRANCH_NAME
+ checkout scm
+ }
+ }
+
+ stage('Build') {
+ steps {
+ echo 'Building'
+ sh 'mvn -U -B -e clean install -DskipTests'
+ }
+ }
+
+ stage('Tests') {
+ steps {
+ echo 'Running tests'
+ // all tests is very very long (10 hours on Apache Jenkins)
+ sh 'mvn -B -e test'
+ }
+ post {
+ always {
+ junit(testResults: '**/surefire-reports/*.xml', allowEmptyResults: true)
+ junit(testResults: '**/failsafe-reports/*.xml', allowEmptyResults: true)
+ }
+ }
+ }
+
+ stage('Deploy') {
+ when {
Review comment:
does it also work in the `post` steps ?
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,170 @@
+#!groovy
+
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def AGENT_LABEL = env.AGENT_LABEL ?: 'ubuntu'
+def JDK_NAME = env.JDK_NAME ?: 'JDK 11 (latest)'
+
+pipeline {
+
+ agent {
+ node {
+ label AGENT_LABEL
+ }
+ }
+
+ environment {
+ // ... setup any environment variables ...
+ MVN_LOCAL_REPO_OPT = '-Dmaven.repo.local=.repository'
+ MVN_TEST_FAIL_IGNORE = '-Dmaven.test.failure.ignore=true'
+ CI=true
+ }
+
+ tools {
+ // ... tell Jenkins what java version, maven version or other tools are required ...
+ maven 'maven_3_latest'
+ jdk JDK_NAME
+ }
+
+ options {
+ // Configure an overall timeout for the build of 4 hours.
+ timeout(time: 4, unit: 'HOURS')
+ // When we have test-fails e.g. we don't need to run the remaining steps
+ skipStagesAfterUnstable()
+ buildDiscarder(
+ logRotator(artifactNumToKeepStr: '5', numToKeepStr: '10')
+ )
+ disableConcurrentBuilds()
+ }
+
+ stages {
+ stage('Initialization') {
+ steps {
+ echo 'Building branch ' + env.BRANCH_NAME
+ echo 'Using PATH ' + env.PATH
+ }
+ }
+
+ stage('Cleanup') {
+ steps {
+ echo 'Cleaning up the workspace'
+ deleteDir()
+ }
+ }
+
+ stage('Checkout') {
+ steps {
+ echo 'Checking out branch ' + env.BRANCH_NAME
+ checkout scm
+ }
+ }
+
+ stage('Build') {
+ steps {
+ echo 'Building'
+ sh 'mvn -U -B -e clean install -DskipTests'
+ }
+ }
+
+ stage('Tests') {
+ steps {
+ echo 'Running tests'
+ // all tests is very very long (10 hours on Apache Jenkins)
+ sh 'mvn -B -e test'
+ }
+ post {
+ always {
+ junit(testResults: '**/surefire-reports/*.xml', allowEmptyResults: true)
+ junit(testResults: '**/failsafe-reports/*.xml', allowEmptyResults: true)
+ }
+ }
+ }
+
+ stage('Deploy') {
+ when {
Review comment:
i fixed up this one
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil edited a comment on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil edited a comment on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-752232783
The CI builds are now reasonnably successful see https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/activity
The PR history is clean and it seems an ~admin enabled build for all PRs~ never mind, github detected the sucessful build for the commit on another branch but it didn't build this branch.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-774474432
We finally got a green run : https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/51/pipeline
to get there we had to drop all attempts at parallelisation: no surefire-forks, no running jobs in parallel and be lucky not to be hit by the [copy on write issue](https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/49/tests) (see the detailed logs for org.apache.james.smtp.SmtpRandomStoringTest and you will find the signature null pointer exception ```
15:53:44.340 [ERROR] o.a.j.t.m.d.MailDispatcher - Error While storing mail.
java.lang.NullPointerException: null
at org.apache.james.server.core.MimeMessageInputStream.<init>(MimeMessageInputStream.java:63)
```
or by timeouts as in https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/50/tests
build 52 where I attempted to run test suites in parallel again failed with a linshare container initialization failure which had been [encountered before](https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/41/tests):
```
java.lang.ExceptionInInitializerError
at org.apache.james.webadmin.integration.memory.vault.MemoryLinshareBlobExportMechanismIntegrationTest.<clinit>(MemoryLinshareBlobExportMechanismIntegrationTest.java:34)
Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed
at org.apache.james.webadmin.integration.memory.vault.MemoryLinshareBlobExportMechanismIntegrationTest.<clinit>(MemoryLinshareBlobExportMechanismIntegrationTest.java:34)
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
at org.apache.james.webadmin.integration.memory.vault.MemoryLinshareBlobExportMechanismIntegrationTest.<clinit>(MemoryLinshareBlobExportMechanismIntegrationTest.java:34)
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
```
I started archiving the test-run.log on failure and logging as much test container related stuff as possible to have more details on such failures: it looks like the container fails to coordinate internally or maybe is killed during initialization by a timeout that is a bit too optimistic for the CI plateform ... (the database receives a fast-shutdown command while it is apply migrations and the backend which is starting fails to connect)
```
03:59:52.056 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> COMMIT
03:59:52.105 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.155 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.222 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.271 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.345 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.397 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.455 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.456 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database>
03:59:52.462 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> LOG: received fast shutdown request
03:59:52.462 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> waiting for server to shut down...LOG: aborting any active transactions
03:59:52.462 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> .LOG: autovacuum launcher shutting down
03:59:52.466 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> LOG: shutting down
03:59:52.706 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> [WARN]:localhost-startStop-1:20210206.035952:org.hibernate.orm.cache:createCache:HHH90001006: Missing cache[default-update-timestamps-region] was created on-the-fly. The created cache will use a provider-specific default configuration: make sure you defined one. You can disable this warning by setting 'hibernate.cache.ehcache.missing_cache_strategy' to 'create'.
03:59:52.706 [DEBUG] o.t.c.output.WaitingConsumer - STDOUT: [WARN]:localhost-startStop-1:20210206.035952:org.hibernate.orm.cache:createCache:HHH90001006: Missing cache[default-update-timestamps-region] was created on-the-fly. The created cache will use a provider-specific default configuration: make sure you defined one. You can disable this warning by setting 'hibernate.cache.ehcache.missing_cache_strategy' to 'create'.efault-query-results-region] was created on-the-fly. The created cache will use a provider-specific default configuration: make sure you defined one. You can disable this warning by setting 'hibernate.cache.ehcache.missing_cache_strategy' to 'create'.
03:59:52.737 [DEBUG] o.t.c.output.WaitingConsumer - STDOUT: [WARN]:localhost-startStop-1:20210206.035952:org.hibernate.orm.cache:createCache:HHH90001006: Missing cache[default-query-results-region] was created on-the-fly. The created cache will use a provider-specific default configuration: make sure you defined one. You can disable this warning by setting 'hibernate.cache.ehcache.missing_cache_strategy' to 'create'..ehcache.missing_cache_strategy' to 'create'.
03:59:56.296 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> 06-Feb-2021 03:59:56.291 WARNING [localhost-startStop-1] org.postgresql.core.v3.ConnectionFactoryImpl.log ConnectException occurred while connecting to linshare_database:5432
03:59:56.296 [DEBUG] o.t.c.output.WaitingConsumer - STDERR: java.net.ConnectException: Connection refused (Connection refused)
03:59:56.296 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> java.net.ConnectException: Connection refused (Connection refused)
03:59:56.296 [DEBUG] o.t.c.output.WaitingConsumer - STDERR: at org.hibernate.tool.schema.internal.exec.ImprovedExtractionContextImpl.getJdbcConnection(ImprovedExtractionContextImpl.java:60)
03:59:56.296 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> at java.net.PlainSocketImpl.socketConnect(Native Method)
03:59:56.296 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] Arsnael commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
Arsnael commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r551108614
##########
File path: Jenkinsfile
##########
@@ -87,36 +87,37 @@ pipeline {
sh 'mvn -U -B -e clean install -DskipTests -T1C'
}
}
-
- stage('Stable Tests') {
- steps {
- echo 'Running tests'
- // all tests run is very very long (10 hours on Apache Jenkins)
- sh 'mvn -B -e -fae test '
- }
- post {
- always {
- junit(testResults: '**/surefire-reports/*.xml', allowEmptyResults: true)
- junit(testResults: '**/failsafe-reports/*.xml', allowEmptyResults: true)
+ stage('Tests') {
+ parallel {
Review comment:
Thanks for the clarification, I might have raised a false alarm :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r529397152
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,175 @@
+#!groovy
+
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def AGENT_LABEL = env.AGENT_LABEL ?: 'ubuntu'
+def JDK_NAME = env.JDK_NAME ?: 'JDK 11 (latest)'
+
+pipeline {
+
+ agent {
+ node {
+ label AGENT_LABEL
+ }
+ }
+
+ environment {
+ // ... setup any environment variables ...
+ MVN_LOCAL_REPO_OPT = '-Dmaven.repo.local=.repository'
+ MVN_TEST_FAIL_IGNORE = '-Dmaven.test.failure.ignore=true'
+ CI=true
+ }
+
+ tools {
+ // ... tell Jenkins what java version, maven version or other tools are required ...
+ maven 'maven_3_latest'
+ jdk JDK_NAME
+ }
+
+ options {
+ // Configure an overall timeout for the build of 4 hours.
+ timeout(time: 4, unit: 'HOURS')
+ // When we have test-fails e.g. we don't need to run the remaining steps
+ skipStagesAfterUnstable()
+ buildDiscarder(
+ logRotator(artifactNumToKeepStr: '5', numToKeepStr: '10')
+ )
+ disableConcurrentBuilds()
+ }
+
+ stages {
+ stage('Initialization') {
+ steps {
+ echo 'Building branch ' + env.BRANCH_NAME
+ echo 'Using PATH ' + env.PATH
+ }
+ }
+
+ stage('Cleanup') {
+ steps {
+ echo 'Cleaning up the workspace'
+ deleteDir()
+ }
+ }
+
+ stage('Checkout') {
+ steps {
+ echo 'Checking out branch ' + env.BRANCH_NAME
+ checkout scm
+ }
+ }
+
+ stage('Build') {
+ steps {
+ echo 'Building'
+ sh 'mvn -U -B -e clean install -DskipTests'
+ }
+ }
+
+ stage('Tests') {
+ steps {
+ echo 'Running tests'
+ // all tests is very very long (10 hours on Apache Jenkins)
+ sh 'mvn -B -e test'
+ }
+ post {
+ always {
+ junit(testResults: '**/surefire-reports/*.xml', allowEmptyResults: true)
+ junit(testResults: '**/failsafe-reports/*.xml', allowEmptyResults: true)
+ }
+ }
+ }
+ /**
+ Unsure what the project's policy is regarding deployment
+ I left a sane sample in the file to facilitate further config
+ by a maintainer
Review comment:
I have updated the PR but I'm not sure if the jenkins job will have the rights to do it (we will get pipeline errors until this is setup or commented again)
also it seems that it doesn't want to build PRs from forks of people which are not maintainers/developpers in the project. can you try duplicating this PR from an account with write access to verify it triggers the build ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
chibenwa commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571860821
##########
File path: backends-common/cassandra/src/test/java/org/apache/james/backends/cassandra/DockerCassandra.java
##########
@@ -118,11 +126,36 @@ private void grantPermissionToTestingUser(Cluster privilegedCluster, String keys
@SuppressWarnings("resource")
public DockerCassandra() {
- this("cassandra_3_11_3", AdditionalDockerFileStep.IDENTITY);
+ this("cassandra_3_11_3-"+ UUID.randomUUID().toString(), AdditionalDockerFileStep.IDENTITY);
Review comment:
> I guess you don't feel the pain of trying to build james on a CI for months.
That's why I approved straight away :-)
> If this PR goes green, we'll definitely want to merge it and try to optimize later.
I perfectly understand that and agree with it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-774979256
I guess we will then have discussions on how to make this build faster.
I see that JMAP draft integration tests takes quit a bunch of time. Do we really need them to run on top of both cassandra-guice and distributed james? (We could reclaim 1h of build time very easily there).
Maybe some features (LinShare for instance?) have little meaning in this Apache project and could be removed + maintained by third parties. Should we plan to audit such features?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] mbaechler closed pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
mbaechler closed pull request #264:
URL: https://github.com/apache/james-project/pull/264
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-774983676
> I think I read in a discussion that JMAP draft was going to be dropped "soon" anyways which is why I didn't spend any time splitting this off.
Yes, we kinda finished implementing JMAP RFC-8621 - missing MDN support, but once done we should be ready for a deprecation/removal.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] mbaechler commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
mbaechler commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-763504186
I can't figure out what breaks.
test this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-775751041
I do agree - hopefully we will quickly shorten the build time to make it less problematic.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] aduprat commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
aduprat commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r529399037
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,170 @@
+#!groovy
+
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def AGENT_LABEL = env.AGENT_LABEL ?: 'ubuntu'
+def JDK_NAME = env.JDK_NAME ?: 'JDK 11 (latest)'
+
+pipeline {
+
+ agent {
+ node {
+ label AGENT_LABEL
+ }
+ }
+
+ environment {
+ // ... setup any environment variables ...
+ MVN_LOCAL_REPO_OPT = '-Dmaven.repo.local=.repository'
+ MVN_TEST_FAIL_IGNORE = '-Dmaven.test.failure.ignore=true'
+ CI=true
+ }
+
+ tools {
+ // ... tell Jenkins what java version, maven version or other tools are required ...
+ maven 'maven_3_latest'
+ jdk JDK_NAME
+ }
+
+ options {
+ // Configure an overall timeout for the build of 4 hours.
+ timeout(time: 4, unit: 'HOURS')
+ // When we have test-fails e.g. we don't need to run the remaining steps
+ skipStagesAfterUnstable()
+ buildDiscarder(
+ logRotator(artifactNumToKeepStr: '5', numToKeepStr: '10')
+ )
+ disableConcurrentBuilds()
+ }
+
+ stages {
+ stage('Initialization') {
+ steps {
+ echo 'Building branch ' + env.BRANCH_NAME
+ echo 'Using PATH ' + env.PATH
+ }
+ }
+
+ stage('Cleanup') {
+ steps {
+ echo 'Cleaning up the workspace'
+ deleteDir()
+ }
+ }
+
+ stage('Checkout') {
+ steps {
+ echo 'Checking out branch ' + env.BRANCH_NAME
+ checkout scm
+ }
+ }
+
+ stage('Build') {
+ steps {
+ echo 'Building'
+ sh 'mvn -U -B -e clean install -DskipTests'
+ }
+ }
+
+ stage('Tests') {
+ steps {
+ echo 'Running tests'
+ // all tests is very very long (10 hours on Apache Jenkins)
+ sh 'mvn -B -e test'
+ }
+ post {
+ always {
+ junit(testResults: '**/surefire-reports/*.xml', allowEmptyResults: true)
+ junit(testResults: '**/failsafe-reports/*.xml', allowEmptyResults: true)
+ }
+ }
+ }
+
+ stage('Deploy') {
+ when {
Review comment:
```
when { branch 'master' }
```
is a shortcut for such expression
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571486569
##########
File path: Jenkinsfile
##########
@@ -87,36 +87,37 @@ pipeline {
sh 'mvn -U -B -e clean install -DskipTests -T1C'
}
}
-
Review comment:
I beg to differ : with sequential stages the build takes over 7hours
with parallel stages
- stable tests take about 5h30
- unstable tests take 2h10
build failure in 52 cannot be attributed to running parallel stages : the test which failed is a webmin integration test which is the last module to run in the stable tests. By then the unstable test stage had been completed for over 3hours...
I'm pretty sure the issue is with the Linshare singleton wait strategy: it looks like some containers start before containers they depend on are fully started
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-774965808
> What are the build time statistic after these changes?
all parallelization optimizations have been disabled the last the [last successful build](https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/51/pipeline) took 7 hours and 2 minutes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-774960870
I rebased on master it's building now
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] mbaechler commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
mbaechler commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-775745608
Ok so the PR is green (sometimes) and the history is clean, would people agree I merge it on master ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil edited a comment on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil edited a comment on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-774474432
We finally got a green run : https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/51/pipeline
to get there we had to drop all attempts at parallelisation: no surefire-forks, no running jobs in parallel and be lucky not to be hit by the [copy on write issue](https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/49/tests) (see the detailed logs for org.apache.james.smtp.SmtpRandomStoringTest and you will find the signature null pointer exception
```
15:53:44.340 [ERROR] o.a.j.t.m.d.MailDispatcher - Error While storing mail.
java.lang.NullPointerException: null
at org.apache.james.server.core.MimeMessageInputStream.<init>(MimeMessageInputStream.java:63)
```
or by [timeouts](https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/50/tests)
build 52 where I attempted to run test suites in parallel again failed with a linshare container initialization failure which had been [encountered before](https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/41/tests):
```
java.lang.ExceptionInInitializerError
at org.apache.james.webadmin.integration.memory.vault.MemoryLinshareBlobExportMechanismIntegrationTest.<clinit>(MemoryLinshareBlobExportMechanismIntegrationTest.java:34)
Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed
at org.apache.james.webadmin.integration.memory.vault.MemoryLinshareBlobExportMechanismIntegrationTest.<clinit>(MemoryLinshareBlobExportMechanismIntegrationTest.java:34)
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
at org.apache.james.webadmin.integration.memory.vault.MemoryLinshareBlobExportMechanismIntegrationTest.<clinit>(MemoryLinshareBlobExportMechanismIntegrationTest.java:34)
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
```
I started archiving the test-run.log on failure and logging as much test container related stuff as possible to have more details on such failures: it looks like the container fails to coordinate internally or maybe is killed during initialization by a timeout that is a bit too optimistic for the CI plateform ... (the database receives a fast-shutdown command while it is apply migrations and the backend which is starting fails to connect)
```
03:59:52.056 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> COMMIT
03:59:52.105 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.155 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.222 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.271 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.345 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.397 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.455 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> CREATE VIEW
03:59:52.456 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database>
03:59:52.462 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> LOG: received fast shutdown request
03:59:52.462 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> waiting for server to shut down...LOG: aborting any active transactions
03:59:52.462 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> .LOG: autovacuum launcher shutting down
03:59:52.466 [DEBUG] org.apache.james.linshare.Linshare - <linshare-database> LOG: shutting down
03:59:52.706 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> [WARN]:localhost-startStop-1:20210206.035952:org.hibernate.orm.cache:createCache:HHH90001006: Missing cache[default-update-timestamps-region] was created on-the-fly. The created cache will use a provider-specific default configuration: make sure you defined one. You can disable this warning by setting 'hibernate.cache.ehcache.missing_cache_strategy' to 'create'.
03:59:52.706 [DEBUG] o.t.c.output.WaitingConsumer - STDOUT: [WARN]:localhost-startStop-1:20210206.035952:org.hibernate.orm.cache:createCache:HHH90001006: Missing cache[default-update-timestamps-region] was created on-the-fly. The created cache will use a provider-specific default configuration: make sure you defined one. You can disable this warning by setting 'hibernate.cache.ehcache.missing_cache_strategy' to 'create'.efault-query-results-region] was created on-the-fly. The created cache will use a provider-specific default configuration: make sure you defined one. You can disable this warning by setting 'hibernate.cache.ehcache.missing_cache_strategy' to 'create'.
03:59:52.737 [DEBUG] o.t.c.output.WaitingConsumer - STDOUT: [WARN]:localhost-startStop-1:20210206.035952:org.hibernate.orm.cache:createCache:HHH90001006: Missing cache[default-query-results-region] was created on-the-fly. The created cache will use a provider-specific default configuration: make sure you defined one. You can disable this warning by setting 'hibernate.cache.ehcache.missing_cache_strategy' to 'create'..ehcache.missing_cache_strategy' to 'create'.
03:59:56.296 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> 06-Feb-2021 03:59:56.291 WARNING [localhost-startStop-1] org.postgresql.core.v3.ConnectionFactoryImpl.log ConnectException occurred while connecting to linshare_database:5432
03:59:56.296 [DEBUG] o.t.c.output.WaitingConsumer - STDERR: java.net.ConnectException: Connection refused (Connection refused)
03:59:56.296 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> java.net.ConnectException: Connection refused (Connection refused)
03:59:56.296 [DEBUG] o.t.c.output.WaitingConsumer - STDERR: at org.hibernate.tool.schema.internal.exec.ImprovedExtractionContextImpl.getJdbcConnection(ImprovedExtractionContextImpl.java:60)
03:59:56.296 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> at java.net.PlainSocketImpl.socketConnect(Native Method)
03:59:56.296 [DEBUG] org.apache.james.linshare.Linshare - <linshare-backend> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r529410639
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,175 @@
+#!groovy
+
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def AGENT_LABEL = env.AGENT_LABEL ?: 'ubuntu'
+def JDK_NAME = env.JDK_NAME ?: 'JDK 11 (latest)'
+
+pipeline {
+
+ agent {
+ node {
+ label AGENT_LABEL
+ }
+ }
+
+ environment {
+ // ... setup any environment variables ...
+ MVN_LOCAL_REPO_OPT = '-Dmaven.repo.local=.repository'
+ MVN_TEST_FAIL_IGNORE = '-Dmaven.test.failure.ignore=true'
+ CI=true
+ }
+
+ tools {
+ // ... tell Jenkins what java version, maven version or other tools are required ...
+ maven 'maven_3_latest'
+ jdk JDK_NAME
+ }
+
+ options {
+ // Configure an overall timeout for the build of 4 hours.
+ timeout(time: 4, unit: 'HOURS')
+ // When we have test-fails e.g. we don't need to run the remaining steps
+ skipStagesAfterUnstable()
+ buildDiscarder(
+ logRotator(artifactNumToKeepStr: '5', numToKeepStr: '10')
+ )
+ disableConcurrentBuilds()
+ }
+
+ stages {
+ stage('Initialization') {
+ steps {
+ echo 'Building branch ' + env.BRANCH_NAME
+ echo 'Using PATH ' + env.PATH
+ }
+ }
+
+ stage('Cleanup') {
+ steps {
+ echo 'Cleaning up the workspace'
+ deleteDir()
+ }
+ }
+
+ stage('Checkout') {
+ steps {
+ echo 'Checking out branch ' + env.BRANCH_NAME
+ checkout scm
+ }
+ }
+
+ stage('Build') {
+ steps {
+ echo 'Building'
+ sh 'mvn -U -B -e clean install -DskipTests'
+ }
+ }
+
+ stage('Tests') {
+ steps {
+ echo 'Running tests'
+ // all tests is very very long (10 hours on Apache Jenkins)
+ sh 'mvn -B -e test'
+ }
+ post {
+ always {
+ junit(testResults: '**/surefire-reports/*.xml', allowEmptyResults: true)
+ junit(testResults: '**/failsafe-reports/*.xml', allowEmptyResults: true)
+ }
+ }
+ }
+ /**
+ Unsure what the project's policy is regarding deployment
+ I left a sane sample in the file to facilitate further config
+ by a maintainer
Review comment:
fixed up
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] mbaechler commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
mbaechler commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-764801771
```
org.apache.james.rrt.cassandra.CassandraRecipientRewriteTableV6Test Time elapsed: 6.512 s <<< ERROR!
java.lang.ExceptionInInitializerError
Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed
Caused by: org.testcontainers.containers.ContainerFetchException: Can't get Docker image: RemoteDockerImage(imageName=<resolving>, imagePullPolicy=DefaultPullPolicy())
Caused by: com.github.dockerjava.api.exception.DockerClientException: Could not build image: unable to find image "sha256:8825a65748992b57919e0610c7b43bb2431cf8467295962d32696af687417745"
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571857751
##########
File path: mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapperTest.java
##########
@@ -59,10 +60,7 @@
import org.apache.james.mailbox.model.search.Wildcard;
import org.apache.james.mailbox.store.MailboxReactorUtils;
import org.assertj.core.api.SoftAssertions;
-import org.junit.jupiter.api.BeforeEach;
-import org.junit.jupiter.api.Disabled;
-import org.junit.jupiter.api.Nested;
-import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.*;
Review comment:
fixed up
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] mbaechler commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
mbaechler commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-762826374
```
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 6.007 s <<< FAILURE! - in org.apache.james.webadmin.routes.CassandraMappingsRoutesTest
[ERROR] org.apache.james.webadmin.routes.CassandraMappingsRoutesTest Time elapsed: 6.007 s <<< ERROR!
java.lang.ExceptionInInitializerError
Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed
Caused by: org.testcontainers.containers.ContainerFetchException: Can't get Docker image: RemoteDockerImage(imageName=<resolving>, imagePullPolicy=DefaultPullPolicy())
Caused by: com.github.dockerjava.api.exception.DockerClientException: Could not build image: unable to find image "sha256:2b4247eb252e9826687d0b7adb6d411963caad11295bc433e8fe3e8f5e714049"
```
Problem using docker
test this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-752232987
test this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] mbaechler commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
mbaechler commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571854219
##########
File path: backends-common/cassandra/src/test/java/org/apache/james/backends/cassandra/DockerCassandra.java
##########
@@ -118,11 +126,36 @@ private void grantPermissionToTestingUser(Cluster privilegedCluster, String keys
@SuppressWarnings("resource")
public DockerCassandra() {
- this("cassandra_3_11_3", AdditionalDockerFileStep.IDENTITY);
+ this("cassandra_3_11_3-"+ UUID.randomUUID().toString(), AdditionalDockerFileStep.IDENTITY);
Review comment:
I guess you don't feel the pain of trying to build james on a CI for months.
If this PR goes green, we'll definitely want to merge it and try to optimize later.
You then will be able to try optimizations on the Apache CI build infrastructure and share the result with us.
We don't really like having a 10hours build ... but not having any build at all if far worst.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-775796225
Congrats!
Now let's work together to get the build below 3h :-)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571676332
##########
File path: Jenkinsfile
##########
@@ -87,36 +87,37 @@ pipeline {
sh 'mvn -U -B -e clean install -DskipTests -T1C'
}
}
-
Review comment:
I ended up dropping the parallel stage for now this is an optimization and it is more unstable than unoptimized builds.
optimization can come in a second step
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
chibenwa commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571849215
##########
File path: mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/mail/CassandraMailboxMapperTest.java
##########
@@ -59,10 +60,7 @@
import org.apache.james.mailbox.model.search.Wildcard;
import org.apache.james.mailbox.store.MailboxReactorUtils;
import org.assertj.core.api.SoftAssertions;
-import org.junit.jupiter.api.BeforeEach;
-import org.junit.jupiter.api.Disabled;
-import org.junit.jupiter.api.Nested;
-import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.*;
Review comment:
Wildcard spotted ;-)
##########
File path: backends-common/cassandra/src/test/java/org/apache/james/backends/cassandra/DockerCassandra.java
##########
@@ -118,11 +126,36 @@ private void grantPermissionToTestingUser(Cluster privilegedCluster, String keys
@SuppressWarnings("resource")
public DockerCassandra() {
- this("cassandra_3_11_3", AdditionalDockerFileStep.IDENTITY);
+ this("cassandra_3_11_3-"+ UUID.randomUUID().toString(), AdditionalDockerFileStep.IDENTITY);
Review comment:
`UUID.randomUUID().toString()` this means that we might end up building a new image for each invocation.
An idea to allow multiple builds to hare the same docket daemon could be to realy on a build ID environment variable as a differenciator.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil edited a comment on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil edited a comment on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-734479249
I have cleaned up and rebased the Jenkinsfile which seems to work in #265
there was an issue with the git describe which was disabled in 8880117 , I think that using the `git describe --always` is better than disabling the plugin entirely :
```
--always
Show uniquely abbreviated commit object as fallback.
```
it preserves the current behavior while providing a working fallback for PRs which seem to be tested in detached head mode. combined with the already present `--dirty` it should never fail. The only downside is that jars generated for these builds will not have useful version information but they are not published either.
Another solution could be to use profiles to only enable the plugin on master / selected branches ...
Also #265 proved that CI only runs for PRs pushed by people who have write access to the repository :D
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil edited a comment on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil edited a comment on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-772602283
```
creatingConcurrentlyMailboxesWithSameParentShouldNotFail – org.apache.james.mailbox.cassandra.CassandraMailboxManagerTest$WithBatchSize
11:02:45.062 [ERROR] o.a.j.u.c.ConcurrentTestRunner - Error caught during concurrent testing (iteration 0, threadNumber 9)
com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency SERIAL (1 responses were required but only 0 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at com.datastax.shaded.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 27 common frames omitted
```
full log at https://builds.apache.org/blue/rest/organizations/jenkins/pipelines/james/pipelines/ApacheJames/branches/PR-264/runs/43/nodes/86/log/?start=0&download=true
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] mbaechler commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
mbaechler commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-763604971
```
12:26:17,264 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project james-server-data-cassandra: There was a timeout or other error in the fork -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project james-server-data-cassandra: There was a timeout or other error in the fork
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
chibenwa commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r529389335
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,175 @@
+#!groovy
+
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def AGENT_LABEL = env.AGENT_LABEL ?: 'ubuntu'
+def JDK_NAME = env.JDK_NAME ?: 'JDK 11 (latest)'
+
+pipeline {
+
+ agent {
+ node {
+ label AGENT_LABEL
+ }
+ }
+
+ environment {
+ // ... setup any environment variables ...
+ MVN_LOCAL_REPO_OPT = '-Dmaven.repo.local=.repository'
+ MVN_TEST_FAIL_IGNORE = '-Dmaven.test.failure.ignore=true'
+ CI=true
+ }
+
+ tools {
+ // ... tell Jenkins what java version, maven version or other tools are required ...
+ maven 'maven_3_latest'
+ jdk JDK_NAME
+ }
+
+ options {
+ // Configure an overall timeout for the build of 4 hours.
+ timeout(time: 4, unit: 'HOURS')
+ // When we have test-fails e.g. we don't need to run the remaining steps
+ skipStagesAfterUnstable()
+ buildDiscarder(
+ logRotator(artifactNumToKeepStr: '5', numToKeepStr: '10')
+ )
+ disableConcurrentBuilds()
+ }
+
+ stages {
+ stage('Initialization') {
+ steps {
+ echo 'Building branch ' + env.BRANCH_NAME
+ echo 'Using PATH ' + env.PATH
+ }
+ }
+
+ stage('Cleanup') {
+ steps {
+ echo 'Cleaning up the workspace'
+ deleteDir()
+ }
+ }
+
+ stage('Checkout') {
+ steps {
+ echo 'Checking out branch ' + env.BRANCH_NAME
+ checkout scm
+ }
+ }
+
+ stage('Build') {
+ steps {
+ echo 'Building'
+ sh 'mvn -U -B -e clean install -DskipTests'
+ }
+ }
+
+ stage('Tests') {
+ steps {
+ echo 'Running tests'
+ // all tests is very very long (10 hours on Apache Jenkins)
+ sh 'mvn -B -e test'
+ }
+ post {
+ always {
+ junit(testResults: '**/surefire-reports/*.xml', allowEmptyResults: true)
+ junit(testResults: '**/failsafe-reports/*.xml', allowEmptyResults: true)
+ }
+ }
+ }
+ /**
+ Unsure what the project's policy is regarding deployment
+ I left a sane sample in the file to facilitate further config
+ by a maintainer
Review comment:
I agree with deploying snapshots upon master pushes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571868651
##########
File path: backends-common/cassandra/src/test/java/org/apache/james/backends/cassandra/DockerCassandra.java
##########
@@ -118,11 +126,36 @@ private void grantPermissionToTestingUser(Cluster privilegedCluster, String keys
@SuppressWarnings("resource")
public DockerCassandra() {
- this("cassandra_3_11_3", AdditionalDockerFileStep.IDENTITY);
+ this("cassandra_3_11_3-"+ UUID.randomUUID().toString(), AdditionalDockerFileStep.IDENTITY);
Review comment:
I have a few ideas of things to try but indeed I would really like to stop maintaining this extremely long lived branch :(
There are 2 reasons to use unique names:
- it allows for concurrent runs on the same machine (which is a possible cause for [docker cache errors](https://github.com/apache/james-project/pull/264#issuecomment-762826374) )
- it makes it easier to diagnose other docker cache related failures. Our working hypothesis here is that there may be a garbage collection process on apache CI which removes images that are not currently in use. it might be a bit too aggressive and collect images that were just build right before we start them. We added detailed logging of docker events in 28d0de15 to diagnose this further
There are a few more ideas for optimization one of the things I intend to try is running in [dockerfile](https://www.jenkins.io/doc/book/pipeline/docker/#dockerfile) to be closer to how linagora's build works
I also want to merge the [COW fix](https://github.com/apache/james-project/pull/282) and revert "unstable" in a specific branch to see how it impacts both build time and stability
last but not least I hope to find some time in the future to investigate the root cause for the `OutOfMemoryError : Direct buffer memory` which forced me to drop surefire fork reuse. I did some preliminary research and it is most likely a memory leak of off-heap memory. The leak is "resolved" by dropping forks but likely indicates a deeper issue probably with incorrectly sized buffers. Once the leak is resolved we should be able to reactivate reuseFork whih would massively speed up the build (it suceeded once with reuseFork and took around 3h)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-772336138
After :
- using a unique name for the cassandra singleton image to avoid clashes between parallel maven runs overwriting each other's docker cache
- dropping fork reuse entirely to get rid of memory leaks inducing `OutOfMemoryError: Direct buffer memory`
last failure is :
```
23:31:06.935 [ERROR] ?.3.2] - Could not start container
java.lang.IllegalStateException: Container exited with code 1
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:469)
at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:325)
at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:323)
at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:311)
at org.apache.james.linshare.Linshare.start(Linshare.java:71)
at org.apache.james.linshare.LinshareSingleton.<clinit>(LinshareSingleton.java:27)
at org.apache.james.linshare.LinshareExtension.<clinit>(LinshareExtension.java:62)
at org.apache.james.modules.LinshareGuiceExtension.<init>(LinshareGuiceExtension.java:39)
at org.apache.james.LinshareBlobExportMechanismProvidingTest.<clinit>(LinshareBlobExportMechanismProvidingTest.java:29)
23:31:06.973 [ERROR] ?.3.2] - Log output from the failed container:
wait-for-it.sh: waiting 15 seconds for ${LS_SERVER_HOST}:${LS_SERVER_PORT}
wait-for-it.sh: timeout occurred after waiting 15 seconds for ${LS_SERVER_HOST}:${LS_SERVER_PORT}
INFO:init.sh: Checking all required env variables...
LS_SERVER_HOST : backend
LS_SERVER_PORT : 8080
LS_LDAP_URL : ldap://ldap:389
LS_USER_URL : https://user.linshare.local
LS_EXTERNAL_URL : https://user.linshare.local
LS_PASSWORD : admi...
LS_NO_REPLY_ADDRESS : no-reply@linshare.org
INFO:init.sh: All env variables checked
INFO:init.sh: Checking all optional env variables...
LS_LDAP_DN : cn=linshare,dc=linshare,dc=org
LS_LDAP_PW : lins...
LS_LDAP_BASE_DN : ou=People,dc=linshare,dc=org
LS_LDAP_NAME : ldap-local
LS_FORCE_INIT : 0
LS_DOMAIN_PATTERN_NAME : openldap-local
LS_DOMAIN_PATTERN_MODEL : 868400c0-c12e-456a-8c3c-19e985290586
LS_EXTRA_INIT_SCRIPT :
LS_JWT_PUB_KEY : /linagora/data/public.pem
LS_JWT_PUB_KEY_NAME :
INFO:init.sh: All env variables checked
23:21:22 WARNING : Current password is not valid.
23:21:22 INFO : Trying to authenticate with the new provided password
23:21:22 ERROR : Can't update the current password.
```
Since Linshare is yet another singleton which generates a docker image, I'll try assigning it a unique image name just in case.
see you tonight for the results (build takes >6h)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r550431463
##########
File path: Jenkinsfile
##########
@@ -87,36 +87,37 @@ pipeline {
sh 'mvn -U -B -e clean install -DskipTests -T1C'
}
}
-
- stage('Stable Tests') {
- steps {
- echo 'Running tests'
- // all tests run is very very long (10 hours on Apache Jenkins)
- sh 'mvn -B -e -fae test '
- }
- post {
- always {
- junit(testResults: '**/surefire-reports/*.xml', allowEmptyResults: true)
- junit(testResults: '**/failsafe-reports/*.xml', allowEmptyResults: true)
+ stage('Tests') {
+ parallel {
Review comment:
I'm not sure what you are referring to :
* [PR 264#1](https://ci-builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/1/tests) failures are on OOM and duplicate JMX bindings that I suspect are leftovers bindings of the OOM failures
* [PR 268#51](https://ci-builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-268/51/tests) failures are all related to a DockerCassandraSingleton startup failure which I can't explain at this stage : it occurred in both stable and unstable stages, both of which spawn different JVM and are thus supposed to build their own copy of the image on the first initialization of the singleton.
* [PR 268#47](https://ci-builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-268/47/tests) failed because of new tests added on master (MailboxChangeTest) I assume I was unlucky to get my PR merged with a new commit which had bad tests, this was fixed after a rebase (I assume the tests were fixed in between :) )
* [PR 268#46](https://ci-builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-268/46/tests) same as previous + a weird cassandra error (I should probably have tagged the CassandraSubscriptionManagerTest as unstable)
* [PR 268#44](https://ci-builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-268/44/tests) fails on a flaky test
*[PR 268#43](https://ci-builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-268/43/pipeline) failed on a surefire timeout the logs are inconclusive :/
Also of note:
* using parallel for the unstable tests reduces the build from 3h40 to 2h57
* tests should not bind to fixed ports but maybe there are constraints I'm not aware of ...
##########
File path: Jenkinsfile
##########
@@ -87,36 +87,37 @@ pipeline {
sh 'mvn -U -B -e clean install -DskipTests -T1C'
}
}
-
- stage('Stable Tests') {
- steps {
- echo 'Running tests'
- // all tests run is very very long (10 hours on Apache Jenkins)
- sh 'mvn -B -e -fae test '
- }
- post {
- always {
- junit(testResults: '**/surefire-reports/*.xml', allowEmptyResults: true)
- junit(testResults: '**/failsafe-reports/*.xml', allowEmptyResults: true)
+ stage('Tests') {
+ parallel {
Review comment:
I'm not sure what you are referring to :
* [PR 264#1](https://ci-builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-264/1/tests) failures are on OOM and duplicate JMX bindings that I suspect are leftovers bindings of the OOM failures
* [PR 268#51](https://ci-builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-268/51/tests) failures are all related to a DockerCassandraSingleton startup failure which I can't explain at this stage : it occurred in both stable and unstable stages, both of which spawn different JVM and are thus supposed to build their own copy of the image on the first initialization of the singleton.
* [PR 268#47](https://ci-builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-268/47/tests) failed because of new tests added on master (MailboxChangeTest) I assume I was unlucky to get my PR merged with a new commit which had bad tests, this was fixed after a rebase (I assume the tests were fixed in between :) )
* [PR 268#46](https://ci-builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-268/46/tests) same as previous + a weird cassandra error (I should probably have tagged the CassandraSubscriptionManagerTest as unstable)
* [PR 268#44](https://ci-builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-268/44/tests) fails on a flaky test
* [PR 268#43](https://ci-builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/detail/PR-268/43/pipeline) failed on a surefire timeout the logs are inconclusive :/
Also of note:
* using parallel for the unstable tests reduces the build from 3h40 to 2h57
* tests should not bind to fixed ports but maybe there are constraints I'm not aware of ...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] mbaechler commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
mbaechler commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571836726
##########
File path: backends-common/cassandra/src/test/java/org/apache/james/backends/cassandra/DockerCassandra.java
##########
@@ -118,12 +120,12 @@ private void grantPermissionToTestingUser(Cluster privilegedCluster, String keys
@SuppressWarnings("resource")
public DockerCassandra() {
- this("cassandra_3_11_3", AdditionalDockerFileStep.IDENTITY);
+ this("cassandra_3_11_3-"+ UUID.randomUUID().toString(), AdditionalDockerFileStep.IDENTITY);
}
public DockerCassandra(String imageName, AdditionalDockerFileStep additionalSteps) {
client = DockerClientFactory.instance().client();
- boolean doNotDeleteImageAfterUsage = false;
+ boolean doNotDeleteImageAfterUsage = true;
Review comment:
If you revert it, no
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571486075
##########
File path: Jenkinsfile
##########
@@ -47,7 +47,7 @@ pipeline {
options {
// Configure an overall timeout for the build of 4 hours.
- timeout(time: 4, unit: 'HOURS')
Review comment:
Build 51 (the only one green at this stage ) took over 7 hours to complete.
4 hours was appropriate with surefire forkReuse=true
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] Arsnael commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
Arsnael commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r550379533
##########
File path: Jenkinsfile
##########
@@ -87,36 +87,37 @@ pipeline {
sh 'mvn -U -B -e clean install -DskipTests -T1C'
}
}
-
- stage('Stable Tests') {
- steps {
- echo 'Running tests'
- // all tests run is very very long (10 hours on Apache Jenkins)
- sh 'mvn -B -e -fae test '
- }
- post {
- always {
- junit(testResults: '**/surefire-reports/*.xml', allowEmptyResults: true)
- junit(testResults: '**/failsafe-reports/*.xml', allowEmptyResults: true)
+ stage('Tests') {
+ parallel {
Review comment:
I would be careful with parallel, that might be why I saw some server already exist exception in some errors in the latest build
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] mbaechler commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
mbaechler commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-764462588
```
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 6.468 s <<< FAILURE! - in org.apache.james.mailbox.cassandra.mail.CassandraMailboxCounterDAOTest
org.apache.james.mailbox.cassandra.mail.CassandraMailboxCounterDAOTest Time elapsed: 6.468 s <<< ERROR!
java.lang.ExceptionInInitializerError
Caused by: org.testcontainers.containers.ContainerLaunchException: Container startup failed
Caused by: org.testcontainers.containers.ContainerFetchException: Can't get Docker image: RemoteDockerImage(imageName=<resolving>, imagePullPolicy=DefaultPullPolicy())
Caused by: com.github.dockerjava.api.exception.DockerClientException: Could not build image: unable to find image "sha256:8c0edf3bbb37eda4019a560c1a68292c991a9e493f74b47266f64d2caf19cd85"
```
Image caching is probably the root of our problem
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-734479249
I have cleaned up and rebased the Jenkinsfile which seems to work in #265
there was an issue with the git describe which was disabled in 8880117 , I think that using the `git describe --always` is better than disabling the plugin entirely :
```
--always
Show uniquely abbreviated commit object as fallback.
```
it preserves the current behavior while providing a working fallback for PRs which seem to be tested in detached head mode. combined with the already present `--dirty` it should never fail. The only downside is that jars generated for these builds will not have useful version information but they are not published either.
Another solution could be to use profiles to only enable the plugin on master / selected branches ...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-774981905
>I see that JMAP draft integration tests takes quit a bunch of time. Do we really need them to run on top of both cassandra-guice and distributed james? (We could reclaim 1h of build time very easily there).
I think I read in a discussion that JMAP draft was going to be dropped "soon" anyways which is why I didn't spend any time splitting this off.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571487218
##########
File path: backends-common/cassandra/src/test/java/org/apache/james/backends/cassandra/DockerCassandra.java
##########
@@ -118,12 +120,12 @@ private void grantPermissionToTestingUser(Cluster privilegedCluster, String keys
@SuppressWarnings("resource")
public DockerCassandra() {
- this("cassandra_3_11_3", AdditionalDockerFileStep.IDENTITY);
+ this("cassandra_3_11_3-"+ UUID.randomUUID().toString(), AdditionalDockerFileStep.IDENTITY);
}
public DockerCassandra(String imageName, AdditionalDockerFileStep additionalSteps) {
client = DockerClientFactory.instance().client();
- boolean doNotDeleteImageAfterUsage = false;
+ boolean doNotDeleteImageAfterUsage = true;
Review comment:
this chance was reverted afterwards in a vaguely related commit. I have rewritten the history to clean up my mess.
Should I still rename the variable?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-752232783
The CI builds are now reasonnably successful see https://builds.apache.org/blue/organizations/jenkins/james%2FApacheJames/activity
The PR history is clean and it seems an admin enabled build for all PRs
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-772602283
```
11:02:45.062 [ERROR] o.a.j.u.c.ConcurrentTestRunner - Error caught during concurrent testing (iteration 0, threadNumber 9)
com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency SERIAL (1 responses were required but only 0 replica responded)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:91)
at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:66)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297)
at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268)
at com.datastax.shaded.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88)
... 27 common frames omitted
```
full log at https://builds.apache.org/blue/rest/organizations/jenkins/pipelines/james/pipelines/ApacheJames/branches/PR-264/runs/43/nodes/86/log/?start=0&download=true
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571838905
##########
File path: backends-common/cassandra/src/test/java/org/apache/james/backends/cassandra/DockerCassandra.java
##########
@@ -118,12 +120,12 @@ private void grantPermissionToTestingUser(Cluster privilegedCluster, String keys
@SuppressWarnings("resource")
public DockerCassandra() {
- this("cassandra_3_11_3", AdditionalDockerFileStep.IDENTITY);
+ this("cassandra_3_11_3-"+ UUID.randomUUID().toString(), AdditionalDockerFileStep.IDENTITY);
}
public DockerCassandra(String imageName, AdditionalDockerFileStep additionalSteps) {
client = DockerClientFactory.instance().client();
- boolean doNotDeleteImageAfterUsage = false;
+ boolean doNotDeleteImageAfterUsage = true;
Review comment:
changes to this variable were reverted, I'll resove this :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] mbaechler commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
mbaechler commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r550309315
##########
File path: mailbox/cassandra/src/test/java/org/apache/james/mailbox/cassandra/CassandraMailboxManagerConsistencyTest.java
##########
@@ -34,10 +35,7 @@
import org.apache.james.mailbox.model.search.Wildcard;
import org.apache.james.mailbox.store.PreDeletionHooks;
import org.assertj.core.api.SoftAssertions;
-import org.junit.jupiter.api.BeforeEach;
-import org.junit.jupiter.api.Disabled;
-import org.junit.jupiter.api.Nested;
-import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.*;
Review comment:
it's supposed to break checkstyle, we don't allow wildcard import
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,167 @@
+#!groovy
+
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def AGENT_LABEL = env.AGENT_LABEL ?: 'ubuntu'
+def JDK_NAME = env.JDK_NAME ?: 'jdk_11_latest'
+
+pipeline {
+
+ agent {
+ node {
+ label AGENT_LABEL
+ }
+ }
+
+ environment {
+ // ... setup any environment variables ...
+ MVN_LOCAL_REPO_OPT = '-Dmaven.repo.local=.repository'
+ MVN_TEST_FAIL_IGNORE = '-Dmaven.test.failure.ignore=true'
+ CI=true
+ LC_CTYPE = 'en_US.UTF-8'
+ }
+
+ tools {
+ // ... tell Jenkins what java version, maven version or other tools are required ...
+ maven 'maven_3_latest'
+ jdk JDK_NAME
+ }
+
+ options {
+ // Configure an overall timeout for the build of 4 hours.
+ timeout(time: 4, unit: 'HOURS')
+ // When we have test-fails e.g. we don't need to run the remaining steps
+ skipStagesAfterUnstable()
+ buildDiscarder(
+ logRotator(artifactNumToKeepStr: '5', numToKeepStr: '10')
+ )
+ disableConcurrentBuilds()
+ }
+
+ stages {
+ stage('Initialization') {
+ steps {
+ echo 'Building branch ' + env.BRANCH_NAME
+ echo 'Using PATH ' + env.PATH
+ }
+ }
+
+ stage('Cleanup') {
+ steps {
+ echo 'Cleaning up the workspace'
+ deleteDir()
+ }
+ }
+
+ stage('Checkout') {
+ steps {
+ echo 'Checking out branch ' + env.BRANCH_NAME
+ checkout scm
+ }
+ }
+
+ stage('Build') {
+ steps {
+ echo 'Building'
+ sh 'mvn -U -B -e clean install -DskipTests'
+ }
+ }
+
+ stage('Tests') {
+ steps {
+ echo 'Running tests'
+ // all tests is very very long (10 hours on Apache Jenkins)
Review comment:
really ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571485916
##########
File path: backends-common/elasticsearch/pom.xml
##########
@@ -112,7 +112,7 @@
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
- <reuseForks>true</reuseForks>
Review comment:
fixedup
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] mbaechler commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
mbaechler commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-764462588
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-732825265
I checked and my intuition was correct, the builds don't trigger because I am (rightly ;) ) considered an untrusted source
```
Getting remote pull request #264...
Checking pull request #264
Checking pull request #264
(not from a trusted source)
‘Jenkinsfile’ not found
Does not meet criteria
1 pull requests were processed
```
(source : https://builds.apache.org/job/james/job/ApacheJames/indexing/events)
I'm pretty sure the `Jenkinsfile` is there so I assume it gets filtered away because I'm not to be trusted. This means the `Jenkinsfile` has not even been linted for now. A maintainer will have to recreate/push the branch in its own repo or possibly in this repo and reopen a PR for it to be picked up
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] mbaechler commented on a change in pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
mbaechler commented on a change in pull request #264:
URL: https://github.com/apache/james-project/pull/264#discussion_r571473120
##########
File path: Jenkinsfile
##########
@@ -87,36 +87,37 @@ pipeline {
sh 'mvn -U -B -e clean install -DskipTests -T1C'
}
}
-
Review comment:
we should remove this commit from the history
##########
File path: Jenkinsfile
##########
@@ -47,7 +47,7 @@ pipeline {
options {
// Configure an overall timeout for the build of 4 hours.
- timeout(time: 4, unit: 'HOURS')
Review comment:
we should drop this commit (if the current duration is less than 4 hours)
##########
File path: backends-common/elasticsearch/pom.xml
##########
@@ -112,7 +112,7 @@
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
- <reuseForks>true</reuseForks>
Review comment:
this commit lack JIRA number
##########
File path: backends-common/cassandra/src/test/java/org/apache/james/backends/cassandra/DockerCassandra.java
##########
@@ -118,12 +120,12 @@ private void grantPermissionToTestingUser(Cluster privilegedCluster, String keys
@SuppressWarnings("resource")
public DockerCassandra() {
- this("cassandra_3_11_3", AdditionalDockerFileStep.IDENTITY);
+ this("cassandra_3_11_3-"+ UUID.randomUUID().toString(), AdditionalDockerFileStep.IDENTITY);
}
public DockerCassandra(String imageName, AdditionalDockerFileStep additionalSteps) {
client = DockerClientFactory.instance().client();
- boolean doNotDeleteImageAfterUsage = false;
+ boolean doNotDeleteImageAfterUsage = true;
Review comment:
we should rename the variable to `deleteImageAfterUsage`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] jeantil commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
jeantil commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-756223519
The latest crash is ... interesting : I have no idea what to do about it ...
```
[WARNING] Corrupted STDOUT by directly writing to native stream in forked JVM 8. See FAQ web page and the dump file /home/jenkins/workspace/james_ApacheJames_PR-264/server/protocols/webadmin/webadmin-mailbox/target/surefire-reports/2021-01-07T12-25-01_355-jvmRun8.dumpstream
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org
[GitHub] [james-project] chibenwa commented on pull request #264: JAMES-3225 configures CI for builds.apache.org
Posted by GitBox <gi...@apache.org>.
chibenwa commented on pull request #264:
URL: https://github.com/apache/james-project/pull/264#issuecomment-752384991
\o/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@james.apache.org
For additional commands, e-mail: notifications-help@james.apache.org