You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/08/27 18:04:25 UTC
[GitHub] [kafka] mumrah opened a new pull request #9226: [DO NOT MERGE] Jenkinsfile testing
mumrah opened a new pull request #9226:
URL: https://github.com/apache/kafka/pull/9226
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mumrah commented on a change in pull request #9226: KAFKA-10444: Jenkinsfile testing
Posted by GitBox <gi...@apache.org>.
mumrah commented on a change in pull request #9226:
URL: https://github.com/apache/kafka/pull/9226#discussion_r480452574
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,200 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def setupGradle() {
+ // Delete gradle cache to workaround cache corruption bugs, see KAFKA-3167
+ dir('.gradle') {
+ deleteDir()
+ }
+ sh './gradlew -version'
+}
+
+def doValidation() {
+ try {
+ sh '''
+ ./gradlew -PscalaVersion=$SCALA_VERSION clean compileJava compileScala compileTestJava compileTestScala \
+ spotlessScalaCheck checkstyleMain checkstyleTest spotbugsMain rat \
+ --profile --no-daemon --continue -PxmlSpotBugsReport=true \"$@\"
+ '''
+ } catch(err) {
+ error('Validation checks failed, aborting this build')
+ }
+}
+
+def doTest() {
+ try {
+ sh '''
+ ./gradlew -PscalaVersion=$SCALA_VERSION unitTest integrationTest \
+ --profile --no-daemon --continue -PtestLoggingEvents=started,passed,skipped,failed "$@"
+ '''
+ } catch(err) {
+ echo 'Some tests failed, marking this build UNSTABLE'
+ currentBuild.result = 'UNSTABLE'
Review comment:
So, this seems to have caused us to lose which build had which results in the summary.
Before we had:
Now we have:
I'll try moving the `junit` call inside the actual stage and see if that helps
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mumrah commented on pull request #9226: KAFKA-10444: Jenkinsfile testing
Posted by GitBox <gi...@apache.org>.
mumrah commented on pull request #9226:
URL: https://github.com/apache/kafka/pull/9226#issuecomment-684175203
Example failed build with a compile error: https://ci-builds.apache.org/job/Kafka/job/kafka-pr/view/change-requests/job/PR-9226/67
One with a rat failure: https://ci-builds.apache.org/job/Kafka/job/kafka-pr/view/change-requests/job/PR-9226/66/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on a change in pull request #9226: KAFKA-10444: Jenkinsfile testing
Posted by GitBox <gi...@apache.org>.
ijuma commented on a change in pull request #9226:
URL: https://github.com/apache/kafka/pull/9226#discussion_r480364696
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,200 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def setupGradle() {
+ // Delete gradle cache to workaround cache corruption bugs, see KAFKA-3167
+ dir('.gradle') {
+ deleteDir()
+ }
+ sh './gradlew -version'
Review comment:
Is this informational so that we know the gradle version? Or is there more to it?
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,200 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def setupGradle() {
+ // Delete gradle cache to workaround cache corruption bugs, see KAFKA-3167
+ dir('.gradle') {
+ deleteDir()
+ }
+ sh './gradlew -version'
+}
+
+def doValidation() {
+ try {
+ sh '''
+ ./gradlew -PscalaVersion=$SCALA_VERSION clean compileJava compileScala compileTestJava compileTestScala \
+ spotlessScalaCheck checkstyleMain checkstyleTest spotbugsMain rat \
+ --profile --no-daemon --continue -PxmlSpotBugsReport=true \"$@\"
+ '''
+ } catch(err) {
+ error('Validation checks failed, aborting this build')
Review comment:
Should we include `err` in the output somehow? Or is that not needed?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mumrah commented on a change in pull request #9226: KAFKA-10444: Jenkinsfile testing
Posted by GitBox <gi...@apache.org>.
mumrah commented on a change in pull request #9226:
URL: https://github.com/apache/kafka/pull/9226#discussion_r480391239
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,200 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def setupGradle() {
+ // Delete gradle cache to workaround cache corruption bugs, see KAFKA-3167
+ dir('.gradle') {
+ deleteDir()
+ }
+ sh './gradlew -version'
+}
+
+def doValidation() {
+ try {
+ sh '''
+ ./gradlew -PscalaVersion=$SCALA_VERSION clean compileJava compileScala compileTestJava compileTestScala \
+ spotlessScalaCheck checkstyleMain checkstyleTest spotbugsMain rat \
+ --profile --no-daemon --continue -PxmlSpotBugsReport=true \"$@\"
+ '''
+ } catch(err) {
+ error('Validation checks failed, aborting this build')
Review comment:
I think the err here will be like an ExectuionException -- i'm going to try something different than this try/catch actually
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mumrah commented on a change in pull request #9226: KAFKA-10444: Jenkinsfile testing
Posted by GitBox <gi...@apache.org>.
mumrah commented on a change in pull request #9226:
URL: https://github.com/apache/kafka/pull/9226#discussion_r480388916
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,200 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def setupGradle() {
+ // Delete gradle cache to workaround cache corruption bugs, see KAFKA-3167
+ dir('.gradle') {
+ deleteDir()
+ }
+ sh './gradlew -version'
Review comment:
Yes informational to see the gradle version and jdk that's running
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mumrah commented on a change in pull request #9226: KAFKA-10444: Jenkinsfile testing
Posted by GitBox <gi...@apache.org>.
mumrah commented on a change in pull request #9226:
URL: https://github.com/apache/kafka/pull/9226#discussion_r481202138
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,200 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def setupGradle() {
+ // Delete gradle cache to workaround cache corruption bugs, see KAFKA-3167
+ dir('.gradle') {
+ deleteDir()
+ }
+ sh './gradlew -version'
+}
+
+def doValidation() {
+ try {
+ sh '''
+ ./gradlew -PscalaVersion=$SCALA_VERSION clean compileJava compileScala compileTestJava compileTestScala \
+ spotlessScalaCheck checkstyleMain checkstyleTest spotbugsMain rat \
+ --profile --no-daemon --continue -PxmlSpotBugsReport=true \"$@\"
+ '''
+ } catch(err) {
+ error('Validation checks failed, aborting this build')
+ }
+}
+
+def doTest() {
+ try {
+ sh '''
+ ./gradlew -PscalaVersion=$SCALA_VERSION unitTest integrationTest \
+ --profile --no-daemon --continue -PtestLoggingEvents=started,passed,skipped,failed "$@"
+ '''
+ } catch(err) {
+ echo 'Some tests failed, marking this build UNSTABLE'
+ currentBuild.result = 'UNSTABLE'
Review comment:
Moving the junit into the stage seemed to work
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] lbradstreet commented on a change in pull request #9226: KAFKA-10444: Jenkinsfile testing
Posted by GitBox <gi...@apache.org>.
lbradstreet commented on a change in pull request #9226:
URL: https://github.com/apache/kafka/pull/9226#discussion_r480392178
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,200 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def setupGradle() {
+ // Delete gradle cache to workaround cache corruption bugs, see KAFKA-3167
+ dir('.gradle') {
+ deleteDir()
+ }
+ sh './gradlew -version'
+}
+
+def doValidation() {
+ try {
+ sh '''
+ ./gradlew -PscalaVersion=$SCALA_VERSION clean compileJava compileScala compileTestJava compileTestScala \
+ spotlessScalaCheck checkstyleMain checkstyleTest spotbugsMain rat \
+ --profile --no-daemon --continue -PxmlSpotBugsReport=true \"$@\"
+ '''
+ } catch(err) {
+ error('Validation checks failed, aborting this build')
+ }
+}
+
+def doTest() {
+ try {
+ sh '''
+ ./gradlew -PscalaVersion=$SCALA_VERSION unitTest integrationTest \
+ --profile --no-daemon --continue -PtestLoggingEvents=started,passed,skipped,failed "$@"
+ '''
+ } catch(err) {
+ echo 'Some tests failed, marking this build UNSTABLE'
+ currentBuild.result = 'UNSTABLE'
Review comment:
I think this may mark a build as UNSTABLE if one of gradle's executors exited. We're trying to achieve the property that the build won't fail completely if one of the tests fails, right? If so, I think the best way is to supply `-PignoreFailures=true` at https://github.com/apache/kafka/pull/9226/files#diff-58231b16fdee45a03a4ee3cf94a9f2c3R44, and add:
```
task integrationTest(type: Test, dependsOn: compileJava) {
ignoreFailures = userIgnoreFailures
```
```
task unitTest(type: Test, dependsOn: compileJava) {
ignoreFailures = userIgnoreFailures
```
```
ext {
userIgnoreFailures = project.hasProperty('ignoreFailures') ? ignoreFailures : false
```
to build.gradle.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] lbradstreet commented on a change in pull request #9226: KAFKA-10444: Jenkinsfile testing
Posted by GitBox <gi...@apache.org>.
lbradstreet commented on a change in pull request #9226:
URL: https://github.com/apache/kafka/pull/9226#discussion_r480404009
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,200 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def setupGradle() {
+ // Delete gradle cache to workaround cache corruption bugs, see KAFKA-3167
+ dir('.gradle') {
+ deleteDir()
+ }
+ sh './gradlew -version'
+}
+
+def doValidation() {
+ try {
+ sh '''
+ ./gradlew -PscalaVersion=$SCALA_VERSION clean compileJava compileScala compileTestJava compileTestScala \
+ spotlessScalaCheck checkstyleMain checkstyleTest spotbugsMain rat \
+ --profile --no-daemon --continue -PxmlSpotBugsReport=true \"$@\"
+ '''
+ } catch(err) {
+ error('Validation checks failed, aborting this build')
+ }
+}
+
+def doTest() {
+ try {
+ sh '''
+ ./gradlew -PscalaVersion=$SCALA_VERSION unitTest integrationTest \
+ --profile --no-daemon --continue -PtestLoggingEvents=started,passed,skipped,failed "$@"
+ '''
+ } catch(err) {
+ echo 'Some tests failed, marking this build UNSTABLE'
+ currentBuild.result = 'UNSTABLE'
Review comment:
Correct, it'll be picked up when the test output files are picked up.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mumrah commented on pull request #9226: KAFKA-10444: Jenkinsfile testing
Posted by GitBox <gi...@apache.org>.
mumrah commented on pull request #9226:
URL: https://github.com/apache/kafka/pull/9226#issuecomment-684931052
Closing in favor of #9238 for a clean slate
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mumrah commented on a change in pull request #9226: KAFKA-10444: Jenkinsfile testing
Posted by GitBox <gi...@apache.org>.
mumrah commented on a change in pull request #9226:
URL: https://github.com/apache/kafka/pull/9226#discussion_r480393840
##########
File path: Jenkinsfile
##########
@@ -0,0 +1,200 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+def setupGradle() {
+ // Delete gradle cache to workaround cache corruption bugs, see KAFKA-3167
+ dir('.gradle') {
+ deleteDir()
+ }
+ sh './gradlew -version'
+}
+
+def doValidation() {
+ try {
+ sh '''
+ ./gradlew -PscalaVersion=$SCALA_VERSION clean compileJava compileScala compileTestJava compileTestScala \
+ spotlessScalaCheck checkstyleMain checkstyleTest spotbugsMain rat \
+ --profile --no-daemon --continue -PxmlSpotBugsReport=true \"$@\"
+ '''
+ } catch(err) {
+ error('Validation checks failed, aborting this build')
+ }
+}
+
+def doTest() {
+ try {
+ sh '''
+ ./gradlew -PscalaVersion=$SCALA_VERSION unitTest integrationTest \
+ --profile --no-daemon --continue -PtestLoggingEvents=started,passed,skipped,failed "$@"
+ '''
+ } catch(err) {
+ echo 'Some tests failed, marking this build UNSTABLE'
+ currentBuild.result = 'UNSTABLE'
Review comment:
I see, so in this case we'll let the `junit` directive set the build to unstable?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] mumrah closed pull request #9226: KAFKA-10444: Jenkinsfile testing
Posted by GitBox <gi...@apache.org>.
mumrah closed pull request #9226:
URL: https://github.com/apache/kafka/pull/9226
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org