You are viewing a plain text version of this content. The canonical link for it is here.
Posted to pr@cassandra.apache.org by "michaelsembwever (via GitHub)" <gi...@apache.org> on 2025/06/25 06:34:29 UTC

[PR] Improve jenkinsfile parse performance [cassandra]

michaelsembwever opened a new pull request, #4211:
URL: https://github.com/apache/cassandra/pull/4211

   Only read the supported and default jdks once.  Also add debug when starting the pipeline on the parameters that will be used for building and testing.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] Improve jenkinsfile parse performance [cassandra]

Posted by "michaelsembwever (via GitHub)" <gi...@apache.org>.
michaelsembwever commented on PR #4211:
URL: https://github.com/apache/cassandra/pull/4211#issuecomment-3003700129

   still needs a little work (throwing a groovy error).  but basic idea to the improvement is there.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] Improve jenkinsfile parse performance [cassandra]

Posted by "michaelsembwever (via GitHub)" <gi...@apache.org>.
michaelsembwever commented on code in PR #4211:
URL: https://github.com/apache/cassandra/pull/4211#discussion_r2557830296


##########
.jenkins/Jenkinsfile:
##########
@@ -412,12 +416,15 @@ def test(command, cell) {
           dir("build") {
             sh """
                 mkdir -p test/output/${cell.step}
-                find test/output -type f -name TEST*.xml -execdir mkdir -p jdk_${cell.jdk}/${cell.arch} ';' -execdir mv {} jdk_${cell.jdk}/${cell.arch}/{} ';'
+                find test/output -type f -name "TEST*.xml" -execdir mkdir -p jdk_${cell.jdk}/${cell.arch} ';' -execdir mv {} jdk_${cell.jdk}/${cell.arch}/{} ';'
                 find test/output -name cqlshlib.xml -execdir mv cqlshlib.xml ${cell.step}/cqlshlib${cell_suffix}.xml ';'
                 find test/output -name nosetests.xml -execdir mv nosetests.xml ${cell.step}/nosetests${cell_suffix}.xml ';'
               """
             junit testResults: "test/**/TEST-*.xml,test/**/cqlshlib*.xml,test/**/nosetests*.xml", testDataPublishers: [[$class: 'StabilityTestDataPublisher']]
-            sh "find test/output -type f -name *.xml -exec sh -c 'xz -f {} &' ';' ; wait ; find test/output -type f -name *.xml.xz | wc -l"
+            sh """
+                find test/output -type f -name "*.xml" -print0 | xargs -0 -r -n1 -P"\$(nproc)" xz -f
+                echo "test result files compressed"; find test/output -type f -name "*.xml.xz" | wc -l
+              """

Review Comment:
   this is the bugfix for missing test results in the summary file.
   
   `wait` wasn't waiting for backgrounded child tasks inside the `sh -c …`
   it was simpler to replace it with `xargs -P …`
   
   ref: https://the-asf.slack.com/archives/CK23JSY2K/p1763714160975359



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org


Re: [PR] Improve jenkinsfile parse performance [cassandra]

Posted by "michaelsembwever (via GitHub)" <gi...@apache.org>.
michaelsembwever commented on PR #4211:
URL: https://github.com/apache/cassandra/pull/4211#issuecomment-3572862396

   CI here: http://ci-cassandra.infra.datastax.com/job/cassandra-5.0/50/ 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: pr-unsubscribe@cassandra.apache.org
For additional commands, e-mail: pr-help@cassandra.apache.org