You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/05/10 20:58:39 UTC

[GitHub] [beam] kileys opened a new pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

kileys opened a new pull request #14774:
URL: https://github.com/apache/beam/pull/14774


   Create and update Grafana dashboards
   
   **Please** add a meaningful description for your change here
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
    - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
   
   `ValidatesRunner` compliance status (on master branch)
   --------------------------------------------------------
   
   <table>
     <thead>
       <tr>
         <th>Lang</th>
         <th>ULR</th>
         <th>Dataflow</th>
         <th>Flink</th>
         <th>Samza</th>
         <th>Spark</th>
         <th>Twister2</th>
       </tr>
     </thead>
     <tbody>
       <tr>
         <td>Go</td>
         <td>---</td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
           </a>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>---</td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>---</td>
       </tr>
       <tr>
         <td>Java</td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/badge/icon?subject=V1+Streaming">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon?subject=V1+Java+11">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/badge/icon?subject=V2+Streaming">
           </a><br>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon?subject=Java+8">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon?subject=Java+11">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon?subject=Portable">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon?subject=Portable+Streaming">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon?subject=Portable">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon?subject=Structured+Streaming">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/badge/icon">
           </a>
         </td>
       </tr>
       <tr>
         <td>Python</td>
         <td>---</td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon?subject=ValCont">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon?subject=Portable">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>---</td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>---</td>
       </tr>
       <tr>
         <td>XLang</td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>---</td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>---</td>
       </tr>
     </tbody>
   </table>
   
   Examples testing status on various runners
   --------------------------------------------------------
   
   <table>
     <thead>
       <tr>
         <th>Lang</th>
         <th>ULR</th>
         <th>Dataflow</th>
         <th>Flink</th>
         <th>Samza</th>
         <th>Spark</th>
         <th>Twister2</th>
       </tr>
     </thead>
     <tbody>
       <tr>
         <td>Go</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
       </tr>
       <tr>
         <td>Java</td>
         <td>---</td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/badge/icon?subject=V1">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/badge/icon?subject=V1+Java11">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
           </a><br>
         </td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
       </tr>
       <tr>
         <td>Python</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
       </tr>
       <tr>
         <td>XLang</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
       </tr>
     </tbody>
   </table>
   
   Post-Commit SDK/Transform Integration Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   <table>
     <thead>
       <tr>
         <th>Go</th>
         <th>Java</th>
         <th>Python</th>
       </tr>
     </thead>
     <tbody>
       <tr>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon?subject=3.6">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon?subject=3.7">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/badge/icon?subject=3.8">
           </a>
         </td>
       </tr>
     </tbody>
   </table>
   
   Pre-Commit Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   
   <table>
     <thead>
       <tr>
         <th>---</th>
         <th>Java</th>
         <th>Python</th>
         <th>Go</th>
         <th>Website</th>
         <th>Whitespace</th>
         <th>Typescript</th>
       </tr>
     </thead>
     <tbody>
       <tr>
         <td>Non-portable</td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon">
           </a><br>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon?subject=Tests">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon?subject=Lint">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/badge/icon?subject=Docker">
           </a><br>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/badge/icon?subject=Docs">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/badge/icon">
           </a>
         </td>
       </tr>
       <tr>
         <td>Portable</td>
         <td>---</td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>
           <a href="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/">
             <img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/badge/icon">
           </a>
         </td>
         <td>---</td>
         <td>---</td>
         <td>---</td>
       </tr>
     </tbody>
   </table>
   
   See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
   
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [BEAM-12334] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-843665043


   > Looks like this broke `beam_PreCommit_SQL_Java11`: [BEAM-12364](https://issues.apache.org/jira/browse/BEAM-12364)
   
   PR for the fix is here https://github.com/apache/beam/pull/14838


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] amaliujia commented on a change in pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
amaliujia commented on a change in pull request #14774:
URL: https://github.com/apache/beam/pull/14774#discussion_r629741596



##########
File path: .test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow_V2.groovy
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+import CommonTestProperties.Runner
+import CommonTestProperties.SDK
+import CommonTestProperties.TriggeringContext
+import NexmarkBuilder as Nexmark
+import NoPhraseTriggeringPostCommitBuilder
+import PhraseTriggeringPostCommitBuilder
+import InfluxDBCredentialsHelper
+
+import static NexmarkDatabaseProperties.nexmarkBigQueryArgs
+import static NexmarkDatabaseProperties.nexmarkInfluxDBArgs
+
+def final JOB_SPECIFIC_OPTIONS = [
+  'influxTags' : '{"runnerVersion":"V2", "javaVersion":"8"}',
+  'exportSummaryToBigQuery' : false,
+  'region' : 'us-central1',
+  'suite' : 'STRESS',
+  'numWorkers' : 4,
+  'maxNumWorkers' : 4,
+  'autoscalingAlgorithm' : 'NONE',
+  'nexmarkParallel' : 16,
+  'enforceEncodability' : true,
+  'enforceImmutability' : true
+]
+
+def final JOB_SPECIFIC_SWITCHES = [
+  '-Pnexmark.runner.version="V2"'
+]
+
+// This job runs the suite of Nexmark tests against the Dataflow runner V2.
+NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_Dataflow_V2',
+    'Dataflow Runner V2 Nexmark Tests', this) {

Review comment:
       Ah I guess it should from `PhraseTriggeringPostCommitBuilder.postCommitJob` and use the second command. 
   
   Per SQL postcommit  job:
   
   ![Screen Shot 2021-05-10 at 4 15 29 PM](https://user-images.githubusercontent.com/1938382/117735795-12ed5600-b1ab-11eb-89df-92199d1acf27.png)
   
   
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-839133109


   Run Dataflow Runner V2 Nexmark Tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] ibzib commented on pull request #14774: [BEAM-12334] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
ibzib commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-843664238


   Looks like this broke `beam_PreCommit_SQL_Java11`: BEAM-12364


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-839260202






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-839986900


   Run Dataflow Runner V2 Nexmark Tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [BEAM-12334] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-842628701


   Run Java PreCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] amaliujia commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-839299198


   Run Seed Job


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] amaliujia commented on a change in pull request #14774: [BEAM-12334] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
amaliujia commented on a change in pull request #14774:
URL: https://github.com/apache/beam/pull/14774#discussion_r632066917



##########
File path: sdks/java/testing/nexmark/build.gradle
##########
@@ -102,6 +109,39 @@ if (shouldProvideSpark) {
   }
 }
 
+def getNexmarkArgs = {
+  def nexmarkArgsStr =  project.findProperty(nexmarkArgsProperty) ?: ""
+  def nexmarkArgsList = new ArrayList<String>()
+  Collections.addAll(nexmarkArgsList, nexmarkArgsStr.split())
+
+  if (isDataflowRunner) {
+    if (isDataflowRunnerV2) {
+      nexmarkArgsList.add("--experiments=beam_fn_api,use_unified_worker,use_runner_v2,shuffle_mode=appliance")

Review comment:
       Same for Java8 setup




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-837472959


   Run Dataflow Runner V2 Nexmark Tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] amaliujia commented on a change in pull request #14774: [BEAM-12334] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
amaliujia commented on a change in pull request #14774:
URL: https://github.com/apache/beam/pull/14774#discussion_r632177582



##########
File path: sdks/java/testing/nexmark/build.gradle
##########
@@ -102,6 +109,39 @@ if (shouldProvideSpark) {
   }
 }
 
+def getNexmarkArgs = {
+  def nexmarkArgsStr =  project.findProperty(nexmarkArgsProperty) ?: ""
+  def nexmarkArgsList = new ArrayList<String>()
+  Collections.addAll(nexmarkArgsList, nexmarkArgsStr.split())
+
+  if (isDataflowRunner) {
+    if (isDataflowRunnerV2) {
+      nexmarkArgsList.add("--experiments=beam_fn_api,use_unified_worker,use_runner_v2,shuffle_mode=appliance")
+      def sdkContainerImage = project.findProperty('sdkContainerImage') ?: project(":runners:google-cloud-dataflow-java").dockerImageName
+      nexmarkArgsList.add("--sdkContainerImage=${sdkContainerImage}")

Review comment:
       Flags I have been using with runner v2 is `--experiments=runner_harness_container_image=${image}` and `--worker_harness_container_image=${image}`




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] amaliujia commented on a change in pull request #14774: [BEAM-12334] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
amaliujia commented on a change in pull request #14774:
URL: https://github.com/apache/beam/pull/14774#discussion_r632061480



##########
File path: .test-infra/jenkins/NexmarkBuilder.groovy
##########
@@ -81,16 +87,46 @@ class NexmarkBuilder {
   }
 
 
-  static void suite(context, String title, Runner runner, SDK sdk, Map<String, Object> options) {
+  static void suite(context, String title, Runner runner, SDK sdk, Map<String, Object> options, List<String> jobSpecificSwitches, String javaRuntimeVersion) {
+    if (javaRuntimeVersion == JAVA_11_RUNTIME_VERSION) {
+      java11Suite(context, title, runner, sdk, options, jobSpecificSwitches)
+    } else {
+      InfluxDBCredentialsHelper.useCredentials(context)

Review comment:
       create a `java8suite` and move this block into taht suite.

##########
File path: .test-infra/jenkins/NexmarkBuilder.groovy
##########
@@ -81,16 +87,46 @@ class NexmarkBuilder {
   }
 
 
-  static void suite(context, String title, Runner runner, SDK sdk, Map<String, Object> options) {
+  static void suite(context, String title, Runner runner, SDK sdk, Map<String, Object> options, List<String> jobSpecificSwitches, String javaRuntimeVersion) {
+    if (javaRuntimeVersion == JAVA_11_RUNTIME_VERSION) {
+      java11Suite(context, title, runner, sdk, options, jobSpecificSwitches)
+    } else {
+      InfluxDBCredentialsHelper.useCredentials(context)
+      context.steps {
+        shell("echo \"*** RUN ${title} ***\"")

Review comment:
       Run with java 8?

##########
File path: sdks/java/testing/nexmark/build.gradle
##########
@@ -102,6 +109,39 @@ if (shouldProvideSpark) {
   }
 }
 
+def getNexmarkArgs = {
+  def nexmarkArgsStr =  project.findProperty(nexmarkArgsProperty) ?: ""
+  def nexmarkArgsList = new ArrayList<String>()
+  Collections.addAll(nexmarkArgsList, nexmarkArgsStr.split())
+
+  if (isDataflowRunner) {
+    if (isDataflowRunnerV2) {
+      nexmarkArgsList.add("--experiments=beam_fn_api,use_unified_worker,use_runner_v2,shuffle_mode=appliance")
+      def sdkContainerImage = project.findProperty('sdkContainerImage') ?: project(":runners:google-cloud-dataflow-java").dockerImageName
+      nexmarkArgsList.add("--sdkContainerImage=${sdkContainerImage}")

Review comment:
       Is this line trying to provide images for runner v2? IIRC usually I use two different flags to specify images for runner v2. Are you referring to other v2 setup to use this flag?

##########
File path: .test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow_V2.groovy
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+import CommonTestProperties.Runner
+import CommonTestProperties.SDK
+import CommonTestProperties.TriggeringContext
+import NexmarkBuilder as Nexmark
+import NoPhraseTriggeringPostCommitBuilder
+import PhraseTriggeringPostCommitBuilder
+import InfluxDBCredentialsHelper
+
+import static NexmarkDatabaseProperties.nexmarkBigQueryArgs
+import static NexmarkDatabaseProperties.nexmarkInfluxDBArgs
+
+def final JOB_SPECIFIC_OPTIONS = [
+  'influxTags' : '{\\\"runnerVersion\\\":\\\"V2\\\",\\\"javaVersion\\\":\\\"8\\\"}',
+  'exportSummaryToBigQuery' : false,
+  'region' : 'us-central1',
+  'suite' : 'STRESS',

Review comment:
       Question: there are also other modes. E.g. smoke test. Why choose `STRESS` (for performance measurement purpose)?

##########
File path: sdks/java/testing/nexmark/build.gradle
##########
@@ -102,6 +109,39 @@ if (shouldProvideSpark) {
   }
 }
 
+def getNexmarkArgs = {
+  def nexmarkArgsStr =  project.findProperty(nexmarkArgsProperty) ?: ""
+  def nexmarkArgsList = new ArrayList<String>()
+  Collections.addAll(nexmarkArgsList, nexmarkArgsStr.split())
+
+  if (isDataflowRunner) {
+    if (isDataflowRunnerV2) {
+      nexmarkArgsList.add("--experiments=beam_fn_api,use_unified_worker,use_runner_v2,shuffle_mode=appliance")

Review comment:
       I would suggest you use `shuffle_mode=service`. Currently we just focus on runnver v2 + shuffle service. 

##########
File path: sdks/java/testing/nexmark/build.gradle
##########
@@ -102,6 +109,39 @@ if (shouldProvideSpark) {
   }
 }
 
+def getNexmarkArgs = {
+  def nexmarkArgsStr =  project.findProperty(nexmarkArgsProperty) ?: ""
+  def nexmarkArgsList = new ArrayList<String>()
+  Collections.addAll(nexmarkArgsList, nexmarkArgsStr.split())
+
+  if (isDataflowRunner) {
+    if (isDataflowRunnerV2) {
+      nexmarkArgsList.add("--experiments=beam_fn_api,use_unified_worker,use_runner_v2,shuffle_mode=appliance")
+      def sdkContainerImage = project.findProperty('sdkContainerImage') ?: project(":runners:google-cloud-dataflow-java").dockerImageName
+      nexmarkArgsList.add("--sdkContainerImage=${sdkContainerImage}")
+      // TODO(BEAM-12295) enable all queries once issues with runner V2 is fixed.
+      if (nexmarkArgsList.contains("--streaming=true")) {
+        nexmarkArgsList.add("--skipQueries=AVERAGE_PRICE_FOR_CATEGORY,AVERAGE_SELLING_PRICE_BY_SELLER,WINNING_BIDS,BOUNDED_SIDE_INPUT_JOIN,SESSION_SIDE_INPUT_JOIN,PORTABILITY_BATCH") // 4, 6, 9, 13, 14, 15
+      } else {
+        nexmarkArgsList.add("--skipQueries=LOCAL_ITEM_SUGGESTION,AVERAGE_PRICE_FOR_CATEGORY,AVERAGE_SELLING_PRICE_BY_SELLER,HIGHEST_BID,WINNING_BIDS,SESSION_SIDE_INPUT_JOIN,BOUNDED_SIDE_INPUT_JOIN") // 3, 4, 6, 7, 9, 13, 14, 15
+      }
+    } else {
+      def dataflowWorkerJar = project.findProperty('dataflowWorkerJar') ?: project(":runners:google-cloud-dataflow-java:worker:legacy-worker").shadowJar.archivePath
+      // Provide job with a customizable worker jar.
+      // With legacy worker jar, containerImage is set to empty (i.e. to use the internal build).
+      // More context and discussions can be found in PR#6694.
+      nexmarkArgsList.add("--dataflowWorkerJar=${dataflowWorkerJar}".toString())
+      nexmarkArgsList.add('--workerHarnessContainerImage=')

Review comment:
       This flag seems not set?

##########
File path: .test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow_V2.groovy
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+import CommonTestProperties.Runner
+import CommonTestProperties.SDK
+import CommonTestProperties.TriggeringContext
+import NexmarkBuilder as Nexmark
+import NoPhraseTriggeringPostCommitBuilder
+import PhraseTriggeringPostCommitBuilder
+import InfluxDBCredentialsHelper
+
+import static NexmarkDatabaseProperties.nexmarkBigQueryArgs
+import static NexmarkDatabaseProperties.nexmarkInfluxDBArgs
+
+def final JOB_SPECIFIC_OPTIONS = [
+  'influxTags' : '{\\\"runnerVersion\\\":\\\"V2\\\",\\\"javaVersion\\\":\\\"8\\\"}',
+  'exportSummaryToBigQuery' : false,
+  'region' : 'us-central1',
+  'suite' : 'STRESS',
+  'numWorkers' : 4,
+  'maxNumWorkers' : 4,
+  'autoscalingAlgorithm' : 'NONE',
+  'nexmarkParallel' : 16,
+  'enforceEncodability' : true,
+  'enforceImmutability' : true
+]
+
+def final JOB_SPECIFIC_SWITCHES = [
+  '-Pnexmark.runner.version="V2"'
+]
+
+// This job runs the suite of Nexmark tests against the Dataflow runner V2.
+NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_Dataflow_V2',

Review comment:
       Question: why need a `NoPhraseTriggeringPostCommitBuilder`?

##########
File path: .test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow_V2.groovy
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+import CommonTestProperties.Runner
+import CommonTestProperties.SDK
+import CommonTestProperties.TriggeringContext
+import NexmarkBuilder as Nexmark
+import NoPhraseTriggeringPostCommitBuilder
+import PhraseTriggeringPostCommitBuilder
+import InfluxDBCredentialsHelper
+
+import static NexmarkDatabaseProperties.nexmarkBigQueryArgs
+import static NexmarkDatabaseProperties.nexmarkInfluxDBArgs
+
+def final JOB_SPECIFIC_OPTIONS = [
+  'influxTags' : '{\\\"runnerVersion\\\":\\\"V2\\\",\\\"javaVersion\\\":\\\"8\\\"}',
+  'exportSummaryToBigQuery' : false,
+  'region' : 'us-central1',
+  'suite' : 'STRESS',
+  'numWorkers' : 4,
+  'maxNumWorkers' : 4,
+  'autoscalingAlgorithm' : 'NONE',
+  'nexmarkParallel' : 16,
+  'enforceEncodability' : true,
+  'enforceImmutability' : true
+]
+
+def final JOB_SPECIFIC_SWITCHES = [
+  '-Pnexmark.runner.version="V2"'

Review comment:
       Ah to confirm, this flag enables runner v2?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-840009490


   Run Dataflow Runner V2 Java 11 Nexmark Tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-840747664


   Dataflow Runner Nexmark Tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on a change in pull request #14774: [BEAM-12334] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on a change in pull request #14774:
URL: https://github.com/apache/beam/pull/14774#discussion_r632829014



##########
File path: .test-infra/jenkins/NexmarkBuilder.groovy
##########
@@ -81,16 +87,46 @@ class NexmarkBuilder {
   }
 
 
-  static void suite(context, String title, Runner runner, SDK sdk, Map<String, Object> options) {
+  static void suite(context, String title, Runner runner, SDK sdk, Map<String, Object> options, List<String> jobSpecificSwitches, String javaRuntimeVersion) {
+    if (javaRuntimeVersion == JAVA_11_RUNTIME_VERSION) {
+      java11Suite(context, title, runner, sdk, options, jobSpecificSwitches)
+    } else {
+      InfluxDBCredentialsHelper.useCredentials(context)

Review comment:
       done

##########
File path: .test-infra/jenkins/NexmarkBuilder.groovy
##########
@@ -81,16 +87,46 @@ class NexmarkBuilder {
   }
 
 
-  static void suite(context, String title, Runner runner, SDK sdk, Map<String, Object> options) {
+  static void suite(context, String title, Runner runner, SDK sdk, Map<String, Object> options, List<String> jobSpecificSwitches, String javaRuntimeVersion) {
+    if (javaRuntimeVersion == JAVA_11_RUNTIME_VERSION) {
+      java11Suite(context, title, runner, sdk, options, jobSpecificSwitches)
+    } else {
+      InfluxDBCredentialsHelper.useCredentials(context)
+      context.steps {
+        shell("echo \"*** RUN ${title} ***\"")

Review comment:
       done

##########
File path: .test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow_V2.groovy
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+import CommonTestProperties.Runner
+import CommonTestProperties.SDK
+import CommonTestProperties.TriggeringContext
+import NexmarkBuilder as Nexmark
+import NoPhraseTriggeringPostCommitBuilder
+import PhraseTriggeringPostCommitBuilder
+import InfluxDBCredentialsHelper
+
+import static NexmarkDatabaseProperties.nexmarkBigQueryArgs
+import static NexmarkDatabaseProperties.nexmarkInfluxDBArgs
+
+def final JOB_SPECIFIC_OPTIONS = [
+  'influxTags' : '{\\\"runnerVersion\\\":\\\"V2\\\",\\\"javaVersion\\\":\\\"8\\\"}',
+  'exportSummaryToBigQuery' : false,
+  'region' : 'us-central1',
+  'suite' : 'STRESS',
+  'numWorkers' : 4,
+  'maxNumWorkers' : 4,
+  'autoscalingAlgorithm' : 'NONE',
+  'nexmarkParallel' : 16,
+  'enforceEncodability' : true,
+  'enforceImmutability' : true
+]
+
+def final JOB_SPECIFIC_SWITCHES = [
+  '-Pnexmark.runner.version="V2"'
+]
+
+// This job runs the suite of Nexmark tests against the Dataflow runner V2.
+NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_Dataflow_V2',

Review comment:
       This is to separate the PR triggered and cron metrics. PR triggered metrics shouldn't affect the dashboards

##########
File path: sdks/java/testing/nexmark/build.gradle
##########
@@ -102,6 +109,39 @@ if (shouldProvideSpark) {
   }
 }
 
+def getNexmarkArgs = {
+  def nexmarkArgsStr =  project.findProperty(nexmarkArgsProperty) ?: ""
+  def nexmarkArgsList = new ArrayList<String>()
+  Collections.addAll(nexmarkArgsList, nexmarkArgsStr.split())
+
+  if (isDataflowRunner) {
+    if (isDataflowRunnerV2) {
+      nexmarkArgsList.add("--experiments=beam_fn_api,use_unified_worker,use_runner_v2,shuffle_mode=appliance")
+      def sdkContainerImage = project.findProperty('sdkContainerImage') ?: project(":runners:google-cloud-dataflow-java").dockerImageName
+      nexmarkArgsList.add("--sdkContainerImage=${sdkContainerImage}")

Review comment:
       This is option: https://github.com/apache/beam/blob/018f5e7746b12dc42c77482227e0da429a550248/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/options/DataflowPipelineWorkerPoolOptions.java#L134

##########
File path: sdks/java/testing/nexmark/build.gradle
##########
@@ -102,6 +109,39 @@ if (shouldProvideSpark) {
   }
 }
 
+def getNexmarkArgs = {
+  def nexmarkArgsStr =  project.findProperty(nexmarkArgsProperty) ?: ""
+  def nexmarkArgsList = new ArrayList<String>()
+  Collections.addAll(nexmarkArgsList, nexmarkArgsStr.split())
+
+  if (isDataflowRunner) {
+    if (isDataflowRunnerV2) {
+      nexmarkArgsList.add("--experiments=beam_fn_api,use_unified_worker,use_runner_v2,shuffle_mode=appliance")

Review comment:
       done

##########
File path: .test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow_V2.groovy
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+import CommonTestProperties.Runner
+import CommonTestProperties.SDK
+import CommonTestProperties.TriggeringContext
+import NexmarkBuilder as Nexmark
+import NoPhraseTriggeringPostCommitBuilder
+import PhraseTriggeringPostCommitBuilder
+import InfluxDBCredentialsHelper
+
+import static NexmarkDatabaseProperties.nexmarkBigQueryArgs
+import static NexmarkDatabaseProperties.nexmarkInfluxDBArgs
+
+def final JOB_SPECIFIC_OPTIONS = [
+  'influxTags' : '{\\\"runnerVersion\\\":\\\"V2\\\",\\\"javaVersion\\\":\\\"8\\\"}',
+  'exportSummaryToBigQuery' : false,
+  'region' : 'us-central1',
+  'suite' : 'STRESS',
+  'numWorkers' : 4,
+  'maxNumWorkers' : 4,
+  'autoscalingAlgorithm' : 'NONE',
+  'nexmarkParallel' : 16,
+  'enforceEncodability' : true,
+  'enforceImmutability' : true
+]
+
+def final JOB_SPECIFIC_SWITCHES = [
+  '-Pnexmark.runner.version="V2"'

Review comment:
       I use this as a flag in the build.gradle to run v2 specific tasks/flags

##########
File path: sdks/java/testing/nexmark/build.gradle
##########
@@ -102,6 +109,39 @@ if (shouldProvideSpark) {
   }
 }
 
+def getNexmarkArgs = {
+  def nexmarkArgsStr =  project.findProperty(nexmarkArgsProperty) ?: ""
+  def nexmarkArgsList = new ArrayList<String>()
+  Collections.addAll(nexmarkArgsList, nexmarkArgsStr.split())
+
+  if (isDataflowRunner) {
+    if (isDataflowRunnerV2) {
+      nexmarkArgsList.add("--experiments=beam_fn_api,use_unified_worker,use_runner_v2,shuffle_mode=appliance")
+      def sdkContainerImage = project.findProperty('sdkContainerImage') ?: project(":runners:google-cloud-dataflow-java").dockerImageName
+      nexmarkArgsList.add("--sdkContainerImage=${sdkContainerImage}")
+      // TODO(BEAM-12295) enable all queries once issues with runner V2 is fixed.
+      if (nexmarkArgsList.contains("--streaming=true")) {
+        nexmarkArgsList.add("--skipQueries=AVERAGE_PRICE_FOR_CATEGORY,AVERAGE_SELLING_PRICE_BY_SELLER,WINNING_BIDS,BOUNDED_SIDE_INPUT_JOIN,SESSION_SIDE_INPUT_JOIN,PORTABILITY_BATCH") // 4, 6, 9, 13, 14, 15
+      } else {
+        nexmarkArgsList.add("--skipQueries=LOCAL_ITEM_SUGGESTION,AVERAGE_PRICE_FOR_CATEGORY,AVERAGE_SELLING_PRICE_BY_SELLER,HIGHEST_BID,WINNING_BIDS,SESSION_SIDE_INPUT_JOIN,BOUNDED_SIDE_INPUT_JOIN") // 3, 4, 6, 7, 9, 13, 14, 15
+      }
+    } else {
+      def dataflowWorkerJar = project.findProperty('dataflowWorkerJar') ?: project(":runners:google-cloud-dataflow-java:worker:legacy-worker").shadowJar.archivePath
+      // Provide job with a customizable worker jar.
+      // With legacy worker jar, containerImage is set to empty (i.e. to use the internal build).
+      // More context and discussions can be found in PR#6694.
+      nexmarkArgsList.add("--dataflowWorkerJar=${dataflowWorkerJar}".toString())
+      nexmarkArgsList.add('--workerHarnessContainerImage=')

Review comment:
       It's in the comment above, but it's used in the other runner v2 tests as well

##########
File path: .test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow_V2.groovy
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+import CommonTestProperties.Runner
+import CommonTestProperties.SDK
+import CommonTestProperties.TriggeringContext
+import NexmarkBuilder as Nexmark
+import NoPhraseTriggeringPostCommitBuilder
+import PhraseTriggeringPostCommitBuilder
+import InfluxDBCredentialsHelper
+
+import static NexmarkDatabaseProperties.nexmarkBigQueryArgs
+import static NexmarkDatabaseProperties.nexmarkInfluxDBArgs
+
+def final JOB_SPECIFIC_OPTIONS = [
+  'influxTags' : '{\\\"runnerVersion\\\":\\\"V2\\\",\\\"javaVersion\\\":\\\"8\\\"}',
+  'exportSummaryToBigQuery' : false,
+  'region' : 'us-central1',
+  'suite' : 'STRESS',

Review comment:
       It's what the other nexmark tests are being run at. Also, we do want to use this to measure the performance of java 8 vs 11. I could drop the PR triggered ones to SMOKE since we don't need the full performance test




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] amaliujia merged pull request #14774: [BEAM-12334] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
amaliujia merged pull request #14774:
URL: https://github.com/apache/beam/pull/14774


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-837333813


   @amaliujia can you run the seed job so I can see if the Jenkins jobs runs?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-839133572


   Run Dataflow Runner V2 Java 11 Nexmark Tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] amaliujia commented on pull request #14774: [BEAM-12334] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-841593684


   LGTM


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-839986852


   Run Dataflow Runner V2 Java 11 Nexmark Tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-840747786






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] amaliujia commented on a change in pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
amaliujia commented on a change in pull request #14774:
URL: https://github.com/apache/beam/pull/14774#discussion_r629740733



##########
File path: .test-infra/jenkins/job_PostCommit_Java_Nexmark_Dataflow_V2.groovy
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+import CommonTestProperties.Runner
+import CommonTestProperties.SDK
+import CommonTestProperties.TriggeringContext
+import NexmarkBuilder as Nexmark
+import NoPhraseTriggeringPostCommitBuilder
+import PhraseTriggeringPostCommitBuilder
+import InfluxDBCredentialsHelper
+
+import static NexmarkDatabaseProperties.nexmarkBigQueryArgs
+import static NexmarkDatabaseProperties.nexmarkInfluxDBArgs
+
+def final JOB_SPECIFIC_OPTIONS = [
+  'influxTags' : '{"runnerVersion":"V2", "javaVersion":"8"}',
+  'exportSummaryToBigQuery' : false,
+  'region' : 'us-central1',
+  'suite' : 'STRESS',
+  'numWorkers' : 4,
+  'maxNumWorkers' : 4,
+  'autoscalingAlgorithm' : 'NONE',
+  'nexmarkParallel' : 16,
+  'enforceEncodability' : true,
+  'enforceImmutability' : true
+]
+
+def final JOB_SPECIFIC_SWITCHES = [
+  '-Pnexmark.runner.version="V2"'
+]
+
+// This job runs the suite of Nexmark tests against the Dataflow runner V2.
+NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Nexmark_Dataflow_V2',
+    'Dataflow Runner V2 Nexmark Tests', this) {

Review comment:
       Which should be used in the jenkins job command? Is it `beam_PostCommit_Java_Nexmark_Dataflow_V2` or `ataflow Runner V2 Nexmark Tests`?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] amaliujia commented on a change in pull request #14774: [BEAM-12334] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
amaliujia commented on a change in pull request #14774:
URL: https://github.com/apache/beam/pull/14774#discussion_r632061480



##########
File path: .test-infra/jenkins/NexmarkBuilder.groovy
##########
@@ -81,16 +87,46 @@ class NexmarkBuilder {
   }
 
 
-  static void suite(context, String title, Runner runner, SDK sdk, Map<String, Object> options) {
+  static void suite(context, String title, Runner runner, SDK sdk, Map<String, Object> options, List<String> jobSpecificSwitches, String javaRuntimeVersion) {
+    if (javaRuntimeVersion == JAVA_11_RUNTIME_VERSION) {
+      java11Suite(context, title, runner, sdk, options, jobSpecificSwitches)
+    } else {
+      InfluxDBCredentialsHelper.useCredentials(context)

Review comment:
       create a `java8suite` and move this block into that suite.

##########
File path: .test-infra/jenkins/NexmarkBuilder.groovy
##########
@@ -81,16 +87,46 @@ class NexmarkBuilder {
   }
 
 
-  static void suite(context, String title, Runner runner, SDK sdk, Map<String, Object> options) {
+  static void suite(context, String title, Runner runner, SDK sdk, Map<String, Object> options, List<String> jobSpecificSwitches, String javaRuntimeVersion) {
+    if (javaRuntimeVersion == JAVA_11_RUNTIME_VERSION) {
+      java11Suite(context, title, runner, sdk, options, jobSpecificSwitches)
+    } else {
+      InfluxDBCredentialsHelper.useCredentials(context)

Review comment:
       create a `java8suite` and move this block into that suite?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-839218905






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] amaliujia commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-837456148


   Run Seed Job


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-839313253


   Run Dataflow Runner V2 Java 11 Nexmark Tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] amaliujia commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-839034420


   Run Seed Job


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-840055212


   Run Dataflow Runner V2 Java 11 Nexmark Tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] amaliujia commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
amaliujia commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-837480974


   Run Seed Job


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-840751544






-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [BEAM-12334] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-840791238


   R: @amaliujia, @kennknowles 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] amaliujia commented on a change in pull request #14774: [BEAM-12334] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
amaliujia commented on a change in pull request #14774:
URL: https://github.com/apache/beam/pull/14774#discussion_r632066200



##########
File path: sdks/java/testing/nexmark/build.gradle
##########
@@ -102,6 +109,39 @@ if (shouldProvideSpark) {
   }
 }
 
+def getNexmarkArgs = {
+  def nexmarkArgsStr =  project.findProperty(nexmarkArgsProperty) ?: ""
+  def nexmarkArgsList = new ArrayList<String>()
+  Collections.addAll(nexmarkArgsList, nexmarkArgsStr.split())
+
+  if (isDataflowRunner) {
+    if (isDataflowRunnerV2) {
+      nexmarkArgsList.add("--experiments=beam_fn_api,use_unified_worker,use_runner_v2,shuffle_mode=appliance")
+      def sdkContainerImage = project.findProperty('sdkContainerImage') ?: project(":runners:google-cloud-dataflow-java").dockerImageName
+      nexmarkArgsList.add("--sdkContainerImage=${sdkContainerImage}")

Review comment:
       Is this line trying to provide images for runner v2? IIRC usually I use two different flags to specify images for runner v2. Are you referring to other v2 setup which use this flag?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kileys commented on pull request #14774: [WIP] Run Jenkins jobs for Nexmark Dataflow V2 Java 8 and 11

Posted by GitBox <gi...@apache.org>.
kileys commented on pull request #14774:
URL: https://github.com/apache/beam/pull/14774#issuecomment-839312958


   Run Dataflow Runner V2 Nexmark Tests


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org