[GitHub] [beam] Abacn opened a new issue, #26479: [Task]: Reduce running time of beam_PostCommit_Java_VR_Dataflow_V2_Streaming

["Abacn (via GitHub)" <gi...@apache.org>]

Abacn opened a new issue, #26479:
URL: https://github.com/apache/beam/issues/26479

   ### What needs to happen?
   
   Found when doing #26416.  
   
   The test was added back in #13251 by duplicating the batch VR test suite (and include some streaming only tags). Since then the timeout has been bumped twice 330 min -> 450 min -> 510 min. In #14747 it was commented that
   
   > We should then revisit exactly what tests we need to run and how to speed them up and consume fewer resources.
   
   For Dataflow runner v1, batch and streaming jobs use different workers, but for runner v2, they uses the same workers. Many VR tests are testing basic transform functionalities (e.g. create, view) not involving batch/streaming differences. And most tests PCollection are bounded and not Timestamps. Does it suffice to run batch VR for them?
   
   If not, we should probably reduce the trigger interval for this test at least, by now it consistently costs 6.5-7 h to run: https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/ even exceeds the postcommit trigger interval (6 h). 
   
   ### Issue Priority
   
   Priority: 3 (nice-to-have improvement)
   
   ### Issue Components
   
   - [ ] Component: Python SDK
   - [ ] Component: Java SDK
   - [ ] Component: Go SDK
   - [ ] Component: Typescript SDK
   - [ ] Component: IO connector
   - [ ] Component: Beam examples
   - [ ] Component: Beam playground
   - [ ] Component: Beam katas
   - [ ] Component: Website
   - [ ] Component: Spark Runner
   - [ ] Component: Flink Runner
   - [ ] Component: Samza Runner
   - [ ] Component: Twister2 Runner
   - [ ] Component: Hazelcast Jet Runner
   - [ ] Component: Google Cloud Dataflow Runner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org