You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Yichi Zhang (Jira)" <ji...@apache.org> on 2022/05/04 21:56:00 UTC
[jira] [Commented] (BEAM-14416) ParDo LoadTest performance regression on java streaming dataflow runner v2
[ https://issues.apache.org/jira/browse/BEAM-14416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531961#comment-17531961 ]
Yichi Zhang commented on BEAM-14416:
------------------------------------
Some how there seems to be a large variance on number of records generated in pipeline.
55k records:
[https://pantheon.corp.google.com/dataflow/jobs/us-central1/2022-04-14_05_45_57-426277949241977369;step=Read%20input%2FParDo%28StripIds%29;mainTab=JOB_GRAPH?project=apache-beam-testing&pageState=(%22dfTime%22:(%22l%22:%22dfJobMaxTime%22))]
110k records:
[https://pantheon.corp.google.com/dataflow/jobs/us-central1/2022-04-15_05_45_35-2483161966189012574;step=Read%20input%2FParDo%28StripIds%29;mainTab=JOB_GRAPH;bottomTab=WORKER_LOGS;logsSeverity=INFO?project=apache-beam-testing]
190k records:
https://pantheon.corp.google.com/dataflow/jobs/us-central1/2022-04-30_05_45_24-15806191066837497192;step=Read%20input?project=apache-beam-testing&pageState=(%22dfTime%22:(%22l%22:%22dfJobMaxTime%22))
> ParDo LoadTest performance regression on java streaming dataflow runner v2
> --------------------------------------------------------------------------
>
> Key: BEAM-14416
> URL: https://issues.apache.org/jira/browse/BEAM-14416
> Project: Beam
> Issue Type: Test
> Components: testing
> Reporter: Yichi Zhang
> Assignee: Yichi Zhang
> Priority: P2
> Fix For: 2.39.0
>
>
> [http://metrics.beam.apache.org/d/MOi-kf3Zk/pardo-load-tests?orgId=1&var-processingType=streaming&var-sdk=java&from=now-60d&to=now]
>
> it seems that there is a trend of runtime going up from 04/13
>
> [https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java11/]
>
> https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_V2_Streaming_Java17/
--
This message was sent by Atlassian Jira
(v8.20.7#820007)