You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2023/11/19 22:13:07 UTC
[I] Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs:mean_load_model_latency_milli_secs [beam]
github-actions[bot] opened a new issue, #29492:
URL: https://github.com/apache/beam/issues/29492
Performance change found in the
test: `pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs` for the metric: `mean_load_model_latency_milli_secs`.
For more information on how to triage the alerts, please look at
`Triage performance alert issues` section of the [README](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/testing/analyzers/README.md#triage-performance-alert-issues).
`Test description:` Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU.
Test link - https://github.com/apache/beam/blob/42d0a6e3564d8b9c5d912428a6de18fb22a13ac1/.test-infra/jenkins/job_InferenceBenchmarkTests_Python.groovy#L151
Test dashboard - http://metrics.beam.apache.org/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1&viewPanel=7
```
timestamp: Sun Nov 19 06:36:43 2023, metric_value: 209499.95
timestamp: Sat Nov 18 06:29:46 2023, metric_value: 212258.56
timestamp: Fri Nov 17 06:31:43 2023, metric_value: 209102.95 <---- Anomaly
timestamp: Wed Nov 15 06:30:16 2023, metric_value: 168926.49
timestamp: Tue Nov 14 06:26:12 2023, metric_value: 165426.66
timestamp: Mon Nov 13 06:26:33 2023, metric_value: 173417.09
timestamp: Sun Nov 12 06:26:54 2023, metric_value: 162621.34
timestamp: Sat Nov 11 06:26:23 2023, metric_value: 168426.61
timestamp: Fri Nov 10 06:26:40 2023, metric_value: 168759.09
timestamp: Thu Nov 9 06:29:53 2023, metric_value: 171794.31
timestamp: Tue Nov 7 07:13:42 2023, metric_value: 167979.26
timestamp: Tue Nov 7 06:28:45 2023, metric_value: 172938.72
timestamp: Mon Nov 6 06:35:23 2023, metric_value: 170587.09
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs:mean_load_model_latency_milli_secs [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad closed issue #29492:
Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs:mean_load_model_latency_milli_secs
URL: https://github.com/apache/beam/issues/29492
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs:mean_load_model_latency_milli_secs [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #29492:
URL: https://github.com/apache/beam/issues/29492#issuecomment-1872616189
Performance change found in the
test: `pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs` for the metric: `mean_load_model_latency_milli_secs`.
For more information on how to triage the alerts, please look at
`Triage performance alert issues` section of the [README](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/testing/analyzers/README.md#triage-performance-alert-issues).
`Test description:` Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU.
Test link - https://github.com/apache/beam/blob/42d0a6e3564d8b9c5d912428a6de18fb22a13ac1/.test-infra/jenkins/job_InferenceBenchmarkTests_Python.groovy#L151
Test dashboard - http://metrics.beam.apache.org/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1&viewPanel=7
```
timestamp: Sat Dec 30 06:46:27 2023, metric_value: 202143.13
timestamp: Fri Dec 29 06:45:37 2023, metric_value: 210222.32
timestamp: Thu Dec 28 00:16:10 2023, metric_value: 207040.46
timestamp: Wed Dec 27 06:44:56 2023, metric_value: 217219.49
timestamp: Tue Dec 26 06:44:21 2023, metric_value: 203994.03
timestamp: Sun Dec 24 07:04:48 2023, metric_value: 205072.90
timestamp: Sat Dec 23 06:58:13 2023, metric_value: 205543.48
timestamp: Fri Dec 22 06:52:50 2023, metric_value: 209199.66 <---- Anomaly
timestamp: Thu Dec 21 06:52:04 2023, metric_value: 221007.77
timestamp: Wed Dec 20 07:04:02 2023, metric_value: 223566.49
timestamp: Tue Dec 19 06:54:26 2023, metric_value: 215618.13
timestamp: Mon Dec 18 16:43:04 2023, metric_value: 231189.45
timestamp: Fri Dec 15 06:54:00 2023, metric_value: 217229.54
timestamp: Thu Dec 14 06:45:17 2023, metric_value: 207335.30
timestamp: Wed Dec 13 06:55:32 2023, metric_value: 213023.18
timestamp: Tue Dec 12 06:40:05 2023, metric_value: 213582.44
timestamp: Mon Dec 11 06:37:05 2023, metric_value: 218185.46
timestamp: Sun Dec 10 06:32:59 2023, metric_value: 209092.40
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs:mean_load_model_latency_milli_secs [beam]
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on issue #29492:
URL: https://github.com/apache/beam/issues/29492#issuecomment-1819318191
Similar to https://github.com/apache/beam/issues/29491.
cc: @tvalentyn current python interrupts if you could take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs:mean_load_model_latency_milli_secs [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #29492:
URL: https://github.com/apache/beam/issues/29492#issuecomment-1928216086
Performance change found in the
test: `pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs` for the metric: `mean_load_model_latency_milli_secs`.
For more information on how to triage the alerts, please look at
`Triage performance alert issues` section of the [README](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/testing/analyzers/README.md#triage-performance-alert-issues).
`Test description:` Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU.
Test link - https://github.com/apache/beam/blob/42d0a6e3564d8b9c5d912428a6de18fb22a13ac1/.test-infra/jenkins/job_InferenceBenchmarkTests_Python.groovy#L151
Test dashboard - http://metrics.beam.apache.org/d/ZpS8Uf44z/python-ml-runinference-benchmarks?orgId=1&viewPanel=7
```
timestamp: Mon Feb 5 06:48:47 2024, metric_value: 219060.07
timestamp: Sat Feb 3 06:45:12 2024, metric_value: 225485.20
timestamp: Thu Feb 1 06:48:04 2024, metric_value: 221938.53 <---- Anomaly
timestamp: Tue Jan 30 06:47:30 2024, metric_value: 209624.08
timestamp: Mon Jan 29 06:45:18 2024, metric_value: 209521.78
timestamp: Sun Jan 28 06:39:42 2024, metric_value: 200487.08
timestamp: Sat Jan 27 06:43:41 2024, metric_value: 222538.60
timestamp: Fri Jan 26 06:45:21 2024, metric_value: 210453.72
timestamp: Thu Jan 25 06:44:42 2024, metric_value: 208565.78
timestamp: Wed Jan 24 06:55:31 2024, metric_value: 211543.25
timestamp: Mon Jan 22 06:44:51 2024, metric_value: 209070.38
timestamp: Sun Jan 21 06:45:18 2024, metric_value: 201586.58
timestamp: Sat Jan 20 06:38:09 2024, metric_value: 196935.87
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Performance Regression or Improvement: pytorch_image_classification_benchmarks-resnet152-GPU-mean_load_model_latency_milli_secs:mean_load_model_latency_milli_secs [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #29492:
URL: https://github.com/apache/beam/issues/29492#issuecomment-1928696471
I think this benchmark readings have too much variance, doesn't seem to be an anomaly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org