You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2023/03/27 22:10:16 UTC
[GitHub] [beam] github-actions[bot] opened a new issue, #26000: Performance Regression or Improvement: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU:mean_inference_batch_latency_micro_secs
github-actions[bot] opened a new issue, #26000:
URL: https://github.com/apache/beam/issues/26000
Performance change found in the
test: `Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU:apache_beam.testing.benchmarks.inference.pytorch_image_classification_benchmarks` for the metric: `mean_inference_batch_latency_micro_secs`.
For more information on how to triage the alerts, please look at
`Triage performance alert issues` section of the [README](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/testing/analyzers/README.md#triage-performance-alert-issues).
timestamp: Mon Mar 27 18:25:38 2023, metric_value: `72532.29442691903`
timestamp: Sun Mar 26 18:21:12 2023, metric_value: `67661.97871621621`
timestamp: Sat Mar 25 18:27:50 2023, metric_value: `73059.60363774157`
timestamp: Fri Mar 24 18:30:46 2023, metric_value: `70512.15529411765`
timestamp: Thu Mar 23 18:28:27 2023, metric_value: `67625.78046142754`
timestamp: Wed Mar 22 18:35:18 2023, metric_value: `72893.40243055555`
timestamp: Tue Mar 21 18:34:17 2023, metric_value: `77126.60742268042`
timestamp: Mon Mar 20 18:38:44 2023, metric_value: `72422.41292442498`
timestamp: Fri Mar 17 18:54:55 2023, metric_value: `72757.09090909091`
timestamp: Wed Mar 15 18:39:24 2023, metric_value: `307992.8282527881`
timestamp: Tue Mar 14 18:20:28 2023, metric_value: `274270.1772339637` <---- Anomaly
timestamp: Mon Mar 13 18:52:40 2023, metric_value: `307627.037796542`
timestamp: Sun Mar 12 18:17:56 2023, metric_value: `296421.48533834587`
timestamp: Sat Mar 11 18:19:42 2023, metric_value: `334912.551322277`
timestamp: Fri Mar 10 18:21:33 2023, metric_value: `345194.726580009`
timestamp: Thu Mar 9 18:26:05 2023, metric_value: `391700.1058052434`
timestamp: Wed Mar 8 18:25:08 2023, metric_value: `319142.57038123166`
timestamp: Tue Mar 7 18:23:57 2023, metric_value: `401203.8418560606`
timestamp: Tue Mar 7 01:20:20 2023, metric_value: `374878.1331090822`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #26000: Performance Regression or Improvement: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU:mean_inference_batch_latency_micro_secs
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #26000:
URL: https://github.com/apache/beam/issues/26000#issuecomment-1487321257
Also: in README it may make sense to move the triaging section higher to the top of the narrative
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #26000: Performance Regression or Improvement: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU:mean_inference_batch_latency_micro_secs
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on issue #26000:
URL: https://github.com/apache/beam/issues/26000#issuecomment-1487470056
.close-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #26000: Performance Regression or Improvement: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU:mean_inference_batch_latency_micro_secs
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on issue #26000:
URL: https://github.com/apache/beam/issues/26000#issuecomment-1487454435
We can close this issue as done @tvalentyn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #26000: Performance Regression or Improvement: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU:mean_inference_batch_latency_micro_secs
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on issue #26000:
URL: https://github.com/apache/beam/issues/26000#issuecomment-1486198209
@tvalentyn this is the alert filed by performance alerting tool
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #26000: Performance Regression or Improvement: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU:mean_inference_batch_latency_micro_secs
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #26000:
URL: https://github.com/apache/beam/issues/26000#issuecomment-1505871963
@AnandInguva did we file an issue to rootcause mismatched anomaly index pointer?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] closed issue #26000: Performance Regression or Improvement: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU:mean_inference_batch_latency_micro_secs
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #26000:
Performance Regression or Improvement: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU:mean_inference_batch_latency_micro_secs
URL: https://github.com/apache/beam/issues/26000
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #26000: Performance Regression or Improvement: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU:mean_inference_batch_latency_micro_secs
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #26000:
URL: https://github.com/apache/beam/issues/26000#issuecomment-1487320317
Nice, it would be good to add pointers how to find actual runs: either in bug message or in triage guidebook
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #26000: Performance Regression or Improvement: Pytorch image classification on 50k images of size 224 x 224 with resnet 152 with Tesla T4 GPU:mean_inference_batch_latency_micro_secs
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on issue #26000:
URL: https://github.com/apache/beam/issues/26000#issuecomment-1487442930
I triaged this perf alert. It was due to the newer release of torch.
Pytorch 2.0 is faster as stated here https://pytorch.org/get-started/pytorch-2.0/.
On [March 17th](https://ci-beam.apache.org/job/beam_Inference_Python_Benchmarks_Dataflow/193/consoleFull), Pytorch 2.0 was installed but on [March 15th](https://ci-beam.apache.org/job/beam_Inference_Python_Benchmarks_Dataflow/191/consoleFull), Pytorch 1.13.1 was installed in the tests.
Note: March 16th test was not registered due to a failure caused by Jenkins.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org