You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/14 17:38:23 UTC
[GitHub] [beam] AnandInguva opened a new issue, #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
AnandInguva opened a new issue, #21859:
URL: https://github.com/apache/beam/issues/21859
### What needs to happen?
Torch IT test uses Imagenet dataset and torch hub pre-trained models. Sklearn IT test uses dataset from Kaggle.
Until the licenses for these are figured out, sickbay all the IT tests that uses publicly available data and models.
### Issue Priority
Priority: 2
### Issue Component
Component: sdk-py-core
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #21859:
URL: https://github.com/apache/beam/issues/21859#issuecomment-1224554896
Looks like it's not done: https://github.com/apache/beam/blob/c7f64264451af12ff6c7c0ef4bc95fd7ce0f5418/sdks/python/apache_beam/ml/inference/sklearn_inference_it_test.py#L56
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
Posted by GitBox <gi...@apache.org>.
AnandInguva commented on issue #21859:
URL: https://github.com/apache/beam/issues/21859#issuecomment-1224606966
https://github.com/apache/beam/pull/22825. Added a PR to unskip the change. Once thats merged, we can close this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit closed issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
Posted by GitBox <gi...@apache.org>.
TheNeuralBit closed issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
URL: https://github.com/apache/beam/issues/21859
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #21859:
URL: https://github.com/apache/beam/issues/21859#issuecomment-1232081928
Actually, we can just reuse https://github.com/apache/beam/issues/22858
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
Posted by GitBox <gi...@apache.org>.
AnandInguva commented on issue #21859:
URL: https://github.com/apache/beam/issues/21859#issuecomment-1224570053
> Looks like it's not done:
>
> https://github.com/apache/beam/blob/c7f64264451af12ff6c7c0ef4bc95fd7ce0f5418/sdks/python/apache_beam/ml/inference/sklearn_inference_it_test.py#L56
This used to be run before https://github.com/apache/beam/pull/22088. I think @ryanthompson591 accidentally added skip to that test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on issue #21859:
URL: https://github.com/apache/beam/issues/21859#issuecomment-1224557466
Helpful link: https://ci-beam.apache.org/job/beam_PostCommit_Python39/lastCompletedBuild/testReport/apache_beam.ml.inference.pytorch_inference_it_test/PyTorchInference/
shows that these tests are still skipped
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #21859:
URL: https://github.com/apache/beam/issues/21859#issuecomment-1232081486
@AnandInguva could you please file an issue to see if/how we can export test signal from the RunInference tests and combine it with the PostCommit signal?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] TheNeuralBit commented on issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
Posted by GitBox <gi...@apache.org>.
TheNeuralBit commented on issue #21859:
URL: https://github.com/apache/beam/issues/21859#issuecomment-1224992633
Thank you for the detailed explanation!
I think it's possible for the Jenkins UI to display the output of multiple test suites. e.g. in the Python PreCommit we see duplicate runs of each test, one for each environment:
I'm not sure what is actually enabling this, but we should do it in these PostCommits too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn closed issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
Posted by GitBox <gi...@apache.org>.
tvalentyn closed issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
URL: https://github.com/apache/beam/issues/21859
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
Posted by GitBox <gi...@apache.org>.
AnandInguva commented on issue #21859:
URL: https://github.com/apache/beam/issues/21859#issuecomment-1224592363
> Helpful link: https://ci-beam.apache.org/job/beam_PostCommit_Python39/lastCompletedBuild/testReport/apache_beam.ml.inference.pytorch_inference_it_test/PyTorchInference/
>
> shows that these tests are still skipped
Jenkins UI only shows the IT tests that are part of Dataflow suite( these are written to xml files and the UI is created from that). We have 5~6 IT post commit suite. Inference tests runs here https://github.com/apache/beam/blob/94941667e6b472f279501bd81514f204e9f75038/build.gradle.kts#L335 and is not displayed on the UI.
The tests we see in the UI are part of https://github.com/apache/beam/blob/94941667e6b472f279501bd81514f204e9f75038/build.gradle.kts#L331. Here, inference tests are collected because they have the decorator `it_postcommit` and then gets skipped because the dependencies are not installed on the Jenkins node.
The tests actually pass in the https://ci-beam.apache.org/job/beam_PostCommit_Python39/lastCompletedBuild/testReport/apache_beam.ml.inference.pytorch_inference_it_test/PyTorchInference/. It can be found in the console
<img width="1112" alt="Screen Shot 2022-08-24 at 12 11 19 AM" src="https://user-images.githubusercontent.com/34158215/186238468-fa97ce82-83df-4e3c-90d3-6f3e2ffca292.png">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #21859:
URL: https://github.com/apache/beam/issues/21859#issuecomment-1222517440
PLease reopen if this is not done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #21859: Unskip RunInference IT tests once the licensing on the datasets and models is complete.
Posted by GitBox <gi...@apache.org>.
AnandInguva commented on issue #21859:
URL: https://github.com/apache/beam/issues/21859#issuecomment-1155519116
.take-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org