You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "damccorm (via GitHub)" <gi...@apache.org> on 2023/02/17 21:21:52 UTC
[GitHub] [beam] damccorm opened a new pull request, #25548: Add dependencies needed for some ml integration tests
damccorm opened a new pull request, #25548:
URL: https://github.com/apache/beam/pull/25548
**Please** add a meaningful description for your change here
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm closed pull request #25548: Add dependencies needed for some ml integration tests
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed pull request #25548: Add dependencies needed for some ml integration tests
URL: https://github.com/apache/beam/pull/25548
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on pull request #25548: Add dependencies needed for some ml integration tests
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on PR #25548:
URL: https://github.com/apache/beam/pull/25548#issuecomment-1435716414
Looking at #25547, the tests are actually running but not visible in the Jenkins UI. You can look in the console log and search for `InferencePostCommitIT` in python 37 https://github.com/apache/beam/blob/977e53104f493124101c8f06d63cca0ec0994990/build.gradle.kts#L447 and python 39 test suite https://github.com/apache/beam/blob/977e53104f493124101c8f06d63cca0ec0994990/build.gradle.kts#L476.
Console log: https://ci-beam.apache.org/job/beam_PostCommit_Python39/1480/consoleFull
The tests we see in the `Test Results` UI of jenkins is of the gradle task https://github.com/apache/beam/blob/977e53104f493124101c8f06d63cca0ec0994990/build.gradle.kts#L470. in the gradle task, we skip all the ML tests since they are run on different gradle task `PostCommitIT`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on pull request #25548: Add dependencies needed for some ml integration tests
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #25548:
URL: https://github.com/apache/beam/pull/25548#issuecomment-1435282295
Run Python 3.8 PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on a diff in pull request #25548: Add dependencies needed for some ml integration tests
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on code in PR #25548:
URL: https://github.com/apache/beam/pull/25548#discussion_r1111059175
##########
sdks/python/setup.py:
##########
@@ -282,11 +282,20 @@ def get_portability_package_data():
'pyyaml>=3.12,<7.0.0',
'requests_mock>=1.7,<2.0',
'tenacity>=5.0.2,<6.0',
+ 'onnxruntime==1.13.1',
+ 'Pillow>=9.0.0',
'pytest>=7.1.2,<8.0',
'pytest-xdist>=2.5.0,<3',
'pytest-timeout>=2.1.0,<3',
'scikit-learn>=0.20.0',
+ 'skl2onnx==1.13',
'sqlalchemy>=1.3,<2.0',
+ 'tensorflow>=2.11.0,<3',
Review Comment:
I am in favor of adding these dependencies to the respective gradle/tox tasks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on a diff in pull request #25548: Add dependencies needed for some ml integration tests
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on code in PR #25548:
URL: https://github.com/apache/beam/pull/25548#discussion_r1111059144
##########
sdks/python/setup.py:
##########
@@ -282,11 +282,20 @@ def get_portability_package_data():
'pyyaml>=3.12,<7.0.0',
'requests_mock>=1.7,<2.0',
'tenacity>=5.0.2,<6.0',
+ 'onnxruntime==1.13.1',
+ 'Pillow>=9.0.0',
'pytest>=7.1.2,<8.0',
'pytest-xdist>=2.5.0,<3',
'pytest-timeout>=2.1.0,<3',
'scikit-learn>=0.20.0',
+ 'skl2onnx==1.13',
'sqlalchemy>=1.3,<2.0',
+ 'tensorflow>=2.11.0,<3',
Review Comment:
Do we want to add these in the test?
Initially, there was a disagreement of adding ML libraries to the apache_beam[test], as it could install lot of unncessary dependencies and also big in size(tensorflow, torch) on to the user machine.
Also, I am removing tensorflow from the python/containers/base_image_requirements_manual.txt since it is blocking the upgrade of protobuf.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org