You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "jrmccluskey (via GitHub)" <gi...@apache.org> on 2023/03/10 20:15:33 UTC
[GitHub] [beam] jrmccluskey opened a new pull request, #25798: Add support for common collections.abc types
jrmccluskey opened a new pull request, #25798:
URL: https://github.com/apache/beam/pull/25798
Adds support for the `collections.abc` module's `Iterable`, `Iterator`, and `Generator` containers as type hints. The `typing` module equivalents had previously been supported but are deprecated as of Python 3.9, so support for the new recommended container types was necessary.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on pull request #25798: Add support for common collections.abc types
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on PR #25798:
URL: https://github.com/apache/beam/pull/25798#issuecomment-1464366795
R: @tvalentyn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on a diff in pull request #25798: Add support for common collections.abc types
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on code in PR #25798:
URL: https://github.com/apache/beam/pull/25798#discussion_r1134685315
##########
sdks/python/apache_beam/typehints/native_type_compatibility.py:
##########
@@ -187,6 +187,26 @@ def convert_builtin_to_typing(typ):
return typ
+def convert_collections_to_typing(typ):
+ """Converts a given collections.abc type to a typing object.
+
+ Args:
+ typ: an object inheriting from a collections.abc object
+
+ Returns:
+ type: The corresponding typing object.
+ """
+ if hasattr(typ, '__iter__'):
+ print("here")
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on pull request #25798: Add support for common collections.abc types
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on PR #25798:
URL: https://github.com/apache/beam/pull/25798#issuecomment-1466415289
TIL our python codecov is not running on 3.9 or 3.10. Interesting.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on pull request #25798: Add support for common collections.abc types
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on PR #25798:
URL: https://github.com/apache/beam/pull/25798#issuecomment-1464542126
PTAL at test failures.
```
> self.assertEqual(converted_beam_type, expected_beam_type, description)
E AssertionError: collections.abc.Iterable[int] != Iterable[<class 'int'>] : collection iterable
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on pull request #25798: Add support for common collections.abc types
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on PR #25798:
URL: https://github.com/apache/beam/pull/25798#issuecomment-1467035469
> TIL our python codecov is not running on 3.9 or 3.10. Interesting.
Yes, some tests only run on one of the Python versions, typically the lowest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #25798: Add support for common collections.abc types
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25798:
URL: https://github.com/apache/beam/pull/25798#issuecomment-1464368206
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #25798: Add support for common collections.abc types
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #25798:
URL: https://github.com/apache/beam/pull/25798#issuecomment-1466372450
## [Codecov](https://codecov.io/gh/apache/beam/pull/25798?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#25798](https://codecov.io/gh/apache/beam/pull/25798?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (825eac1) into [master](https://codecov.io/gh/apache/beam/commit/f44c553478e6c0dccfcde45579334ee500f3add3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f44c553) will **decrease** coverage by `0.10%`.
> The diff coverage is `18.18%`.
> :exclamation: Current head 825eac1 differs from pull request most recent head 149e21a. Consider uploading reports for the commit 149e21a to get more accurate results
```diff
@@ Coverage Diff @@
## master #25798 +/- ##
==========================================
- Coverage 72.81% 72.72% -0.10%
==========================================
Files 775 777 +2
Lines 102928 103079 +151
==========================================
+ Hits 74946 74962 +16
- Misses 26527 26662 +135
Partials 1455 1455
```
| Flag | Coverage Δ | |
|---|---|---|
| python | `81.80% <18.18%> (-0.16%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/25798?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...apache\_beam/typehints/native\_type\_compatibility.py](https://codecov.io/gh/apache/beam/pull/25798?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL25hdGl2ZV90eXBlX2NvbXBhdGliaWxpdHkucHk=) | `78.73% <18.18%> (-4.09%)` | :arrow_down: |
... and [10 files with indirect coverage changes](https://codecov.io/gh/apache/beam/pull/25798/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on pull request #25798: Add support for common collections.abc types
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on PR #25798:
URL: https://github.com/apache/beam/pull/25798#issuecomment-1467129703
Seeing some segfaults in Python 3.10... is this related to this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on pull request #25798: Add support for common collections.abc types
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on PR #25798:
URL: https://github.com/apache/beam/pull/25798#issuecomment-1468260276
The MacOS 3.10 unit test run appears to be segfaulting at head, not this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey merged pull request #25798: Add support for common collections.abc types
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey merged PR #25798:
URL: https://github.com/apache/beam/pull/25798
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on pull request #25798: Add support for common collections.abc types
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on PR #25798:
URL: https://github.com/apache/beam/pull/25798#issuecomment-1466342943
Looks like the mechanic I was leveraging to check if the type was an iterable was new behavior in 3.11, not older versions. The new routing of checking for an `__iter__` attribute should fix that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jrmccluskey commented on pull request #25798: Add support for common collections.abc types
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on PR #25798:
URL: https://github.com/apache/beam/pull/25798#issuecomment-1466507288
Run Python_Integration PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on a diff in pull request #25798: Add support for common collections.abc types
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on code in PR #25798:
URL: https://github.com/apache/beam/pull/25798#discussion_r1134666381
##########
sdks/python/apache_beam/typehints/native_type_compatibility.py:
##########
@@ -187,6 +187,26 @@ def convert_builtin_to_typing(typ):
return typ
+def convert_collections_to_typing(typ):
+ """Converts a given collections.abc type to a typing object.
+
+ Args:
+ typ: an object inheriting from a collections.abc object
+
+ Returns:
+ type: The corresponding typing object.
+ """
+ if hasattr(typ, '__iter__'):
+ print("here")
Review Comment:
leftover
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org