You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2020/09/24 09:37:02 UTC
[GitHub] [beam] kamilwu opened a new pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
kamilwu opened a new pull request #12927:
URL: https://github.com/apache/beam/pull/12927
Dependencies updated:
- fastavro
- cachetools
- oauth2client
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2
--- | --- | --- | --- | --- | --- | ---
Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | ---
Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/)
Python | [](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/) | ---
XLang | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/) | ---
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
--- |Java | Python | Go | Website | Whitespace | Typescript
--- | --- | --- | --- | --- | --- | ---
Non-portable | [](https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/)<br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/) <br>[](https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/)
Portable | --- | [](https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/) | --- | --- | --- | ---
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698241541
# [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=h1) Report
> Merging [#12927](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d48a4d6294a0a5db5ceae3ba79376abe412d0103?el=desc) will **increase** coverage by `41.74%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12927 +/- ##
===========================================
+ Coverage 40.23% 81.98% +41.74%
===========================================
Files 455 461 +6
Lines 53731 54913 +1182
===========================================
+ Hits 21619 45019 +23400
+ Misses 32112 9894 -22218
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/avroio.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXZyb2lvLnB5) | `61.76% <ø> (+28.43%)` | :arrow_up: |
| [setup.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc25vd2ZsYWtlLnB5) | `64.15% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/portability/python\_urns.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvcHl0aG9uX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_util.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya191dGlsLnB5) | `0.00% <0.00%> (ø)` | |
| [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
| [...ache\_beam/runners/interactive/pipeline\_analyzer.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9hbmFseXplci5weQ==) | | |
| [sdks/python/apache\_beam/io/external/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZXh0ZXJuYWwvc25vd2ZsYWtlLnB5) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/dataframe/io.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2lvLnB5) | `92.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_perf.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19wZXJmLnB5) | `0.00% <0.00%> (ø)` | |
| ... and [284 more](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=footer). Last update [d48a4d6...bf6cc32](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698241541
# [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=h1) Report
> Merging [#12927](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d48a4d6294a0a5db5ceae3ba79376abe412d0103?el=desc) will **increase** coverage by `41.75%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12927 +/- ##
===========================================
+ Coverage 40.23% 81.99% +41.75%
===========================================
Files 455 461 +6
Lines 53731 54970 +1239
===========================================
+ Hits 21619 45071 +23452
+ Misses 32112 9899 -22213
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [setup.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc25vd2ZsYWtlLnB5) | `64.15% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/portability/python\_urns.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvcHl0aG9uX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_util.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya191dGlsLnB5) | `0.00% <0.00%> (ø)` | |
| [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
| [...ache\_beam/runners/interactive/pipeline\_analyzer.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9hbmFseXplci5weQ==) | | |
| [...python/apache\_beam/examples/wordcount\_dataframe.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X2RhdGFmcmFtZS5weQ==) | `92.59% <0.00%> (ø)` | |
| [...beam/testing/benchmarks/nexmark/queries/query10.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTEwLnB5) | `0.00% <0.00%> (ø)` | |
| [...\_beam/runners/portability/sdk\_container\_builder.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9zZGtfY29udGFpbmVyX2J1aWxkZXIucHk=) | `31.21% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/external/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZXh0ZXJuYWwvc25vd2ZsYWtlLnB5) | `0.00% <0.00%> (ø)` | |
| ... and [283 more](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=footer). Last update [d48a4d6...d466a4a](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698234783
Run Python 3.8 PostCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on a change in pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
kamilwu commented on a change in pull request #12927:
URL: https://github.com/apache/beam/pull/12927#discussion_r496712359
##########
File path: sdks/python/apache_beam/io/avroio.py
##########
@@ -627,11 +627,19 @@ def write_record(self, writer, value):
writer.append(value)
+class _FastAvroWriter(Writer):
+ """An adapter class which exposes a file handle so that it can be closed
+ by the sink. """
+ def __init__(self, file_handle, schema, codec):
+ super(_FastAvroWriter, self).__init__(file_handle, schema, codec)
Review comment:
It turned out this change is unnecessary -- I'll remove it then.
The problem was with pyenv and Python interpreter on my machine. It lacked libpython.so. As a result, I couldn't import cythonized code. Instead, a pure Python version, `_write_py.py`, was being used: https://github.com/fastavro/fastavro/blob/master/fastavro/write.py#L3. That pure Python version has some differences, but most importantly, the `Writer` class doesn't set `self.fo` field:
https://github.com/fastavro/fastavro/blob/master/fastavro/_write_py.py#L411 (pure Python)
vs.
https://github.com/fastavro/fastavro/blob/master/fastavro/_write.pyx#L559 (cython)
I wrongly assumed it was the fault of updating the package.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on a change in pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on a change in pull request #12927:
URL: https://github.com/apache/beam/pull/12927#discussion_r498461732
##########
File path: sdks/python/apache_beam/io/avroio.py
##########
@@ -627,11 +627,19 @@ def write_record(self, writer, value):
writer.append(value)
+class _FastAvroWriter(Writer):
+ """An adapter class which exposes a file handle so that it can be closed
+ by the sink. """
+ def __init__(self, file_handle, schema, codec):
+ super(_FastAvroWriter, self).__init__(file_handle, schema, codec)
Review comment:
Ah that's right. Cython presence only matters if we are installing fastavro from sources.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698241541
# [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=h1) Report
> Merging [#12927](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d48a4d6294a0a5db5ceae3ba79376abe412d0103?el=desc) will **increase** coverage by `41.74%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12927 +/- ##
===========================================
+ Coverage 40.23% 81.98% +41.74%
===========================================
Files 455 461 +6
Lines 53731 54913 +1182
===========================================
+ Hits 21619 45019 +23400
+ Misses 32112 9894 -22218
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/avroio.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXZyb2lvLnB5) | `61.76% <ø> (+28.43%)` | :arrow_up: |
| [setup.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc25vd2ZsYWtlLnB5) | `64.15% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/portability/python\_urns.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvcHl0aG9uX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_util.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya191dGlsLnB5) | `0.00% <0.00%> (ø)` | |
| [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
| [...ache\_beam/runners/interactive/pipeline\_analyzer.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9hbmFseXplci5weQ==) | | |
| [sdks/python/apache\_beam/dataframe/io.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2lvLnB5) | `92.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_perf.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19wZXJmLnB5) | `0.00% <0.00%> (ø)` | |
| [...\_beam/runners/portability/sdk\_container\_builder.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9zZGtfY29udGFpbmVyX2J1aWxkZXIucHk=) | `31.21% <0.00%> (ø)` | |
| ... and [284 more](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=footer). Last update [d48a4d6...bf6cc32](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on a change in pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on a change in pull request #12927:
URL: https://github.com/apache/beam/pull/12927#discussion_r496100904
##########
File path: sdks/python/setup.py
##########
@@ -141,7 +141,7 @@ def get_version():
# server, therefore list of allowed versions is very narrow.
# See: https://github.com/uqfoundation/dill/issues/341.
'dill>=0.3.1.1,<0.3.2',
- 'fastavro>=0.21.4,<0.24',
+ 'fastavro>=1.0.0,<2',
Review comment:
Do we need to increase the lower bound (is the change not compatible with earlier fastavro releases)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn merged pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
tvalentyn merged pull request #12927:
URL: https://github.com/apache/beam/pull/12927
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698241541
# [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=h1) Report
> Merging [#12927](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d48a4d6294a0a5db5ceae3ba79376abe412d0103?el=desc) will **increase** coverage by `41.75%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12927 +/- ##
===========================================
+ Coverage 40.23% 81.99% +41.75%
===========================================
Files 455 461 +6
Lines 53731 54970 +1239
===========================================
+ Hits 21619 45071 +23452
+ Misses 32112 9899 -22213
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [setup.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc25vd2ZsYWtlLnB5) | `64.15% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/portability/python\_urns.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvcHl0aG9uX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_util.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya191dGlsLnB5) | `0.00% <0.00%> (ø)` | |
| [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
| [...ache\_beam/runners/interactive/pipeline\_analyzer.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9hbmFseXplci5weQ==) | | |
| [sdks/python/apache\_beam/utils/histogram.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaGlzdG9ncmFtLnB5) | `94.28% <0.00%> (ø)` | |
| [...beam/testing/benchmarks/nexmark/queries/query10.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTEwLnB5) | `0.00% <0.00%> (ø)` | |
| [...python/apache\_beam/examples/wordcount\_dataframe.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X2RhdGFmcmFtZS5weQ==) | `92.59% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/external/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZXh0ZXJuYWwvc25vd2ZsYWtlLnB5) | `0.00% <0.00%> (ø)` | |
| ... and [283 more](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=footer). Last update [d48a4d6...d466a4a](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on a change in pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on a change in pull request #12927:
URL: https://github.com/apache/beam/pull/12927#discussion_r497092701
##########
File path: sdks/python/apache_beam/io/avroio.py
##########
@@ -627,11 +627,19 @@ def write_record(self, writer, value):
writer.append(value)
+class _FastAvroWriter(Writer):
+ """An adapter class which exposes a file handle so that it can be closed
+ by the sink. """
+ def __init__(self, file_handle, schema, codec):
+ super(_FastAvroWriter, self).__init__(file_handle, schema, codec)
Review comment:
Actually, it is a change in 0.22.0: https://github.com/fastavro/fastavro/pull/343/commits/6c55c318735278d097b05da7077b23fead50a039#diff-96a4a8649900b3ff9f22a57de25baca4R498
https://github.com/fastavro/fastavro/blob/f468264b0f86661f1893c8b32bc592fb44e39c75/ChangeLog#L91
Cython is an optional dependency in Beam, so we should not have to rely on presence of cython.
It seems strange that fastavro public api is now different in cythonized and non-cythonized version. Are you interested in reaching out to fastavro folks to clarify whether this is something they want to fix?
We explicitly close the filehandle due to: https://issues.apache.org/jira/browse/BEAM-7747. Perhaps there is a better way?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on a change in pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
kamilwu commented on a change in pull request #12927:
URL: https://github.com/apache/beam/pull/12927#discussion_r496669473
##########
File path: sdks/python/setup.py
##########
@@ -141,7 +141,7 @@ def get_version():
# server, therefore list of allowed versions is very narrow.
# See: https://github.com/uqfoundation/dill/issues/341.
'dill>=0.3.1.1,<0.3.2',
- 'fastavro>=0.21.4,<0.24',
+ 'fastavro>=1.0.0,<2',
Review comment:
Not really. I'll restore the previous bound
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on a change in pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on a change in pull request #12927:
URL: https://github.com/apache/beam/pull/12927#discussion_r497096331
##########
File path: sdks/python/apache_beam/io/avroio.py
##########
@@ -627,11 +627,19 @@ def write_record(self, writer, value):
writer.append(value)
+class _FastAvroWriter(Writer):
+ """An adapter class which exposes a file handle so that it can be closed
+ by the sink. """
+ def __init__(self, file_handle, schema, codec):
+ super(_FastAvroWriter, self).__init__(file_handle, schema, codec)
Review comment:
Asked on https://github.com/fastavro/fastavro/pull/343#discussion_r497095926
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-699889741
R: @tvalentyn
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on a change in pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
kamilwu commented on a change in pull request #12927:
URL: https://github.com/apache/beam/pull/12927#discussion_r497520564
##########
File path: sdks/python/apache_beam/io/avroio.py
##########
@@ -627,11 +627,19 @@ def write_record(self, writer, value):
writer.append(value)
+class _FastAvroWriter(Writer):
+ """An adapter class which exposes a file handle so that it can be closed
+ by the sink. """
+ def __init__(self, file_handle, schema, codec):
+ super(_FastAvroWriter, self).__init__(file_handle, schema, codec)
Review comment:
Thanks. We've already got an answer, let's wait until we know if it's possible for the `fo` attribute to become available.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698241541
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698291447
R: @chamikaramj
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on a change in pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
kamilwu commented on a change in pull request #12927:
URL: https://github.com/apache/beam/pull/12927#discussion_r498217598
##########
File path: sdks/python/apache_beam/io/avroio.py
##########
@@ -627,11 +627,19 @@ def write_record(self, writer, value):
writer.append(value)
+class _FastAvroWriter(Writer):
+ """An adapter class which exposes a file handle so that it can be closed
+ by the sink. """
+ def __init__(self, file_handle, schema, codec):
+ super(_FastAvroWriter, self).__init__(file_handle, schema, codec)
Review comment:
> Was it a unit test failure that occurred locally?
Yes.
> I wonder why our non-cython test suite does not catch it.
I think it doesn't really depend on whether cython is installed or not. `pip install fastavro` downloads a wheel built on the specific platform, which already contains libraries that can be imported. For example, on my Linux machine:
```
pip install --force-reinstall fastavro==0.23.6
Collecting fastavro==0.23.6
Using cached fastavro-0.23.6-cp37-cp37m-manylinux2010_x86_64.whl (1.4 MB)
```
```
ls site-packages/fastavro | grep write
json_write.py
_logical_writers.cpython-37m-x86_64-linux-gnu.so
logical_writers.py
_logical_writers_py.py
_write.cpython-37m-x86_64-linux-gnu.so <-- this is the library I'm talking about
write.py
_write_py.py
```
I'm pretty sure the same happens on Jenkins and on our users' machines. So how did I catch the bug? That `.so` library links with `libpython3.x.so`:
```
ldd site-packages/fastavro/_write.cpython-37m-x86_64-linux-gnu.so
linux-vdso.so.1 (0x00007ffd7abf7000)
libpython3.7m.so.1.0 => not found
libc.so.6 => /lib64/libc.so.6 (0x00007f8c548e3000)
/lib64/ld-linux-x86-64.so.2 (0x00007f8c54d07000)
```
which I didn't have on my machine due to misconfiguration (I already know it's pyenv fault, but this is a different story). That's why the first `import` statement failed, `ImportError` was raised, and pure Python version was used instead.
To sum up, it is a problem, but the chances that someone will encounter this are pretty low. Anyway, I think it still makes sense to file a JIRA and test the workaround that you proposed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on a change in pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on a change in pull request #12927:
URL: https://github.com/apache/beam/pull/12927#discussion_r497830780
##########
File path: sdks/python/apache_beam/io/avroio.py
##########
@@ -627,11 +627,19 @@ def write_record(self, writer, value):
writer.append(value)
+class _FastAvroWriter(Writer):
+ """An adapter class which exposes a file handle so that it can be closed
+ by the sink. """
+ def __init__(self, file_handle, schema, codec):
+ super(_FastAvroWriter, self).__init__(file_handle, schema, codec)
Review comment:
Let's do the following:
1. File a JIRA for this issue, mention in the Jira the error message that you observed. Let's mention in the JIRA the workaround (install cython or downgrade fastavro to 0.21.24) in case users discover this error.
2. Add a workaround in code:
```
try:
writer.fo.close()
except AttributeError:
# We can clean this up after fastavro lower bound includes a fix for https://github.com/fastavro/fastavro/issues/472.
writer.encoder.close()
```
3. Test that the workaround works.
Thanks for spotting this. How did you catch it by the way? Was it a unit test failure that occurred locally? I wonder why our non-cython test suite does not catch it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698241541
# [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=h1) Report
> Merging [#12927](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d48a4d6294a0a5db5ceae3ba79376abe412d0103?el=desc) will **increase** coverage by `41.74%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12927 +/- ##
===========================================
+ Coverage 40.23% 81.98% +41.74%
===========================================
Files 455 461 +6
Lines 53731 54913 +1182
===========================================
+ Hits 21619 45019 +23400
+ Misses 32112 9894 -22218
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/avroio.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXZyb2lvLnB5) | `61.76% <ø> (+28.43%)` | :arrow_up: |
| [setup.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc25vd2ZsYWtlLnB5) | `64.15% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/portability/python\_urns.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvcHl0aG9uX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_util.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya191dGlsLnB5) | `0.00% <0.00%> (ø)` | |
| [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
| [...ache\_beam/runners/interactive/pipeline\_analyzer.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9hbmFseXplci5weQ==) | | |
| [...beam/testing/benchmarks/nexmark/queries/query10.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTEwLnB5) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/external/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZXh0ZXJuYWwvc25vd2ZsYWtlLnB5) | `0.00% <0.00%> (ø)` | |
| [...\_beam/runners/portability/sdk\_container\_builder.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9zZGtfY29udGFpbmVyX2J1aWxkZXIucHk=) | `31.21% <0.00%> (ø)` | |
| ... and [284 more](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=footer). Last update [d48a4d6...bf6cc32](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698241541
# [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=h1) Report
> Merging [#12927](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d48a4d6294a0a5db5ceae3ba79376abe412d0103?el=desc) will **increase** coverage by `41.74%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12927 +/- ##
===========================================
+ Coverage 40.23% 81.98% +41.74%
===========================================
Files 455 461 +6
Lines 53731 54913 +1182
===========================================
+ Hits 21619 45019 +23400
+ Misses 32112 9894 -22218
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/avroio.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXZyb2lvLnB5) | `61.76% <ø> (+28.43%)` | :arrow_up: |
| [setup.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc25vd2ZsYWtlLnB5) | `64.15% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/portability/python\_urns.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvcHl0aG9uX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_util.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya191dGlsLnB5) | `0.00% <0.00%> (ø)` | |
| [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
| [...ache\_beam/runners/interactive/pipeline\_analyzer.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9hbmFseXplci5weQ==) | | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_perf.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19wZXJmLnB5) | `0.00% <0.00%> (ø)` | |
| [...\_beam/runners/portability/sdk\_container\_builder.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9zZGtfY29udGFpbmVyX2J1aWxkZXIucHk=) | `31.21% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/histogram.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaGlzdG9ncmFtLnB5) | `94.28% <0.00%> (ø)` | |
| ... and [284 more](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=footer). Last update [d48a4d6...bf6cc32](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698241541
# [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=h1) Report
> Merging [#12927](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d48a4d6294a0a5db5ceae3ba79376abe412d0103?el=desc) will **increase** coverage by `41.74%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12927 +/- ##
===========================================
+ Coverage 40.23% 81.98% +41.74%
===========================================
Files 455 461 +6
Lines 53731 54913 +1182
===========================================
+ Hits 21619 45019 +23400
+ Misses 32112 9894 -22218
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/avroio.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXZyb2lvLnB5) | `61.76% <ø> (+28.43%)` | :arrow_up: |
| [setup.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc25vd2ZsYWtlLnB5) | `64.15% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/portability/python\_urns.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvcHl0aG9uX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_util.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya191dGlsLnB5) | `0.00% <0.00%> (ø)` | |
| [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
| [...ache\_beam/runners/interactive/pipeline\_analyzer.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9hbmFseXplci5weQ==) | | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_perf.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19wZXJmLnB5) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/histogram.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaGlzdG9ncmFtLnB5) | `94.28% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/external/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZXh0ZXJuYWwvc25vd2ZsYWtlLnB5) | `0.00% <0.00%> (ø)` | |
| ... and [284 more](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=footer). Last update [d48a4d6...d466a4a](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698241541
# [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=h1) Report
> Merging [#12927](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d48a4d6294a0a5db5ceae3ba79376abe412d0103?el=desc) will **increase** coverage by `41.75%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12927 +/- ##
===========================================
+ Coverage 40.23% 81.99% +41.75%
===========================================
Files 455 461 +6
Lines 53731 54970 +1239
===========================================
+ Hits 21619 45071 +23452
+ Misses 32112 9899 -22213
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [setup.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc25vd2ZsYWtlLnB5) | `64.15% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/portability/python\_urns.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvcHl0aG9uX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_util.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya191dGlsLnB5) | `0.00% <0.00%> (ø)` | |
| [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
| [...ache\_beam/runners/interactive/pipeline\_analyzer.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9hbmFseXplci5weQ==) | | |
| [...beam/testing/benchmarks/nexmark/queries/query10.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTEwLnB5) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/external/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZXh0ZXJuYWwvc25vd2ZsYWtlLnB5) | `0.00% <0.00%> (ø)` | |
| [...\_beam/runners/portability/sdk\_container\_builder.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9zZGtfY29udGFpbmVyX2J1aWxkZXIucHk=) | `31.21% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/histogram.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaGlzdG9ncmFtLnB5) | `94.28% <0.00%> (ø)` | |
| ... and [283 more](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=footer). Last update [d48a4d6...d466a4a](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698241541
# [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=h1) Report
> Merging [#12927](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d48a4d6294a0a5db5ceae3ba79376abe412d0103?el=desc) will **increase** coverage by `41.75%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12927 +/- ##
===========================================
+ Coverage 40.23% 81.99% +41.75%
===========================================
Files 455 461 +6
Lines 53731 54970 +1239
===========================================
+ Hits 21619 45071 +23452
+ Misses 32112 9899 -22213
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [setup.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc25vd2ZsYWtlLnB5) | `64.15% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/portability/python\_urns.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvcHl0aG9uX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_util.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya191dGlsLnB5) | `0.00% <0.00%> (ø)` | |
| [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
| [...ache\_beam/runners/interactive/pipeline\_analyzer.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9hbmFseXplci5weQ==) | | |
| [sdks/python/apache\_beam/dataframe/io.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2lvLnB5) | `92.00% <0.00%> (ø)` | |
| [...python/apache\_beam/examples/wordcount\_dataframe.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X2RhdGFmcmFtZS5weQ==) | `92.59% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/histogram.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaGlzdG9ncmFtLnB5) | `94.28% <0.00%> (ø)` | |
| [...\_beam/runners/portability/sdk\_container\_builder.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9zZGtfY29udGFpbmVyX2J1aWxkZXIucHk=) | `31.21% <0.00%> (ø)` | |
| ... and [283 more](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=footer). Last update [d48a4d6...d466a4a](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698241541
# [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=h1) Report
> Merging [#12927](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d48a4d6294a0a5db5ceae3ba79376abe412d0103?el=desc) will **increase** coverage by `41.74%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12927 +/- ##
===========================================
+ Coverage 40.23% 81.98% +41.74%
===========================================
Files 455 461 +6
Lines 53731 54913 +1182
===========================================
+ Hits 21619 45019 +23400
+ Misses 32112 9894 -22218
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/avroio.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXZyb2lvLnB5) | `61.76% <ø> (+28.43%)` | :arrow_up: |
| [setup.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc25vd2ZsYWtlLnB5) | `64.15% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/portability/python\_urns.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvcHl0aG9uX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_util.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya191dGlsLnB5) | `0.00% <0.00%> (ø)` | |
| [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
| [...ache\_beam/runners/interactive/pipeline\_analyzer.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9hbmFseXplci5weQ==) | | |
| [sdks/python/apache\_beam/dataframe/io.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2lvLnB5) | `92.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_perf.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19wZXJmLnB5) | `0.00% <0.00%> (ø)` | |
| [...\_beam/runners/portability/sdk\_container\_builder.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9zZGtfY29udGFpbmVyX2J1aWxkZXIucHk=) | `31.21% <0.00%> (ø)` | |
| ... and [284 more](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=footer). Last update [d48a4d6...bf6cc32](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on a change in pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on a change in pull request #12927:
URL: https://github.com/apache/beam/pull/12927#discussion_r496099516
##########
File path: sdks/python/apache_beam/io/avroio.py
##########
@@ -627,11 +627,19 @@ def write_record(self, writer, value):
writer.append(value)
+class _FastAvroWriter(Writer):
+ """An adapter class which exposes a file handle so that it can be closed
+ by the sink. """
+ def __init__(self, file_handle, schema, codec):
+ super(_FastAvroWriter, self).__init__(file_handle, schema, codec)
Review comment:
What is the context for this change (do you by chance have a pertaining fastavro commit that changes this behavior)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698241541
# [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=h1) Report
> Merging [#12927](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/d48a4d6294a0a5db5ceae3ba79376abe412d0103?el=desc) will **increase** coverage by `41.74%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #12927 +/- ##
===========================================
+ Coverage 40.23% 81.98% +41.74%
===========================================
Files 455 461 +6
Lines 53731 54913 +1182
===========================================
+ Hits 21619 45019 +23400
+ Misses 32112 9894 -22218
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/io/avroio.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXZyb2lvLnB5) | `61.76% <ø> (+28.43%)` | :arrow_up: |
| [setup.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2V0dXAucHk=) | `0.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc25vd2ZsYWtlLnB5) | `64.15% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/portability/python\_urns.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvcHl0aG9uX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_util.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya191dGlsLnB5) | `0.00% <0.00%> (ø)` | |
| [...eam/testing/benchmarks/nexmark/nexmark\_launcher.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19sYXVuY2hlci5weQ==) | `0.00% <0.00%> (ø)` | |
| [...ache\_beam/runners/interactive/pipeline\_analyzer.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9hbmFseXplci5weQ==) | | |
| [sdks/python/apache\_beam/dataframe/io.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2lvLnB5) | `92.00% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/external/snowflake.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZXh0ZXJuYWwvc25vd2ZsYWtlLnB5) | `0.00% <0.00%> (ø)` | |
| [...he\_beam/testing/benchmarks/nexmark/nexmark\_perf.py](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbmV4bWFya19wZXJmLnB5) | `0.00% <0.00%> (ø)` | |
| ... and [284 more](https://codecov.io/gh/apache/beam/pull/12927/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=footer). Last update [d48a4d6...bf6cc32](https://codecov.io/gh/apache/beam/pull/12927?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-698234783
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on a change in pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on a change in pull request #12927:
URL: https://github.com/apache/beam/pull/12927#discussion_r497830780
##########
File path: sdks/python/apache_beam/io/avroio.py
##########
@@ -627,11 +627,19 @@ def write_record(self, writer, value):
writer.append(value)
+class _FastAvroWriter(Writer):
+ """An adapter class which exposes a file handle so that it can be closed
+ by the sink. """
+ def __init__(self, file_handle, schema, codec):
+ super(_FastAvroWriter, self).__init__(file_handle, schema, codec)
Review comment:
Let's do the following:
1. File a JIRA for this issue, mention in the the error message that you observed. Let's mention in the JIRA the workaround (install cython or downgrade fastavro to 0.21.24) in case users discover this error.
2. Add a workaround in code:
```
try:
writer.fo.close()
except AttributeError:
# We can clean this up after fastavro lower bound includes a fix for https://github.com/fastavro/fastavro/issues/472.
writer.encoder.close()
```
3. Test that the workaround works.
Thanks for spotting this. How did you catch it by the way? Was it a unit test failure that occurred locally? I wonder why our non-cython test suite does not catch it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kamilwu commented on pull request #12927: Update some of the Python dependencies from Beam Dependency Check Report
Posted by GitBox <gi...@apache.org>.
kamilwu commented on pull request #12927:
URL: https://github.com/apache/beam/pull/12927#issuecomment-700664756
Run Python 3.8 PostCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org