You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/06/18 20:55:51 UTC
[GitHub] [beam] pabloem opened a new pull request #14690: [BEAM-12445] Moving streaming inserts with new client
pabloem opened a new pull request #14690:
URL: https://github.com/apache/beam/pull/14690
**Please** add a meaningful description for your change here
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
`ValidatesRunner` compliance status (on master branch)
--------------------------------------------------------
<table>
<thead>
<tr>
<th>Lang</th>
<th>ULR</th>
<th>Dataflow</th>
<th>Flink</th>
<th>Samza</th>
<th>Spark</th>
<th>Twister2</th>
</tr>
</thead>
<tbody>
<tr>
<td>Go</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
</a>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
<tr>
<td>Java</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/badge/icon?subject=V1+Streaming">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon?subject=V1+Java+11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/badge/icon?subject=V2+Streaming">
</a><br>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon?subject=Java+8">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon?subject=Java+11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon?subject=Portable+Streaming">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon?subject=Structured+Streaming">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/badge/icon">
</a>
</td>
</tr>
<tr>
<td>Python</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon?subject=ValCont">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
<tr>
<td>XLang</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
</tbody>
</table>
Examples testing status on various runners
--------------------------------------------------------
<table>
<thead>
<tr>
<th>Lang</th>
<th>ULR</th>
<th>Dataflow</th>
<th>Flink</th>
<th>Samza</th>
<th>Spark</th>
<th>Twister2</th>
</tr>
</thead>
<tbody>
<tr>
<td>Go</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>Java</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/badge/icon?subject=V1+Java11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>Python</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>XLang</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
</tbody>
</table>
Post-Commit SDK/Transform Integration Tests Status (on master branch)
------------------------------------------------------------------------------------------------
<table>
<thead>
<tr>
<th>Go</th>
<th>Java</th>
<th>Python</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon?subject=3.6">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon?subject=3.7">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/badge/icon?subject=3.8">
</a>
</td>
</tr>
</tbody>
</table>
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
<table>
<thead>
<tr>
<th>---</th>
<th>Java</th>
<th>Python</th>
<th>Go</th>
<th>Website</th>
<th>Whitespace</th>
<th>Typescript</th>
</tr>
</thead>
<tbody>
<tr>
<td>Non-portable</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon">
</a><br>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon?subject=Tests">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon?subject=Lint">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/badge/icon?subject=Docker">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/badge/icon?subject=Docs">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
</tr>
<tr>
<td>Portable</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
</tbody>
</table>
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (61a332d) into [master](https://codecov.io/gh/apache/beam/commit/000ac07f2883d4708b6abbfc935fd004405aae19?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (000ac07) will **increase** coverage by `1.19%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
+ Coverage 82.55% 83.75% +1.19%
==========================================
Files 455 439 -16
Lines 55143 59104 +3961
==========================================
+ Hits 45526 49502 +3976
+ Misses 9617 9602 -15
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...python/apache\_beam/examples/wordcount\_dataframe.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X2RhdGFmcmFtZS5weQ==) | `0.00% <0.00%> (-92.60%)` | :arrow_down: |
| [...s/python/apache\_beam/examples/snippets/snippets.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvc25pcHBldHMucHk=) | `76.70% <0.00%> (-12.83%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/kafka.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8va2Fma2EucHk=) | `79.16% <0.00%> (-6.55%)` | :arrow_down: |
| [...n/apache\_beam/runners/direct/test\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9kaXJlY3RfcnVubmVyLnB5) | `37.50% <0.00%> (-4.81%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `74.72% <0.00%> (-4.81%)` | :arrow_down: |
| [test\_config.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-dGVzdF9jb25maWcucHk=) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/test/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy90ZXN0L19faW5pdF9fLnB5) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/gcp/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL19faW5pdF9fLnB5) | `80.00% <0.00%> (-4.62%)` | :arrow_down: |
| [...thon/apache\_beam/runners/worker/channel\_factory.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvY2hhbm5lbF9mYWN0b3J5LnB5) | `75.00% <0.00%> (-3.95%)` | :arrow_down: |
| [...thon/apache\_beam/runners/worker/sdk\_worker\_main.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlcl9tYWluLnB5) | `72.26% <0.00%> (-3.66%)` | :arrow_down: |
| ... and [457 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [67518ce...61a332d](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831570231
Run Python PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on a change in pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on a change in pull request #14690:
URL: https://github.com/apache/beam/pull/14690#discussion_r646830329
##########
File path: sdks/python/apache_beam/io/gcp/bigquery_test.py
##########
@@ -809,15 +808,14 @@ def test_dofn_client_process_performs_batching(self):
fn.process(('project_id:dataset_id.table_id', {'month': 1}))
# InsertRows not called as batch size is not hit yet
- self.assertFalse(client.tabledata.InsertAll.called)
+ self.assertFalse(client.insert_rows_json.called)
Review comment:
no. The API formats everything into JSON. Other API calls use automatic JSON conversions.
##########
File path: sdks/python/apache_beam/io/gcp/bigquery.py
##########
@@ -1312,10 +1312,11 @@ def _flush_batch(self, destination):
skip_invalid_rows=True)
self.batch_latency_metric.update((time.time() - start) * 1000)
- failed_rows = [rows[entry.index] for entry in errors]
+ failed_rows = [rows[entry['index']] for entry in errors]
Review comment:
that's right.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (eb3e7a8) into [master](https://codecov.io/gh/apache/beam/commit/30afbcb75ac8eefeaf6e549f2a8b78f619866806?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (30afbcb) will **increase** coverage by `0.17%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
+ Coverage 83.47% 83.65% +0.17%
==========================================
Files 898 440 -458
Lines 117918 58997 -58921
==========================================
- Hits 98433 49354 -49079
+ Misses 19485 9643 -9842
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...n/apache\_beam/runners/direct/test\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RpcmVjdC90ZXN0X2RpcmVjdF9ydW5uZXIucHk=) | `37.50% <0.00%> (-4.81%)` | :arrow_down: |
| [...s/sdks/python/apache\_beam/runners/test/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3Rlc3QvX19pbml0X18ucHk=) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [...ache\_beam/runners/dataflow/test\_dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L3Rlc3RfZGF0YWZsb3dfcnVubmVyLnB5) | `32.65% <0.00%> (-2.65%)` | :arrow_down: |
| [...ild/srcs/sdks/python/apache\_beam/transforms/sql.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90cmFuc2Zvcm1zL3NxbC5weQ==) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
| [...ers/dataflow/internal/clients/dataflow/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvZGF0YWZsb3cvX19pbml0X18ucHk=) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [...s/dataflow/internal/clients/cloudbuild/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvY2xvdWRidWlsZC9fX2luaXRfXy5weQ==) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [.../srcs/sdks/python/apache\_beam/runners/job/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2pvYi91dGlscy5weQ==) | `69.23% <0.00%> (-2.20%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/spark\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L3NwYXJrX3J1bm5lci5weQ==) | `53.65% <0.00%> (-2.16%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/flink\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L2ZsaW5rX3J1bm5lci5weQ==) | `59.61% <0.00%> (-1.50%)` | :arrow_down: |
| [.../build/srcs/sdks/python/apache\_beam/tools/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90b29scy91dGlscy5weQ==) | `67.74% <0.00%> (-1.49%)` | :arrow_down: |
| ... and [656 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [43fba5f...eb3e7a8](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem merged pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem merged pull request #14690:
URL: https://github.com/apache/beam/pull/14690
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-865306322
hoping this will improve performance significantly. We may still need to make improvements, but we'll see : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-864291983
Run Python PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94f2401) into [master](https://codecov.io/gh/apache/beam/commit/d1042c0e23626e76174d775d077e988463c8c20d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d1042c0) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
- Coverage 83.66% 83.66% -0.01%
==========================================
Files 440 440
Lines 59053 59047 -6
==========================================
- Hits 49406 49400 -6
Misses 9647 9647
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...srcs/sdks/python/apache\_beam/transforms/display.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9kaXNwbGF5LnB5) | | |
| [.../srcs/sdks/python/apache\_beam/transforms/window.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy93aW5kb3cucHk=) | | |
| [...rcs/sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | | |
| [...he\_beam/portability/api/external\_transforms\_pb2.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2V4dGVybmFsX3RyYW5zZm9ybXNfcGIyLnB5) | | |
| [.../runners/interactive/testing/pipeline\_assertion.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS90ZXN0aW5nL3BpcGVsaW5lX2Fzc2VydGlvbi5weQ==) | | |
| [...eam/runners/interactive/caching/streaming\_cache.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9jYWNoaW5nL3N0cmVhbWluZ19jYWNoZS5weQ==) | | |
| [...ild/srcs/sdks/python/apache\_beam/utils/sentinel.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc2VudGluZWwucHk=) | | |
| [...e\_beam/io/gcp/big\_query\_query\_to\_table\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ19xdWVyeV9xdWVyeV90b190YWJsZV9waXBlbGluZS5weQ==) | | |
| [...ld/srcs/sdks/python/apache\_beam/transforms/core.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9jb3JlLnB5) | | |
| [...beam/portability/api/beam\_artifact\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fYXJ0aWZhY3RfYXBpX3BiMl9ncnBjLnB5) | | |
| ... and [870 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [db62cc7...94f2401](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94f2401) into [master](https://codecov.io/gh/apache/beam/commit/d1042c0e23626e76174d775d077e988463c8c20d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d1042c0) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
- Coverage 83.66% 83.66% -0.01%
==========================================
Files 440 440
Lines 59053 59047 -6
==========================================
- Hits 49406 49400 -6
Misses 9647 9647
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...e\_beam/portability/api/beam\_runner\_api\_pb2\_urns.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcnVubmVyX2FwaV9wYjJfdXJucy5weQ==) | | |
| [...s/python/apache\_beam/utils/thread\_pool\_executor.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvdGhyZWFkX3Bvb2xfZXhlY3V0b3IucHk=) | | |
| [...cs/sdks/python/apache\_beam/runners/direct/clock.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvY2xvY2sucHk=) | | |
| [...ks/python/apache\_beam/runners/worker/opcounters.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvb3Bjb3VudGVycy5weQ==) | | |
| [...beam/testing/load\_tests/load\_test\_metrics\_utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9sb2FkX3Rlc3RzL2xvYWRfdGVzdF9tZXRyaWNzX3V0aWxzLnB5) | | |
| [...beam/runners/portability/local\_job\_service\_main.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9sb2NhbF9qb2Jfc2VydmljZV9tYWluLnB5) | | |
| [...cs/sdks/python/apache\_beam/typehints/decorators.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL2RlY29yYXRvcnMucHk=) | | |
| [.../examples/snippets/transforms/elementwise/regex.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9lbGVtZW50d2lzZS9yZWdleC5weQ==) | | |
| [.../srcs/sdks/python/apache\_beam/dataframe/convert.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2NvbnZlcnQucHk=) | | |
| [...nners/direct/consumer\_tracking\_pipeline\_visitor.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvY29uc3VtZXJfdHJhY2tpbmdfcGlwZWxpbmVfdmlzaXRvci5weQ==) | | |
| ... and [870 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [db62cc7...94f2401](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0013482) into [master](https://codecov.io/gh/apache/beam/commit/30afbcb75ac8eefeaf6e549f2a8b78f619866806?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (30afbcb) will **increase** coverage by `0.13%`.
> The diff coverage is `n/a`.
> :exclamation: Current head 0013482 differs from pull request most recent head eb3e7a8. Consider uploading reports for the commit eb3e7a8 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
+ Coverage 83.47% 83.61% +0.13%
==========================================
Files 898 442 -456
Lines 117918 59249 -58669
==========================================
- Hits 98433 49540 -48893
+ Misses 19485 9709 -9776
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...n/apache\_beam/runners/direct/test\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RpcmVjdC90ZXN0X2RpcmVjdF9ydW5uZXIucHk=) | `37.50% <0.00%> (-4.81%)` | :arrow_down: |
| [...s/sdks/python/apache\_beam/runners/test/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3Rlc3QvX19pbml0X18ucHk=) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [...ache\_beam/runners/dataflow/test\_dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L3Rlc3RfZGF0YWZsb3dfcnVubmVyLnB5) | `32.65% <0.00%> (-2.65%)` | :arrow_down: |
| [...ild/srcs/sdks/python/apache\_beam/transforms/sql.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90cmFuc2Zvcm1zL3NxbC5weQ==) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
| [...ers/dataflow/internal/clients/dataflow/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvZGF0YWZsb3cvX19pbml0X18ucHk=) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [...s/dataflow/internal/clients/cloudbuild/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvY2xvdWRidWlsZC9fX2luaXRfXy5weQ==) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [.../srcs/sdks/python/apache\_beam/runners/job/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2pvYi91dGlscy5weQ==) | `69.23% <0.00%> (-2.20%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/spark\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L3NwYXJrX3J1bm5lci5weQ==) | `53.65% <0.00%> (-2.16%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/flink\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L2ZsaW5rX3J1bm5lci5weQ==) | `59.61% <0.00%> (-1.50%)` | :arrow_down: |
| [.../build/srcs/sdks/python/apache\_beam/tools/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90b29scy91dGlscy5weQ==) | `67.74% <0.00%> (-1.49%)` | :arrow_down: |
| ... and [658 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [43fba5f...eb3e7a8](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-833870675
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] quentin-sommer edited a comment on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
quentin-sommer edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-865047288
Thank you so much for this ❤️ I was about to write a ticket mentioning the poor performance of the internal bq client
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-857155082
Run Python 3.8 PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-832106066
r: @chamikaramj
cc: @ihji
This change switches the streaming inserts client to the newer google.cloud.bigquery library. On two quick benchmark runs, I was able to double the throughput by making this change.
A shortcoming of this change is that we lose some metadata being passed in the API requests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831513141
Run Python 3.8 PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (eb3e7a8) into [master](https://codecov.io/gh/apache/beam/commit/30afbcb75ac8eefeaf6e549f2a8b78f619866806?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (30afbcb) will **increase** coverage by `0.17%`.
> The diff coverage is `n/a`.
> :exclamation: Current head eb3e7a8 differs from pull request most recent head 060cc18. Consider uploading reports for the commit 060cc18 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
+ Coverage 83.47% 83.65% +0.17%
==========================================
Files 898 440 -458
Lines 117918 58997 -58921
==========================================
- Hits 98433 49354 -49079
+ Misses 19485 9643 -9842
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...n/apache\_beam/runners/direct/test\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RpcmVjdC90ZXN0X2RpcmVjdF9ydW5uZXIucHk=) | `37.50% <0.00%> (-4.81%)` | :arrow_down: |
| [...s/sdks/python/apache\_beam/runners/test/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3Rlc3QvX19pbml0X18ucHk=) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [...ache\_beam/runners/dataflow/test\_dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L3Rlc3RfZGF0YWZsb3dfcnVubmVyLnB5) | `32.65% <0.00%> (-2.65%)` | :arrow_down: |
| [...ild/srcs/sdks/python/apache\_beam/transforms/sql.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90cmFuc2Zvcm1zL3NxbC5weQ==) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
| [...ers/dataflow/internal/clients/dataflow/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvZGF0YWZsb3cvX19pbml0X18ucHk=) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [...s/dataflow/internal/clients/cloudbuild/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvY2xvdWRidWlsZC9fX2luaXRfXy5weQ==) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [.../srcs/sdks/python/apache\_beam/runners/job/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2pvYi91dGlscy5weQ==) | `69.23% <0.00%> (-2.20%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/spark\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L3NwYXJrX3J1bm5lci5weQ==) | `53.65% <0.00%> (-2.16%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/flink\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L2ZsaW5rX3J1bm5lci5weQ==) | `59.61% <0.00%> (-1.50%)` | :arrow_down: |
| [.../build/srcs/sdks/python/apache\_beam/tools/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90b29scy91dGlscy5weQ==) | `67.74% <0.00%> (-1.49%)` | :arrow_down: |
| ... and [656 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [43fba5f...060cc18](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] quentin-sommer commented on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
quentin-sommer commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-865047288
Thank you so much for this ❤️ I was about to write a ticket mentioning the poor performance of the internal client
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0013482) into [master](https://codecov.io/gh/apache/beam/commit/30afbcb75ac8eefeaf6e549f2a8b78f619866806?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (30afbcb) will **increase** coverage by `0.13%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
+ Coverage 83.47% 83.61% +0.13%
==========================================
Files 898 442 -456
Lines 117918 59249 -58669
==========================================
- Hits 98433 49540 -48893
+ Misses 19485 9709 -9776
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...n/apache\_beam/runners/direct/test\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RpcmVjdC90ZXN0X2RpcmVjdF9ydW5uZXIucHk=) | `37.50% <0.00%> (-4.81%)` | :arrow_down: |
| [...s/sdks/python/apache\_beam/runners/test/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3Rlc3QvX19pbml0X18ucHk=) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [...ache\_beam/runners/dataflow/test\_dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L3Rlc3RfZGF0YWZsb3dfcnVubmVyLnB5) | `32.65% <0.00%> (-2.65%)` | :arrow_down: |
| [...ild/srcs/sdks/python/apache\_beam/transforms/sql.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90cmFuc2Zvcm1zL3NxbC5weQ==) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
| [...ers/dataflow/internal/clients/dataflow/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvZGF0YWZsb3cvX19pbml0X18ucHk=) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [...s/dataflow/internal/clients/cloudbuild/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvY2xvdWRidWlsZC9fX2luaXRfXy5weQ==) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [.../srcs/sdks/python/apache\_beam/runners/job/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2pvYi91dGlscy5weQ==) | `69.23% <0.00%> (-2.20%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/spark\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L3NwYXJrX3J1bm5lci5weQ==) | `53.65% <0.00%> (-2.16%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/flink\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L2ZsaW5rX3J1bm5lci5weQ==) | `59.61% <0.00%> (-1.50%)` | :arrow_down: |
| [.../build/srcs/sdks/python/apache\_beam/tools/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90b29scy91dGlscy5weQ==) | `67.74% <0.00%> (-1.49%)` | :arrow_down: |
| ... and [658 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [43fba5f...0013482](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-857162574
@chamikaramj PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] quentin-sommer commented on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
quentin-sommer commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-865047288
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on a change in pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on a change in pull request #14690:
URL: https://github.com/apache/beam/pull/14690#discussion_r646831126
##########
File path: sdks/python/apache_beam/io/gcp/bigquery_test.py
##########
@@ -950,12 +943,10 @@ def store_callback(arg):
with open(file_name_2, 'w') as f:
json.dump(json_output, f)
- res = mock.Mock()
- res.insertErrors = []
Review comment:
because `insert_rows_json` returns a list of errors (empty when no errors), and `client.tabledata.InsertAll` returns an object that contains the list of errors within it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #14690: Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #14690:
URL: https://github.com/apache/beam/pull/14690#discussion_r640771695
##########
File path: sdks/python/apache_beam/io/gcp/bigquery.py
##########
@@ -1312,10 +1312,11 @@ def _flush_batch(self, destination):
skip_invalid_rows=True)
self.batch_latency_metric.update((time.time() - start) * 1000)
- failed_rows = [rows[entry.index] for entry in errors]
+ failed_rows = [rows[entry['index']] for entry in errors]
Review comment:
Is this due to the API change ?
##########
File path: sdks/python/apache_beam/io/gcp/bigquery_test.py
##########
@@ -950,12 +943,10 @@ def store_callback(arg):
with open(file_name_2, 'w') as f:
json.dump(json_output, f)
- res = mock.Mock()
- res.insertErrors = []
Review comment:
Why the change to return value ?
##########
File path: sdks/python/apache_beam/io/gcp/bigquery_test.py
##########
@@ -809,15 +808,14 @@ def test_dofn_client_process_performs_batching(self):
fn.process(('project_id:dataset_id.table_id', {'month': 1}))
# InsertRows not called as batch size is not hit yet
- self.assertFalse(client.tabledata.InsertAll.called)
+ self.assertFalse(client.insert_rows_json.called)
Review comment:
Do you know if the API support other formats (for example, Avro) that are more efficient ?
##########
File path: sdks/python/apache_beam/io/gcp/bigquery_tools.py
##########
@@ -632,22 +639,21 @@ def _insert_all_rows(
base_labels=labels)
started_millis = int(time.time() * 1000)
- response = None
try:
- response = self.client.tabledata.InsertAll(request)
- if not response.insertErrors:
+ table_ref = gcp_bigquery.DatasetReference(project_id,
+ dataset_id).table(table_id)
+ errors = self.gcp_bq_client.insert_rows_json(
+ table_ref, json_rows=rows, row_ids=insert_ids, skip_invalid_rows=True)
+ if not errors:
service_call_metric.call('ok')
- for insert_error in response.insertErrors:
- for error in insert_error.errors:
- service_call_metric.call(error.reason)
+ for insert_error in errors:
+ service_call_metric.call(insert_error['errors'][0])
except HttpError as e:
Review comment:
You probably have to rebase here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e1386ce) into [master](https://codecov.io/gh/apache/beam/commit/1ee9a42ce20097179a9bd513b941264594005243?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1ee9a42) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
- Coverage 83.67% 83.66% -0.01%
==========================================
Files 440 440
Lines 59053 59047 -6
==========================================
- Hits 49412 49402 -10
- Misses 9641 9645 +4
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...srcs/sdks/python/apache\_beam/metrics/metricbase.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWV0cmljcy9tZXRyaWNiYXNlLnB5) | | |
| [...ild/srcs/sdks/python/apache\_beam/utils/sentinel.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc2VudGluZWwucHk=) | | |
| [...ild/srcs/sdks/python/apache\_beam/transforms/sql.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9zcWwucHk=) | | |
| [...rcs/sdks/python/apache\_beam/testing/test\_stream.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbS5weQ==) | | |
| [...m/portability/api/beam\_interactive\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1faW50ZXJhY3RpdmVfYXBpX3BiMl9ncnBjLnB5) | | |
| [...\_beam/testing/benchmarks/nexmark/queries/query0.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTAucHk=) | | |
| [...d/srcs/sdks/python/apache\_beam/runners/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9fX2luaXRfXy5weQ==) | | |
| [...s/python/apache\_beam/examples/snippets/snippets.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvc25pcHBldHMucHk=) | | |
| [...e\_beam/portability/api/beam\_interactive\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1faW50ZXJhY3RpdmVfYXBpX3BiMi5weQ==) | | |
| [...dks/python/apache\_beam/testing/extra\_assertions.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9leHRyYV9hc3NlcnRpb25zLnB5) | | |
| ... and [870 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8676ec1...e1386ce](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (61a332d) into [master](https://codecov.io/gh/apache/beam/commit/000ac07f2883d4708b6abbfc935fd004405aae19?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (000ac07) will **increase** coverage by `1.19%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
+ Coverage 82.55% 83.75% +1.19%
==========================================
Files 455 439 -16
Lines 55143 59104 +3961
==========================================
+ Hits 45526 49502 +3976
+ Misses 9617 9602 -15
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...python/apache\_beam/examples/wordcount\_dataframe.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X2RhdGFmcmFtZS5weQ==) | `0.00% <0.00%> (-92.60%)` | :arrow_down: |
| [...s/python/apache\_beam/examples/snippets/snippets.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvc25pcHBldHMucHk=) | `76.70% <0.00%> (-12.83%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/kafka.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8va2Fma2EucHk=) | `79.16% <0.00%> (-6.55%)` | :arrow_down: |
| [...n/apache\_beam/runners/direct/test\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9kaXJlY3RfcnVubmVyLnB5) | `37.50% <0.00%> (-4.81%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `74.72% <0.00%> (-4.81%)` | :arrow_down: |
| [test\_config.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-dGVzdF9jb25maWcucHk=) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/test/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy90ZXN0L19faW5pdF9fLnB5) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/gcp/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL19faW5pdF9fLnB5) | `80.00% <0.00%> (-4.62%)` | :arrow_down: |
| [...thon/apache\_beam/runners/worker/channel\_factory.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvY2hhbm5lbF9mYWN0b3J5LnB5) | `75.00% <0.00%> (-3.95%)` | :arrow_down: |
| [...thon/apache\_beam/runners/worker/sdk\_worker\_main.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlcl9tYWluLnB5) | `72.26% <0.00%> (-3.66%)` | :arrow_down: |
| ... and [457 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [67518ce...61a332d](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] quentin-sommer edited a comment on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
quentin-sommer edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-865047288
Thank you so much for this ❤️ I was about to write a ticket mentioning the poor performance of the internal bq client
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (61a332d) into [master](https://codecov.io/gh/apache/beam/commit/000ac07f2883d4708b6abbfc935fd004405aae19?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (000ac07) will **increase** coverage by `1.18%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
+ Coverage 82.55% 83.74% +1.18%
==========================================
Files 455 439 -16
Lines 55143 59104 +3961
==========================================
+ Hits 45526 49497 +3971
+ Misses 9617 9607 -10
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...python/apache\_beam/examples/wordcount\_dataframe.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X2RhdGFmcmFtZS5weQ==) | `0.00% <0.00%> (-92.60%)` | :arrow_down: |
| [...s/python/apache\_beam/examples/snippets/snippets.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvc25pcHBldHMucHk=) | `76.70% <0.00%> (-12.83%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/kafka.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8va2Fma2EucHk=) | `79.16% <0.00%> (-6.55%)` | :arrow_down: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `87.80% <0.00%> (-5.06%)` | :arrow_down: |
| [...n/apache\_beam/runners/direct/test\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9kaXJlY3RfcnVubmVyLnB5) | `37.50% <0.00%> (-4.81%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `74.72% <0.00%> (-4.81%)` | :arrow_down: |
| [test\_config.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-dGVzdF9jb25maWcucHk=) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/test/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy90ZXN0L19faW5pdF9fLnB5) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/gcp/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL19faW5pdF9fLnB5) | `80.00% <0.00%> (-4.62%)` | :arrow_down: |
| [...thon/apache\_beam/runners/worker/channel\_factory.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvY2hhbm5lbF9mYWN0b3J5LnB5) | `75.00% <0.00%> (-3.95%)` | :arrow_down: |
| ... and [457 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [67518ce...61a332d](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] quentin-sommer removed a comment on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
quentin-sommer removed a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-865047288
Thank you so much for this ❤️ I was about to write a ticket mentioning the poor performance of the internal bq client
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (eb3e7a8) into [master](https://codecov.io/gh/apache/beam/commit/1ee9a42ce20097179a9bd513b941264594005243?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1ee9a42) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.
> :exclamation: Current head eb3e7a8 differs from pull request most recent head e1386ce. Consider uploading reports for the commit e1386ce to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
- Coverage 83.67% 83.65% -0.02%
==========================================
Files 440 440
Lines 59053 58997 -56
==========================================
- Hits 49412 49354 -58
- Misses 9641 9643 +2
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...che\_beam/runners/interactive/augmented\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9hdWdtZW50ZWRfcGlwZWxpbmUucHk=) | | |
| [...n/apache\_beam/runners/direct/test\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9kaXJlY3RfcnVubmVyLnB5) | | |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | | |
| [...beam/portability/api/beam\_artifact\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fYXJ0aWZhY3RfYXBpX3BiMl9ncnBjLnB5) | | |
| [...m/runners/portability/spark\_uber\_jar\_job\_server.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9zcGFya191YmVyX2phcl9qb2Jfc2VydmVyLnB5) | | |
| [...m/portability/api/beam\_interactive\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1faW50ZXJhY3RpdmVfYXBpX3BiMl9ncnBjLnB5) | | |
| [...dks/python/apache\_beam/transforms/external\_java.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbF9qYXZhLnB5) | | |
| [...n/apache\_beam/examples/complete/game/user\_score.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvZ2FtZS91c2VyX3Njb3JlLnB5) | | |
| [.../srcs/sdks/python/apache\_beam/runners/sdf\_utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9zZGZfdXRpbHMucHk=) | | |
| [...python/apache\_beam/typehints/typehints\_test\_py3.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3R5cGVoaW50c190ZXN0X3B5My5weQ==) | | |
| ... and [870 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8676ec1...e1386ce](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0013482) into [master](https://codecov.io/gh/apache/beam/commit/30afbcb75ac8eefeaf6e549f2a8b78f619866806?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (30afbcb) will **increase** coverage by `0.13%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
+ Coverage 83.47% 83.61% +0.13%
==========================================
Files 898 442 -456
Lines 117918 59249 -58669
==========================================
- Hits 98433 49540 -48893
+ Misses 19485 9709 -9776
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...n/apache\_beam/runners/direct/test\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RpcmVjdC90ZXN0X2RpcmVjdF9ydW5uZXIucHk=) | `37.50% <0.00%> (-4.81%)` | :arrow_down: |
| [...s/sdks/python/apache\_beam/runners/test/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3Rlc3QvX19pbml0X18ucHk=) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [...ache\_beam/runners/dataflow/test\_dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L3Rlc3RfZGF0YWZsb3dfcnVubmVyLnB5) | `32.65% <0.00%> (-2.65%)` | :arrow_down: |
| [...ild/srcs/sdks/python/apache\_beam/transforms/sql.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90cmFuc2Zvcm1zL3NxbC5weQ==) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
| [...ers/dataflow/internal/clients/dataflow/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvZGF0YWZsb3cvX19pbml0X18ucHk=) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [...s/dataflow/internal/clients/cloudbuild/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvY2xvdWRidWlsZC9fX2luaXRfXy5weQ==) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [.../srcs/sdks/python/apache\_beam/runners/job/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2pvYi91dGlscy5weQ==) | `69.23% <0.00%> (-2.20%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/spark\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L3NwYXJrX3J1bm5lci5weQ==) | `53.65% <0.00%> (-2.16%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/flink\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L2ZsaW5rX3J1bm5lci5weQ==) | `59.61% <0.00%> (-1.50%)` | :arrow_down: |
| [.../build/srcs/sdks/python/apache\_beam/tools/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX1BocmFzZS9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90b29scy91dGlscy5weQ==) | `67.74% <0.00%> (-1.49%)` | :arrow_down: |
| ... and [658 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [43fba5f...0013482](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e1386ce) into [master](https://codecov.io/gh/apache/beam/commit/1ee9a42ce20097179a9bd513b941264594005243?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1ee9a42) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
- Coverage 83.67% 83.66% -0.01%
==========================================
Files 440 440
Lines 59053 59047 -6
==========================================
- Hits 49412 49402 -10
- Misses 9641 9645 +4
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...d/srcs/sdks/python/apache\_beam/internal/pickler.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvcGlja2xlci5weQ==) | | |
| [...amples/snippets/transforms/elementwise/tostring.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9lbGVtZW50d2lzZS90b3N0cmluZy5weQ==) | | |
| [...ers/dataflow/dataflow\_exercise\_metrics\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19leGVyY2lzZV9tZXRyaWNzX3BpcGVsaW5lLnB5) | | |
| [...\_beam/testing/benchmarks/nexmark/queries/query3.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTMucHk=) | | |
| [...build/srcs/sdks/python/apache\_beam/utils/shared.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc2hhcmVkLnB5) | | |
| [...\_beam/testing/benchmarks/nexmark/queries/query1.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTEucHk=) | | |
| [...thon/apache\_beam/testing/metric\_result\_matchers.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9tZXRyaWNfcmVzdWx0X21hdGNoZXJzLnB5) | | |
| [...flow/internal/clients/dataflow/message\_matchers.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9jbGllbnRzL2RhdGFmbG93L21lc3NhZ2VfbWF0Y2hlcnMucHk=) | | |
| [...cs/sdks/python/apache\_beam/utils/windowed\_value.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvd2luZG93ZWRfdmFsdWUucHk=) | | |
| [...ks/python/apache\_beam/internal/metrics/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9fX2luaXRfXy5weQ==) | | |
| ... and [870 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8676ec1...e1386ce](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem closed pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem closed pull request #14690:
URL: https://github.com/apache/beam/pull/14690
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94f2401) into [master](https://codecov.io/gh/apache/beam/commit/a4abc91436eadf78abda304664a3e0f64019ea61?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a4abc91) will **decrease** coverage by `0.09%`.
> The diff coverage is `n/a`.
> :exclamation: Current head 94f2401 differs from pull request most recent head 881eb18. Consider uploading reports for the commit 881eb18 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
- Coverage 83.75% 83.66% -0.10%
==========================================
Files 436 440 +4
Lines 58442 59047 +605
==========================================
+ Hits 48950 49400 +450
- Misses 9492 9647 +155
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...hon/apache\_beam/examples/cookbook/mergecontacts.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29va2Jvb2svbWVyZ2Vjb250YWN0cy5weQ==) | | |
| [...srcs/sdks/python/apache\_beam/transforms/trigger.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy90cmlnZ2VyLnB5) | | |
| [...m/testing/benchmarks/nexmark/models/auction\_bid.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbW9kZWxzL2F1Y3Rpb25fYmlkLnB5) | | |
| [...amples/snippets/transforms/aggregation/distinct.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9kaXN0aW5jdC5weQ==) | | |
| [.../apache\_beam/options/pipeline\_options\_validator.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vb3B0aW9ucy9waXBlbGluZV9vcHRpb25zX3ZhbGlkYXRvci5weQ==) | | |
| [.../srcs/sdks/python/apache\_beam/utils/proto\_utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcHJvdG9fdXRpbHMucHk=) | | |
| [...nippets/transforms/aggregation/groupintobatches.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9ncm91cGludG9iYXRjaGVzLnB5) | | |
| [.../srcs/sdks/python/apache\_beam/transforms/window.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy93aW5kb3cucHk=) | | |
| [...\_beam/testing/benchmarks/nexmark/queries/query5.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTUucHk=) | | |
| [.../apache\_beam/testing/benchmarks/nexmark/monitor.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbW9uaXRvci5weQ==) | | |
| ... and [866 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [3f2351f...881eb18](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] quentin-sommer commented on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
quentin-sommer commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-865049597
Thank you for this! I was about to write a ticket mentioning the poor performance of the internal bq client
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on a change in pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
aaltay commented on a change in pull request #14690:
URL: https://github.com/apache/beam/pull/14690#discussion_r674140743
##########
File path: sdks/python/apache_beam/io/gcp/bigquery_tools.py
##########
@@ -292,6 +301,7 @@ def __init__(self, client=None, temp_dataset_id=None):
http=get_new_http(),
credentials=auth.get_service_credentials(),
response_encoding='utf8')
+ self.gcp_bq_client = client or gcp_bigquery.Client()
Review comment:
Is this a backward incompatible change, because users might pass a different type of a client that does not support `insert_rows_json` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-843370498
@chamikaramj @ihji PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-863582833
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] quentin-sommer removed a comment on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
quentin-sommer removed a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-865047288
Thank you so much for this ❤️ I was about to write a ticket mentioning the poor performance of the internal bq client
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (eb3e7a8) into [master](https://codecov.io/gh/apache/beam/commit/30afbcb75ac8eefeaf6e549f2a8b78f619866806?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (30afbcb) will **increase** coverage by `0.17%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
+ Coverage 83.47% 83.65% +0.17%
==========================================
Files 898 440 -458
Lines 117918 58997 -58921
==========================================
- Hits 98433 49354 -49079
+ Misses 19485 9643 -9842
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...n/apache\_beam/runners/direct/test\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RpcmVjdC90ZXN0X2RpcmVjdF9ydW5uZXIucHk=) | `37.50% <0.00%> (-4.81%)` | :arrow_down: |
| [...s/sdks/python/apache\_beam/runners/test/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3Rlc3QvX19pbml0X18ucHk=) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [...ache\_beam/runners/dataflow/test\_dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L3Rlc3RfZGF0YWZsb3dfcnVubmVyLnB5) | `32.65% <0.00%> (-2.65%)` | :arrow_down: |
| [...ild/srcs/sdks/python/apache\_beam/transforms/sql.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90cmFuc2Zvcm1zL3NxbC5weQ==) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
| [...ers/dataflow/internal/clients/dataflow/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvZGF0YWZsb3cvX19pbml0X18ucHk=) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [...s/dataflow/internal/clients/cloudbuild/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvY2xvdWRidWlsZC9fX2luaXRfXy5weQ==) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [.../srcs/sdks/python/apache\_beam/runners/job/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2pvYi91dGlscy5weQ==) | `69.23% <0.00%> (-2.20%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/spark\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L3NwYXJrX3J1bm5lci5weQ==) | `53.65% <0.00%> (-2.16%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/flink\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L2ZsaW5rX3J1bm5lci5weQ==) | `59.61% <0.00%> (-1.50%)` | :arrow_down: |
| [.../build/srcs/sdks/python/apache\_beam/tools/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90b29scy91dGlscy5weQ==) | `67.74% <0.00%> (-1.49%)` | :arrow_down: |
| ... and [656 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [43fba5f...eb3e7a8](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-829534131
Run Python 3.8 PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e1386ce) into [master](https://codecov.io/gh/apache/beam/commit/1ee9a42ce20097179a9bd513b941264594005243?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1ee9a42) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
- Coverage 83.67% 83.66% -0.01%
==========================================
Files 440 440
Lines 59053 59047 -6
==========================================
- Hits 49412 49402 -10
- Misses 9641 9645 +4
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ld/srcs/sdks/python/apache\_beam/io/filesystemio.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZXN5c3RlbWlvLnB5) | | |
| [...ache\_beam/portability/api/beam\_job\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fam9iX2FwaV9wYjJfZ3JwYy5weQ==) | | |
| [...ild/srcs/sdks/python/apache\_beam/transforms/sql.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9zcWwucHk=) | | |
| [...srcs/sdks/python/apache\_beam/dataframe/doctests.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2RvY3Rlc3RzLnB5) | | |
| [...ython/apache\_beam/examples/kafkataxi/kafka\_taxi.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMva2Fma2F0YXhpL2thZmthX3RheGkucHk=) | | |
| [...rcs/sdks/python/apache\_beam/transforms/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9fX2luaXRfXy5weQ==) | | |
| [...ild/srcs/sdks/python/apache\_beam/utils/sentinel.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc2VudGluZWwucHk=) | | |
| [...thon/apache\_beam/runners/worker/operation\_specs.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvb3BlcmF0aW9uX3NwZWNzLnB5) | | |
| [...gcp/datastore/v1new/datastore\_write\_it\_pipeline.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2RhdGFzdG9yZS92MW5ldy9kYXRhc3RvcmVfd3JpdGVfaXRfcGlwZWxpbmUucHk=) | | |
| [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | | |
| ... and [870 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [8676ec1...e1386ce](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ihji commented on a change in pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
ihji commented on a change in pull request #14690:
URL: https://github.com/apache/beam/pull/14690#discussion_r652270505
##########
File path: sdks/python/apache_beam/io/gcp/bigquery_tools.py
##########
@@ -1114,29 +1112,16 @@ def insert_rows(
# BigQuery will do a best-effort if unique IDs are provided. This situation
# can happen during retries on failures.
# TODO(silviuc): Must add support to writing TableRow's instead of dicts.
- final_rows = []
- for i, row in enumerate(rows):
- json_row = self._convert_to_json_row(row)
- insert_id = str(self.unique_row_id) if not insert_ids else insert_ids[i]
- final_rows.append(
- bigquery.TableDataInsertAllRequest.RowsValueListEntry(
- insertId=insert_id, json=json_row))
+ insert_ids = [
+ str(self.unique_row_id) if not insert_ids else insert_ids[i] for i,
+ _ in enumerate(rows)
+ ]
+ rows = [json.loads(json.dumps(r, default=default_encoder)) for r in rows]
Review comment:
any reason to dump and load? sanity check?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] aaltay commented on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
aaltay commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-926977098
It seems like orjson dependency has some issue when installed on Windows. Maybe consider making it optional. (See: https://stackoverflow.com/questions/69305399/apache-beam2-32-0-installation-fails)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94f2401) into [master](https://codecov.io/gh/apache/beam/commit/d1042c0e23626e76174d775d077e988463c8c20d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d1042c0) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
- Coverage 83.66% 83.66% -0.01%
==========================================
Files 440 440
Lines 59053 59047 -6
==========================================
- Hits 49406 49400 -6
Misses 9647 9647
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...che\_beam/runners/interactive/caching/read\_cache.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9jYWNoaW5nL3JlYWRfY2FjaGUucHk=) | | |
| [...m/portability/api/beam\_interactive\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1faW50ZXJhY3RpdmVfYXBpX3BiMl9ncnBjLnB5) | | |
| [.../python/apache\_beam/examples/windowed\_wordcount.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd2luZG93ZWRfd29yZGNvdW50LnB5) | | |
| [...eam/runners/portability/fn\_api\_runner/fn\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL2ZuX3J1bm5lci5weQ==) | | |
| [...cs/sdks/python/apache\_beam/runners/direct/clock.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvY2xvY2sucHk=) | | |
| [...python/apache\_beam/examples/wordcount\_debugging.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X2RlYnVnZ2luZy5weQ==) | | |
| [...apache\_beam/examples/complete/game/leader\_board.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvZ2FtZS9sZWFkZXJfYm9hcmQucHk=) | | |
| [...on/apache\_beam/runners/direct/watermark\_manager.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3Qvd2F0ZXJtYXJrX21hbmFnZXIucHk=) | | |
| [...am/testing/benchmarks/chicago\_taxi/process\_tfma.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL2NoaWNhZ29fdGF4aS9wcm9jZXNzX3RmbWEucHk=) | | |
| [...sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | | |
| ... and [870 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [db62cc7...94f2401](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-865306322
hoping this will improve performance significantly. We may still need to make improvements, but we'll see : )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94f2401) into [master](https://codecov.io/gh/apache/beam/commit/d1042c0e23626e76174d775d077e988463c8c20d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d1042c0) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
- Coverage 83.66% 83.66% -0.01%
==========================================
Files 440 440
Lines 59053 59047 -6
==========================================
- Hits 49406 49400 -6
Misses 9647 9647
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../srcs/sdks/python/apache\_beam/typehints/schemas.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3NjaGVtYXMucHk=) | | |
| [...on/apache\_beam/portability/api/beam\_job\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fam9iX2FwaV9wYjIucHk=) | | |
| [...s/snippets/transforms/aggregation/combinevalues.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvdHJhbnNmb3Jtcy9hZ2dyZWdhdGlvbi9jb21iaW5ldmFsdWVzLnB5) | | |
| [...he\_beam/examples/cookbook/multiple\_output\_pardo.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29va2Jvb2svbXVsdGlwbGVfb3V0cHV0X3BhcmRvLnB5) | | |
| [...ld/srcs/sdks/python/apache\_beam/ml/gcp/visionml.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvZ2NwL3Zpc2lvbm1sLnB5) | | |
| [...srcs/sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | | |
| [...che\_beam/examples/flink/flink\_streaming\_impulse.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvZmxpbmsvZmxpbmtfc3RyZWFtaW5nX2ltcHVsc2UucHk=) | | |
| [...apache\_beam/examples/cookbook/custom\_ptransform.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29va2Jvb2svY3VzdG9tX3B0cmFuc2Zvcm0ucHk=) | | |
| [.../srcs/sdks/python/apache\_beam/transforms/window.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy93aW5kb3cucHk=) | | |
| [...uild/srcs/sdks/python/apache\_beam/io/filesystem.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZXN5c3RlbS5weQ==) | | |
| ... and [870 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [db62cc7...94f2401](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-861002713
PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-833871250
I was able to double throughput with this change on the standard json library. Throughput should increase further for those able to use orjson
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (e1386ce) into [master](https://codecov.io/gh/apache/beam/commit/d1042c0e23626e76174d775d077e988463c8c20d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d1042c0) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.
> :exclamation: Current head e1386ce differs from pull request most recent head 94f2401. Consider uploading reports for the commit 94f2401 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
=======================================
Coverage 83.66% 83.66%
=======================================
Files 440 440
Lines 59053 59047 -6
=======================================
- Hits 49406 49402 -4
+ Misses 9647 9645 -2
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...m/testing/benchmarks/nexmark/models/auction\_bid.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvbW9kZWxzL2F1Y3Rpb25fYmlkLnB5) | | |
| [...s/dataflow/internal/clients/cloudbuild/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9pbnRlcm5hbC9jbGllbnRzL2Nsb3VkYnVpbGQvX19pbml0X18ucHk=) | | |
| [...am/testing/benchmarks/chicago\_taxi/trainer/task.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL2NoaWNhZ29fdGF4aS90cmFpbmVyL3Rhc2sucHk=) | | |
| [...ites/tox/py38/build/srcs/sdks/python/gen\_protos.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vZ2VuX3Byb3Rvcy5weQ==) | | |
| [...ld/srcs/sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | | |
| [...ild/srcs/sdks/python/apache\_beam/tools/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdG9vbHMvX19pbml0X18ucHk=) | | |
| [.../srcs/sdks/python/apache\_beam/utils/proto\_utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvcHJvdG9fdXRpbHMucHk=) | | |
| [...uild/srcs/sdks/python/apache\_beam/internal/util.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvdXRpbC5weQ==) | | |
| [...s/python/apache\_beam/io/gcp/bigquery\_avro\_tools.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5X2F2cm9fdG9vbHMucHk=) | | |
| [...thon/apache\_beam/examples/complete/autocomplete.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvYXV0b2NvbXBsZXRlLnB5) | | |
| ... and [870 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [db62cc7...94f2401](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (eb3e7a8) into [master](https://codecov.io/gh/apache/beam/commit/30afbcb75ac8eefeaf6e549f2a8b78f619866806?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (30afbcb) will **increase** coverage by `0.17%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
+ Coverage 83.47% 83.65% +0.17%
==========================================
Files 898 440 -458
Lines 117918 58997 -58921
==========================================
- Hits 98433 49354 -49079
+ Misses 19485 9643 -9842
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...n/apache\_beam/runners/direct/test\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RpcmVjdC90ZXN0X2RpcmVjdF9ydW5uZXIucHk=) | `37.50% <0.00%> (-4.81%)` | :arrow_down: |
| [...s/sdks/python/apache\_beam/runners/test/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3Rlc3QvX19pbml0X18ucHk=) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [...ache\_beam/runners/dataflow/test\_dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L3Rlc3RfZGF0YWZsb3dfcnVubmVyLnB5) | `32.65% <0.00%> (-2.65%)` | :arrow_down: |
| [...ild/srcs/sdks/python/apache\_beam/transforms/sql.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90cmFuc2Zvcm1zL3NxbC5weQ==) | `83.33% <0.00%> (-2.39%)` | :arrow_down: |
| [...ers/dataflow/internal/clients/dataflow/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvZGF0YWZsb3cvX19pbml0X18ucHk=) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [...s/dataflow/internal/clients/cloudbuild/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2RhdGFmbG93L2ludGVybmFsL2NsaWVudHMvY2xvdWRidWlsZC9fX2luaXRfXy5weQ==) | `77.77% <0.00%> (-2.23%)` | :arrow_down: |
| [.../srcs/sdks/python/apache\_beam/runners/job/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL2pvYi91dGlscy5weQ==) | `69.23% <0.00%> (-2.20%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/spark\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L3NwYXJrX3J1bm5lci5weQ==) | `53.65% <0.00%> (-2.16%)` | :arrow_down: |
| [...on/apache\_beam/runners/portability/flink\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS9ydW5uZXJzL3BvcnRhYmlsaXR5L2ZsaW5rX3J1bm5lci5weQ==) | `59.61% <0.00%> (-1.50%)` | :arrow_down: |
| [.../build/srcs/sdks/python/apache\_beam/tools/utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0NvbW1pdC9zcmMvc2Rrcy9weXRob24vdGVzdC1zdWl0ZXMvdG94L3B5MzgvYnVpbGQvc3Jjcy9zZGtzL3B5dGhvbi9hcGFjaGVfYmVhbS90b29scy91dGlscy5weQ==) | `67.74% <0.00%> (-1.49%)` | :arrow_down: |
| ... and [656 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [43fba5f...eb3e7a8](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (94f2401) into [master](https://codecov.io/gh/apache/beam/commit/d1042c0e23626e76174d775d077e988463c8c20d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d1042c0) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
- Coverage 83.66% 83.66% -0.01%
==========================================
Files 440 440
Lines 59053 59047 -6
==========================================
- Hits 49406 49400 -6
Misses 9647 9647
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...am/testing/benchmarks/chicago\_taxi/process\_tfma.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL2NoaWNhZ29fdGF4aS9wcm9jZXNzX3RmbWEucHk=) | | |
| [...beam/testing/benchmarks/nexmark/queries/query11.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy9iZW5jaG1hcmtzL25leG1hcmsvcXVlcmllcy9xdWVyeTExLnB5) | | |
| [...thon/apache\_beam/io/azure/blobstoragefilesystem.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vYXp1cmUvYmxvYnN0b3JhZ2VmaWxlc3lzdGVtLnB5) | | |
| [...les/complete/juliaset/juliaset/juliaset\_test\_it.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvY29tcGxldGUvanVsaWFzZXQvanVsaWFzZXQvanVsaWFzZXRfdGVzdF9pdC5weQ==) | | |
| [...pache\_beam/typehints/trivial\_inference\_test\_py3.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3RyaXZpYWxfaW5mZXJlbmNlX3Rlc3RfcHkzLnB5) | | |
| [...m/runners/portability/spark\_uber\_jar\_job\_server.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9zcGFya191YmVyX2phcl9qb2Jfc2VydmVyLnB5) | | |
| [...d/srcs/sdks/python/apache\_beam/internal/pickler.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvcGlja2xlci5weQ==) | | |
| [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | | |
| [...e\_beam/portability/api/beam\_interactive\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1faW50ZXJhY3RpdmVfYXBpX3BiMi5weQ==) | | |
| [...ython/apache\_beam/runners/direct/direct\_metrics.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmVhbV9QcmVDb21taXRfUHl0aG9uX0Nyb24vc3JjL3Nka3MvcHl0aG9uL3Rlc3Qtc3VpdGVzL3RveC9weTM4L2J1aWxkL3NyY3Mvc2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZGlyZWN0X21ldHJpY3MucHk=) | | |
| ... and [870 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [db62cc7...94f2401](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #14690:
URL: https://github.com/apache/beam/pull/14690#discussion_r652329059
##########
File path: sdks/python/apache_beam/io/gcp/bigquery_tools.py
##########
@@ -632,22 +639,21 @@ def _insert_all_rows(
base_labels=labels)
started_millis = int(time.time() * 1000)
- response = None
try:
- response = self.client.tabledata.InsertAll(request)
- if not response.insertErrors:
+ table_ref = gcp_bigquery.DatasetReference(project_id,
+ dataset_id).table(table_id)
+ errors = self.gcp_bq_client.insert_rows_json(
+ table_ref, json_rows=rows, row_ids=insert_ids, skip_invalid_rows=True)
+ if not errors:
service_call_metric.call('ok')
- for insert_error in response.insertErrors:
- for error in insert_error.errors:
- service_call_metric.call(error.reason)
+ for insert_error in errors:
+ service_call_metric.call(insert_error['errors'][0])
except HttpError as e:
Review comment:
Not rebased ? (https://github.com/apache/beam/commit/e959d621c1553a556eea8ed0b6bc9e27bd0452ac#diff-5427a5d3887eb695cefde082c58575a2372972996b547d55961abeb4f7bc3deb)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] pabloem commented on pull request #14690: [wip] Exploring streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
pabloem commented on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-832106348
TODO before merging: Add metadata to requests by adding it to the python-bigquery l;ibrary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #14690: [BEAM-12445] Moving streaming inserts with new client
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14690:
URL: https://github.com/apache/beam/pull/14690#issuecomment-831576662
# [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14690](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (61a332d) into [master](https://codecov.io/gh/apache/beam/commit/000ac07f2883d4708b6abbfc935fd004405aae19?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (000ac07) will **increase** coverage by `1.18%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14690 +/- ##
==========================================
+ Coverage 82.55% 83.74% +1.18%
==========================================
Files 455 439 -16
Lines 55143 59104 +3961
==========================================
+ Hits 45526 49497 +3971
+ Misses 9617 9607 -10
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...python/apache\_beam/examples/wordcount\_dataframe.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvd29yZGNvdW50X2RhdGFmcmFtZS5weQ==) | `0.00% <0.00%> (-92.60%)` | :arrow_down: |
| [...s/python/apache\_beam/examples/snippets/snippets.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvc25pcHBldHMvc25pcHBldHMucHk=) | `76.70% <0.00%> (-12.83%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/kafka.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8va2Fma2EucHk=) | `79.16% <0.00%> (-6.55%)` | :arrow_down: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `87.80% <0.00%> (-5.06%)` | :arrow_down: |
| [...n/apache\_beam/runners/direct/test\_direct\_runner.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9kaXJlY3RfcnVubmVyLnB5) | `37.50% <0.00%> (-4.81%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `74.72% <0.00%> (-4.81%)` | :arrow_down: |
| [test\_config.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-dGVzdF9jb25maWcucHk=) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/test/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy90ZXN0L19faW5pdF9fLnB5) | `66.66% <0.00%> (-4.77%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/gcp/\_\_init\_\_.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL19faW5pdF9fLnB5) | `80.00% <0.00%> (-4.62%)` | :arrow_down: |
| [...thon/apache\_beam/runners/worker/channel\_factory.py](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvY2hhbm5lbF9mYWN0b3J5LnB5) | `75.00% <0.00%> (-3.95%)` | :arrow_down: |
| ... and [457 more](https://codecov.io/gh/apache/beam/pull/14690/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [67518ce...61a332d](https://codecov.io/gh/apache/beam/pull/14690?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org