You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by GitBox <gi...@apache.org> on 2021/09/17 01:22:05 UTC
[GitHub] [gobblin] jack-moseley opened a new pull request #3397: [GOBBLIN-1546] Don't contact schema registry for every record read by job status monitor
jack-moseley opened a new pull request #3397:
URL: https://github.com/apache/gobblin/pull/3397
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
### JIRA
- [x] My PR addresses the following [Gobblin JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-1546
### Description
- [x] Here are some details about my PR, including screenshots (if applicable):
The kafka job status monitor is currently calling `readSchemaVersioningInformation` to advance the input stream of the kafka messages it reads to the record portion, which calls the schema registry every time, but the result of it is ignored. This PR adds another API that just advances the input stream without contacting the schema registry, to remove a point of failure.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:
tested in local gaas deployment
### Commits
- [x] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3397: [GOBBLIN-1546] Don't contact schema registry for every record read by job status monitor
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3397:
URL: https://github.com/apache/gobblin/pull/3397#issuecomment-921381279
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3397](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9e13ec6) into [master](https://codecov.io/gh/apache/gobblin/commit/47707df00a6884ada5974a5f5203408ce1efb890?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (47707df) will **increase** coverage by `1.80%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3397 +/- ##
============================================
+ Coverage 46.54% 48.34% +1.80%
+ Complexity 10243 7620 -2623
============================================
Files 2062 1429 -633
Lines 80413 56437 -23976
Branches 8981 6517 -2464
============================================
- Hits 37429 27287 -10142
+ Misses 39516 26594 -12922
+ Partials 3468 2556 -912
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...etrics/reporter/util/FixedSchemaVersionWriter.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9yZXBvcnRlci91dGlsL0ZpeGVkU2NoZW1hVmVyc2lvbldyaXRlci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...metrics/reporter/util/NoopSchemaVersionWriter.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9yZXBvcnRlci91dGlsL05vb3BTY2hlbWFWZXJzaW9uV3JpdGVyLmphdmE=) | `50.00% <0.00%> (-16.67%)` | :arrow_down: |
| [.../org/apache/gobblin/async/AsyncDataDispatcher.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2FzeW5jL0FzeW5jRGF0YURpc3BhdGNoZXIuamF2YQ==) | `79.66% <0.00%> (-8.48%)` | :arrow_down: |
| [...che/gobblin/runtime/TaskStateCollectorService.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvVGFza1N0YXRlQ29sbGVjdG9yU2VydmljZS5qYXZh) | `59.48% <0.00%> (-3.82%)` | :arrow_down: |
| [...kafka/schemareg/GobblinHttpMethodRetryHandler.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2thZmthL3NjaGVtYXJlZy9Hb2JibGluSHR0cE1ldGhvZFJldHJ5SGFuZGxlci5qYXZh) | | |
| [...oubleshooter/MySqlMultiContextIssueRepository.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy90cm91Ymxlc2hvb3Rlci9NeVNxbE11bHRpQ29udGV4dElzc3VlUmVwb3NpdG9yeS5qYXZh) | | |
| [...oogle/webmaster/GoogleWebmasterDayPartitioner.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvb2dsZS1pbmdlc3Rpb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vaW5nZXN0aW9uL2dvb2dsZS93ZWJtYXN0ZXIvR29vZ2xlV2VibWFzdGVyRGF5UGFydGl0aW9uZXIuamF2YQ==) | | |
| [...gobblin/kafka/schemareg/LiKafkaSchemaRegistry.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2thZmthL3NjaGVtYXJlZy9MaUthZmthU2NoZW1hUmVnaXN0cnkuamF2YQ==) | | |
| [...gobblin/service/modules/spec/JobExecutionPlan.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9zcGVjL0pvYkV4ZWN1dGlvblBsYW4uamF2YQ==) | | |
| [...gobblin/compliance/purger/HivePurgerExtractor.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tY29tcGxpYW5jZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9jb21wbGlhbmNlL3B1cmdlci9IaXZlUHVyZ2VyRXh0cmFjdG9yLmphdmE=) | | |
| ... and [629 more](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [47707df...9e13ec6](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3397: [GOBBLIN-1546] Don't contact schema registry for every record read by job status monitor
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3397:
URL: https://github.com/apache/gobblin/pull/3397#issuecomment-921381279
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3397](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9e13ec6) into [master](https://codecov.io/gh/apache/gobblin/commit/47707df00a6884ada5974a5f5203408ce1efb890?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (47707df) will **decrease** coverage by `0.00%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3397 +/- ##
============================================
- Coverage 46.54% 46.53% -0.01%
- Complexity 10243 10245 +2
============================================
Files 2062 2062
Lines 80413 80426 +13
Branches 8981 8983 +2
============================================
Hits 37429 37429
- Misses 39516 39530 +14
+ Partials 3468 3467 -1
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...etrics/reporter/util/FixedSchemaVersionWriter.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9yZXBvcnRlci91dGlsL0ZpeGVkU2NoZW1hVmVyc2lvbldyaXRlci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...metrics/reporter/util/NoopSchemaVersionWriter.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9yZXBvcnRlci91dGlsL05vb3BTY2hlbWFWZXJzaW9uV3JpdGVyLmphdmE=) | `50.00% <0.00%> (-16.67%)` | :arrow_down: |
| [...ics/reporter/util/SchemaRegistryVersionWriter.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL21ldHJpY3MvcmVwb3J0ZXIvdXRpbC9TY2hlbWFSZWdpc3RyeVZlcnNpb25Xcml0ZXIuamF2YQ==) | `27.45% <0.00%> (-1.72%)` | :arrow_down: |
| [.../service/monitoring/KafkaAvroJobStatusMonitor.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9uaXRvcmluZy9LYWZrYUF2cm9Kb2JTdGF0dXNNb25pdG9yLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../org/apache/gobblin/async/AsyncDataDispatcher.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2FzeW5jL0FzeW5jRGF0YURpc3BhdGNoZXIuamF2YQ==) | `79.66% <0.00%> (-8.48%)` | :arrow_down: |
| [...re/ControllerUserDefinedMessageHandlerFactory.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9jb3JlL0NvbnRyb2xsZXJVc2VyRGVmaW5lZE1lc3NhZ2VIYW5kbGVyRmFjdG9yeS5qYXZh) | `34.37% <0.00%> (-4.69%)` | :arrow_down: |
| [...che/gobblin/runtime/TaskStateCollectorService.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvVGFza1N0YXRlQ29sbGVjdG9yU2VydmljZS5qYXZh) | `59.48% <0.00%> (-3.82%)` | :arrow_down: |
| [...main/java/org/apache/gobblin/util/HadoopUtils.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvSGFkb29wVXRpbHMuamF2YQ==) | `30.59% <0.00%> (+0.65%)` | :arrow_up: |
| [...lin/restli/throttling/ZookeeperLeaderElection.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL1pvb2tlZXBlckxlYWRlckVsZWN0aW9uLmphdmE=) | `72.22% <0.00%> (+2.22%)` | :arrow_up: |
| [...lin/util/filesystem/FileSystemInstrumentation.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsZXN5c3RlbS9GaWxlU3lzdGVtSW5zdHJ1bWVudGF0aW9uLmphdmE=) | `100.00% <0.00%> (+7.14%)` | :arrow_up: |
| ... and [2 more](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [47707df...9e13ec6](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter commented on pull request #3397: [GOBBLIN-1546] Don't contact schema registry for every record read by job status monitor
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #3397:
URL: https://github.com/apache/gobblin/pull/3397#issuecomment-921381279
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3397](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9e13ec6) into [master](https://codecov.io/gh/apache/gobblin/commit/47707df00a6884ada5974a5f5203408ce1efb890?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (47707df) will **decrease** coverage by `3.18%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3397 +/- ##
============================================
- Coverage 46.54% 43.35% -3.19%
+ Complexity 10243 2008 -8235
============================================
Files 2062 401 -1661
Lines 80413 17182 -63231
Branches 8981 2114 -6867
============================================
- Hits 37429 7450 -29979
+ Misses 39516 8896 -30620
+ Partials 3468 836 -2632
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...blin/converter/filter/AvroProjectionConverter.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vY29udmVydGVyL2ZpbHRlci9BdnJvUHJvamVjdGlvbkNvbnZlcnRlci5qYXZh) | | |
| [...rsion/hive/query/HiveValidationQueryGenerator.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvbnZlcnNpb24vaGl2ZS9xdWVyeS9IaXZlVmFsaWRhdGlvblF1ZXJ5R2VuZXJhdG9yLmphdmE=) | | |
| [...va/org/apache/gobblin/writer/WatermarkManager.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vd3JpdGVyL1dhdGVybWFya01hbmFnZXIuamF2YQ==) | | |
| [...actor/extract/kafka/KafkaIngestionHealthCheck.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9leHRyYWN0b3IvZXh0cmFjdC9rYWZrYS9LYWZrYUluZ2VzdGlvbkhlYWx0aENoZWNrLmphdmE=) | | |
| [...cation/DataFlowTopologyPickerByHadoopFsSource.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvcmVwbGljYXRpb24vRGF0YUZsb3dUb3BvbG9neVBpY2tlckJ5SGFkb29wRnNTb3VyY2UuamF2YQ==) | | |
| [...blin/compaction/verify/InputRecordCountHelper.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb21wYWN0aW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbXBhY3Rpb24vdmVyaWZ5L0lucHV0UmVjb3JkQ291bnRIZWxwZXIuamF2YQ==) | | |
| [...rg/apache/gobblin/service/FlowConfigsResource.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi1mbG93LWNvbmZpZy1zZXJ2aWNlL2dvYmJsaW4tZmxvdy1jb25maWctc2VydmljZS1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vc2VydmljZS9GbG93Q29uZmlnc1Jlc291cmNlLmphdmE=) | | |
| [...g/apache/gobblin/compat/hadoop/TextSerializer.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vY29tcGF0L2hhZG9vcC9UZXh0U2VyaWFsaXplci5qYXZh) | | |
| [...in/java/gobblin/source/workunit/MultiWorkUnit.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9nb2JibGluL3NvdXJjZS93b3JrdW5pdC9NdWx0aVdvcmtVbml0LmphdmE=) | | |
| [...pache/gobblin/runtime/api/SpecCatalogListener.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvYXBpL1NwZWNDYXRhbG9nTGlzdGVuZXIuamF2YQ==) | | |
| ... and [1650 more](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [47707df...9e13ec6](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] aplex merged pull request #3397: [GOBBLIN-1546] Don't contact schema registry for every record read by job status monitor
Posted by GitBox <gi...@apache.org>.
aplex merged pull request #3397:
URL: https://github.com/apache/gobblin/pull/3397
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3397: [GOBBLIN-1546] Don't contact schema registry for every record read by job status monitor
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3397:
URL: https://github.com/apache/gobblin/pull/3397#issuecomment-921381279
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3397](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9e13ec6) into [master](https://codecov.io/gh/apache/gobblin/commit/47707df00a6884ada5974a5f5203408ce1efb890?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (47707df) will **decrease** coverage by `0.88%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3397 +/- ##
============================================
- Coverage 46.54% 45.66% -0.89%
+ Complexity 10243 9082 -1161
============================================
Files 2062 1810 -252
Lines 80413 71968 -8445
Branches 8981 8050 -931
============================================
- Hits 37429 32862 -4567
+ Misses 39516 36097 -3419
+ Partials 3468 3009 -459
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...etrics/reporter/util/FixedSchemaVersionWriter.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9yZXBvcnRlci91dGlsL0ZpeGVkU2NoZW1hVmVyc2lvbldyaXRlci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...metrics/reporter/util/NoopSchemaVersionWriter.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9yZXBvcnRlci91dGlsL05vb3BTY2hlbWFWZXJzaW9uV3JpdGVyLmphdmE=) | `50.00% <0.00%> (-16.67%)` | :arrow_down: |
| [...ics/reporter/util/SchemaRegistryVersionWriter.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL21ldHJpY3MvcmVwb3J0ZXIvdXRpbC9TY2hlbWFSZWdpc3RyeVZlcnNpb25Xcml0ZXIuamF2YQ==) | `27.45% <0.00%> (-1.72%)` | :arrow_down: |
| [.../org/apache/gobblin/async/AsyncDataDispatcher.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2FzeW5jL0FzeW5jRGF0YURpc3BhdGNoZXIuamF2YQ==) | `79.66% <0.00%> (-8.48%)` | :arrow_down: |
| [...che/gobblin/runtime/TaskStateCollectorService.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvVGFza1N0YXRlQ29sbGVjdG9yU2VydmljZS5qYXZh) | `59.48% <0.00%> (-3.82%)` | :arrow_down: |
| [...modules/flowgraph/DatasetDescriptorConfigKeys.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9mbG93Z3JhcGgvRGF0YXNldERlc2NyaXB0b3JDb25maWdLZXlzLmphdmE=) | | |
| [...va/org/apache/gobblin/source/workunit/Extract.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vc291cmNlL3dvcmt1bml0L0V4dHJhY3QuamF2YQ==) | | |
| [...modules/spec/JobExecutionPlanListDeserializer.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9zcGVjL0pvYkV4ZWN1dGlvblBsYW5MaXN0RGVzZXJpYWxpemVyLmphdmE=) | | |
| [...java/org/apache/gobblin/stream/RecordEnvelope.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vc3RyZWFtL1JlY29yZEVudmVsb3BlLmphdmE=) | | |
| [...a/org/apache/gobblin/writer/DataWriterBuilder.java](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vd3JpdGVyL0RhdGFXcml0ZXJCdWlsZGVyLmphdmE=) | | |
| ... and [250 more](https://codecov.io/gh/apache/gobblin/pull/3397/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [47707df...9e13ec6](https://codecov.io/gh/apache/gobblin/pull/3397?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org