You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by GitBox <gi...@apache.org> on 2021/11/23 01:06:23 UTC
[GitHub] [gobblin] ZihanLi58 opened a new pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
ZihanLi58 opened a new pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
### JIRA
- [ ] My PR addresses the following [Gobblin JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-1582
### Description
- [ ] Here are some details about my PR, including screenshots (if applicable):
Fill low/high watermark info in SourceState for QueryBasedSource
Foe stress test class, add function to make the job always fail
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:
unit tests
### Commits
- [ ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter commented on pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436#issuecomment-977084103
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3436](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (22ad917) into [master](https://codecov.io/gh/apache/gobblin/commit/4a25c886db03dbd267cb3f01b7df1a35096f6064?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (4a25c88) will **increase** coverage by `0.10%`.
> The diff coverage is `26.66%`.
[](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3436 +/- ##
============================================
+ Coverage 46.46% 46.57% +0.10%
- Complexity 10251 10321 +70
============================================
Files 2064 2072 +8
Lines 80614 80863 +249
Branches 9004 9024 +20
============================================
+ Hits 37455 37658 +203
- Misses 39686 39730 +44
- Partials 3473 3475 +2
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...lin/source/extractor/extract/QueryBasedSource.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9leHRyYWN0b3IvZXh0cmFjdC9RdWVyeUJhc2VkU291cmNlLmphdmE=) | `40.37% <0.00%> (-1.79%)` | :arrow_down: |
| [.../apache/gobblin/util/test/StressTestingSource.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvdGVzdC9TdHJlc3NUZXN0aW5nU291cmNlLmphdmE=) | `82.97% <66.66%> (+14.79%)` | :arrow_up: |
| [...he/gobblin/runtime/JobExecutionEventSubmitter.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvSm9iRXhlY3V0aW9uRXZlbnRTdWJtaXR0ZXIuamF2YQ==) | `100.00% <100.00%> (ø)` | |
| [...ache/gobblin/runtime/api/FlowSpecSearchObject.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvYXBpL0Zsb3dTcGVjU2VhcmNoT2JqZWN0LmphdmE=) | `55.12% <0.00%> (-32.38%)` | :arrow_down: |
| [...n/java/org/apache/gobblin/writer/FsDataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3dyaXRlci9Gc0RhdGFXcml0ZXIuamF2YQ==) | `75.00% <0.00%> (-2.00%)` | :arrow_down: |
| [...blin/service/modules/orchestration/DagManager.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9vcmNoZXN0cmF0aW9uL0RhZ01hbmFnZXIuamF2YQ==) | `71.22% <0.00%> (-1.74%)` | :arrow_down: |
| [...gobblin/service/modules/core/GitConfigMonitor.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9jb3JlL0dpdENvbmZpZ01vbml0b3IuamF2YQ==) | `81.35% <0.00%> (-1.70%)` | :arrow_down: |
| [...lin/service/FlowExecutionResourceLocalHandler.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi1mbG93LWNvbmZpZy1zZXJ2aWNlL2dvYmJsaW4tZmxvdy1jb25maWctc2VydmljZS1zZXJ2ZXIvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vc2VydmljZS9GbG93RXhlY3V0aW9uUmVzb3VyY2VMb2NhbEhhbmRsZXIuamF2YQ==) | `4.67% <0.00%> (-1.05%)` | :arrow_down: |
| [.../apache/gobblin/cluster/GobblinClusterManager.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpbkNsdXN0ZXJNYW5hZ2VyLmphdmE=) | `53.27% <0.00%> (-0.65%)` | :arrow_down: |
| [...in/service/modules/core/GobblinServiceManager.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9jb3JlL0dvYmJsaW5TZXJ2aWNlTWFuYWdlci5qYXZh) | `50.20% <0.00%> (-0.41%)` | :arrow_down: |
| ... and [44 more](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [4a25c88...22ad917](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] ZihanLi58 commented on a change in pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
Posted by GitBox <gi...@apache.org>.
ZihanLi58 commented on a change in pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436#discussion_r755435725
##########
File path: gobblin-runtime/src/main/java/org/apache/gobblin/runtime/JobExecutionEventSubmitter.java
##########
@@ -77,6 +78,8 @@ private void submitJobStateEvent(JobState jobState) {
jobMetadataBuilder.put(METADATA_JOB_COMPLETED_TASKS, Integer.toString(jobState.getCompletedTasks()));
jobMetadataBuilder.put(METADATA_JOB_LAUNCHER_TYPE, jobState.getLauncherType().toString());
jobMetadataBuilder.put(METADATA_JOB_TRACKING_URL, jobState.getTrackingURL().or(UNKNOWN_VALUE));
+ jobMetadataBuilder.put(TimingEvent.FlowEventConstants.HIGH_WATERMARK_FIELD, jobState.getProp(TimingEvent.FlowEventConstants.HIGH_WATERMARK_FIELD, ""));
+ jobMetadataBuilder.put(TimingEvent.FlowEventConstants.LOW_WATERMARK_FIELD, jobState.getProp(TimingEvent.FlowEventConstants.LOW_WATERMARK_FIELD, ""));
Review comment:
For me I think UNKONW_VALUE is not expected in most cases and indicate there is something wrong, but "" just indicate we don't set it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] ZihanLi58 merged pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
Posted by GitBox <gi...@apache.org>.
ZihanLi58 merged pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436#issuecomment-977084103
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3436](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (20f38cb) into [master](https://codecov.io/gh/apache/gobblin/commit/9053c3a2cd6fbba94afbfeab85b941865dd95e3c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9053c3a) will **increase** coverage by `2.52%`.
> The diff coverage is `23.52%`.
[](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3436 +/- ##
============================================
+ Coverage 46.57% 49.09% +2.52%
+ Complexity 10326 8855 -1471
============================================
Files 2072 1691 -381
Lines 80849 65324 -15525
Branches 9020 7492 -1528
============================================
- Hits 37655 32073 -5582
+ Misses 39716 30229 -9487
+ Partials 3478 3022 -456
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...lin/source/extractor/extract/QueryBasedSource.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9leHRyYWN0b3IvZXh0cmFjdC9RdWVyeUJhc2VkU291cmNlLmphdmE=) | `40.37% <0.00%> (-1.79%)` | :arrow_down: |
| [...blin/service/modules/orchestration/DagManager.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9vcmNoZXN0cmF0aW9uL0RhZ01hbmFnZXIuamF2YQ==) | `71.00% <0.00%> (-0.23%)` | :arrow_down: |
| [.../apache/gobblin/util/test/StressTestingSource.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvdGVzdC9TdHJlc3NUZXN0aW5nU291cmNlLmphdmE=) | `82.97% <66.66%> (+14.79%)` | :arrow_up: |
| [...he/gobblin/runtime/JobExecutionEventSubmitter.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvSm9iRXhlY3V0aW9uRXZlbnRTdWJtaXR0ZXIuamF2YQ==) | `100.00% <100.00%> (ø)` | |
| [...a/org/apache/gobblin/util/limiter/NoopLimiter.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvbGltaXRlci9Ob29wTGltaXRlci5qYXZh) | `40.00% <0.00%> (-20.00%)` | :arrow_down: |
| [...he/gobblin/source/PartitionAwareFileRetriever.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9QYXJ0aXRpb25Bd2FyZUZpbGVSZXRyaWV2ZXIuamF2YQ==) | `48.14% <0.00%> (-7.41%)` | :arrow_down: |
| [...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh) | `28.26% <0.00%> (-5.08%)` | :arrow_down: |
| [...lin/restli/throttling/ZookeeperLeaderElection.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL1pvb2tlZXBlckxlYWRlckVsZWN0aW9uLmphdmE=) | `70.00% <0.00%> (-2.23%)` | :arrow_down: |
| [.../modules/scheduler/GobblinServiceJobScheduler.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9zY2hlZHVsZXIvR29iYmxpblNlcnZpY2VKb2JTY2hlZHVsZXIuamF2YQ==) | `63.68% <0.00%> (-1.50%)` | :arrow_down: |
| [...a/management/copy/publisher/CopyDataPublisher.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvcHVibGlzaGVyL0NvcHlEYXRhUHVibGlzaGVyLmphdmE=) | `74.17% <0.00%> (-1.33%)` | :arrow_down: |
| ... and [386 more](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [9053c3a...20f38cb](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] phet commented on a change in pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
Posted by GitBox <gi...@apache.org>.
phet commented on a change in pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436#discussion_r754827725
##########
File path: gobblin-utility/src/test/java/org/apache/gobblin/util/test/TestStressTestingSource.java
##########
@@ -168,4 +168,37 @@ public void testRunDuration() throws DataRecordException, IOException {
Assert.assertTrue(Math.abs(timeSpentMicro - (RUN_DURATION_SECS * 1000000)) < (1000000),
"Time spent " + timeSpentMicro);
}
+
+ @Test
+ public void testThrowException() throws DataRecordException, IOException {
+ final int MEM_ALLOC_BYTES = 100;
+ final int NUM_WORK_UNITS = 1;
+ final int SLEEP_TIME_MICRO = 1000;
+ final int NUM_RECORDS = 30; // this config is ignored since the duration is set
+ final int RUN_DURATION_SECS = 5;
+
+ SourceState state = new SourceState();
+ state.setProp(StressTestingSource.NUM_WORK_UNITS_KEY, NUM_WORK_UNITS);
+ state.setProp(StressTestingSource.MEM_ALLOC_BYTES_KEY, MEM_ALLOC_BYTES);
+ state.setProp(StressTestingSource.SLEEP_TIME_MICRO_KEY, SLEEP_TIME_MICRO);
+ state.setProp(StressTestingSource.NUM_RECORDS_KEY, NUM_RECORDS);
+ state.setProp(StressTestingSource.RUN_DURATION_KEY, RUN_DURATION_SECS);
+ state.setProp(StressTestingSource.THROW_EXCEPTION, true);
+
+ StressTestingSource source = new StressTestingSource();
+
+ List<WorkUnit> wus = source.getWorkunits(state);
+ Assert.assertEquals(wus.size(), NUM_WORK_UNITS);
+
+ WorkUnit wu = wus.get(0);
+ WorkUnitState wuState = new WorkUnitState(wu, state);
+ Extractor<String, byte[]> extractor = source.getExtractor(wuState);
+
+ Assert.expectThrows(IOException.class, () -> {
+ byte[] record;
+ while ((record = extractor.readRecord(null)) != null) {
+ Assert.assertEquals(record.length, 100);
Review comment:
minor, but for clarity I might replace with: `Assert.fail("should have thrown!")`
##########
File path: gobblin-core/src/main/java/org/apache/gobblin/source/extractor/extract/QueryBasedSource.java
##########
@@ -241,7 +244,13 @@ public int hashCode() {
addLineageSourceInfo(state, sourceEntity, workunit);
partition.serialize(workunit);
workUnits.add(workunit);
+ highestWaterMark = Math.max(highestWaterMark, partition.getHighWatermark());
+ lowestWaterMark = Math.min(lowestWaterMark, partition.getLowWatermark());
}
+ state.appendToListProp(TimingEvent.FlowEventConstants.HIGH_WATERMARK_FIELD,
+ String.format("%s.%s: %s", sourceEntity.getDatasetName(), sourceEntity.destTableName, highestWaterMark));
+ state.appendToListProp(TimingEvent.FlowEventConstants.LOW_WATERMARK_FIELD,
+ String.format("%s.%s: %s", sourceEntity.getDatasetName(), sourceEntity.destTableName, lowestWaterMark));
Review comment:
could `partitions` ever be empty? if so, I recommend lowest, highest to be `Optional` and only write conditionally, rather than setting the properties to `MAX_VALUE` and `-1`.
##########
File path: gobblin-runtime/src/main/java/org/apache/gobblin/runtime/JobExecutionEventSubmitter.java
##########
@@ -77,6 +78,8 @@ private void submitJobStateEvent(JobState jobState) {
jobMetadataBuilder.put(METADATA_JOB_COMPLETED_TASKS, Integer.toString(jobState.getCompletedTasks()));
jobMetadataBuilder.put(METADATA_JOB_LAUNCHER_TYPE, jobState.getLauncherType().toString());
jobMetadataBuilder.put(METADATA_JOB_TRACKING_URL, jobState.getTrackingURL().or(UNKNOWN_VALUE));
+ jobMetadataBuilder.put(TimingEvent.FlowEventConstants.HIGH_WATERMARK_FIELD, jobState.getProp(TimingEvent.FlowEventConstants.HIGH_WATERMARK_FIELD, ""));
+ jobMetadataBuilder.put(TimingEvent.FlowEventConstants.LOW_WATERMARK_FIELD, jobState.getProp(TimingEvent.FlowEventConstants.LOW_WATERMARK_FIELD, ""));
Review comment:
may not be an issue... just curious: when do we use `""` and when instead `UNKNOWN_VALUE`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436#issuecomment-977084103
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3436](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (54d15f3) into [master](https://codecov.io/gh/apache/gobblin/commit/9053c3a2cd6fbba94afbfeab85b941865dd95e3c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9053c3a) will **decrease** coverage by `0.00%`.
> The diff coverage is `21.05%`.
[](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3436 +/- ##
============================================
- Coverage 46.57% 46.56% -0.01%
+ Complexity 10326 10321 -5
============================================
Files 2072 2072
Lines 80849 80865 +16
Branches 9020 9025 +5
============================================
+ Hits 37655 37657 +2
- Misses 39716 39733 +17
+ Partials 3478 3475 -3
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...lin/source/extractor/extract/QueryBasedSource.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9leHRyYWN0b3IvZXh0cmFjdC9RdWVyeUJhc2VkU291cmNlLmphdmE=) | `40.37% <0.00%> (-1.79%)` | :arrow_down: |
| [...blin/service/modules/orchestration/DagManager.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9vcmNoZXN0cmF0aW9uL0RhZ01hbmFnZXIuamF2YQ==) | `71.00% <0.00%> (-0.23%)` | :arrow_down: |
| [.../apache/gobblin/util/test/StressTestingSource.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvdGVzdC9TdHJlc3NUZXN0aW5nU291cmNlLmphdmE=) | `82.97% <66.66%> (+14.79%)` | :arrow_up: |
| [...he/gobblin/runtime/JobExecutionEventSubmitter.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvSm9iRXhlY3V0aW9uRXZlbnRTdWJtaXR0ZXIuamF2YQ==) | `100.00% <100.00%> (ø)` | |
| [...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh) | `28.26% <0.00%> (-5.08%)` | :arrow_down: |
| [.../modules/scheduler/GobblinServiceJobScheduler.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9zY2hlZHVsZXIvR29iYmxpblNlcnZpY2VKb2JTY2hlZHVsZXIuamF2YQ==) | `63.68% <0.00%> (-1.50%)` | :arrow_down: |
| [...lin/util/filesystem/FileSystemInstrumentation.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsZXN5c3RlbS9GaWxlU3lzdGVtSW5zdHJ1bWVudGF0aW9uLmphdmE=) | `100.00% <0.00%> (+7.14%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [9053c3a...54d15f3](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] arjun4084346 commented on a change in pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
Posted by GitBox <gi...@apache.org>.
arjun4084346 commented on a change in pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436#discussion_r756478504
##########
File path: gobblin-core/src/main/java/org/apache/gobblin/source/extractor/extract/QueryBasedSource.java
##########
@@ -241,6 +244,14 @@ public int hashCode() {
addLineageSourceInfo(state, sourceEntity, workunit);
partition.serialize(workunit);
workUnits.add(workunit);
+ highestWaterMark = highestWaterMark.isPresent() ?
+ highestWaterMark.transform(l -> Math.max(l, partition.getHighWatermark())) : Optional.of(partition.getHighWatermark());
Review comment:
*nit maybe `l` should be renamed to hw ?
and in next line `l` should be `lw ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] phet commented on a change in pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
Posted by GitBox <gi...@apache.org>.
phet commented on a change in pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436#discussion_r755499839
##########
File path: gobblin-utility/src/test/java/org/apache/gobblin/util/test/TestStressTestingSource.java
##########
@@ -168,4 +168,37 @@ public void testRunDuration() throws DataRecordException, IOException {
Assert.assertTrue(Math.abs(timeSpentMicro - (RUN_DURATION_SECS * 1000000)) < (1000000),
"Time spent " + timeSpentMicro);
}
+
+ @Test
+ public void testThrowException() throws DataRecordException, IOException {
+ final int MEM_ALLOC_BYTES = 100;
+ final int NUM_WORK_UNITS = 1;
+ final int SLEEP_TIME_MICRO = 1000;
+ final int NUM_RECORDS = 30; // this config is ignored since the duration is set
+ final int RUN_DURATION_SECS = 5;
+
+ SourceState state = new SourceState();
+ state.setProp(StressTestingSource.NUM_WORK_UNITS_KEY, NUM_WORK_UNITS);
+ state.setProp(StressTestingSource.MEM_ALLOC_BYTES_KEY, MEM_ALLOC_BYTES);
+ state.setProp(StressTestingSource.SLEEP_TIME_MICRO_KEY, SLEEP_TIME_MICRO);
+ state.setProp(StressTestingSource.NUM_RECORDS_KEY, NUM_RECORDS);
+ state.setProp(StressTestingSource.RUN_DURATION_KEY, RUN_DURATION_SECS);
+ state.setProp(StressTestingSource.THROW_EXCEPTION, true);
+
+ StressTestingSource source = new StressTestingSource();
+
+ List<WorkUnit> wus = source.getWorkunits(state);
+ Assert.assertEquals(wus.size(), NUM_WORK_UNITS);
+
+ WorkUnit wu = wus.get(0);
+ WorkUnitState wuState = new WorkUnitState(wu, state);
+ Extractor<String, byte[]> extractor = source.getExtractor(wuState);
+
+ Assert.expectThrows(IOException.class, () -> {
+ byte[] record;
+ while ((record = extractor.readRecord(null)) != null) {
+ Assert.assertEquals(record.length, 100);
Review comment:
actually... looks like it throws just prior to the (final) `extractor.readRecord(null)` call that would first return `null`. so in that case, yes, leave as is and ignore my `Assert.fail` recommendation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] ZihanLi58 commented on a change in pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
Posted by GitBox <gi...@apache.org>.
ZihanLi58 commented on a change in pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436#discussion_r755439349
##########
File path: gobblin-utility/src/test/java/org/apache/gobblin/util/test/TestStressTestingSource.java
##########
@@ -168,4 +168,37 @@ public void testRunDuration() throws DataRecordException, IOException {
Assert.assertTrue(Math.abs(timeSpentMicro - (RUN_DURATION_SECS * 1000000)) < (1000000),
"Time spent " + timeSpentMicro);
}
+
+ @Test
+ public void testThrowException() throws DataRecordException, IOException {
+ final int MEM_ALLOC_BYTES = 100;
+ final int NUM_WORK_UNITS = 1;
+ final int SLEEP_TIME_MICRO = 1000;
+ final int NUM_RECORDS = 30; // this config is ignored since the duration is set
+ final int RUN_DURATION_SECS = 5;
+
+ SourceState state = new SourceState();
+ state.setProp(StressTestingSource.NUM_WORK_UNITS_KEY, NUM_WORK_UNITS);
+ state.setProp(StressTestingSource.MEM_ALLOC_BYTES_KEY, MEM_ALLOC_BYTES);
+ state.setProp(StressTestingSource.SLEEP_TIME_MICRO_KEY, SLEEP_TIME_MICRO);
+ state.setProp(StressTestingSource.NUM_RECORDS_KEY, NUM_RECORDS);
+ state.setProp(StressTestingSource.RUN_DURATION_KEY, RUN_DURATION_SECS);
+ state.setProp(StressTestingSource.THROW_EXCEPTION, true);
+
+ StressTestingSource source = new StressTestingSource();
+
+ List<WorkUnit> wus = source.getWorkunits(state);
+ Assert.assertEquals(wus.size(), NUM_WORK_UNITS);
+
+ WorkUnit wu = wus.get(0);
+ WorkUnitState wuState = new WorkUnitState(wu, state);
+ Extractor<String, byte[]> extractor = source.getExtractor(wuState);
+
+ Assert.expectThrows(IOException.class, () -> {
+ byte[] record;
+ while ((record = extractor.readRecord(null)) != null) {
+ Assert.assertEquals(record.length, 100);
Review comment:
In this case I want to test whether the job can fail as expected. But assert.fail will not test anything and directly fail the test which is not what I want. Please let me know if I miss something here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] phet commented on a change in pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
Posted by GitBox <gi...@apache.org>.
phet commented on a change in pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436#discussion_r755489266
##########
File path: gobblin-runtime/src/main/java/org/apache/gobblin/runtime/JobExecutionEventSubmitter.java
##########
@@ -77,6 +78,8 @@ private void submitJobStateEvent(JobState jobState) {
jobMetadataBuilder.put(METADATA_JOB_COMPLETED_TASKS, Integer.toString(jobState.getCompletedTasks()));
jobMetadataBuilder.put(METADATA_JOB_LAUNCHER_TYPE, jobState.getLauncherType().toString());
jobMetadataBuilder.put(METADATA_JOB_TRACKING_URL, jobState.getTrackingURL().or(UNKNOWN_VALUE));
+ jobMetadataBuilder.put(TimingEvent.FlowEventConstants.HIGH_WATERMARK_FIELD, jobState.getProp(TimingEvent.FlowEventConstants.HIGH_WATERMARK_FIELD, ""));
+ jobMetadataBuilder.put(TimingEvent.FlowEventConstants.LOW_WATERMARK_FIELD, jobState.getProp(TimingEvent.FlowEventConstants.LOW_WATERMARK_FIELD, ""));
Review comment:
sounds reasonable
##########
File path: gobblin-core/src/main/java/org/apache/gobblin/source/extractor/extract/QueryBasedSource.java
##########
@@ -241,6 +244,14 @@ public int hashCode() {
addLineageSourceInfo(state, sourceEntity, workunit);
partition.serialize(workunit);
workUnits.add(workunit);
+ highestWaterMark = highestWaterMark.isPresent() ?
+ Optional.of(Math.max(highestWaterMark.get(), partition.getHighWatermark())) : Optional.of(partition.getHighWatermark());
Review comment:
`Optional.map` would be more idiomatic
##########
File path: gobblin-utility/src/test/java/org/apache/gobblin/util/test/TestStressTestingSource.java
##########
@@ -168,4 +168,37 @@ public void testRunDuration() throws DataRecordException, IOException {
Assert.assertTrue(Math.abs(timeSpentMicro - (RUN_DURATION_SECS * 1000000)) < (1000000),
"Time spent " + timeSpentMicro);
}
+
+ @Test
+ public void testThrowException() throws DataRecordException, IOException {
+ final int MEM_ALLOC_BYTES = 100;
+ final int NUM_WORK_UNITS = 1;
+ final int SLEEP_TIME_MICRO = 1000;
+ final int NUM_RECORDS = 30; // this config is ignored since the duration is set
+ final int RUN_DURATION_SECS = 5;
+
+ SourceState state = new SourceState();
+ state.setProp(StressTestingSource.NUM_WORK_UNITS_KEY, NUM_WORK_UNITS);
+ state.setProp(StressTestingSource.MEM_ALLOC_BYTES_KEY, MEM_ALLOC_BYTES);
+ state.setProp(StressTestingSource.SLEEP_TIME_MICRO_KEY, SLEEP_TIME_MICRO);
+ state.setProp(StressTestingSource.NUM_RECORDS_KEY, NUM_RECORDS);
+ state.setProp(StressTestingSource.RUN_DURATION_KEY, RUN_DURATION_SECS);
+ state.setProp(StressTestingSource.THROW_EXCEPTION, true);
+
+ StressTestingSource source = new StressTestingSource();
+
+ List<WorkUnit> wus = source.getWorkunits(state);
+ Assert.assertEquals(wus.size(), NUM_WORK_UNITS);
+
+ WorkUnit wu = wus.get(0);
+ WorkUnitState wuState = new WorkUnitState(wu, state);
+ Extractor<String, byte[]> extractor = source.getExtractor(wuState);
+
+ Assert.expectThrows(IOException.class, () -> {
+ byte[] record;
+ while ((record = extractor.readRecord(null)) != null) {
+ Assert.assertEquals(record.length, 100);
Review comment:
I figured with `THROW_EXCEPTION == true` it would throw in the first `extractor.readRecord(null)` call. therefore the `Assert.fail` immediately alerts us to begin debugging if that doesn't happen. it's also self-documenting, so future maintainers know, "that loop body should never run".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436#issuecomment-977084103
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3436](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (20f38cb) into [master](https://codecov.io/gh/apache/gobblin/commit/9053c3a2cd6fbba94afbfeab85b941865dd95e3c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9053c3a) will **decrease** coverage by `0.01%`.
> The diff coverage is `23.52%`.
[](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3436 +/- ##
============================================
- Coverage 46.57% 46.56% -0.02%
+ Complexity 10326 10319 -7
============================================
Files 2072 2072
Lines 80849 80865 +16
Branches 9020 9025 +5
============================================
- Hits 37655 37652 -3
- Misses 39716 39738 +22
+ Partials 3478 3475 -3
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...lin/source/extractor/extract/QueryBasedSource.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9leHRyYWN0b3IvZXh0cmFjdC9RdWVyeUJhc2VkU291cmNlLmphdmE=) | `40.37% <0.00%> (-1.79%)` | :arrow_down: |
| [...blin/service/modules/orchestration/DagManager.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9vcmNoZXN0cmF0aW9uL0RhZ01hbmFnZXIuamF2YQ==) | `71.00% <0.00%> (-0.23%)` | :arrow_down: |
| [.../apache/gobblin/util/test/StressTestingSource.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvdGVzdC9TdHJlc3NUZXN0aW5nU291cmNlLmphdmE=) | `82.97% <66.66%> (+14.79%)` | :arrow_up: |
| [...he/gobblin/runtime/JobExecutionEventSubmitter.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvSm9iRXhlY3V0aW9uRXZlbnRTdWJtaXR0ZXIuamF2YQ==) | `100.00% <100.00%> (ø)` | |
| [...a/org/apache/gobblin/util/limiter/NoopLimiter.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvbGltaXRlci9Ob29wTGltaXRlci5qYXZh) | `40.00% <0.00%> (-20.00%)` | :arrow_down: |
| [...he/gobblin/source/PartitionAwareFileRetriever.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9QYXJ0aXRpb25Bd2FyZUZpbGVSZXRyaWV2ZXIuamF2YQ==) | `48.14% <0.00%> (-7.41%)` | :arrow_down: |
| [...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh) | `28.26% <0.00%> (-5.08%)` | :arrow_down: |
| [...lin/restli/throttling/ZookeeperLeaderElection.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL1pvb2tlZXBlckxlYWRlckVsZWN0aW9uLmphdmE=) | `70.00% <0.00%> (-2.23%)` | :arrow_down: |
| [.../modules/scheduler/GobblinServiceJobScheduler.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9zY2hlZHVsZXIvR29iYmxpblNlcnZpY2VKb2JTY2hlZHVsZXIuamF2YQ==) | `63.68% <0.00%> (-1.50%)` | :arrow_down: |
| [...a/management/copy/publisher/CopyDataPublisher.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvcHVibGlzaGVyL0NvcHlEYXRhUHVibGlzaGVyLmphdmE=) | `74.17% <0.00%> (-1.33%)` | :arrow_down: |
| ... and [6 more](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [9053c3a...20f38cb](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436#issuecomment-977084103
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3436](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (54d15f3) into [master](https://codecov.io/gh/apache/gobblin/commit/9053c3a2cd6fbba94afbfeab85b941865dd95e3c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9053c3a) will **increase** coverage by `2.53%`.
> The diff coverage is `21.05%`.
[](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3436 +/- ##
============================================
+ Coverage 46.57% 49.10% +2.53%
+ Complexity 10326 8857 -1469
============================================
Files 2072 1691 -381
Lines 80849 65324 -15525
Branches 9020 7492 -1528
============================================
- Hits 37655 32079 -5576
+ Misses 39716 30224 -9492
+ Partials 3478 3021 -457
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...lin/source/extractor/extract/QueryBasedSource.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9leHRyYWN0b3IvZXh0cmFjdC9RdWVyeUJhc2VkU291cmNlLmphdmE=) | `40.37% <0.00%> (-1.79%)` | :arrow_down: |
| [...blin/service/modules/orchestration/DagManager.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9vcmNoZXN0cmF0aW9uL0RhZ01hbmFnZXIuamF2YQ==) | `71.00% <0.00%> (-0.23%)` | :arrow_down: |
| [.../apache/gobblin/util/test/StressTestingSource.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvdGVzdC9TdHJlc3NUZXN0aW5nU291cmNlLmphdmE=) | `82.97% <66.66%> (+14.79%)` | :arrow_up: |
| [...he/gobblin/runtime/JobExecutionEventSubmitter.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvSm9iRXhlY3V0aW9uRXZlbnRTdWJtaXR0ZXIuamF2YQ==) | `100.00% <100.00%> (ø)` | |
| [...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh) | `28.26% <0.00%> (-5.08%)` | :arrow_down: |
| [.../modules/scheduler/GobblinServiceJobScheduler.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9zY2hlZHVsZXIvR29iYmxpblNlcnZpY2VKb2JTY2hlZHVsZXIuamF2YQ==) | `63.68% <0.00%> (-1.50%)` | :arrow_down: |
| [...rce/extractor/extract/kafka/KafkaSimpleSource.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9leHRyYWN0b3IvZXh0cmFjdC9rYWZrYS9LYWZrYVNpbXBsZVNvdXJjZS5qYXZh) | | |
| [...he/gobblin/kafka/serialize/GsonSerializerBase.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2thZmthL3NlcmlhbGl6ZS9Hc29uU2VyaWFsaXplckJhc2UuamF2YQ==) | | |
| [.../org/apache/gobblin/metrics/kafka/KafkaPusher.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4ta2Fma2EtMDgvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9rYWZrYS9LYWZrYVB1c2hlci5qYXZh) | | |
| [...blin/writer/commands/TeradataBufferedInserter.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tc3FsL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3dyaXRlci9jb21tYW5kcy9UZXJhZGF0YUJ1ZmZlcmVkSW5zZXJ0ZXIuamF2YQ==) | | |
| ... and [380 more](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [9053c3a...54d15f3](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3436: [GOBBLIN-1582] Fill low/high watermark info in SourceState for QueryBasedSource
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3436:
URL: https://github.com/apache/gobblin/pull/3436#issuecomment-977084103
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3436](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (54d15f3) into [master](https://codecov.io/gh/apache/gobblin/commit/9053c3a2cd6fbba94afbfeab85b941865dd95e3c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9053c3a) will **decrease** coverage by `3.22%`.
> The diff coverage is `66.66%`.
[](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3436 +/- ##
============================================
- Coverage 46.57% 43.34% -3.23%
+ Complexity 10326 2021 -8305
============================================
Files 2072 402 -1670
Lines 80849 17334 -63515
Branches 9020 2123 -6897
============================================
- Hits 37655 7514 -30141
+ Misses 39716 8984 -30732
+ Partials 3478 836 -2642
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../apache/gobblin/util/test/StressTestingSource.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvdGVzdC9TdHJlc3NUZXN0aW5nU291cmNlLmphdmE=) | `82.97% <66.66%> (+14.79%)` | :arrow_up: |
| [...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh) | `28.26% <0.00%> (-5.08%)` | :arrow_down: |
| [...lin/source/extractor/extract/QueryBasedSource.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9leHRyYWN0b3IvZXh0cmFjdC9RdWVyeUJhc2VkU291cmNlLmphdmE=) | | |
| [...he/gobblin/runtime/JobExecutionEventSubmitter.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvSm9iRXhlY3V0aW9uRXZlbnRTdWJtaXR0ZXIuamF2YQ==) | | |
| [...blin/service/modules/orchestration/DagManager.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9vcmNoZXN0cmF0aW9uL0RhZ01hbmFnZXIuamF2YQ==) | | |
| [...in/compaction/audit/PinotAuditCountHttpClient.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb21wYWN0aW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbXBhY3Rpb24vYXVkaXQvUGlub3RBdWRpdENvdW50SHR0cENsaWVudC5qYXZh) | | |
| [...che/gobblin/types/AvroGenericRecordTypeMapper.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vdHlwZXMvQXZyb0dlbmVyaWNSZWNvcmRUeXBlTWFwcGVyLmphdmE=) | | |
| [...dules/orchestration/AzkabanProjectFlowsStatus.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tYXprYWJhbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9zZXJ2aWNlL21vZHVsZXMvb3JjaGVzdHJhdGlvbi9BemthYmFuUHJvamVjdEZsb3dzU3RhdHVzLmphdmE=) | | |
| [...pache/gobblin/compliance/HivePartitionVersion.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tY29tcGxpYW5jZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9jb21wbGlhbmNlL0hpdmVQYXJ0aXRpb25WZXJzaW9uLmphdmE=) | | |
| [...ava/org/apache/gobblin/metastore/ZkStateStore.java](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4taGVsaXgvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL1prU3RhdGVTdG9yZS5qYXZh) | | |
| ... and [1664 more](https://codecov.io/gh/apache/gobblin/pull/3436/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [9053c3a...54d15f3](https://codecov.io/gh/apache/gobblin/pull/3436?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org