You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by GitBox <gi...@apache.org> on 2021/11/16 16:10:43 UTC
[GitHub] [gobblin] vikrambohra opened a new pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
vikrambohra opened a new pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426
…n column
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
### JIRA
- [ ] My PR addresses the following [Gobblin JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-1574
### Description
- [ ] Here are some details about my PR, including screenshots (if applicable):
- Added whitelist for iceberg tables to which to add new partition column to schema and pspec
- completenessCheck config controls only if audit system need to be checked
- streamlined code to store db and table name to TableMetadata.
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:
### Commits
- [ ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#issuecomment-970443681
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3426](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c4ff266) into [master](https://codecov.io/gh/apache/gobblin/commit/5bc22a5502ddeb810a35b0fb996dd9dfc8c81121?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5bc22a5) will **increase** coverage by `0.04%`.
> The diff coverage is `70.58%`.
[](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3426 +/- ##
============================================
+ Coverage 43.29% 43.33% +0.04%
- Complexity 2017 2021 +4
============================================
Files 402 402
Lines 17325 17330 +5
Branches 2121 2123 +2
============================================
+ Hits 7500 7510 +10
+ Misses 8987 8980 -7
- Partials 838 840 +2
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ceberg/writer/IcebergMetadataWriterConfigKeys.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlckNvbmZpZ0tleXMuamF2YQ==) | `0.00% <ø> (ø)` | |
| [.../gobblin/iceberg/writer/IcebergMetadataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlci5qYXZh) | `72.18% <70.58%> (-0.12%)` | :arrow_down: |
| [...a/org/apache/gobblin/cluster/GobblinHelixTask.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpbkhlbGl4VGFzay5qYXZh) | `60.21% <0.00%> (-2.16%)` | :arrow_down: |
| [...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh) | `33.33% <0.00%> (+5.07%)` | :arrow_up: |
| [...lin/util/filesystem/FileSystemInstrumentation.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsZXN5c3RlbS9GaWxlU3lzdGVtSW5zdHJ1bWVudGF0aW9uLmphdmE=) | `100.00% <0.00%> (+7.14%)` | :arrow_up: |
| [...a/org/apache/gobblin/util/limiter/NoopLimiter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvbGltaXRlci9Ob29wTGltaXRlci5qYXZh) | `60.00% <0.00%> (+20.00%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [5bc22a5...c4ff266](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] vikrambohra commented on a change in pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
vikrambohra commented on a change in pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#discussion_r751882526
##########
File path: gobblin-iceberg/src/main/java/org/apache/gobblin/iceberg/writer/IcebergMetadataWriter.java
##########
@@ -284,9 +287,8 @@ private Long getAndPersistCurrentWatermark(TableIdentifier tid, String topicPart
* information increases the memory footprints, therefore we would like to flush them eagerly).
*/
public void write(GobblinMetadataChangeEvent gmce, Map<String, Collection<HiveSpec>> newSpecsMap,
- Map<String, Collection<HiveSpec>> oldSpecsMap, HiveSpec tableSpec) throws IOException {
- TableIdentifier tid = TableIdentifier.of(tableSpec.getTable().getDbName(), tableSpec.getTable().getTableName());
- TableMetadata tableMetadata = tableMetadataMap.computeIfAbsent(tid, t -> new TableMetadata());
+ Map<String, Collection<HiveSpec>> oldSpecsMap, HiveSpec tableSpec, TableMetadata tableMetadata) throws IOException {
Review comment:
tid is an iceberg concept with namespace and name rather than db and table. Namespace can be dbName or a multilevel name like hivedb.dbname which could lead to errors. Other option is to pass tableSpec (which has db and table name) to addFiles method. Parsing db and table name everytime we need is not ideal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter commented on pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#issuecomment-970443681
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3426](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9d64d8c) into [master](https://codecov.io/gh/apache/gobblin/commit/5bc22a5502ddeb810a35b0fb996dd9dfc8c81121?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5bc22a5) will **increase** coverage by `7.39%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3426 +/- ##
============================================
+ Coverage 43.29% 50.68% +7.39%
- Complexity 2017 5670 +3653
============================================
Files 402 1035 +633
Lines 17325 39447 +22122
Branches 2121 4417 +2296
============================================
+ Hits 7500 19993 +12493
- Misses 8987 17727 +8740
- Partials 838 1727 +889
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../gobblin/iceberg/writer/IcebergMetadataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlci5qYXZh) | | |
| [...ceberg/writer/IcebergMetadataWriterConfigKeys.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlckNvbmZpZ0tleXMuamF2YQ==) | | |
| [.../apache/gobblin/cluster/GobblinTaskStateModel.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpblRhc2tTdGF0ZU1vZGVsLmphdmE=) | | |
| [.../apache/gobblin/util/concurrent/TaskScheduler.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvY29uY3VycmVudC9UYXNrU2NoZWR1bGVyLmphdmE=) | | |
| [...completeness/verifier/KafkaAuditCountVerifier.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb21wbGV0ZW5lc3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vY29tcGxldGVuZXNzL3ZlcmlmaWVyL0thZmthQXVkaXRDb3VudFZlcmlmaWVyLmphdmE=) | | |
| [.../gobblin/cluster/HelixRetriggeringJobCallable.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhSZXRyaWdnZXJpbmdKb2JDYWxsYWJsZS5qYXZh) | | |
| [...uest\_allocation/AllocatedRequestsIteratorBase.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvcmVxdWVzdF9hbGxvY2F0aW9uL0FsbG9jYXRlZFJlcXVlc3RzSXRlcmF0b3JCYXNlLmphdmE=) | | |
| [...obblin/metastore/FileContextBasedFsStateStore.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tZXRhc3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL0ZpbGVDb250ZXh0QmFzZWRGc1N0YXRlU3RvcmUuamF2YQ==) | | |
| [.../apache/gobblin/aws/GobblinAWSClusterLauncher.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hd3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vYXdzL0dvYmJsaW5BV1NDbHVzdGVyTGF1bmNoZXIuamF2YQ==) | | |
| [.../org/apache/gobblin/metastore/MysqlStateStore.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tZXRhc3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL015c3FsU3RhdGVTdG9yZS5qYXZh) | | |
| ... and [1427 more](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [5bc22a5...9d64d8c](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] ZihanLi58 commented on a change in pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
ZihanLi58 commented on a change in pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#discussion_r752493742
##########
File path: gobblin-iceberg/src/main/java/org/apache/gobblin/iceberg/writer/IcebergMetadataWriter.java
##########
@@ -284,9 +287,8 @@ private Long getAndPersistCurrentWatermark(TableIdentifier tid, String topicPart
* information increases the memory footprints, therefore we would like to flush them eagerly).
*/
public void write(GobblinMetadataChangeEvent gmce, Map<String, Collection<HiveSpec>> newSpecsMap,
- Map<String, Collection<HiveSpec>> oldSpecsMap, HiveSpec tableSpec) throws IOException {
- TableIdentifier tid = TableIdentifier.of(tableSpec.getTable().getDbName(), tableSpec.getTable().getTableName());
- TableMetadata tableMetadata = tableMetadataMap.computeIfAbsent(tid, t -> new TableMetadata());
+ Map<String, Collection<HiveSpec>> oldSpecsMap, HiveSpec tableSpec, TableMetadata tableMetadata) throws IOException {
Review comment:
I agree that dbName and tableName are not necessary to be tid, but here you only use those info to construct tid, does that mean directly pass tid will be more accurate?
##########
File path: gobblin-iceberg/src/main/java/org/apache/gobblin/iceberg/writer/IcebergMetadataWriter.java
##########
@@ -684,7 +687,8 @@ protected void addFiles(GobblinMetadataChangeEvent gmce, Map<String, Collection<
Collection<HiveSpec> hiveSpecs = newSpecsMap.get(new Path(file.getFilePath()).getParent().toString());
StructLike partition = getIcebergPartitionVal(hiveSpecs, file.getFilePath(), partitionSpec);
- if(tableMetadata.completenessEnabled && gmce.getOperationType() == OperationType.add_files) {
+ if(this.newPartitionTableWhitelistBlacklist.acceptTable(tableMetadata.dbName, tableMetadata.tableName)
+ && gmce.getOperationType() == OperationType.add_files) {
tableMetadata.prevCompletenessWatermark = Long.parseLong(table.properties().getOrDefault(COMPLETION_WATERMARK_KEY,
Review comment:
Do we want to make this more general? we can just add one place holder here and address in another PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#issuecomment-970443681
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3426](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0457284) into [master](https://codecov.io/gh/apache/gobblin/commit/5bc22a5502ddeb810a35b0fb996dd9dfc8c81121?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5bc22a5) will **increase** coverage by `3.28%`.
> The diff coverage is `76.47%`.
[](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3426 +/- ##
============================================
+ Coverage 43.29% 46.57% +3.28%
- Complexity 2017 10320 +8303
============================================
Files 402 2071 +1669
Lines 17325 80845 +63520
Branches 2121 9019 +6898
============================================
+ Hits 7500 37650 +30150
- Misses 8987 39716 +30729
- Partials 838 3479 +2641
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ceberg/writer/IcebergMetadataWriterConfigKeys.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlckNvbmZpZ0tleXMuamF2YQ==) | `0.00% <ø> (ø)` | |
| [.../gobblin/iceberg/writer/IcebergMetadataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlci5qYXZh) | `72.51% <76.47%> (+0.21%)` | :arrow_up: |
| [.../java/org/apache/gobblin/cluster/SleepingTask.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvU2xlZXBpbmdUYXNrLmphdmE=) | `39.39% <0.00%> (-6.07%)` | :arrow_down: |
| [...a/org/apache/gobblin/cluster/GobblinHelixTask.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpbkhlbGl4VGFzay5qYXZh) | `60.21% <0.00%> (-2.16%)` | :arrow_down: |
| [...bblin/couchbase/writer/CouchbaseWriterBuilder.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tY291Y2hiYXNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvdWNoYmFzZS93cml0ZXIvQ291Y2hiYXNlV3JpdGVyQnVpbGRlci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...e/gobblin/runtime/std/DefaultConfigurableImpl.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvc3RkL0RlZmF1bHRDb25maWd1cmFibGVJbXBsLmphdmE=) | `100.00% <0.00%> (ø)` | |
| [...me/spec\_executorInstance/AbstractSpecExecutor.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvc3BlY19leGVjdXRvckluc3RhbmNlL0Fic3RyYWN0U3BlY0V4ZWN1dG9yLmphdmE=) | `45.23% <0.00%> (ø)` | |
| [...nt/copy/writer/FileAwareInputStreamDataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvd3JpdGVyL0ZpbGVBd2FyZUlucHV0U3RyZWFtRGF0YVdyaXRlci5qYXZh) | `74.37% <0.00%> (ø)` | |
| [...ata/management/copy/CopyableGlobDatasetFinder.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvQ29weWFibGVHbG9iRGF0YXNldEZpbmRlci5qYXZh) | `100.00% <0.00%> (ø)` | |
| [...itychecker/task/TaskLevelPolicyCheckerBuilder.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3F1YWxpdHljaGVja2VyL3Rhc2svVGFza0xldmVsUG9saWN5Q2hlY2tlckJ1aWxkZXIuamF2YQ==) | `80.64% <0.00%> (ø)` | |
| ... and [1666 more](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [5bc22a5...0457284](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] ZihanLi58 commented on a change in pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
ZihanLi58 commented on a change in pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#discussion_r751810712
##########
File path: gobblin-iceberg/src/main/java/org/apache/gobblin/iceberg/writer/IcebergMetadataWriter.java
##########
@@ -1025,11 +1031,13 @@ public void writeEnvelope(RecordEnvelope<GenericRecord> recordEnvelope, Map<Stri
Optional.of(currentOffset - 1);
}
tableMetadataMap.get(tid).setDatasetName(gmce.getDatasetIdentifier().getNativeName());
Review comment:
We want to set those only when low water mark is not set (which means this is the first time we compute this table since last failure/ restart), so we can move those lines inside the block of line 1028?
##########
File path: gobblin-iceberg/src/main/java/org/apache/gobblin/iceberg/writer/IcebergMetadataWriter.java
##########
@@ -1078,6 +1086,10 @@ public void close() throws IOException {
long newCompletenessWatermark = DEFAULT_COMPLETION_WATERMARK;
SortedSet<ZonedDateTime> datePartitions = new TreeSet<>(Collections.reverseOrder());
+ @Setter
+ String dbName;
Review comment:
Why this two arg needed here?
##########
File path: gobblin-iceberg/src/main/java/org/apache/gobblin/iceberg/writer/IcebergMetadataWriter.java
##########
@@ -284,9 +287,8 @@ private Long getAndPersistCurrentWatermark(TableIdentifier tid, String topicPart
* information increases the memory footprints, therefore we would like to flush them eagerly).
*/
public void write(GobblinMetadataChangeEvent gmce, Map<String, Collection<HiveSpec>> newSpecsMap,
- Map<String, Collection<HiveSpec>> oldSpecsMap, HiveSpec tableSpec) throws IOException {
- TableIdentifier tid = TableIdentifier.of(tableSpec.getTable().getDbName(), tableSpec.getTable().getTableName());
- TableMetadata tableMetadata = tableMetadataMap.computeIfAbsent(tid, t -> new TableMetadata());
+ Map<String, Collection<HiveSpec>> oldSpecsMap, HiveSpec tableSpec, TableMetadata tableMetadata) throws IOException {
Review comment:
We can directly parse tid instead of tableMetadata here?
##########
File path: gobblin-iceberg/src/main/java/org/apache/gobblin/iceberg/writer/IcebergMetadataWriter.java
##########
@@ -684,7 +687,8 @@ protected void addFiles(GobblinMetadataChangeEvent gmce, Map<String, Collection<
Collection<HiveSpec> hiveSpecs = newSpecsMap.get(new Path(file.getFilePath()).getParent().toString());
StructLike partition = getIcebergPartitionVal(hiveSpecs, file.getFilePath(), partitionSpec);
- if(tableMetadata.completenessEnabled && gmce.getOperationType() == OperationType.add_files) {
+ if(this.newPartitionTableWhitelistBlacklist.acceptTable(tableMetadata.dbName, tableMetadata.tableName)
+ && gmce.getOperationType() == OperationType.add_files) {
tableMetadata.prevCompletenessWatermark = Long.parseLong(table.properties().getOrDefault(COMPLETION_WATERMARK_KEY,
Review comment:
What will happen if it's not add_files operation? i.e. what will happen for compaction?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#issuecomment-970443681
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3426](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9d64d8c) into [master](https://codecov.io/gh/apache/gobblin/commit/5bc22a5502ddeb810a35b0fb996dd9dfc8c81121?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5bc22a5) will **increase** coverage by `4.17%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3426 +/- ##
============================================
+ Coverage 43.29% 47.46% +4.17%
- Complexity 2017 8301 +6284
============================================
Files 402 1669 +1267
Lines 17325 63512 +46187
Branches 2121 6896 +4775
============================================
+ Hits 7500 30143 +22643
- Misses 8987 30731 +21744
- Partials 838 2638 +1800
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../gobblin/iceberg/writer/IcebergMetadataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlci5qYXZh) | | |
| [...ceberg/writer/IcebergMetadataWriterConfigKeys.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlckNvbmZpZ0tleXMuamF2YQ==) | | |
| [.../apache/gobblin/aws/AWSClusterSecurityManager.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hd3Mvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vYXdzL0FXU0NsdXN0ZXJTZWN1cml0eU1hbmFnZXIuamF2YQ==) | | |
| [...va/org/apache/gobblin/util/ExponentialBackoff.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvRXhwb25lbnRpYWxCYWNrb2ZmLmphdmE=) | | |
| [...n/java/org/apache/gobblin/salesforce/FileIdVO.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zYWxlc2ZvcmNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NhbGVzZm9yY2UvRmlsZUlkVk8uamF2YQ==) | | |
| [...in/java/org/apache/gobblin/util/DownloadUtils.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvRG93bmxvYWRVdGlscy5qYXZh) | | |
| [.../apache/gobblin/util/io/BatchedMeterDecorator.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvaW8vQmF0Y2hlZE1ldGVyRGVjb3JhdG9yLmphdmE=) | | |
| [...g/apache/gobblin/util/jdbc/DataSourceProvider.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvamRiYy9EYXRhU291cmNlUHJvdmlkZXIuamF2YQ==) | | |
| [...obblin/iceberg/writer/GobblinMCEWriterBuilder.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0dvYmJsaW5NQ0VXcml0ZXJCdWlsZGVyLmphdmE=) | | |
| [...org/apache/gobblin/config/client/ConfigClient.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb25maWctbWFuYWdlbWVudC9nb2JibGluLWNvbmZpZy1jbGllbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vY29uZmlnL2NsaWVudC9Db25maWdDbGllbnQuamF2YQ==) | | |
| ... and [2061 more](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [5bc22a5...9d64d8c](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#issuecomment-970443681
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3426](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0457284) into [master](https://codecov.io/gh/apache/gobblin/commit/5bc22a5502ddeb810a35b0fb996dd9dfc8c81121?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5bc22a5) will **increase** coverage by `5.82%`.
> The diff coverage is `76.47%`.
[](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3426 +/- ##
============================================
+ Coverage 43.29% 49.11% +5.82%
- Complexity 2017 8856 +6839
============================================
Files 402 1690 +1288
Lines 17325 65304 +47979
Branches 2121 7486 +5365
============================================
+ Hits 7500 32071 +24571
- Misses 8987 30207 +21220
- Partials 838 3026 +2188
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ceberg/writer/IcebergMetadataWriterConfigKeys.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlckNvbmZpZ0tleXMuamF2YQ==) | `0.00% <ø> (ø)` | |
| [.../gobblin/iceberg/writer/IcebergMetadataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlci5qYXZh) | `72.51% <76.47%> (+0.21%)` | :arrow_up: |
| [.../java/org/apache/gobblin/cluster/SleepingTask.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvU2xlZXBpbmdUYXNrLmphdmE=) | `39.39% <0.00%> (-6.07%)` | :arrow_down: |
| [...a/org/apache/gobblin/cluster/GobblinHelixTask.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpbkhlbGl4VGFzay5qYXZh) | `60.21% <0.00%> (-2.16%)` | :arrow_down: |
| [...estli/throttling/ThrottlingGuiceServletConfig.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL1Rocm90dGxpbmdHdWljZVNlcnZsZXRDb25maWcuamF2YQ==) | `70.88% <0.00%> (ø)` | |
| [...gobblin/runtime/std/JobLifecycleListenersList.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvc3RkL0pvYkxpZmVjeWNsZUxpc3RlbmVyc0xpc3QuamF2YQ==) | `58.69% <0.00%> (ø)` | |
| [...org/apache/gobblin/runtime/api/FsSpecProducer.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvYXBpL0ZzU3BlY1Byb2R1Y2VyLmphdmE=) | `74.07% <0.00%> (ø)` | |
| [...va/org/apache/gobblin/util/ClassAliasResolver.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vdXRpbC9DbGFzc0FsaWFzUmVzb2x2ZXIuamF2YQ==) | `80.00% <0.00%> (ø)` | |
| [...va/org/apache/gobblin/typedconfig/TypedConfig.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vdHlwZWRjb25maWcvVHlwZWRDb25maWcuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...gobblin/runtime/listeners/AbstractJobListener.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvbGlzdGVuZXJzL0Fic3RyYWN0Sm9iTGlzdGVuZXIuamF2YQ==) | `76.19% <0.00%> (ø)` | |
| ... and [1285 more](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [5bc22a5...0457284](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#issuecomment-970443681
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3426](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c4ff266) into [master](https://codecov.io/gh/apache/gobblin/commit/5bc22a5502ddeb810a35b0fb996dd9dfc8c81121?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5bc22a5) will **increase** coverage by `5.83%`.
> The diff coverage is `70.58%`.
[](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3426 +/- ##
============================================
+ Coverage 43.29% 49.12% +5.83%
- Complexity 2017 8860 +6843
============================================
Files 402 1690 +1288
Lines 17325 65301 +47976
Branches 2121 7486 +5365
============================================
+ Hits 7500 32077 +24577
- Misses 8987 30202 +21215
- Partials 838 3022 +2184
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ceberg/writer/IcebergMetadataWriterConfigKeys.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlckNvbmZpZ0tleXMuamF2YQ==) | `0.00% <ø> (ø)` | |
| [.../gobblin/iceberg/writer/IcebergMetadataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlci5qYXZh) | `72.18% <70.58%> (-0.12%)` | :arrow_down: |
| [...a/org/apache/gobblin/cluster/GobblinHelixTask.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpbkhlbGl4VGFzay5qYXZh) | `60.21% <0.00%> (-2.16%)` | :arrow_down: |
| [...ent/conversion/hive/events/EventWorkunitUtils.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvbnZlcnNpb24vaGl2ZS9ldmVudHMvRXZlbnRXb3JrdW5pdFV0aWxzLmphdmE=) | `61.11% <0.00%> (ø)` | |
| [...org/apache/gobblin/scheduler/SchedulerService.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NjaGVkdWxlci9TY2hlZHVsZXJTZXJ2aWNlLmphdmE=) | `72.72% <0.00%> (ø)` | |
| [...va/org/apache/gobblin/writer/WatermarkManager.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vd3JpdGVyL1dhdGVybWFya01hbmFnZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...org/apache/gobblin/publisher/DataPublisherKey.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3B1Ymxpc2hlci9EYXRhUHVibGlzaGVyS2V5LmphdmE=) | `66.66% <0.00%> (ø)` | |
| [...e/gobblin/hive/HiveRegistrationUnitComparator.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1oaXZlLXJlZ2lzdHJhdGlvbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9oaXZlL0hpdmVSZWdpc3RyYXRpb25Vbml0Q29tcGFyYXRvci5qYXZh) | `17.85% <0.00%> (ø)` | |
| [...he/gobblin/restli/throttling/CountBasedPolicy.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL0NvdW50QmFzZWRQb2xpY3kuamF2YQ==) | `86.95% <0.00%> (ø)` | |
| [...lin/service/modules/flowgraph/FlowEdgeFactory.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9mbG93Z3JhcGgvRmxvd0VkZ2VGYWN0b3J5LmphdmE=) | `0.00% <0.00%> (ø)` | |
| ... and [1284 more](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [5bc22a5...c4ff266](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#issuecomment-970443681
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3426](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (74bbc27) into [master](https://codecov.io/gh/apache/gobblin/commit/5bc22a5502ddeb810a35b0fb996dd9dfc8c81121?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5bc22a5) will **increase** coverage by `5.81%`.
> The diff coverage is `80.00%`.
[](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3426 +/- ##
============================================
+ Coverage 43.29% 49.10% +5.81%
- Complexity 2017 8856 +6839
============================================
Files 402 1690 +1288
Lines 17325 65303 +47978
Branches 2121 7486 +5365
============================================
+ Hits 7500 32069 +24569
- Misses 8987 30212 +21225
- Partials 838 3022 +2184
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ceberg/writer/IcebergMetadataWriterConfigKeys.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlckNvbmZpZ0tleXMuamF2YQ==) | `0.00% <ø> (ø)` | |
| [.../gobblin/iceberg/writer/IcebergMetadataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlci5qYXZh) | `72.46% <80.00%> (+0.17%)` | :arrow_up: |
| [.../java/org/apache/gobblin/cluster/SleepingTask.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvU2xlZXBpbmdUYXNrLmphdmE=) | `39.39% <0.00%> (-6.07%)` | :arrow_down: |
| [...obblin/runtime/job\_catalog/InMemoryJobCatalog.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvam9iX2NhdGFsb2cvSW5NZW1vcnlKb2JDYXRhbG9nLmphdmE=) | `100.00% <0.00%> (ø)` | |
| [...main/java/org/apache/gobblin/yarn/YarnService.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi15YXJuL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3lhcm4vWWFyblNlcnZpY2UuamF2YQ==) | `15.46% <0.00%> (ø)` | |
| [...rg/apache/gobblin/runtime/FsDatasetStateStore.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvRnNEYXRhc2V0U3RhdGVTdG9yZS5qYXZh) | `74.40% <0.00%> (ø)` | |
| [...n/mapreduce/avro/AvroKeyCompactorOutputFormat.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb21wYWN0aW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbXBhY3Rpb24vbWFwcmVkdWNlL2F2cm8vQXZyb0tleUNvbXBhY3Rvck91dHB1dEZvcm1hdC5qYXZh) | `100.00% <0.00%> (ø)` | |
| [...main/java/gobblin/configuration/WorkUnitState.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9nb2JibGluL2NvbmZpZ3VyYXRpb24vV29ya1VuaXRTdGF0ZS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [.../org/apache/gobblin/dataset/DatasetDescriptor.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YXNldC9EYXRhc2V0RGVzY3JpcHRvci5qYXZh) | `91.30% <0.00%> (ø)` | |
| [...set/test/SimplePartitionableDatasetForTesting.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YXNldC90ZXN0L1NpbXBsZVBhcnRpdGlvbmFibGVEYXRhc2V0Rm9yVGVzdGluZy5qYXZh) | `0.00% <0.00%> (ø)` | |
| ... and [1284 more](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [5bc22a5...74bbc27](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#issuecomment-970443681
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3426](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (74bbc27) into [master](https://codecov.io/gh/apache/gobblin/commit/5bc22a5502ddeb810a35b0fb996dd9dfc8c81121?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5bc22a5) will **increase** coverage by `5.14%`.
> The diff coverage is `80.00%`.
[](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3426 +/- ##
============================================
+ Coverage 43.29% 48.43% +5.14%
- Complexity 2017 7689 +5672
============================================
Files 402 1437 +1035
Lines 17325 56779 +39454
Branches 2121 6540 +4419
============================================
+ Hits 7500 27500 +20000
- Misses 8987 26715 +17728
- Partials 838 2564 +1726
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ceberg/writer/IcebergMetadataWriterConfigKeys.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlckNvbmZpZ0tleXMuamF2YQ==) | `0.00% <ø> (ø)` | |
| [.../gobblin/iceberg/writer/IcebergMetadataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlci5qYXZh) | `72.46% <80.00%> (+0.17%)` | :arrow_up: |
| [.../java/org/apache/gobblin/cluster/SleepingTask.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvU2xlZXBpbmdUYXNrLmphdmE=) | `39.39% <0.00%> (-6.07%)` | :arrow_down: |
| [.../gobblin/metrics/context/ContextWeakReference.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9jb250ZXh0L0NvbnRleHRXZWFrUmVmZXJlbmNlLmphdmE=) | `100.00% <0.00%> (ø)` | |
| [...e/gobblin/runtime/retention/DatasetCleanerCli.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vcnVudGltZS9yZXRlbnRpb24vRGF0YXNldENsZWFuZXJDbGkuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [.../gobblin/source/DatePartitionedJsonFileSource.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9EYXRlUGFydGl0aW9uZWRKc29uRmlsZVNvdXJjZS5qYXZh) | `66.66% <0.00%> (ø)` | |
| [...ment/copy/replication/HadoopFsEndPointFactory.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvcmVwbGljYXRpb24vSGFkb29wRnNFbmRQb2ludEZhY3RvcnkuamF2YQ==) | `100.00% <0.00%> (ø)` | |
| [...onverter/avro/JsonIntermediateToAvroConverter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbnZlcnRlci9hdnJvL0pzb25JbnRlcm1lZGlhdGVUb0F2cm9Db252ZXJ0ZXIuamF2YQ==) | `34.28% <0.00%> (ø)` | |
| [...in/java/org/apache/gobblin/runtime/JobContext.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvSm9iQ29udGV4dC5qYXZh) | `77.24% <0.00%> (ø)` | |
| [...pache/gobblin/runtime/mapreduce/MRTaskFactory.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvbWFwcmVkdWNlL01SVGFza0ZhY3RvcnkuamF2YQ==) | `100.00% <0.00%> (ø)` | |
| ... and [1031 more](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [5bc22a5...74bbc27](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#issuecomment-970443681
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3426](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c4ff266) into [master](https://codecov.io/gh/apache/gobblin/commit/5bc22a5502ddeb810a35b0fb996dd9dfc8c81121?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5bc22a5) will **increase** coverage by `3.44%`.
> The diff coverage is `70.58%`.
[](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3426 +/- ##
============================================
+ Coverage 43.29% 46.73% +3.44%
- Complexity 2017 3190 +1173
============================================
Files 402 655 +253
Lines 17325 25854 +8529
Branches 2121 3069 +948
============================================
+ Hits 7500 12083 +4583
- Misses 8987 12474 +3487
- Partials 838 1297 +459
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ceberg/writer/IcebergMetadataWriterConfigKeys.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlckNvbmZpZ0tleXMuamF2YQ==) | `0.00% <ø> (ø)` | |
| [.../gobblin/iceberg/writer/IcebergMetadataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlci5qYXZh) | `72.18% <70.58%> (-0.12%)` | :arrow_down: |
| [...a/org/apache/gobblin/cluster/GobblinHelixTask.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpbkhlbGl4VGFzay5qYXZh) | `60.21% <0.00%> (-2.16%)` | :arrow_down: |
| [...lin/restli/throttling/ThrottlingPolicyFactory.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL1Rocm90dGxpbmdQb2xpY3lGYWN0b3J5LmphdmE=) | `73.33% <0.00%> (ø)` | |
| [...lin/service/modules/flowgraph/FlowEdgeFactory.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9mbG93Z3JhcGgvRmxvd0VkZ2VGYWN0b3J5LmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...ervice/modules/core/GobblinServiceGuiceModule.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9jb3JlL0dvYmJsaW5TZXJ2aWNlR3VpY2VNb2R1bGUuamF2YQ==) | `89.42% <0.00%> (ø)` | |
| [...n/service/modules/template/StaticFlowTemplate.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy90ZW1wbGF0ZS9TdGF0aWNGbG93VGVtcGxhdGUuamF2YQ==) | `87.67% <0.00%> (ø)` | |
| [...he/gobblin/broker/StringNameSharedResourceKey.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vYnJva2VyL1N0cmluZ05hbWVTaGFyZWRSZXNvdXJjZUtleS5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...rg/apache/gobblin/dataset/PartitionDescriptor.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YXNldC9QYXJ0aXRpb25EZXNjcmlwdG9yLmphdmE=) | `38.88% <0.00%> (ø)` | |
| [...service/modules/orchestration/FSDagStateStore.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9vcmNoZXN0cmF0aW9uL0ZTRGFnU3RhdGVTdG9yZS5qYXZh) | `84.44% <0.00%> (ø)` | |
| ... and [249 more](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [5bc22a5...c4ff266](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] ZihanLi58 merged pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
ZihanLi58 merged pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] vikrambohra commented on a change in pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
vikrambohra commented on a change in pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#discussion_r751882607
##########
File path: gobblin-iceberg/src/main/java/org/apache/gobblin/iceberg/writer/IcebergMetadataWriter.java
##########
@@ -1078,6 +1086,10 @@ public void close() throws IOException {
long newCompletenessWatermark = DEFAULT_COMPLETION_WATERMARK;
SortedSet<ZonedDateTime> datePartitions = new TreeSet<>(Collections.reverseOrder());
+ @Setter
+ String dbName;
Review comment:
same comment as below
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#issuecomment-970443681
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3426](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c4ff266) into [master](https://codecov.io/gh/apache/gobblin/commit/5bc22a5502ddeb810a35b0fb996dd9dfc8c81121?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5bc22a5) will **increase** coverage by `3.28%`.
> The diff coverage is `70.58%`.
[](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3426 +/- ##
============================================
+ Coverage 43.29% 46.57% +3.28%
- Complexity 2017 10324 +8307
============================================
Files 402 2071 +1669
Lines 17325 80842 +63517
Branches 2121 9019 +6898
============================================
+ Hits 7500 37656 +30156
- Misses 8987 39711 +30724
- Partials 838 3475 +2637
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ceberg/writer/IcebergMetadataWriterConfigKeys.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlckNvbmZpZ0tleXMuamF2YQ==) | `0.00% <ø> (ø)` | |
| [.../gobblin/iceberg/writer/IcebergMetadataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlci5qYXZh) | `72.18% <70.58%> (-0.12%)` | :arrow_down: |
| [...a/org/apache/gobblin/cluster/GobblinHelixTask.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpbkhlbGl4VGFzay5qYXZh) | `60.21% <0.00%> (-2.16%)` | :arrow_down: |
| [...obblin/ingestion/google/webmaster/UrlTrieNode.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvb2dsZS1pbmdlc3Rpb24vc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vaW5nZXN0aW9uL2dvb2dsZS93ZWJtYXN0ZXIvVXJsVHJpZU5vZGUuamF2YQ==) | `90.62% <0.00%> (ø)` | |
| [...untime/listeners/AbstractCloseableJobListener.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvbGlzdGVuZXJzL0Fic3RyYWN0Q2xvc2VhYmxlSm9iTGlzdGVuZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [...lin/data/management/retention/RetentionEvents.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L3JldGVudGlvbi9SZXRlbnRpb25FdmVudHMuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [.../org/apache/gobblin/writer/AsyncWriterManager.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vd3JpdGVyL0FzeW5jV3JpdGVyTWFuYWdlci5qYXZh) | `0.00% <0.00%> (ø)` | |
| [...ersion/finder/ModDateTimeDatasetVersionFinder.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L3ZlcnNpb24vZmluZGVyL01vZERhdGVUaW1lRGF0YXNldFZlcnNpb25GaW5kZXIuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [.../apache/gobblin/http/ApacheHttpRequestBuilder.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4taHR0cC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9odHRwL0FwYWNoZUh0dHBSZXF1ZXN0QnVpbGRlci5qYXZh) | `59.45% <0.00%> (ø)` | |
| [...ache/gobblin/runtime/plugins/PluginStaticKeys.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvcGx1Z2lucy9QbHVnaW5TdGF0aWNLZXlzLmphdmE=) | `0.00% <0.00%> (ø)` | |
| ... and [1665 more](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [5bc22a5...c4ff266](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#issuecomment-970443681
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3426](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (74bbc27) into [master](https://codecov.io/gh/apache/gobblin/commit/5bc22a5502ddeb810a35b0fb996dd9dfc8c81121?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5bc22a5) will **increase** coverage by `3.27%`.
> The diff coverage is `80.00%`.
[](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3426 +/- ##
============================================
+ Coverage 43.29% 46.56% +3.27%
- Complexity 2017 10320 +8303
============================================
Files 402 2071 +1669
Lines 17325 80844 +63519
Branches 2121 9019 +6898
============================================
+ Hits 7500 37648 +30148
- Misses 8987 39721 +30734
- Partials 838 3475 +2637
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...ceberg/writer/IcebergMetadataWriterConfigKeys.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlckNvbmZpZ0tleXMuamF2YQ==) | `0.00% <ø> (ø)` | |
| [.../gobblin/iceberg/writer/IcebergMetadataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlci5qYXZh) | `72.46% <80.00%> (+0.17%)` | :arrow_up: |
| [.../java/org/apache/gobblin/cluster/SleepingTask.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvU2xlZXBpbmdUYXNrLmphdmE=) | `39.39% <0.00%> (-6.07%)` | :arrow_down: |
| [...runtime/plugins/email/EmailNotificationPlugin.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvcGx1Z2lucy9lbWFpbC9FbWFpbE5vdGlmaWNhdGlvblBsdWdpbi5qYXZh) | `41.30% <0.00%> (ø)` | |
| [...dataset/test/SimpleDatasetPartitionForTesting.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YXNldC90ZXN0L1NpbXBsZURhdGFzZXRQYXJ0aXRpb25Gb3JUZXN0aW5nLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [.../management/copy/replication/HadoopFsEndPoint.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvcmVwbGljYXRpb24vSGFkb29wRnNFbmRQb2ludC5qYXZh) | `9.52% <0.00%> (ø)` | |
| [...ersion/finder/ModDateTimeDatasetVersionFinder.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L3JldGVudGlvbi92ZXJzaW9uL2ZpbmRlci9Nb2REYXRlVGltZURhdGFzZXRWZXJzaW9uRmluZGVyLmphdmE=) | `0.00% <0.00%> (ø)` | |
| [...bblin/runtime/spec\_serde/GenericGsonSpecSerDe.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvc3BlY19zZXJkZS9HZW5lcmljR3NvblNwZWNTZXJEZS5qYXZh) | `50.00% <0.00%> (ø)` | |
| [...etention/policy/predicates/WhitelistPredicate.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L3JldGVudGlvbi9wb2xpY3kvcHJlZGljYXRlcy9XaGl0ZWxpc3RQcmVkaWNhdGUuamF2YQ==) | `0.00% <0.00%> (ø)` | |
| [.../org/apache/gobblin/configuration/SourceState.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vY29uZmlndXJhdGlvbi9Tb3VyY2VTdGF0ZS5qYXZh) | `7.89% <0.00%> (ø)` | |
| ... and [1665 more](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [5bc22a5...74bbc27](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] codecov-commenter edited a comment on pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
codecov-commenter edited a comment on pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#issuecomment-970443681
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3426](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9d64d8c) into [master](https://codecov.io/gh/apache/gobblin/commit/5bc22a5502ddeb810a35b0fb996dd9dfc8c81121?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5bc22a5) will **increase** coverage by `7.91%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3426 +/- ##
============================================
+ Coverage 43.29% 51.20% +7.91%
- Complexity 2017 6837 +4820
============================================
Files 402 1288 +886
Lines 17325 47971 +30646
Branches 2121 5363 +3242
============================================
+ Hits 7500 24564 +17064
- Misses 8987 21222 +12235
- Partials 838 2185 +1347
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../gobblin/iceberg/writer/IcebergMetadataWriter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlci5qYXZh) | | |
| [...ceberg/writer/IcebergMetadataWriterConfigKeys.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1pY2ViZXJnL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2ljZWJlcmcvd3JpdGVyL0ljZWJlcmdNZXRhZGF0YVdyaXRlckNvbmZpZ0tleXMuamF2YQ==) | | |
| [.../java/org/apache/gobblin/admin/AdminWebServer.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hZG1pbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9hZG1pbi9BZG1pbldlYlNlcnZlci5qYXZh) | | |
| [...rg/apache/gobblin/util/io/MeteredOutputStream.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvaW8vTWV0ZXJlZE91dHB1dFN0cmVhbS5qYXZh) | | |
| [...a/org/apache/gobblin/cluster/HelixJobsMapping.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhKb2JzTWFwcGluZy5qYXZh) | | |
| [...ava/org/apache/gobblin/util/ForkOperatorUtils.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvRm9ya09wZXJhdG9yVXRpbHMuamF2YQ==) | | |
| [.../org/apache/gobblin/metastore/MetaStoreModule.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tZXRhc3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL01ldGFTdG9yZU1vZHVsZS5qYXZh) | | |
| [...obblin/metastore/JobHistoryDataSourceProvider.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tZXRhc3RvcmUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0YXN0b3JlL0pvYkhpc3RvcnlEYXRhU291cmNlUHJvdmlkZXIuamF2YQ==) | | |
| [...org/apache/gobblin/util/filters/AndPathFilter.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsdGVycy9BbmRQYXRoRmlsdGVyLmphdmE=) | | |
| [...ava/org/apache/gobblin/util/AutoCloseableLock.java](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvQXV0b0Nsb3NlYWJsZUxvY2suamF2YQ==) | | |
| ... and [1680 more](https://codecov.io/gh/apache/gobblin/pull/3426/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [5bc22a5...9d64d8c](https://codecov.io/gh/apache/gobblin/pull/3426?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [gobblin] vikrambohra commented on a change in pull request #3426: [GOBBLIN-1574] Added whitelist for iceberg tables to add new partitio…
Posted by GitBox <gi...@apache.org>.
vikrambohra commented on a change in pull request #3426:
URL: https://github.com/apache/gobblin/pull/3426#discussion_r751882838
##########
File path: gobblin-iceberg/src/main/java/org/apache/gobblin/iceberg/writer/IcebergMetadataWriter.java
##########
@@ -684,7 +687,8 @@ protected void addFiles(GobblinMetadataChangeEvent gmce, Map<String, Collection<
Collection<HiveSpec> hiveSpecs = newSpecsMap.get(new Path(file.getFilePath()).getParent().toString());
StructLike partition = getIcebergPartitionVal(hiveSpecs, file.getFilePath(), partitionSpec);
- if(tableMetadata.completenessEnabled && gmce.getOperationType() == OperationType.add_files) {
+ if(this.newPartitionTableWhitelistBlacklist.acceptTable(tableMetadata.dbName, tableMetadata.tableName)
+ && gmce.getOperationType() == OperationType.add_files) {
tableMetadata.prevCompletenessWatermark = Long.parseLong(table.properties().getOrDefault(COMPLETION_WATERMARK_KEY,
Review comment:
we will only white list kafka_streaming_hourly tables and re_write files will never occur for them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org