You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/11/18 02:10:00 UTC
[jira] [Work logged] (GOBBLIN-1578) deregistration of hive table should not delete data
[ https://issues.apache.org/jira/browse/GOBBLIN-1578?focusedWorklogId=683038&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-683038 ]
ASF GitHub Bot logged work on GOBBLIN-1578:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 18/Nov/21 02:09
Start Date: 18/Nov/21 02:09
Worklog Time Spent: 10m
Work Description: codecov-commenter commented on pull request #3431:
URL: https://github.com/apache/gobblin/pull/3431#issuecomment-972435745
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3431?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3431](https://codecov.io/gh/apache/gobblin/pull/3431?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25b22d6) into [master](https://codecov.io/gh/apache/gobblin/commit/62a434474d96cdf63bd4913846f6ca07f8c9e5e9?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (62a4344) will **decrease** coverage by `3.26%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/gobblin/pull/3431?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #3431 +/- ##
============================================
- Coverage 46.57% 43.30% -3.27%
+ Complexity 10316 2019 -8297
============================================
Files 2071 402 -1669
Lines 80805 17325 -63480
Branches 9009 2121 -6888
============================================
- Hits 37631 7503 -30128
+ Misses 39694 8983 -30711
+ Partials 3480 839 -2641
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3431?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...a/org/apache/gobblin/util/limiter/NoopLimiter.java](https://codecov.io/gh/apache/gobblin/pull/3431/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvbGltaXRlci9Ob29wTGltaXRlci5qYXZh) | `40.00% <0.00%> (-20.00%)` | :arrow_down: |
| [.../org/apache/gobblin/cluster/GobblinTaskRunner.java](https://codecov.io/gh/apache/gobblin/pull/3431/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpblRhc2tSdW5uZXIuamF2YQ==) | `63.46% <0.00%> (-0.33%)` | :arrow_down: |
| [...lin/util/filesystem/FileSystemInstrumentation.java](https://codecov.io/gh/apache/gobblin/pull/3431/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsZXN5c3RlbS9GaWxlU3lzdGVtSW5zdHJ1bWVudGF0aW9uLmphdmE=) | `92.85% <0.00%> (ø)` | |
| [...lin/hive/metastore/HiveMetaStoreBasedRegister.java](https://codecov.io/gh/apache/gobblin/pull/3431/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1oaXZlLXJlZ2lzdHJhdGlvbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9oaXZlL21ldGFzdG9yZS9IaXZlTWV0YVN0b3JlQmFzZWRSZWdpc3Rlci5qYXZh) | | |
| [.../gobblin/configuration/ConfigurationException.java](https://codecov.io/gh/apache/gobblin/pull/3431/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vY29uZmlndXJhdGlvbi9Db25maWd1cmF0aW9uRXhjZXB0aW9uLmphdmE=) | | |
| [.../service/modules/orchestration/AzkabanSuccess.java](https://codecov.io/gh/apache/gobblin/pull/3431/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tYXprYWJhbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9zZXJ2aWNlL21vZHVsZXMvb3JjaGVzdHJhdGlvbi9BemthYmFuU3VjY2Vzcy5qYXZh) | | |
| [.../org/apache/gobblin/commit/FsRenameCommitStep.java](https://codecov.io/gh/apache/gobblin/pull/3431/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NvbW1pdC9Gc1JlbmFtZUNvbW1pdFN0ZXAuamF2YQ==) | | |
| [...a/org/apache/gobblin/hive/spec/SimpleHiveSpec.java](https://codecov.io/gh/apache/gobblin/pull/3431/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1oaXZlLXJlZ2lzdHJhdGlvbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9oaXZlL3NwZWMvU2ltcGxlSGl2ZVNwZWMuamF2YQ==) | | |
| [...service/modules/dataset/HiveDatasetDescriptor.java](https://codecov.io/gh/apache/gobblin/pull/3431/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9kYXRhc2V0L0hpdmVEYXRhc2V0RGVzY3JpcHRvci5qYXZh) | | |
| [.../extractor/filebased/FileBasedHelperException.java](https://codecov.io/gh/apache/gobblin/pull/3431/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NvdXJjZS9leHRyYWN0b3IvZmlsZWJhc2VkL0ZpbGVCYXNlZEhlbHBlckV4Y2VwdGlvbi5qYXZh) | | |
| ... and [1662 more](https://codecov.io/gh/apache/gobblin/pull/3431/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/gobblin/pull/3431?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/gobblin/pull/3431?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [62a4344...25b22d6](https://codecov.io/gh/apache/gobblin/pull/3431?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 683038)
Remaining Estimate: 0h
Time Spent: 10m
> deregistration of hive table should not delete data
> ---------------------------------------------------
>
> Key: GOBBLIN-1578
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1578
> Project: Apache Gobblin
> Issue Type: Bug
> Reporter: Arjun Singh Bora
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I found that Gobblin actually is not respecting the config hive.dataset.copy.deregister.fileDeleteMethod for unpartitioned tables
> Here we call dropTable with default params [https://github.com/apache/gobblin/blob/master/gobblin-hive-registration/src/main/j[…]g/apache/gobblin/hive/metastore/HiveMetaStoreBasedRegister.java|https://github.com/apache/gobblin/blob/master/gobblin-hive-registration/src/main/java/org/apache/gobblin/hive/metastore/HiveMetaStoreBasedRegister.java#L528], which deletes the data as well. We should be calling it with deleteData=false like in [https://github.com/apache/gobblin/blob/master/gobblin-hive-registration/src/main/j[…]g/apache/gobblin/hive/metastore/HiveMetaStoreBasedRegister.java|https://github.com/apache/gobblin/blob/master/gobblin-hive-registration/src/main/java/org/apache/gobblin/hive/metastore/HiveMetaStoreBasedRegister.java#L549]
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)