You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/06/03 23:55:00 UTC
[jira] [Work logged] (GOBBLIN-1179) Add a typed config to replace
properties
[ https://issues.apache.org/jira/browse/GOBBLIN-1179?focusedWorklogId=441035&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441035 ]
ASF GitHub Bot logged work on GOBBLIN-1179:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 03/Jun/20 23:54
Start Date: 03/Jun/20 23:54
Worklog Time Spent: 10m
Work Description: codecov-commenter commented on pull request #2910:
URL: https://github.com/apache/incubator-gobblin/pull/2910#issuecomment-638520374
# [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2910?src=pr&el=h1) Report
> Merging [#2910](https://codecov.io/gh/apache/incubator-gobblin/pull/2910?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/469a56d9161c404c6dc73e8fe927fe2bb8922ddd&el=desc) will **decrease** coverage by `0.07%`.
> The diff coverage is `0.00%`.
[](https://codecov.io/gh/apache/incubator-gobblin/pull/2910?src=pr&el=tree)
```diff
@@ Coverage Diff @@
## master #2910 +/- ##
============================================
- Coverage 45.72% 45.65% -0.08%
+ Complexity 9295 9292 -3
============================================
Files 1951 1955 +4
Lines 74255 74366 +111
Branches 8218 8246 +28
============================================
- Hits 33953 33950 -3
- Misses 37129 37243 +114
Partials 3173 3173
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2910?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
|---|---|---|---|
| [...che/gobblin/salesforce/QueryBasedSourceConfig.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2910/diff?src=pr&el=tree#diff-Z29iYmxpbi1zYWxlc2ZvcmNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NhbGVzZm9yY2UvUXVlcnlCYXNlZFNvdXJjZUNvbmZpZy5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | |
| [...apache/gobblin/salesforce/SalesforceExtractor.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2910/diff?src=pr&el=tree#diff-Z29iYmxpbi1zYWxlc2ZvcmNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NhbGVzZm9yY2UvU2FsZXNmb3JjZUV4dHJhY3Rvci5qYXZh) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (ø)` | |
| [...n/java/org/apache/gobblin/salesforce/SfConfig.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2910/diff?src=pr&el=tree#diff-Z29iYmxpbi1zYWxlc2ZvcmNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NhbGVzZm9yY2UvU2ZDb25maWcuamF2YQ==) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | |
| [...org/apache/gobblin/typedconfig/ConstraintUtil.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2910/diff?src=pr&el=tree#diff-Z29iYmxpbi1zYWxlc2ZvcmNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3R5cGVkY29uZmlnL0NvbnN0cmFpbnRVdGlsLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | |
| [...va/org/apache/gobblin/typedconfig/TypedConfig.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2910/diff?src=pr&el=tree#diff-Z29iYmxpbi1zYWxlc2ZvcmNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3R5cGVkY29uZmlnL1R5cGVkQ29uZmlnLmphdmE=) | `0.00% <0.00%> (ø)` | `0.00 <0.00> (?)` | |
| [...lin/util/filesystem/FileSystemInstrumentation.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2910/diff?src=pr&el=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsZXN5c3RlbS9GaWxlU3lzdGVtSW5zdHJ1bWVudGF0aW9uLmphdmE=) | `85.71% <0.00%> (-7.15%)` | `3.00% <0.00%> (ø%)` | |
| [...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2910/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh) | `34.92% <0.00%> (-5.56%)` | `14.00% <0.00%> (-2.00%)` | |
| [.../org/apache/gobblin/metrics/RootMetricContext.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2910/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9Sb290TWV0cmljQ29udGV4dC5qYXZh) | `78.12% <0.00%> (-1.57%)` | `15.00% <0.00%> (-1.00%)` | |
| ... and [5 more](https://codecov.io/gh/apache/incubator-gobblin/pull/2910/diff?src=pr&el=tree-more) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2910?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2910?src=pr&el=footer). Last update [469a56d...5f662b6](https://codecov.io/gh/apache/incubator-gobblin/pull/2910?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 441035)
Remaining Estimate: 0h
Time Spent: 10m
> Add a typed config to replace properties
> ----------------------------------------
>
> Key: GOBBLIN-1179
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1179
> Project: Apache Gobblin
> Issue Type: Task
> Reporter: Alex Li
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Add a typed config to replace *_properties.get(“ini.file.userName”)_* with *config.userName*
> The gobblin config file is an ini file. Java loads the ini file to a properties instance. The way to use the config information is to get from the properties.
> |workUnitState.getPropAsBoolean(BULK_API_USE_QUERY_ALL)|
> |workUnitState.getPropAsInt(FETCH_RETRY_LIMIT_KEY, DEFAULT_FETCH_RETRY_LIMIT)|
> |Math.max(MIN_SIZE,Math.min(MAX_SIZE, workUnitState.getPropAsInt(PARTITION_SIZE, DEFAULT_SIZE)));
> // partition size must be >= min and <= max, otherwise use default|
> Problems
> # No consistent key naming model
> * A long dot-separated key string is used, easy to run into typos. The config code is pretty verbose: We use *properties.getProp(key, default)*
> * Key collision if the same type is used in multiple places, e.g kafka.brokers
> # No ownership management
> * in gobblin connector package: We have multiple constant static classes. GobblinKeys, QueryBaseKeys, GaapKeys, and SalesforceConnectorKeys.We can even directly read config values by state.getProp(*“my.key”*) without creating any constant key.
> # No static validation
> * Required & default value
> * Type check
> * Date range
> * Enum
> # No dependency check
> * If users set to *useGaap=true*, there must be *gaap.url* and *gaap.credential*. And this needs to be verified at both runtime and compile time.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)