You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by GitBox <gi...@apache.org> on 2020/02/14 02:24:14 UTC

[GitHub] [incubator-gobblin] linweihs opened a new pull request #2891: Emit Helix Leader Metrics

linweihs opened a new pull request #2891: Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891
 
 
   Dear Gobblin maintainers,
   
   Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
   
   
   ### JIRA
   - [ ] My PR addresses the following [Gobblin JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
       - https://issues.apache.org/jira/browse/GOBBLIN-1051
   
   
   ### Description
   - [ ] Here are some details about my PR, including screenshots (if applicable):
   
   ![Uploading Screen Shot 2020-02-13 at 5.58.49 PM.png…]()
   
   
   ### Tests
   - [ ] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason:
   
   Test locally
   
   ### Commits
   - [ ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
       1. Subject is separated from body by a blank line
       2. Subject is limited to 50 characters
       3. Subject does not end with a period
       4. Subject uses the imperative mood ("add", not "adding")
       5. Body wraps at 72 characters
       6. Body explains "what" and "why", not "how"
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-gobblin] arjun4084346 commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics

Posted by GitBox <gi...@apache.org>.
arjun4084346 commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891#discussion_r379702360
 
 

 ##########
 File path: gobblin-service/src/main/java/org/apache/gobblin/service/modules/core/GobblinServiceManager.java
 ##########
 @@ -58,15 +35,30 @@
 import com.linkedin.restli.server.resources.BaseResource;
 import com.typesafe.config.Config;
 import com.typesafe.config.ConfigFactory;
-
+import java.io.IOException;
+import java.lang.reflect.InvocationTargetException;
 
 Review comment:
   this is not correct. please follow https://gobblin.readthedocs.io/en/latest/developer-guide/CodingStyle/ and use gobblin's codestyle

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-gobblin] codecov-io edited a comment on issue #2891: [GOBBLIN-1051] Emit Helix Leader Metrics

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #2891: [GOBBLIN-1051] Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891#issuecomment-586523868
 
 
   # [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=h1) Report
   > Merging [#2891](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/0b059870755bbab93f11d1cf620862e7502b2d4c?src=pr&el=desc) will **increase** coverage by `1.05%`.
   > The diff coverage is `79.16%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/graphs/tree.svg?width=650&token=4MgURJ0bGc&height=150&src=pr)](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #2891      +/-   ##
   ============================================
   + Coverage     44.78%   45.83%   +1.05%     
   - Complexity     8958     9151     +193     
   ============================================
     Files          1926     1929       +3     
     Lines         72555    72649      +94     
     Branches       7992     7999       +7     
   ============================================
   + Hits          32491    33296     +805     
   + Misses        37073    36314     -759     
   - Partials       2991     3039      +48
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [...org/apache/gobblin/metrics/ServiceMetricNames.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL21ldHJpY3MvU2VydmljZU1ldHJpY05hbWVzLmphdmE=) | `0% <ø> (ø)` | `0 <0> (ø)` | :arrow_down: |
   | [...in/service/modules/core/GobblinServiceManager.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9jb3JlL0dvYmJsaW5TZXJ2aWNlTWFuYWdlci5qYXZh) | `56.74% <79.16%> (+2.03%)` | `25 <0> (ø)` | :arrow_down: |
   | [...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh) | `36.28% <0%> (-2.97%)` | `13% <0%> (ø)` | |
   | [...lin/restli/throttling/ZookeeperLeaderElection.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL1pvb2tlZXBlckxlYWRlckVsZWN0aW9uLmphdmE=) | `70% <0%> (-2.23%)` | `13% <0%> (ø)` | |
   | [...lin/elasticsearch/writer/FutureCallbackHolder.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4tZWxhc3RpY3NlYXJjaC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9lbGFzdGljc2VhcmNoL3dyaXRlci9GdXR1cmVDYWxsYmFja0hvbGRlci5qYXZh) | `61.42% <0%> (-1.43%)` | `4% <0%> (ø)` | |
   | [...main/java/org/apache/gobblin/util/HadoopUtils.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvSGFkb29wVXRpbHMuamF2YQ==) | `30.2% <0%> (-0.68%)` | `24% <0%> (-1%)` | |
   | [...rg/apache/gobblin/publisher/BaseDataPublisher.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3B1Ymxpc2hlci9CYXNlRGF0YVB1Ymxpc2hlci5qYXZh) | `67.74% <0%> (-0.31%)` | `61% <0%> (-1%)` | |
   | [.../org/apache/gobblin/cluster/GobblinTaskRunner.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpblRhc2tSdW5uZXIuamF2YQ==) | `64.62% <0%> (-0.29%)` | `28% <0%> (-1%)` | |
   | [...ava/org/apache/gobblin/metrics/GobblinMetrics.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL21ldHJpY3MvR29iYmxpbk1ldHJpY3MuamF2YQ==) | `18.8% <0%> (-0.06%)` | `15% <0%> (ø)` | |
   | [...lin/util/filesystem/FileSystemInstrumentation.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsZXN5c3RlbS9GaWxlU3lzdGVtSW5zdHJ1bWVudGF0aW9uLmphdmE=) | `92.85% <0%> (ø)` | `4% <0%> (+1%)` | :arrow_up: |
   | ... and [56 more](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=footer). Last update [0b05987...fb4381e](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-gobblin] linweihs commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics

Posted by GitBox <gi...@apache.org>.
linweihs commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891#discussion_r381038818
 
 

 ##########
 File path: gobblin-service/src/main/java/org/apache/gobblin/service/modules/core/GobblinServiceManager.java
 ##########
 @@ -25,6 +28,9 @@
 import java.util.Properties;
 import java.util.concurrent.TimeUnit;
 
+import org.slf4j.Logger;
 
 Review comment:
   no. these are used on here 
   `private static final Logger LOGGER = LoggerFactory.getLogger(GobblinServiceManager.class);`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-gobblin] arjun4084346 commented on issue #2891: [GOBBLIN-1051] Emit Helix Leader Metrics

Posted by GitBox <gi...@apache.org>.
arjun4084346 commented on issue #2891: [GOBBLIN-1051] Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891#issuecomment-586540415
 
 
   LGTM +1

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-gobblin] linweihs commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics

Posted by GitBox <gi...@apache.org>.
linweihs commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891#discussion_r379704682
 
 

 ##########
 File path: gobblin-service/src/main/java/org/apache/gobblin/service/modules/core/GobblinServiceManager.java
 ##########
 @@ -58,15 +35,30 @@
 import com.linkedin.restli.server.resources.BaseResource;
 import com.typesafe.config.Config;
 import com.typesafe.config.ConfigFactory;
-
+import java.io.IOException;
+import java.lang.reflect.InvocationTargetException;
 
 Review comment:
   import order restored by code style

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-gobblin] sv2000 commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics

Posted by GitBox <gi...@apache.org>.
sv2000 commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891#discussion_r380895329
 
 

 ##########
 File path: gobblin-service/src/main/java/org/apache/gobblin/service/modules/core/GobblinServiceManager.java
 ##########
 @@ -214,6 +223,14 @@ public GobblinServiceManager(String serviceName, String serviceId, Config config
       this.isGitConfigMonitorEnabled = false;
     }
 
+    // Initialize Helix leader guage
+    helixLeaderGauges = Optional.of(new HelixLeaderState());
+    String helixLeaderStateGaugeName =
+        MetricRegistry.name(MetricReportUtils.GOBBLIN_SERVICE_METRICS_PREFIX, ServiceMetricNames.HELIX_LEADER_STATE);
+    ContextAwareGauge<Integer> gauge = metricContext.newContextAwareGauge(helixLeaderStateGaugeName, () -> helixLeaderGauges.get().state.value);
 
 Review comment:
   Use helixLeaderGauges.get().state.getValue() with Getter on value field?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-gobblin] asfgit closed pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics

Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-gobblin] sv2000 commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics

Posted by GitBox <gi...@apache.org>.
sv2000 commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891#discussion_r380895014
 
 

 ##########
 File path: gobblin-service/src/main/java/org/apache/gobblin/service/modules/core/GobblinServiceManager.java
 ##########
 @@ -645,4 +678,21 @@ private static void testGobblinService(GobblinServiceManager gobblinServiceManag
       throw new RuntimeException(e);
     }
   }
+
+  @Setter
+  private static class HelixLeaderState {
+    private LeaderState state = LeaderState.UNKNOWN;
+  }
+
+  private enum LeaderState {
+    UNKNOWN(-1),
+    SLAVE(0),
+    MASTER(1);
+
+    public int value;
 
 Review comment:
   Annotate with @Getter?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-gobblin] linweihs commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics

Posted by GitBox <gi...@apache.org>.
linweihs commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891#discussion_r381038920
 
 

 ##########
 File path: gobblin-service/src/main/java/org/apache/gobblin/service/modules/core/GobblinServiceManager.java
 ##########
 @@ -214,6 +223,14 @@ public GobblinServiceManager(String serviceName, String serviceId, Config config
       this.isGitConfigMonitorEnabled = false;
     }
 
+    // Initialize Helix leader guage
+    helixLeaderGauges = Optional.of(new HelixLeaderState());
+    String helixLeaderStateGaugeName =
+        MetricRegistry.name(MetricReportUtils.GOBBLIN_SERVICE_METRICS_PREFIX, ServiceMetricNames.HELIX_LEADER_STATE);
+    ContextAwareGauge<Integer> gauge = metricContext.newContextAwareGauge(helixLeaderStateGaugeName, () -> helixLeaderGauges.get().state.value);
 
 Review comment:
   updated.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-gobblin] sv2000 commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics

Posted by GitBox <gi...@apache.org>.
sv2000 commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891#discussion_r380894548
 
 

 ##########
 File path: gobblin-service/src/main/java/org/apache/gobblin/service/modules/core/GobblinServiceManager.java
 ##########
 @@ -25,6 +28,9 @@
 import java.util.Properties;
 import java.util.concurrent.TimeUnit;
 
+import org.slf4j.Logger;
 
 Review comment:
   Are these unused imports? Don't see them being used in this PR?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-gobblin] codecov-io edited a comment on issue #2891: [GOBBLIN-1051] Emit Helix Leader Metrics

Posted by GitBox <gi...@apache.org>.
codecov-io edited a comment on issue #2891: [GOBBLIN-1051] Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891#issuecomment-586523868
 
 
   # [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=h1) Report
   > Merging [#2891](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/0b059870755bbab93f11d1cf620862e7502b2d4c?src=pr&el=desc) will **increase** coverage by `1.03%`.
   > The diff coverage is `82.6%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/graphs/tree.svg?width=650&token=4MgURJ0bGc&height=150&src=pr)](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #2891      +/-   ##
   ============================================
   + Coverage     44.78%   45.81%   +1.03%     
   - Complexity     8958     9143     +185     
   ============================================
     Files          1926     1929       +3     
     Lines         72555    72625      +70     
     Branches       7992     7997       +5     
   ============================================
   + Hits          32491    33273     +782     
   + Misses        37073    36311     -762     
   - Partials       2991     3041      +50
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [...org/apache/gobblin/metrics/ServiceMetricNames.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL21ldHJpY3MvU2VydmljZU1ldHJpY05hbWVzLmphdmE=) | `0% <ø> (ø)` | `0 <0> (ø)` | :arrow_down: |
   | [...in/service/modules/core/GobblinServiceManager.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9jb3JlL0dvYmJsaW5TZXJ2aWNlTWFuYWdlci5qYXZh) | `56.94% <82.6%> (+2.22%)` | `24 <0> (-1)` | :arrow_down: |
   | [...lin/util/filesystem/FileSystemInstrumentation.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsZXN5c3RlbS9GaWxlU3lzdGVtSW5zdHJ1bWVudGF0aW9uLmphdmE=) | `85.71% <0%> (-7.15%)` | `3% <0%> (ø)` | |
   | [...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh) | `36.28% <0%> (-2.97%)` | `13% <0%> (ø)` | |
   | [...lin/restli/throttling/ZookeeperLeaderElection.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL1pvb2tlZXBlckxlYWRlckVsZWN0aW9uLmphdmE=) | `70% <0%> (-2.23%)` | `13% <0%> (ø)` | |
   | [.../org/apache/gobblin/metrics/RootMetricContext.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzLWJhc2Uvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vbWV0cmljcy9Sb290TWV0cmljQ29udGV4dC5qYXZh) | `78.12% <0%> (-1.57%)` | `15% <0%> (-1%)` | |
   | [.../apache/gobblin/runtime/api/JobExecutionState.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvYXBpL0pvYkV4ZWN1dGlvblN0YXRlLmphdmE=) | `79.43% <0%> (-0.94%)` | `24% <0%> (ø)` | |
   | [...main/java/org/apache/gobblin/util/HadoopUtils.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvSGFkb29wVXRpbHMuamF2YQ==) | `30.53% <0%> (-0.34%)` | `24% <0%> (-1%)` | |
   | [...rg/apache/gobblin/publisher/BaseDataPublisher.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3B1Ymxpc2hlci9CYXNlRGF0YVB1Ymxpc2hlci5qYXZh) | `67.74% <0%> (-0.31%)` | `61% <0%> (-1%)` | |
   | [...ava/org/apache/gobblin/metrics/GobblinMetrics.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL21ldHJpY3MvR29iYmxpbk1ldHJpY3MuamF2YQ==) | `18.8% <0%> (-0.06%)` | `15% <0%> (ø)` | |
   | ... and [52 more](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=footer). Last update [0b05987...e913ac6](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-gobblin] codecov-io commented on issue #2891: [GOBBLIN-1051] Emit Helix Leader Metrics

Posted by GitBox <gi...@apache.org>.
codecov-io commented on issue #2891: [GOBBLIN-1051] Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891#issuecomment-586523868
 
 
   # [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=h1) Report
   > Merging [#2891](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=desc) into [master](https://codecov.io/gh/apache/incubator-gobblin/commit/0b059870755bbab93f11d1cf620862e7502b2d4c?src=pr&el=desc) will **increase** coverage by `1.04%`.
   > The diff coverage is `82.6%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/graphs/tree.svg?width=650&token=4MgURJ0bGc&height=150&src=pr)](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=tree)
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #2891      +/-   ##
   ============================================
   + Coverage     44.78%   45.82%   +1.04%     
   - Complexity     8958     9146     +188     
   ============================================
     Files          1926     1929       +3     
     Lines         72555    72625      +70     
     Branches       7992     7997       +5     
   ============================================
   + Hits          32491    33279     +788     
   + Misses        37073    36306     -767     
   - Partials       2991     3040      +49
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=tree) | Coverage Δ | Complexity Δ | |
   |---|---|---|---|
   | [...org/apache/gobblin/metrics/ServiceMetricNames.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL21ldHJpY3MvU2VydmljZU1ldHJpY05hbWVzLmphdmE=) | `0% <ø> (ø)` | `0 <0> (ø)` | :arrow_down: |
   | [...in/service/modules/core/GobblinServiceManager.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9kdWxlcy9jb3JlL0dvYmJsaW5TZXJ2aWNlTWFuYWdlci5qYXZh) | `56.94% <82.6%> (+2.22%)` | `25 <0> (ø)` | :arrow_down: |
   | [...lin/util/filesystem/FileSystemInstrumentation.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsZXN5c3RlbS9GaWxlU3lzdGVtSW5zdHJ1bWVudGF0aW9uLmphdmE=) | `85.71% <0%> (-7.15%)` | `3% <0%> (ø)` | |
   | [...lin/restli/throttling/ZookeeperLeaderElection.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1yZXN0bGkvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2UvZ29iYmxpbi10aHJvdHRsaW5nLXNlcnZpY2Utc2VydmVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3Jlc3RsaS90aHJvdHRsaW5nL1pvb2tlZXBlckxlYWRlckVsZWN0aW9uLmphdmE=) | `70% <0%> (-2.23%)` | `13% <0%> (ø)` | |
   | [...in/java/org/apache/gobblin/cluster/HelixUtils.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSGVsaXhVdGlscy5qYXZh) | `38.93% <0%> (-0.32%)` | `14% <0%> (+1%)` | |
   | [...rg/apache/gobblin/publisher/BaseDataPublisher.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1jb3JlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3B1Ymxpc2hlci9CYXNlRGF0YVB1Ymxpc2hlci5qYXZh) | `67.74% <0%> (-0.31%)` | `61% <0%> (-1%)` | |
   | [...ava/org/apache/gobblin/metrics/GobblinMetrics.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1tZXRyaWNzLWxpYnMvZ29iYmxpbi1tZXRyaWNzL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL21ldHJpY3MvR29iYmxpbk1ldHJpY3MuamF2YQ==) | `18.8% <0%> (-0.06%)` | `15% <0%> (ø)` | |
   | [...va/org/apache/gobblin/publisher/DataPublisher.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vcHVibGlzaGVyL0RhdGFQdWJsaXNoZXIuamF2YQ==) | `0% <0%> (ø)` | `0% <0%> (ø)` | :arrow_down: |
   | [...a/org/apache/gobblin/runtime/util/ForkMetrics.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1ydW50aW1lL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3J1bnRpbWUvdXRpbC9Gb3JrTWV0cmljcy5qYXZh) | `100% <0%> (ø)` | `5% <0%> (-1%)` | :arrow_down: |
   | [...ache/gobblin/cluster/InMemorySingleTaskRunner.java](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvSW5NZW1vcnlTaW5nbGVUYXNrUnVubmVyLmphdmE=) | `100% <0%> (ø)` | `3% <0%> (?)` | |
   | ... and [50 more](https://codecov.io/gh/apache/incubator-gobblin/pull/2891/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=footer). Last update [0b05987...d81409a](https://codecov.io/gh/apache/incubator-gobblin/pull/2891?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [incubator-gobblin] linweihs commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics

Posted by GitBox <gi...@apache.org>.
linweihs commented on a change in pull request #2891: [GOBBLIN-1051] Emit Helix Leader Metrics
URL: https://github.com/apache/incubator-gobblin/pull/2891#discussion_r381038891
 
 

 ##########
 File path: gobblin-service/src/main/java/org/apache/gobblin/service/modules/core/GobblinServiceManager.java
 ##########
 @@ -645,4 +678,21 @@ private static void testGobblinService(GobblinServiceManager gobblinServiceManag
       throw new RuntimeException(e);
     }
   }
+
+  @Setter
+  private static class HelixLeaderState {
+    private LeaderState state = LeaderState.UNKNOWN;
+  }
+
+  private enum LeaderState {
+    UNKNOWN(-1),
+    SLAVE(0),
+    MASTER(1);
+
+    public int value;
 
 Review comment:
   good point. added

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services