You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gobblin.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2022/10/13 18:26:00 UTC
[jira] [Work logged] (GOBBLIN-1721) Give option to cancel helix workflow through Delete API to avoid job hanging
[ https://issues.apache.org/jira/browse/GOBBLIN-1721?focusedWorklogId=816747&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-816747 ]
ASF GitHub Bot logged work on GOBBLIN-1721:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 13/Oct/22 18:25
Start Date: 13/Oct/22 18:25
Worklog Time Spent: 10m
Work Description: codecov-commenter commented on PR #3580:
URL: https://github.com/apache/gobblin/pull/3580#issuecomment-1278011638
# [Codecov](https://codecov.io/gh/apache/gobblin/pull/3580?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#3580](https://codecov.io/gh/apache/gobblin/pull/3580?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (d024c60) into [master](https://codecov.io/gh/apache/gobblin/commit/fafd40b25fdd3e8f8299f9e939794ad7a2492314?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fafd40b) will **increase** coverage by `1.78%`.
> The diff coverage is `40.00%`.
```diff
@@ Coverage Diff @@
## master #3580 +/- ##
============================================
+ Coverage 46.87% 48.66% +1.78%
+ Complexity 10640 7851 -2789
============================================
Files 2113 1467 -646
Lines 82798 57816 -24982
Branches 9220 6653 -2567
============================================
- Hits 38813 28134 -10679
+ Misses 40425 27063 -13362
+ Partials 3560 2619 -941
```
| [Impacted Files](https://codecov.io/gh/apache/gobblin/pull/3580?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...bblin/cluster/GobblinClusterConfigurationKeys.java](https://codecov.io/gh/apache/gobblin/pull/3580/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpbkNsdXN0ZXJDb25maWd1cmF0aW9uS2V5cy5qYXZh) | `0.00% <ø> (ø)` | |
| [...ache/gobblin/cluster/GobblinHelixJobScheduler.java](https://codecov.io/gh/apache/gobblin/pull/3580/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpbkhlbGl4Sm9iU2NoZWR1bGVyLmphdmE=) | `53.67% <40.00%> (-0.93%)` | :arrow_down: |
| [...a/org/apache/gobblin/util/limiter/NoopLimiter.java](https://codecov.io/gh/apache/gobblin/pull/3580/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvbGltaXRlci9Ob29wTGltaXRlci5qYXZh) | `40.00% <0.00%> (-20.00%)` | :arrow_down: |
| [...lin/util/filesystem/FileSystemInstrumentation.java](https://codecov.io/gh/apache/gobblin/pull/3580/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi11dGlsaXR5L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3V0aWwvZmlsZXN5c3RlbS9GaWxlU3lzdGVtSW5zdHJ1bWVudGF0aW9uLmphdmE=) | `85.71% <0.00%> (-7.15%)` | :arrow_down: |
| [.../java/org/apache/gobblin/cluster/SleepingTask.java](https://codecov.io/gh/apache/gobblin/pull/3580/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvU2xlZXBpbmdUYXNrLmphdmE=) | `39.39% <0.00%> (-6.07%)` | :arrow_down: |
| [...a/management/copy/publisher/CopyDataPublisher.java](https://codecov.io/gh/apache/gobblin/pull/3580/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvcHVibGlzaGVyL0NvcHlEYXRhUHVibGlzaGVyLmphdmE=) | `74.17% <0.00%> (-1.33%)` | :arrow_down: |
| [.../org/apache/gobblin/cluster/GobblinTaskRunner.java](https://codecov.io/gh/apache/gobblin/pull/3580/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1jbHVzdGVyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL2NsdXN0ZXIvR29iYmxpblRhc2tSdW5uZXIuamF2YQ==) | `63.63% <0.00%> (-0.32%)` | :arrow_down: |
| [...anagement/copy/replication/ConfigBasedDataset.java](https://codecov.io/gh/apache/gobblin/pull/3580/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1kYXRhLW1hbmFnZW1lbnQvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2dvYmJsaW4vZGF0YS9tYW5hZ2VtZW50L2NvcHkvcmVwbGljYXRpb24vQ29uZmlnQmFzZWREYXRhc2V0LmphdmE=) | `68.87% <0.00%> (ø)` | |
| [...gobblin/service/monitoring/FsFlowGraphMonitor.java](https://codecov.io/gh/apache/gobblin/pull/3580/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1zZXJ2aWNlL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9nb2JibGluL3NlcnZpY2UvbW9uaXRvcmluZy9Gc0Zsb3dHcmFwaE1vbml0b3IuamF2YQ==) | | |
| [.../apache/gobblin/http/ApacheHttpResponseStatus.java](https://codecov.io/gh/apache/gobblin/pull/3580/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Z29iYmxpbi1tb2R1bGVzL2dvYmJsaW4taHR0cC9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvZ29iYmxpbi9odHRwL0FwYWNoZUh0dHBSZXNwb25zZVN0YXR1cy5qYXZh) | | |
| ... and [647 more](https://codecov.io/gh/apache/gobblin/pull/3580/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
Issue Time Tracking
-------------------
Worklog Id: (was: 816747)
Remaining Estimate: 0h
Time Spent: 10m
> Give option to cancel helix workflow through Delete API to avoid job hanging
> ----------------------------------------------------------------------------
>
> Key: GOBBLIN-1721
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1721
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-cluster
> Reporter: Hanghang Liu
> Assignee: Hung Tran
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently when we receive a job restart(handleUpdateJobConfigArrival), GobblinHelixJobLauncher will firstly call helixTaskDriver.waitToStop to stop the workflow, then initiate the new one. We observe the behavior of Helix taking exceptionally long to stop the workflow, making the job state staying in STOPPING status. This will make waitToStop timeout and throw exception all the time, making the new flow never be able to launch.
> We can utilize Delete API in this case since our job is stateless for Helix, to avoid job hanging.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)