You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gearpump.apache.org by manuzhang <gi...@git.apache.org> on 2016/06/01 03:16:11 UTC
[GitHub] incubator-gearpump pull request: fix GEARPUMP-83, show application pending o...
GitHub user manuzhang opened a pull request:
https://github.com/apache/incubator-gearpump/pull/31
fix GEARPUMP-83, show application pending on worker down
Changes include:
1. add `activeAppMasters` set to `masterState` and `deadAppMasters` is changed to set of appIds. Now all existent applications will be in `appMasterRegistry` and in **pending** status by default. Those in `activeAppMasters` will have **active** status and `deadAppMasters` **inactive** status.
2. add a status column to the "apps" page on the dashboard. status will be one of pending, active and inactive.
3. Internally, `TaskManager` will send an `ApplicationReady` message to `AppMaster` on ApplicationReady state(with DAG version larger than -1). In turn, `AppMaster` will send an `ActivateApplication(appId)` message to `Master`(forwarded to `AppManager`). Then the `appId` is added to `activeAppMasters`. It is removed from `activeAppMasters` either when an application is killed or recovered.
4. add UTs and fix a bunch of warnings (typos, suspicious shadowing, etc)
The snapshot of "apps" page with active and inactive applications
The snapshot of "apps" page when all workers down
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/manuzhang/incubator-gearpump appmaster_status
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-gearpump/pull/31.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #31
----
commit 9529796fc319f1ed69cc6cf726d8f1afef47ef1b
Author: manuzhang <ow...@gmail.com>
Date: 2016-05-31T07:47:09Z
fix GEARPUMP-83, show application pending on worker down
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #31: fix GEARPUMP-83, show application pending on w...
Posted by manuzhang <gi...@git.apache.org>.
Github user manuzhang commented on the issue:
https://github.com/apache/incubator-gearpump/pull/31
@huafengw fixed
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request #31: fix GEARPUMP-83, show application pendi...
Posted by huafengw <gi...@git.apache.org>.
Github user huafengw commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/31#discussion_r65495140
--- Diff: daemon/src/main/scala/org/apache/gearpump/cluster/master/AppManager.scala ---
@@ -268,9 +275,9 @@ private[cluster] class AppManager(kvService: ActorRef, launcher: AppMasterLaunch
def terminationWatch: Receive = {
case terminate: Terminated =>
- terminate.getAddressTerminated()
- LOG.info(s"AppMaster(${terminate.actor.path}) is terminiated, " +
- s"network down: ${terminate.getAddressTerminated()}")
+ terminate.getAddressTerminated
--- End diff --
Is this line of code useless?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request: fix GEARPUMP-83, show application pending o...
Posted by manuzhang <gi...@git.apache.org>.
Github user manuzhang commented on the pull request:
https://github.com/apache/incubator-gearpump/pull/31
R: @stanleyxu2005
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #31: fix GEARPUMP-83, show application pending on w...
Posted by huafengw <gi...@git.apache.org>.
Github user huafengw commented on the issue:
https://github.com/apache/incubator-gearpump/pull/31
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request: fix GEARPUMP-83, show application pending o...
Posted by codecov-io <gi...@git.apache.org>.
Github user codecov-io commented on the pull request:
https://github.com/apache/incubator-gearpump/pull/31
## [Current coverage][cc-pull] is **62.77%**
> Merging [#31][cc-pull] into [master][cc-base-branch] will decrease coverage by **1.32%**
1. 2 files in `.../streaming/appmaster` were modified. [more](https://codecov.io/gh/apache/incubator-gearpump/commit/9529796fc319f1ed69cc6cf726d8f1afef47ef1b/changes?src=pr#73747265616D696E672F7372632F6D61696E2F7363616C612F6F72672F6170616368652F6765617270756D702F73747265616D696E672F6170706D6173746572)
1. 2 files in `...rpump/cluster/master` were modified. [more](https://codecov.io/gh/apache/incubator-gearpump/commit/9529796fc319f1ed69cc6cf726d8f1afef47ef1b/changes?src=pr#6461656D6F6E2F7372632F6D61696E2F7363616C612F6F72672F6170616368652F6765617270756D702F636C75737465722F6D6173746572)
- Misses `+71`
- Hits `-71`
1. 2 files (not in diff) in `...mp/streaming/metrics` were modified. [more](https://codecov.io/gh/apache/incubator-gearpump/commit/9529796fc319f1ed69cc6cf726d8f1afef47ef1b/changes?src=pr#73747265616D696E672F7372632F6D61696E2F7363616C612F6F72672F6170616368652F6765617270756D702F73747265616D696E672F6D657472696373)
1. 4 files (not in diff) in `.../streaming/appmaster` were modified. [more](https://codecov.io/gh/apache/incubator-gearpump/commit/9529796fc319f1ed69cc6cf726d8f1afef47ef1b/changes?src=pr#73747265616D696E672F7372632F6D61696E2F7363616C612F6F72672F6170616368652F6765617270756D702F73747265616D696E672F6170706D6173746572)
1. 2 files (not in diff) in `...e/gearpump/streaming` were modified. [more](https://codecov.io/gh/apache/incubator-gearpump/commit/9529796fc319f1ed69cc6cf726d8f1afef47ef1b/changes?src=pr#73747265616D696E672F7372632F6D61696E2F7363616C612F6F72672F6170616368652F6765617270756D702F73747265616D696E67)
- Misses `-1`
- Hits `+1`
1. 2 files (not in diff) in `...he/gearpump/jarstore` were modified. [more](https://codecov.io/gh/apache/incubator-gearpump/commit/9529796fc319f1ed69cc6cf726d8f1afef47ef1b/changes?src=pr#6461656D6F6E2F7372632F6D61696E2F7363616C612F6F72672F6170616368652F6765617270756D702F6A617273746F7265)
- Misses `+8`
- Hits `-8`
1. 2 files (not in diff) in `...mp/cluster/scheduler` were modified. [more](https://codecov.io/gh/apache/incubator-gearpump/commit/9529796fc319f1ed69cc6cf726d8f1afef47ef1b/changes?src=pr#6461656D6F6E2F7372632F6D61696E2F7363616C612F6F72672F6170616368652F6765617270756D702F636C75737465722F7363686564756C6572)
1. 3 files (not in diff) in `...che/gearpump/cluster` were modified. [more](https://codecov.io/gh/apache/incubator-gearpump/commit/9529796fc319f1ed69cc6cf726d8f1afef47ef1b/changes?src=pr#6461656D6F6E2F7372632F6D61696E2F7363616C612F6F72672F6170616368652F6765617270756D702F636C7573746572)
- Misses `+12`
- Hits `-12`
1. 1 files (not in diff) in `.../org/apache/gearpump` were modified. [more](https://codecov.io/gh/apache/incubator-gearpump/commit/9529796fc319f1ed69cc6cf726d8f1afef47ef1b/changes?src=pr#6461656D6F6E2F7372632F6D61696E2F7363616C612F6F72672F6170616368652F6765617270756D70)
1. 5 files (not in diff) in `...apache/gearpump/util` were modified. [more](https://codecov.io/gh/apache/incubator-gearpump/commit/9529796fc319f1ed69cc6cf726d8f1afef47ef1b/changes?src=pr#636F72652F7372632F6D61696E2F7363616C612F6F72672F6170616368652F6765617270756D702F7574696C)
- Misses `-7`
- Hits `+7`
```diff
@@ master #31 diff @@
==========================================
Files 180 180
Lines 5956 5969 +13
Methods 5678 5689 +11
Messages 0 0
Branches 278 280 +2
==========================================
- Hits 3818 3747 -71
- Misses 2138 2222 +84
Partials 0 0
```
[][cc-pull]
> Powered by [Codecov](https://codecov.io?src=pr). Last updated by [32f1507...9529796][cc-compare]
[cc-base-branch]: https://codecov.io/gh/apache/incubator-gearpump/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/apache/incubator-gearpump/compare/32f1507171eb4398312deb323f7872eb0bb429f5...9529796fc319f1ed69cc6cf726d8f1afef47ef1b
[cc-pull]: https://codecov.io/gh/apache/incubator-gearpump/pull/31?src=pr
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request #31: fix GEARPUMP-83, show application pendi...
Posted by huafengw <gi...@git.apache.org>.
Github user huafengw commented on a diff in the pull request:
https://github.com/apache/incubator-gearpump/pull/31#discussion_r65495393
--- Diff: daemon/src/main/scala/org/apache/gearpump/cluster/master/AppManager.scala ---
@@ -310,22 +320,20 @@ private[cluster] class AppManager(kvService: ActorRef, launcher: AppMasterLaunch
case class RecoverApplication(applicationStatus: ApplicationState)
private def cleanApplicationData(appId: Int): Unit = {
- // Add the dead app to dead appMaster
- appMasterRegistry.get(appId).foreach { pair =>
- val (appMasterActor, info) = pair
- deadAppMasters += appId -> (appMasterActor, info.copy(
- finishTime = System.currentTimeMillis()))
- }
-
- appMasterRegistry -= appId
+ // Add the dead app to dead appMasters
+ deadAppMasters += appId
--- End diff --
When the application is down, you don't update its finish time. You can see the dashboard missed that info.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump issue #31: fix GEARPUMP-83, show application pending on w...
Posted by manuzhang <gi...@git.apache.org>.
Github user manuzhang commented on the issue:
https://github.com/apache/incubator-gearpump/pull/31
The snapshot of "apps" page with active and inactive applications
The snapshot of "apps" page when all workers down
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-gearpump pull request #31: fix GEARPUMP-83, show application pendi...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-gearpump/pull/31
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---