You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by KaiXinXiaoLei <gi...@git.apache.org> on 2017/12/13 14:00:45 UTC
[GitHub] spark pull request #19968: [SPARK-22770][CORE] When driver stopping, there i...
GitHub user KaiXinXiaoLei opened a pull request:
https://github.com/apache/spark/pull/19968
[SPARK-22770][CORE] When driver stopping, there is error: Could not find CoarseGrainedScheduler
## What changes were proposed in this pull request?
When driver stopping, there is a error:
17/12/12 18:30:16 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
17/12/12 18:30:16 ERROR TransportRequestHandler: Error while invoking RpcHandler#receive() for one-way message.
org.apache.spark.SparkException: Could not find CoarseGrainedScheduler.
at org.apache.spark.rpc.netty.Dispatcher.postMessage(Dispatcher.scala:154)
at org.apache.spark.rpc.netty.Dispatcher.postOneWayMessage(Dispatcher.scala:134)
at org.apache.spark.rpc.netty.NettyRpcHandler.receive(NettyRpcEnv.scala:570)
at org.apache.spark.network.server.TransportRequestHandler.processOneWayMessage(TransportRequestHandler.java:180)
at org.apache.spark.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:109)
at org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:119)
at org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:51)
Because the endpoint of CoarseGrainedScheduler is stopped, this error is not problem, so i think this error should not be printed.
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Please review http://spark.apache.org/contributing.html before opening a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/KaiXinXiaoLei/spark pointstopped2
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/19968.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #19968
----
commit 8a52dc561f8da527bf2fc0f040c741f5dfdfe1f5
Author: hanghang <58...@qq.com>
Date: 2017-12-13T13:14:06Z
change code
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22770][CORE] When driver stopping, there is error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19968
**[Test build #84855 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84855/testReport)** for PR 19968 at commit [`8a52dc5`](https://github.com/apache/spark/commit/8a52dc561f8da527bf2fc0f040c741f5dfdfe1f5).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19968
**[Test build #84867 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84867/testReport)** for PR 19968 at commit [`39bc683`](https://github.com/apache/spark/commit/39bc683441c0f3a789d2bf0f12bcd0f0af04418a).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19968
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22770][CORE] When driver stopping, there is error...
Posted by KaiXinXiaoLei <gi...@git.apache.org>.
Github user KaiXinXiaoLei commented on the issue:
https://github.com/apache/spark/pull/19968
@srowen i close https://github.com/apache/spark/pull/19965, and update description
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19968
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84856/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22770][CORE] When driver stopping, there is error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19968
**[Test build #84856 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84856/testReport)** for PR 19968 at commit [`af7a021`](https://github.com/apache/spark/commit/af7a0212b394dbde82ffbfc1bfd6c98b6e18d307).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19968
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84855/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19968
**[Test build #84856 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84856/testReport)** for PR 19968 at commit [`af7a021`](https://github.com/apache/spark/commit/af7a0212b394dbde82ffbfc1bfd6c98b6e18d307).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19968
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19968
**[Test build #84867 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84867/testReport)** for PR 19968 at commit [`39bc683`](https://github.com/apache/spark/commit/39bc683441c0f3a789d2bf0f12bcd0f0af04418a).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19968
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84867/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22770][CORE] When driver stopping, there is error...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19968
You already had another PR open for the same issue, which was an exact duplicate of another PR, for another issue that is a duplicate. Please clean it up into one issue -- SPAKR-22769
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19968
**[Test build #84855 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84855/testReport)** for PR 19968 at commit [`8a52dc5`](https://github.com/apache/spark/commit/8a52dc561f8da527bf2fc0f040c741f5dfdfe1f5).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19968: [SPARK-22769][CORE] When driver stopping, there i...
Posted by zsxwing <gi...@git.apache.org>.
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/19968#discussion_r157631169
--- Diff: core/src/main/scala/org/apache/spark/rpc/netty/Dispatcher.scala ---
@@ -100,6 +102,7 @@ private[netty] class Dispatcher(nettyEnv: NettyRpcEnv, numUsableCores: Int) exte
return
}
unregisterRpcEndpoint(rpcEndpointRef.name)
+ endpointsIsStopped.putIfAbsent(rpcEndpointRef.name, true)
--- End diff --
this will be a memory leak. Spark may create some temp RpcEndpoints. You can make `postMessage` return `Some(RpcEndpointNotFoundException)` and ignore it.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19968: [SPARK-22769][CORE] When driver stopping, there i...
Posted by KaiXinXiaoLei <gi...@git.apache.org>.
Github user KaiXinXiaoLei commented on a diff in the pull request:
https://github.com/apache/spark/pull/19968#discussion_r159167935
--- Diff: core/src/main/scala/org/apache/spark/rpc/netty/Dispatcher.scala ---
@@ -100,6 +102,7 @@ private[netty] class Dispatcher(nettyEnv: NettyRpcEnv, numUsableCores: Int) exte
return
}
unregisterRpcEndpoint(rpcEndpointRef.name)
+ endpointsIsStopped.putIfAbsent(rpcEndpointRef.name, true)
--- End diff --
ok thanks
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by srowen <gi...@git.apache.org>.
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/19968
Ping @KaiXinXiaoLei to update or close
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by KaiXinXiaoLei <gi...@git.apache.org>.
Github user KaiXinXiaoLei commented on the issue:
https://github.com/apache/spark/pull/19968
Now this problem， i don't work. Now i close it .
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19968
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by jiangxb1987 <gi...@git.apache.org>.
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/19968
ping @KaiXinXiaoLei are you still working on this?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #19968: [SPARK-22769][CORE] When driver stopping, there i...
Posted by KaiXinXiaoLei <gi...@git.apache.org>.
Github user KaiXinXiaoLei closed the pull request at:
https://github.com/apache/spark/pull/19968
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #19968: [SPARK-22769][CORE] When driver stopping, there is error...
Posted by KaiXinXiaoLei <gi...@git.apache.org>.
Github user KaiXinXiaoLei commented on the issue:
https://github.com/apache/spark/pull/19968
@srowen ok ,i will update, thanks
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org