You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by MaxGekk <gi...@git.apache.org> on 2018/08/30 21:01:19 UTC
[GitHub] spark pull request #22292: [SPARK-25286][CORE] Removing the dangerous parmap
GitHub user MaxGekk opened a pull request:
https://github.com/apache/spark/pull/22292
[SPARK-25286][CORE] Removing the dangerous parmap
## What changes were proposed in this pull request?
I propose to remove the `parmap` method which accepts an execution context as a parameter. The method should be removed to eliminate any deadlocks that can occur if `parmap` is called recursively on thread pools restricted by size.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/MaxGekk/spark-1 remove-overloaded-parmap
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22292.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22292
----
commit 7fbd7b44edec74d07356148c349c1a6fbe163e1b
Author: Maxim Gekk <ma...@...>
Date: 2018-08-30T20:43:22Z
Revert "Porting UnionRDD on parmap"
This reverts commit 72cdfeb765cda13ab03ed8515a83fa24657894ac.
commit 6fd4c0099c85df240ef7fb9d915f0b20c06b0040
Author: Maxim Gekk <ma...@...>
Date: 2018-08-30T20:48:05Z
Removing potentially dangerous method
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by zsxwing <gi...@git.apache.org>.
Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/22292#discussion_r214204549
--- Diff: streaming/src/main/scala/org/apache/spark/streaming/util/FileBasedWriteAheadLog.scala ---
@@ -315,7 +315,9 @@ private[streaming] object FileBasedWriteAheadLog {
implicit val ec = executionContext
--- End diff --
nit: this line is not needed
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22292
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95526/
Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22292
Merged build finished. Test FAILed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by gatorsmile <gi...@git.apache.org>.
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/22292
Thanks! Merged to master.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22292
For future, let us keep a connection with #22233 that is the original one
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22292
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95531/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/22292
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22292
**[Test build #95492 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95492/testReport)** for PR 22292 at commit [`6fd4c00`](https://github.com/apache/spark/commit/6fd4c0099c85df240ef7fb9d915f0b20c06b0040).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22292
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by MaxGekk <gi...@git.apache.org>.
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/22292
@zsxwing @gatorsmile Please, have a look at the PR.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22292
**[Test build #95531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95531/testReport)** for PR 22292 at commit [`e868edb`](https://github.com/apache/spark/commit/e868edb45d6e0ea4dda68a44d1c6e6426f292bd0).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22292
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22292
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22292
**[Test build #95526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95526/testReport)** for PR 22292 at commit [`e868edb`](https://github.com/apache/spark/commit/e868edb45d6e0ea4dda68a44d1c6e6426f292bd0).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22292
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95492/
Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22292
**[Test build #95492 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95492/testReport)** for PR 22292 at commit [`6fd4c00`](https://github.com/apache/spark/commit/6fd4c0099c85df240ef7fb9d915f0b20c06b0040).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22292
**[Test build #95526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95526/testReport)** for PR 22292 at commit [`e868edb`](https://github.com/apache/spark/commit/e868edb45d6e0ea4dda68a44d1c6e6426f292bd0).
* This patch **fails build dependency tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by MaxGekk <gi...@git.apache.org>.
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/22292
jenkins, retest this, please
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22292
Merged build finished. Test PASSed.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22292
**[Test build #95531 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95531/testReport)** for PR 22292 at commit [`e868edb`](https://github.com/apache/spark/commit/e868edb45d6e0ea4dda68a44d1c6e6426f292bd0).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org