You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by kellrott <gi...@git.apache.org> on 2014/06/23 19:37:48 UTC
[GitHub] spark pull request: Fix for SPARK-2228
GitHub user kellrott opened a pull request:
https://github.com/apache/spark/pull/1182
Fix for SPARK-2228
Add check to see if stageId exists in stageIdToPool before attempting to remove it from poolToActiveStages.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kellrott/spark SPARK-2228
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1182.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1182
----
commit 8dbe8218f408f982b7418063af486f85359e42b6
Author: Kyle Ellrott <ke...@gmail.com>
Date: 2014-06-23T13:21:53Z
Adding check to make sure that stageId is found in stageIdToPool
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: Fix for SPARK-2228
Posted by kellrott <gi...@git.apache.org>.
Github user kellrott commented on the pull request:
https://github.com/apache/spark/pull/1182#issuecomment-47335004
My problem code started up calling SVMWithSGD.train in several parallel threads. This matches with your notes about events being generated too fast for the listener.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: Fix for SPARK-2228
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1182#issuecomment-46877112
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: Fix for SPARK-2228
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1182#issuecomment-46877098
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: Fix for SPARK-2228
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/1182#issuecomment-47309794
@kellrott actually, could you describe the workload you are running when you encountered this issue? I had some more thoughts posted in 2228.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: Fix for SPARK-2228
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/1182#issuecomment-47309664
Hey Kyle,
It might be nicer to track down the root cause of this issue. I think this happens to make the exception go away, but I rand the reproduction provided by @bxshi and there are other problems that co-occur with the exception. So I might spend some effort tracking it down.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: Fix for SPARK-2228
Posted by kellrott <gi...@git.apache.org>.
Github user kellrott closed the pull request at:
https://github.com/apache/spark/pull/1182
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: Fix for SPARK-2228
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1182#issuecomment-46883068
Merged build finished. All automated tests passed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: Fix for SPARK-2228
Posted by bxshi <gi...@git.apache.org>.
Github user bxshi commented on the pull request:
https://github.com/apache/spark/pull/1182#issuecomment-46894610
Hi @kellrott , thanks for your help on that issue! I tried such approach, that could eliminate the exception, but if you take a look at SparkUI, there will still be a huge number of active tasks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] spark pull request: Fix for SPARK-2228
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1182#issuecomment-46883069
All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16031/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---