You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by sarutak <gi...@git.apache.org> on 2014/08/07 06:23:08 UTC
[GitHub] spark pull request: [SPARK-2896] Remove redundant error notificati...
GitHub user sarutak opened a pull request:
https://github.com/apache/spark/pull/1823
[SPARK-2896] Remove redundant error notification by results.put() when fetching from remote
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sarutak/spark SPARK-2896
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1823.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1823
----
commit 997720518e8c0f0000707c2b2919498dab5a6eee
Author: Kousuke Saruta <sa...@oss.nttdata.co.jp>
Date: 2014-08-07T04:13:31Z
Removed redundant error notification
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-2896] Remove redundant error notificati...
Posted by rxin <gi...@git.apache.org>.
Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/1823#issuecomment-55078979
Do you mind closing this? This whole code is going to be updated by https://github.com/apache/spark/pull/2330
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-2896] Remove redundant error notificati...
Posted by sarutak <gi...@git.apache.org>.
Github user sarutak closed the pull request at:
https://github.com/apache/spark/pull/1823
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-2896] Remove redundant error notificati...
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/1823#issuecomment-51699829
I took a quick look - I'm not sure we want to change this. Just because only the first result is used right now doesn't mean we won't want to do something more sophisticated in the future. And donwstream consumers of code could make the assumption that all failed fetches will be present.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-2896] Remove redundant error notificati...
Posted by sarutak <gi...@git.apache.org>.
Github user sarutak commented on the pull request:
https://github.com/apache/spark/pull/1823#issuecomment-51699999
Thanks for looking this ,@pwendell .
This change is also for consistency. In #1578, I discussed whether all of failures should be set to results or not with @mateiz and I decided to set only first failure for fast fail.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-2896] Remove redundant error notificati...
Posted by pwendell <gi...@git.apache.org>.
Github user pwendell commented on a diff in the pull request:
https://github.com/apache/spark/pull/1823#discussion_r16028389
--- Diff: core/src/main/scala/org/apache/spark/storage/BlockFetcherIterator.scala ---
@@ -132,9 +132,12 @@ object BlockFetcherIterator {
}
case Failure(exception) => {
logError("Could not get block(s) from " + cmId, exception)
- for ((blockId, size) <- req.blocks) {
- results.put(new FetchResult(blockId, -1, null))
- }
+ /**
+ * In current implementation, only pass the first element is enough
+ * because when iterator get FetchResult which size is -1,
+ * FetchFailedException is thrown immediately.
+ */
+ results.put(new FetchResult(req.blocks(0)._1, -1, null))
--- End diff --
Changing this seems somewhat brittle... what if we want to change the behavior on the receiving end? These results are very small, I don't see a big problem with sending them.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-2896] Remove redundant error notificati...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/1823#issuecomment-51429930
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org