You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by ilganeli <gi...@git.apache.org> on 2015/04/07 19:20:35 UTC
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
GitHub user ilganeli opened a pull request:
https://github.com/apache/spark/pull/5396
[SPARK-6746B] Refactor large functions in DAGScheduler to improve readibility
Hi all. The DAG Scheduler had several very large, complex, and unreadable functions. Specifically,
```handleTaskCompletion```, ```cleanupStateForJobAndIndependentStages```, and ```submitMissingTasks```. This patch is almost exclusively code movement, very little code has changed with the exception of handling return logic when serializing tasks within ```submitMissingTasks```.
I believe the final product is much more readable and makes it much easier to understand what's going on within the DAGScheduler.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ilganeli/spark SPARK-6746B
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5396.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5396
----
commit a63eb72b58014cfbc408be053ab5a22f21abae35
Author: Ilya Ganelin <il...@capitalone.com>
Date: 2015-04-07T17:14:22Z
[SPARK-6746] Refactored submitMissingTasks, cleanupStateForJobAndIndependentStages, and handleTaskCompletion
commit c9db10da94f066109b915d24c3f0f3ec7ee50eea
Author: Ilya Ganelin <il...@capitalone.com>
Date: 2015-04-07T17:19:55Z
Updated shuffle map task
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-91011325
[Test build #29874 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29874/consoleFull) for PR 5396 at commit [`f0dcc7b`](https://github.com/apache/spark/commit/f0dcc7b8b62e7cbb4608b5cc9f3e6fe865c87bd8).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by ilganeli <gi...@git.apache.org>.
Github user ilganeli commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90747691
Mark - you've convinced me. With that said, do you see any way to make the code more modular without sacrificing the protection from scoping? My primary intent with my refactoring was to break things out into discrete function calls. Would moving those internal to where they're used help?
Sent with Good (www.good.com)
-----Original Message-----
From: Mark Hamstra [notifications@github.com<ma...@github.com>]
Sent: Tuesday, April 07, 2015 05:55 PM Eastern Standard Time
To: apache/spark
Cc: Ganelin, Ilya
Subject: Re: [spark] [SPARK-6746B] Refactor large functions in DAGScheduler to improve readibility (#5396)
I guess I just don't see things the same way. I don't have a problem with the DAGScheduler being implemented with scoping idioms that may be unfamiliar to a large number of coders. The DAGScheduler is about as far from being a public API in Spark as you can get, and I don't really think we want to sacrifice the additional safety that nested scopes provide within the DAGScheduler just to make it easy for more developers to make changes to that part of Spark's code. If you want an even more contrary point of view, a high bar of expected familiarity with certain idioms actually can serve us well in restricting just who is making changes to the DAGScheduler and what style and concerns they are maintaining.
Quite simply, I don't find a function declared within the scope of another function, for example, to be at all difficult to read -- it's just a common idiom across several programming languages with which I am familiar. On the other hand, flattening out those carefully nested scopes does actually make it harder for me to read and reason about where and under what preconditions a previously-nested function can and should be used.
I'm not saying that I won't consider any refactoring of the DAGScheduler or the flattening of any scopes; but in order for me not to veto them, any such changes will require a lot more justification than simply that making them produces code that is more readable for a larger audience of developers.
—
Reply to this email directly or view it on GitHub<https://github.com/apache/spark/pull/5396#issuecomment-90743681>.
________________________________________________________
The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-91006985
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29871/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by markhamstra <gi...@git.apache.org>.
Github user markhamstra commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90743681
I guess I just don't see things the same way. I don't have a problem with the DAGScheduler being implemented with scoping idioms that may be unfamiliar to a large number of coders. The DAGScheduler is about as far from being a public API in Spark as you can get, and I don't really think we want to sacrifice the additional safety that nested scopes provide within the DAGScheduler just to make it easy for more developers to make changes to that part of Spark's code. If you want an even more contrary point of view, a high bar of expected familiarity with certain idioms actually can serve us well in restricting just who is making changes to the DAGScheduler and what style and concerns they are maintaining.
Quite simply, I don't find a function declared within the scope of another function, for example, to be at all difficult to read -- it's just a common idiom across several programming languages with which I am familiar. On the other hand, flattening out those carefully nested scopes does actually make it harder for me to read and reason about where and under what preconditions a previously-nested function can and should be used.
I'm not saying that I won't consider any refactoring of the DAGScheduler or the flattening of any scopes; but in order for me not to veto them, any such changes will require a lot more justification than simply that making them produces code that is more readable for a larger audience of developers.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90989108
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29873/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-96797161
[Test build #31024 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/31024/consoleFull) for PR 5396 at commit [`f0dcc7b`](https://github.com/apache/spark/commit/f0dcc7b8b62e7cbb4608b5cc9f3e6fe865c87bd8).
* This patch **fails Spark unit tests**.
* This patch **does not merge cleanly**.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-91006975
[Test build #29871 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29871/consoleFull) for PR 5396 at commit [`3fc6258`](https://github.com/apache/spark/commit/3fc62582559f3c388f7e3a5edddf0d6c0df08d46).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by markhamstra <gi...@git.apache.org>.
Github user markhamstra commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90717805
Ok, in looking further, I see that I'm going to have a more general problem with this PR. I can't see exposing carefully nested and scoped portions of functions to the rest of the DAGScheduler as a good thing. Things are defined and visible where they make sense and are needed, and are intentionally not available elsewhere. To me, exposing additional interfaces to all of the DAGScheduler makes it more complicated to figure out how and when the various pieces should be used, not less.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90695334
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29800/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90984998
[Test build #29871 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29871/consoleFull) for PR 5396 at commit [`3fc6258`](https://github.com/apache/spark/commit/3fc62582559f3c388f7e3a5edddf0d6c0df08d46).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90695319
[Test build #29800 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29800/consoleFull) for PR 5396 at commit [`c9db10d`](https://github.com/apache/spark/commit/c9db10da94f066109b915d24c3f0f3ec7ee50eea).
* This patch **passes all tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90653800
[Test build #29800 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29800/consoleFull) for PR 5396 at commit [`c9db10d`](https://github.com/apache/spark/commit/c9db10da94f066109b915d24c3f0f3ec7ee50eea).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by ilganeli <gi...@git.apache.org>.
Github user ilganeli commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90987956
@markhamstra I've moved the helper functions into the scope of the functions that use them and added clarifying comments to separate the helper functions from the code that uses them. The major gain that I see here is that now process flow for the complex functions is much easier to understand, e.g. in ```handleTaskCompletion``` the operational logic is now condensed to a concise set of case statements and the handling of each case is abstracted to the individual functions.
By modularizing the code, it's now easier to reason about the operation of the DAGScheduler. I agree with the need to minimize uninformed modification of this code but I think even core committers could benefit from not having to consider execution logic that is several hundred lines long.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-91011337
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29874/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by ilganeli <gi...@git.apache.org>.
Github user ilganeli commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-91361642
@markhamstra What do you think of the latest revision? This makes the code more modular and readable (imo) without sacrificing anything by way of code safety.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90988857
[Test build #29873 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29873/consoleFull) for PR 5396 at commit [`c95e15e`](https://github.com/apache/spark/commit/c95e15e62ccf178458d2ff868eb63d4ea889f59b).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by markhamstra <gi...@git.apache.org>.
Github user markhamstra commented on a diff in the pull request:
https://github.com/apache/spark/pull/5396#discussion_r27914814
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ---
@@ -418,6 +418,32 @@ class DAGScheduler(
updateJobIdStageIdMapsList(List(stage))
}
+ /** Clear references to the given stage in stored data structures */
+ private def removeStage(stageId: Int) {
--- End diff --
By moving this you've changed the scope in which removeStage is visible, potentially opening up improper calls to removeStage from elsewhere than intended in the DAGScheduler.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90990453
[Test build #29874 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29874/consoleFull) for PR 5396 at commit [`f0dcc7b`](https://github.com/apache/spark/commit/f0dcc7b8b62e7cbb4608b5cc9f3e6fe865c87bd8).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by ilganeli <gi...@git.apache.org>.
Github user ilganeli commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90731048
Mark - your point is valid. However, then we we get enormous monolithic functions that are near impossible to read , comprehend, or debug without specialized knowledge. The counterpoint I'd make is that any changes throughout DAGScheduler are necessarily faced with heavy scrutiny, I'd argue that opportunity for misuse of the now visible functions is unlikely.
Moreover, because their functionality is now much clearer, it will be easier to reason about any subsequent changes and write cleaner more stable code that uses these pieces effectively.
Sent with Good (www.good.com)
-----Original Message-----
From: Mark Hamstra [notifications@github.com<ma...@github.com>]
Sent: Tuesday, April 07, 2015 04:16 PM Eastern Standard Time
To: apache/spark
Cc: Ganelin, Ilya
Subject: Re: [spark] [SPARK-6746B] Refactor large functions in DAGScheduler to improve readibility (#5396)
Ok, in looking further, I see that I'm going to have a more general problem with this PR. I can't see exposing carefully nested and scoped portions of functions to the rest of the DAGScheduler as a good thing. Things are defined and visible where they make sense and are needed, and are intentionally not available elsewhere. To me, exposing additional interfaces to all of the DAGScheduler makes it more complicated to figure out how and when the various pieces should be used, not less.
—
Reply to this email directly or view it on GitHub<https://github.com/apache/spark/pull/5396#issuecomment-90717805>.
________________________________________________________
The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by ilganeli <gi...@git.apache.org>.
Github user ilganeli closed the pull request at:
https://github.com/apache/spark/pull/5396
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by ilganeli <gi...@git.apache.org>.
Github user ilganeli commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-95245649
All - I assume the lack of response indicates that folks aren't interested in this patch. I'll close this unless I get some further comments.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by markhamstra <gi...@git.apache.org>.
Github user markhamstra commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90750182
Maybe... convince me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-6746B] Refactor large functions in DAGS...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/5396#issuecomment-90989102
[Test build #29873 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/29873/consoleFull) for PR 5396 at commit [`c95e15e`](https://github.com/apache/spark/commit/c95e15e62ccf178458d2ff868eb63d4ea889f59b).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
* This patch does not change any dependencies.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org