You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Owen O'Malley (JIRA)" <ji...@apache.org> on 2009/05/08 09:21:45 UTC
[jira] Created: (HADOOP-5785) The job setup and cleanup tasks
should be optional
The job setup and cleanup tasks should be optional
--------------------------------------------------
Key: HADOOP-5785
URL: https://issues.apache.org/jira/browse/HADOOP-5785
Project: Hadoop Core
Issue Type: Improvement
Components: mapred
Reporter: Owen O'Malley
For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Attachment: patch-5785-2.txt
Patch incorporating review comments.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711158#action_12711158 ]
Vinod K V commented on HADOOP-5785:
-----------------------------------
i think it should be job specific as we specify setup/cleanup via OutputCommitter which itself is job specific.
Also, instead of configuration, as JobSetup and JobCleanup are specified via the job's OutPutCommitter, why not have something in the lines of the following and use them generically to determine if a job needs setup/cleanup.
{code}
OutputCommitter.isJobSetupNeeded()
OutputCommitter.isJobCleanupNeeded()
{code}
Thoughts?
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Attachment: patch-5785-1.txt
Added the temporary directory to be removed in garvageCollect() to cleanupQueue.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721439#action_12721439 ]
Hadoop QA commented on HADOOP-5785:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12411048/patch-5785-5.txt
against trunk revision 785928.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 27 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
-1 release audit. The applied patch generated 496 release audit warnings (more than the trunk's current 495 warnings).
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/530/testReport/
Release audit warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/530/artifact/trunk/patchprocess/releaseAuditDiffWarnings.txt
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/530/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/530/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/530/console
This message is automatically generated.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785-4.txt, patch-5785-5.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "He Yongqiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712975#action_12712975 ]
He Yongqiang commented on HADOOP-5785:
--------------------------------------
This is absolutely what i have wanted for a long time. Great! I vote for job-level.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713424#action_12713424 ]
Vinod K V commented on HADOOP-5785:
-----------------------------------
bq.If not a method, it can be a simple protected boolean OutputCommitter.isJobSetupNeeded over-ridable by subclasses. Does that work?
I take that back. Even reading this variable on the JobTracker needs the class to be loaded which in turn means running user code.
So, +1 for the initial proposal for a configuration property. +1 for job-level configuration.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Status: Open (was: Patch Available)
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Attachment: (was: patch-5785-4.txt)
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Vinod K V (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713018#action_12713018 ]
Vinod K V commented on HADOOP-5785:
-----------------------------------
bq. This cannot be done, because it is again user code. and we don't want to run any user code in JobTracker.
If not a method, it can be a simple protected boolean OutputCommitter.isJobSetupNeeded over-ridable by subclasses. Does that work?
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Status: Patch Available (was: Open)
test-patch and ant test passed on my machine
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785-4.txt, patch-5785-5.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711105#action_12711105 ]
Devaraj Das commented on HADOOP-5785:
-------------------------------------
Justification why it should be cluster-level config: (1) It is probably something that would conflict with the model of having setup/cleanup tasks in the first place, and hence should be applicable for all jobs or none. (2) It is for latency sensitive apps, and such jobs are usually benchmarks (like terasort) for which we probably will have a dedicated cluster, where we would run only such jobs (3) Might add to confusion when it comes to debugging jobs or the cluster itself.
The above is my opinion.. Open for discussion. I don't have a very strong preference for one over the other.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Attachment: patch-5785-5.txt
Updated patch with trunk.
Fixed a test failure with earlier patch
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785-4.txt, patch-5785-5.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Fix Version/s: 0.21.0
Affects Version/s: 0.19.0
Status: Patch Available (was: Open)
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Status: Open (was: Patch Available)
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785-4.txt, patch-5785-5.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719387#action_12719387 ]
Amareshwari Sriramadasu commented on HADOOP-5785:
-------------------------------------------------
Test failures are not related to the patch. All tests passed on my machine.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Status: Patch Available (was: Open)
test patch and ant test passed on my machine
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719000#action_12719000 ]
Hadoop QA commented on HADOOP-5785:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12410301/patch-5785-3.txt
against trunk revision 784042.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 24 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/495/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/495/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/495/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/495/console
This message is automatically generated.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Attachment: patch-5785-4.txt
Patch incorporating Owen's comments.
Added Job.setJobSetupCleanupNeeded and JobContext.getJobSetupCleanupNeeded.
Removed doSetupInline and doCleanupInline.
Added testcase with new api.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711094#action_12711094 ]
Devaraj Das commented on HADOOP-5785:
-------------------------------------
bq. Should the configuration be job-level or cluster-level?
I vote for cluster-level.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712973#action_12712973 ]
Amareshwari Sriramadasu commented on HADOOP-5785:
-------------------------------------------------
bq. Also, instead of configuration, as JobSetup and JobCleanup are specified via the job's OutPutCommitter, why not have something in the lines of the following and use them generically to determine if a job needs setup/cleanup.
This cannot be done, because it is again user code. and we don't want to run any user code in JobTracker.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Status: Open (was: Patch Available)
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716530#action_12716530 ]
Amar Kamat commented on HADOOP-5785:
------------------------------------
Some comments on JobInProgress :
1. remove JobContext. construct during initTasks.
2. Move setup/cleanup tasks initialization to different method
3. introduce methods setupInline/cleanupInline to do setup/cleanup inline in initTasks and garbageCollect.
4. Remove jobSetupCleanupNeeded guards from canLaunchJobCleanup/SetupTask() methods. And Use them in completedTask whenever maps/reduces complete.
5. add an api to FileoutputCommitter to give temp dir.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717694#action_12717694 ]
Hadoop QA commented on HADOOP-5785:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12410205/patch-5785-2.txt
against trunk revision 782881.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 Eclipse classpath. The patch retains Eclipse classpath integrity.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/485/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/485/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/485/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/485/console
This message is automatically generated.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716163#action_12716163 ]
Amareshwari Sriramadasu commented on HADOOP-5785:
-------------------------------------------------
test patch and ant test passed on my machine.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718319#action_12718319 ]
Amareshwari Sriramadasu commented on HADOOP-5785:
-------------------------------------------------
test patch and ant test passed on my machine with latest patch.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Status: Open (was: Patch Available)
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Status: Patch Available (was: Open)
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Attachment: patch-5785-4.txt
attaching correct file
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785-4.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12719423#action_12719423 ]
Owen O'Malley commented on HADOOP-5785:
---------------------------------------
You should define a setter and getter for the variable. In particular, it should go into the new Job and JobContext, respectively.
If the variable is set, no initialization or cleanup should be done. The assumption is that the application needs no setup or cleanup or it will be handled externally from something like Pig, Oozie, or Hive.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Attachment: patch-5785.txt
Attaching patch for review.
Patch does the following:
Reads the configuration mapred.committer.job.setup.cleanup.needed (default value is true). If the configuration is set to false. JobInProgress.initTasks does not initialize setup and cleanup tasks, creates _temporary directory by calling FileOutputCommitter.setupJob(); also changes job state to Running. The directory gets removed in garbageCollect() method.
To inform state change after initTasks, initJob method is added in TaskTrackerManager (implemented by JobTracker). JT.initJob calls JIP.initTasks without JobTracker lock. Once initialization completes/fails, if there is state change. it locks JobTracker and informs listeners.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Sreekanth Ramakrishnan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717987#action_12717987 ]
Sreekanth Ramakrishnan commented on HADOOP-5785:
------------------------------------------------
I tested the patch with respect to {{JobInitializationPoller}}. The changes look fine to me.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716689#action_12716689 ]
Hadoop QA commented on HADOOP-5785:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12409758/patch-5785-1.txt
against trunk revision 781968.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 18 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/467/console
This message is automatically generated.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721635#action_12721635 ]
Amareshwari Sriramadasu commented on HADOOP-5785:
-------------------------------------------------
-1 release audit. looks wrong. All the newly added files have license header. releaseAuditDiffWarnings.txt shows org.apache.hadoop.mapreduce.JobContext.html in the diff, which is not touched.
-1 core tests. Test failure is not related to the patch. Raised HADOOP-6086. Test passed on my machine
-1 contrib tests. Failure is a known issue.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785-4.txt, patch-5785-5.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711092#action_12711092 ]
Amareshwari Sriramadasu commented on HADOOP-5785:
-------------------------------------------------
Should we have a configuration property to speficy whether job setup and job cleanup are needed or not?
If the configuration is set, there will be no user code executed. JIP will create temporary output directory in initTasks() and delete the same in garbageCollect() (same code as pre-3150). Thoughts?
Should the configuration be job-level or cluster-level?
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Attachment: patch-5785-3.txt
Updated patch with trunk.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amar Kamat (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718311#action_12718311 ]
Amar Kamat commented on HADOOP-5785:
------------------------------------
Changes look fine to me.
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu reassigned HADOOP-5785:
-----------------------------------------------
Assignee: Amareshwari Sriramadasu
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5785) The job setup and cleanup tasks
should be optional
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HADOOP-5785:
--------------------------------------------
Status: Patch Available (was: Open)
> The job setup and cleanup tasks should be optional
> --------------------------------------------------
>
> Key: HADOOP-5785
> URL: https://issues.apache.org/jira/browse/HADOOP-5785
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.19.0
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.21.0
>
> Attachments: patch-5785-1.txt, patch-5785-2.txt, patch-5785-3.txt, patch-5785-4.txt, patch-5785.txt
>
>
> For jobs that require low latency and do not require setup or cleanup tasks for the job, it should be possible to turn them off for that job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.