You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Harsh J Chouraria (JIRA)" <ji...@apache.org> on 2010/11/01 21:06:24 UTC
[jira] Created: (MAPREDUCE-2170) Send out last-minute load averages
in TaskTrackerStatus
Send out last-minute load averages in TaskTrackerStatus
-------------------------------------------------------
Key: MAPREDUCE-2170
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: jobtracker
Affects Versions: 0.22.0
Environment: GNU/Linux
Reporter: Harsh J Chouraria
Assignee: Harsh J Chouraria
Priority: Minor
Fix For: 0.22.0
Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Updated] (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2170:
-------------------------------------
Status: Open (was: Patch Available)
Yes please, love to have it for NodeManager. Thanks!
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Critical
> Fix For: 0.23.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff, mapreduce.loadaverage.r6.diff
>
> Original Estimate: 20m
> Remaining Estimate: 20m
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927138#action_12927138 ]
Harsh J Chouraria commented on MAPREDUCE-2170:
----------------------------------------------
Hm, I didn't know about that MXBean thingy! [Just googled that method]
I could make the change, but the Linux resource plugin is only loaded when the os is detected to be "linux", that still makes it (mostly) non-portable doesn't it?
Is there a portable plugin?
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2170) Send out last-minute load averages
in TaskTrackerStatus
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated MAPREDUCE-2170:
-----------------------------------------
Attachment: mapreduce.loadaverage.r3.diff
Attaching patch here as requested by Arun.
Patch also addresses Scott Chen's query. The Linux plugin does not do any /proc/loadavg reading anymore.
P.s. Is it safe to also rely on {{OperatingSystemMXBean.getAvailableProcessors()}} for retrieving the number of CPUs via {{ResourceCalculatorPlugin.getNumProcessors()}}? If it is, I could add this on also.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: mapreduce.loadaverage.r3.diff
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933733#action_12933733 ]
Hadoop QA commented on MAPREDUCE-2170:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12459720/mapreduce.loadaverage.r3.diff
against trunk revision 1036745.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The patch appears to cause tar ant target to fail.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
org.apache.hadoop.cli.TestMRCLI
org.apache.hadoop.conf.TestJobConf
org.apache.hadoop.conf.TestNoDefaultsJobConf
org.apache.hadoop.examples.pi.math.TestSummation
org.apache.hadoop.examples.terasort.TestTeraSort
org.apache.hadoop.examples.TestBaileyBorweinPlouffe
org.apache.hadoop.fs.slive.TestSlive
org.apache.hadoop.fs.TestDFSIO
org.apache.hadoop.fs.TestFileSystem
org.apache.hadoop.fs.TestHarFileSystem
org.apache.hadoop.fs.TestJHLA
org.apache.hadoop.io.TestSequenceFileMergeProgress
org.apache.hadoop.ipc.TestSocketFactory
org.apache.hadoop.mapred.join.TestDatamerge
org.apache.hadoop.mapred.lib.TestDelegatingInputFormat
org.apache.hadoop.mapred.pipes.TestPipesAsDifferentUser
org.apache.hadoop.mapred.pipes.TestPipes
org.apache.hadoop.mapred.TestAdminOperationsProtocolWithServiceAuthorization
org.apache.hadoop.mapred.TestAuditLogger
org.apache.hadoop.mapred.TestBadRecords
org.apache.hadoop.mapred.TestClusterMapReduceTestCase
org.apache.hadoop.mapred.TestClusterMRNotification
org.apache.hadoop.mapred.TestClusterStatus
org.apache.hadoop.mapred.TestCompositeTaskTrackerInstrumentation
org.apache.hadoop.mapred.TestCompressedEmptyMapOutputs
org.apache.hadoop.mapred.TestControlledMapReduceJob
org.apache.hadoop.mapred.TestDebugScriptWithLinuxTaskController
org.apache.hadoop.mapred.TestDebugScript
org.apache.hadoop.mapred.TestEmptyJob
org.apache.hadoop.mapred.TestFieldSelection
org.apache.hadoop.mapred.TestIFile
org.apache.hadoop.mapred.TestInputPath
org.apache.hadoop.mapred.TestIsolationRunner
org.apache.hadoop.mapred.TestJobCleanup
org.apache.hadoop.mapred.TestJobClient
org.apache.hadoop.mapred.TestJobCounters
org.apache.hadoop.mapred.TestJobDirCleanup
org.apache.hadoop.mapred.TestJobExecutionAsDifferentUser
org.apache.hadoop.mapred.TestJobHistory
org.apache.hadoop.mapred.TestJobInProgressListener
org.apache.hadoop.mapred.TestJobKillAndFail
org.apache.hadoop.mapred.TestJobName
org.apache.hadoop.mapred.TestJobQueueClient
org.apache.hadoop.mapred.TestJobQueueInformation
org.apache.hadoop.mapred.TestJobRetire
org.apache.hadoop.mapred.TestJobStatusPersistency
org.apache.hadoop.mapred.TestJobSysDirWithDFS
org.apache.hadoop.mapred.TestJobTrackerInstrumentation
org.apache.hadoop.mapred.TestJobTrackerStart
org.apache.hadoop.mapred.TestJobTrackerXmlJsp
org.apache.hadoop.mapred.TestJvmManager
org.apache.hadoop.mapred.TestJvmReuse
org.apache.hadoop.mapred.TestKeyValueTextInputFormat
org.apache.hadoop.mapred.TestKillSubProcessesWithLinuxTaskController
org.apache.hadoop.mapred.TestKillSubProcesses
org.apache.hadoop.mapred.TestLazyOutput
org.apache.hadoop.mapred.TestLimitTasksPerJobTaskScheduler
org.apache.hadoop.mapred.TestLinuxTaskController
org.apache.hadoop.mapred.TestLocalizationWithLinuxTaskController
org.apache.hadoop.mapred.TestMapProgress
org.apache.hadoop.mapred.TestMapredHeartbeat
org.apache.hadoop.mapred.TestMapredSystemDir
org.apache.hadoop.mapred.TestMiniMRClasspath
org.apache.hadoop.mapred.TestMiniMRLocalFS
org.apache.hadoop.mapred.TestMiniMRWithDFS
org.apache.hadoop.mapred.TestMRServerPorts
org.apache.hadoop.mapred.TestMRWithDistributedCache
org.apache.hadoop.mapred.TestMultiFileInputFormat
org.apache.hadoop.mapred.TestMultiFileSplit
org.apache.hadoop.mapred.TestMultipleLevelCaching
org.apache.hadoop.mapred.TestMultipleTextOutputFormat
org.apache.hadoop.mapred.TestNodeHealthService
org.apache.hadoop.mapred.TestNodeRefresh
org.apache.hadoop.mapred.TestParallelInitialization
org.apache.hadoop.mapred.TestQueueManagerWithJobTracker
org.apache.hadoop.mapred.TestQueueManager
org.apache.hadoop.mapred.TestRecoveryManager
org.apache.hadoop.mapred.TestSetupAndCleanupFailure
org.apache.hadoop.mapred.TestSpecialCharactersInOutputPath
org.apache.hadoop.mapred.TestSubmitJob
org.apache.hadoop.mapred.TestTaskFail
org.apache.hadoop.mapred.TestTaskOutputSize
org.apache.hadoop.mapred.TestTaskTrackerMemoryManager
org.apache.hadoop.mapred.TestTaskTrackerSlotManagement
org.apache.hadoop.mapred.TestTrackerReservation
org.apache.hadoop.mapred.TestTTResourceReporting
org.apache.hadoop.mapred.TestWebUIAuthorization
org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat
org.apache.hadoop.mapreduce.lib.input.TestDelegatingInputFormat
org.apache.hadoop.mapreduce.lib.join.TestJoinDatamerge
org.apache.hadoop.mapreduce.lib.join.TestJoinProperties
org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter
org.apache.hadoop.mapreduce.security.TestBinaryTokenFile
org.apache.hadoop.mapreduce.security.TestTokenCacheOldApi
org.apache.hadoop.mapreduce.security.TestTokenCache
org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken
org.apache.hadoop.mapreduce.security.token.TestDelegationTokenRenewal
org.apache.hadoop.mapreduce.TestChild
org.apache.hadoop.mapreduce.TestJobACLs
org.apache.hadoop.mapreduce.TestMapReduceLazyOutput
org.apache.hadoop.mapreduce.TestMapReduceLocal
org.apache.hadoop.mapreduce.TestMRJobClient
org.apache.hadoop.mapreduce.TestNoJobSetupCleanup
org.apache.hadoop.mapreduce.TestTaskContext
org.apache.hadoop.security.authorize.TestServiceLevelAuthorization
org.apache.hadoop.security.TestMapredGroupMappingServiceRefresh
org.apache.hadoop.tools.rumen.TestRumenJobTraces
org.apache.hadoop.tools.TestCopyFiles
org.apache.hadoop.tools.TestDistCh
org.apache.hadoop.tools.TestHadoopArchives
org.apache.hadoop.tools.TestHarFileSystem
-1 contrib tests. The patch failed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/6//testReport/
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/6//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/6//console
This message is automatically generated.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: mapreduce.loadaverage.r3.diff
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2170) Send out last-minute load averages
in TaskTrackerStatus
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated MAPREDUCE-2170:
-----------------------------------------
Status: In Progress (was: Patch Available)
Fixing the cause that led to a failing core test. [New patch upped -- r5]
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2170) Send out last-minute load averages
in TaskTrackerStatus
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated MAPREDUCE-2170:
-----------------------------------------
Status: Patch Available (was: In Progress)
This should pass the tests. Had not added the TaskTracker.java diff previously, oops.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Ambud Sharma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209262#comment-13209262 ]
Ambud Sharma commented on MAPREDUCE-2170:
-----------------------------------------
Hi,
I am wondering if keeping that(systemaverageload) as a separate RPC call used/made when need would make more sense?
-Ambud
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Critical
> Fix For: 0.24.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff, mapreduce.loadaverage.r6.diff
>
> Original Estimate: 20m
> Remaining Estimate: 20m
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927145#action_12927145 ]
Allen Wittenauer commented on MAPREDUCE-2170:
---------------------------------------------
That's sort of my point: why not promote this to be a Hadoop feature rather than something tied to a specific OS if we can support it in a generic way?
One of the bad things about load avg is that it doesn't give you a why. In other words, a node may be totally fine running more cpu-bound tasks if the box is actually being hammered IO-wise. Since both conditions show as load.... But this would require a lot more smarts in the system and of the user. So while it makes me slightly uncomfortable to use this value, load should be 'good enough' given the current state of the system.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2170) Send out last-minute load averages
in TaskTrackerStatus
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-2170:
-------------------------------------
Status: Patch Available (was: Open)
+1 the patch looks good.
Marking it 'PA" on Harsh's behalf... I'll commit once we get an 'ok' from Hudson.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: mapreduce.loadaverage.r3.diff
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927317#action_12927317 ]
Scott Chen commented on MAPREDUCE-2170:
---------------------------------------
And thank you for adding this feature, Harsh. I think this is a good thing to have :)
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12928752#action_12928752 ]
Scott Chen commented on MAPREDUCE-2170:
---------------------------------------
Hey Harsh, I think it is OK to cast it into float.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2170) Send out last-minute load averages
in TaskTrackerStatus
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated MAPREDUCE-2170:
-----------------------------------------
Attachment: mapreduce.loadaverage.r5.diff
Attaching a new patch that should fix the test case fail.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930227#action_12930227 ]
Scott Chen commented on MAPREDUCE-2170:
---------------------------------------
Hey Harsh,
Why do we need to override getSystemLoadAverage()? The native one should also work for Linux case, right?
{code}
@Override
public float getSystemLoadAverage() {
readProcLoadAverageFile();
return loadAverage;
}
{code}
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933867#action_12933867 ]
Nigel Daley commented on MAPREDUCE-2170:
----------------------------------------
Sorry for the long precommit comment by Hadoop QA. I'm trying to get this running on MR again. Lots of tests failing. Looks like trunk wasn't properly updated after 0.22 branching. I still see references to 0.22 in the ivy/libraries.properties file on trunk.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: mapreduce.loadaverage.r3.diff
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2170) Send out last-minute load averages
in TaskTrackerStatus
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated MAPREDUCE-2170:
-----------------------------------------
Attachment: mapreduce.loadaverage.r4.diff
Updating patch to fix a findbugs BX warning on ResourceCalculatorPlugin:46
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Updated] (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated MAPREDUCE-2170:
-----------------------------------------
Fix Version/s: 0.23.0
Issue Type: New Feature (was: Improvement)
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Critical
> Fix For: 0.23.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff
>
> Original Estimate: 20m
> Remaining Estimate: 20m
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977858#action_12977858 ]
Harsh J Chouraria commented on MAPREDUCE-2170:
----------------------------------------------
Running 'ant test-patch' passes here.
{code}
[exec] +1 overall.
[exec]
[exec] +1 @author. The patch does not contain any @author tags.
[exec]
[exec] +1 tests included. The patch appears to include 6 new or modified tests.
[exec]
[exec] +1 javadoc. The javadoc tool did not generate any warning messages.
[exec]
[exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
[exec]
[exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
[exec]
[exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
[exec]
[exec] +1 system test framework. The patch passed system test framework compile.
[exec]
[exec]
[exec]
[exec]
[exec] ======================================================================
[exec] ======================================================================
[exec] Finished build.
[exec] ======================================================================
[exec] ======================================================================
[exec]
[exec]
BUILD SUCCESSFUL
Total time: 31 minutes 51 seconds
{code}
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927129#action_12927129 ]
Allen Wittenauer commented on MAPREDUCE-2170:
---------------------------------------------
Why not use getSystemLoadAvg() to make it portable?
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047940#comment-13047940 ]
Harsh J commented on MAPREDUCE-2170:
------------------------------------
Well this is strange… Went through about 15 of the failed list, but they do not appear related to my TT status patch at all.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Critical
> Fix For: 0.23.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff, mapreduce.loadaverage.r6.diff
>
> Original Estimate: 20m
> Remaining Estimate: 20m
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-2170:
-------------------------------
Environment: (was: GNU/Linux)
Tags: load average, tasktracker (was: load average, tasktracker, linux)
(Changed env/tags to make it look non-linux-specific since it would work across all platforms).
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Critical
> Fix For: 0.23.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff, mapreduce.loadaverage.r6.diff
>
> Original Estimate: 20m
> Remaining Estimate: 20m
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927150#action_12927150 ]
Harsh J Chouraria commented on MAPREDUCE-2170:
----------------------------------------------
We could add a portable plugin, sure! Are there sufficient functions available as part of the JVM itself, for this?
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Harsh J (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176041#comment-13176041 ]
Harsh J commented on MAPREDUCE-2170:
------------------------------------
I took a look at NodeManager parts, and it appears that the plugin is only used for resource view use? How does the resource plugin come into play when it comes to the resource-based scheduler?
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Critical
> Fix For: 0.24.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff, mapreduce.loadaverage.r6.diff
>
> Original Estimate: 20m
> Remaining Estimate: 20m
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J updated MAPREDUCE-2170:
-------------------------------
Attachment: mapreduce.loadaverage.r6.diff
Rebased patch for trunk (no changes).
Is there still demand for this addition?
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Critical
> Fix For: 0.23.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff, mapreduce.loadaverage.r6.diff
>
> Original Estimate: 20m
> Remaining Estimate: 20m
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927215#action_12927215 ]
Allen Wittenauer commented on MAPREDUCE-2170:
---------------------------------------------
Why does it have to be in a plug-in? Why can't it just be part of the TaskTracker in general?
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12935309#action_12935309 ]
Hadoop QA commented on MAPREDUCE-2170:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12460091/mapreduce.loadaverage.r4.diff
against trunk revision 1038357.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
org.apache.hadoop.mapred.TestControlledMapReduceJob
org.apache.hadoop.mapred.TestTTResourceReporting
org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken
org.apache.hadoop.tools.rumen.TestRumenJobTraces
org.apache.hadoop.tools.TestHarFileSystem
-1 contrib tests. The patch failed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/7//testReport/
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/7//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/7//console
This message is automatically generated.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930068#action_12930068 ]
Harsh J Chouraria commented on MAPREDUCE-2170:
----------------------------------------------
I've updated the patch on the review board.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12999935#comment-12999935 ]
Hadoop QA commented on MAPREDUCE-2170:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12466938/mapreduce.loadaverage.r5.diff
against trunk revision 1074251.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/54//testReport/
Findbugs warnings: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/54//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://hudson.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/54//console
This message is automatically generated.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff
>
> Original Estimate: 20m
> Remaining Estimate: 20m
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927465#action_12927465 ]
Harsh J Chouraria commented on MAPREDUCE-2170:
----------------------------------------------
I began to change this, but have run into a small issue. How do I store a 'double' inside a Configuration? There are only ways to set/get floats from it.
getSystemLoadAverage() in OperatingSystemMXBean returns a double. Is it okay if we change that to a float and use it?
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047939#comment-13047939 ]
Hadoop QA commented on MAPREDUCE-2170:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12482150/mapreduce.loadaverage.r6.diff
against trunk revision 1134180.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 6 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
org.apache.hadoop.cli.TestMRCLI
org.apache.hadoop.conf.TestNoDefaultsJobConf
org.apache.hadoop.examples.terasort.TestTeraSort
org.apache.hadoop.fs.TestDFSIO
org.apache.hadoop.fs.TestFileSystem
org.apache.hadoop.ipc.TestSocketFactory
org.apache.hadoop.mapred.join.TestDatamerge
org.apache.hadoop.mapred.lib.TestDelegatingInputFormat
org.apache.hadoop.mapred.pipes.TestPipes
org.apache.hadoop.mapred.TestBadRecords
org.apache.hadoop.mapred.TestClusterMapReduceTestCase
org.apache.hadoop.mapred.TestCommandLineJobSubmission
org.apache.hadoop.mapred.TestCompressedEmptyMapOutputs
org.apache.hadoop.mapred.TestControlledMapReduceJob
org.apache.hadoop.mapred.TestFileInputFormat
org.apache.hadoop.mapred.TestJobClient
org.apache.hadoop.mapred.TestJobDirCleanup
org.apache.hadoop.mapred.TestJobHistory
org.apache.hadoop.mapred.TestJobName
org.apache.hadoop.mapred.TestJobQueueInformation
org.apache.hadoop.mapred.TestJobStatusPersistency
org.apache.hadoop.mapred.TestJobSysDirWithDFS
org.apache.hadoop.mapred.TestJobTrackerXmlJsp
org.apache.hadoop.mapred.TestLazyOutput
org.apache.hadoop.mapred.TestMapredHeartbeat
org.apache.hadoop.mapred.TestMapredSystemDir
org.apache.hadoop.mapred.TestMiniMRChildTask
org.apache.hadoop.mapred.TestMiniMRClasspath
org.apache.hadoop.mapred.TestMiniMRDFSCaching
org.apache.hadoop.mapred.TestMiniMRDFSSort
org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
org.apache.hadoop.mapred.TestMiniMRWithDFS
org.apache.hadoop.mapred.TestMultipleLevelCaching
org.apache.hadoop.mapred.TestNodeRefresh
org.apache.hadoop.mapred.TestRecoveryManager
org.apache.hadoop.mapred.TestReduceFetchFromPartialMem
org.apache.hadoop.mapred.TestReduceFetch
org.apache.hadoop.mapred.TestSetupAndCleanupFailure
org.apache.hadoop.mapred.TestSeveral
org.apache.hadoop.mapred.TestSpecialCharactersInOutputPath
org.apache.hadoop.mapred.TestSubmitJob
org.apache.hadoop.mapred.TestTaskFail
org.apache.hadoop.mapred.TestWebUIAuthorization
org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat
org.apache.hadoop.mapreduce.lib.input.TestDelegatingInputFormat
org.apache.hadoop.mapreduce.lib.join.TestJoinDatamerge
org.apache.hadoop.mapreduce.lib.join.TestJoinProperties
org.apache.hadoop.mapreduce.security.TestBinaryTokenFile
org.apache.hadoop.mapreduce.security.TestTokenCacheOldApi
org.apache.hadoop.mapreduce.security.TestTokenCache
org.apache.hadoop.mapreduce.TestMapReduceLazyOutput
org.apache.hadoop.mapreduce.TestMRJobClient
org.apache.hadoop.security.authorize.TestServiceLevelAuthorization
org.apache.hadoop.security.TestMapredGroupMappingServiceRefresh
org.apache.hadoop.tools.TestCopyFiles
org.apache.hadoop.tools.TestDistCh
org.apache.hadoop.tools.TestHadoopArchives
org.apache.hadoop.tools.TestHarFileSystem
-1 contrib tests. The patch failed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/375//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/375//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/375//console
This message is automatically generated.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Critical
> Fix For: 0.23.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff, mapreduce.loadaverage.r6.diff
>
> Original Estimate: 20m
> Remaining Estimate: 20m
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Philip Zeyliger (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176042#comment-13176042 ]
Philip Zeyliger commented on MAPREDUCE-2170:
--------------------------------------------
I will return to the office on January 17th. For urgent matters,
please contact Aparna or Philip L. The week between Christmas and New
Years', your best bet is Chris L.
Thanks,
-- Philip
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Critical
> Fix For: 0.24.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff, mapreduce.loadaverage.r6.diff
>
> Original Estimate: 20m
> Remaining Estimate: 20m
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930194#action_12930194 ]
Arun C Murthy commented on MAPREDUCE-2170:
------------------------------------------
Harsh, can you please put up the patch here on jira and grant license to ASF for inclusion? Thanks.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Harsh J (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209275#comment-13209275 ]
Harsh J commented on MAPREDUCE-2170:
------------------------------------
Ambud,
What gain do you see in disconnecting it off of other stats (CPU/Memory/etc.) in the regular heartbeats?
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Critical
> Fix For: 0.24.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff, mapreduce.loadaverage.r6.diff
>
> Original Estimate: 20m
> Remaining Estimate: 20m
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Scott Chen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927315#action_12927315 ]
Scott Chen commented on MAPREDUCE-2170:
---------------------------------------
+1 on Allen's idea of using getSystemLoadAvg() in ResourceCalculatorPlugin.getSystemLoadAverage(). So if in the future people have added a new ResourceCalculatorPlugin for other platform, they can directly use this method.
I think it is better to put this in ResourceCalculatorPlugin to keep similar tools in the same class.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Harsh J (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098572#comment-13098572 ]
Harsh J commented on MAPREDUCE-2170:
------------------------------------
If there's still interest for this, I can update it up for the hadoop-yarn framework (ContainersMonitor).
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobtracker
> Affects Versions: 0.22.0
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Critical
> Fix For: 0.23.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff, mapreduce.loadaverage.r6.diff
>
> Original Estimate: 20m
> Remaining Estimate: 20m
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927301#action_12927301 ]
Harsh J Chouraria commented on MAPREDUCE-2170:
----------------------------------------------
Plugins could help if there are better detailed, platform-specific ways of getting resource statistics?
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Minor
> Fix For: 0.22.0
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-2170) Send out last-minute load averages
in TaskTrackerStatus
Posted by "Harsh J Chouraria (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J Chouraria updated MAPREDUCE-2170:
-----------------------------------------
Priority: Blocker (was: Minor)
Setting to BLOCKER as per Nigel's mail.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Blocker
> Fix For: 0.22.0
>
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Updated] (MAPREDUCE-2170) Send out last-minute load
averages in TaskTrackerStatus
Posted by "Nigel Daley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nigel Daley updated MAPREDUCE-2170:
-----------------------------------
Priority: Critical (was: Blocker)
Fix Version/s: (was: 0.22.0)
This isn't a blocker for 0.22. Removing from 0.22.
> Send out last-minute load averages in TaskTrackerStatus
> -------------------------------------------------------
>
> Key: MAPREDUCE-2170
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Affects Versions: 0.22.0
> Environment: GNU/Linux
> Reporter: Harsh J Chouraria
> Assignee: Harsh J Chouraria
> Priority: Critical
> Attachments: mapreduce.loadaverage.r3.diff, mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff
>
> Original Estimate: 20m
> Remaining Estimate: 20m
>
> Load averages could be useful in scheduling. This patch looks to extend the existing Linux resource plugin (via /proc/loadavg file) to allow transmitting load averages of the last one minute via the TaskTrackerStatus.
> Patch is up for review, with test cases added, at: https://reviews.apache.org/r/20/
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira