You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Runping Qi (JIRA)" <ji...@apache.org> on 2007/10/09 18:48:51 UTC
[jira] Created: (HADOOP-2015) Job tracker should report the number
of splits that are local to some task trackers
Job tracker should report the number of splits that are local to some task trackers
-----------------------------------------------------------------------------------
Key: HADOOP-2015
URL: https://issues.apache.org/jira/browse/HADOOP-2015
Project: Hadoop
Issue Type: Improvement
Components: mapred
Reporter: Runping Qi
Right now, jon tracker keeps track the number of launched mappers with local data.
However, it is not clear how many mappers that are potentially be launched with data locality.
This information is readily available in Job Tracker. It is just a matter to create a separate global counter
and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2015) Job tracker should report the number
of splits that are local to some task trackers
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley updated HADOOP-2015:
----------------------------------
Status: Open (was: Patch Available)
I meant to cancel this patch, until it logs rather than use a user counter to record the information.
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
> Attachments: hadoop-2015.txt
>
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2015) Job tracker should report the number
of splits that are local to some task trackers
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Runping Qi updated HADOOP-2015:
-------------------------------
Status: Open (was: Patch Available)
Need some more change
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2015) Job tracker should report the number
of splits that are local to some task trackers
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Runping Qi updated HADOOP-2015:
-------------------------------
Attachment: hadoop-2015.txt
a patch that adds a counter for the total number of tasks
that may be launched with local data
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
> Attachments: hadoop-2015.txt
>
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2015) Job tracker should report the number
of splits that are local to some task trackers
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Runping Qi updated HADOOP-2015:
-------------------------------
Attachment: (was: hadoop-2015.txt)
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2015) Job tracker should report the number
of splits that are local to some task trackers
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Runping Qi updated HADOOP-2015:
-------------------------------
Status: Patch Available (was: Open)
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
> Attachments: hadoop-2015.txt
>
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2015) Job tracker should report the number
of splits that are local to some task trackers
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Runping Qi updated HADOOP-2015:
-------------------------------
Status: Patch Available (was: Open)
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
> Attachments: hadoop-2015.txt
>
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2015) Job tracker should report the
number of splits that are local to some task trackers
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12537455 ]
Owen O'Malley commented on HADOOP-2015:
---------------------------------------
I think for most users this is unwanted information. I'd be much happier if the information was just logged on the JT where an interested sysadmin could mine the data, but where it wouldn't confuse the user.
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
> Attachments: hadoop-2015.txt
>
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2015) Job tracker should report the number
of splits that are local to some task trackers
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Runping Qi updated HADOOP-2015:
-------------------------------
Attachment: (was: hadoop-2015.txt)
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
> Attachments: hadoop-2015.txt
>
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-2015) Job tracker should report the number
of splits that are local to some task trackers
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Runping Qi reassigned HADOOP-2015:
----------------------------------
Assignee: Runping Qi
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2015) Job tracker should report the number
of splits that are local to some task trackers
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Runping Qi updated HADOOP-2015:
-------------------------------
Status: Open (was: Patch Available)
a more optimized patch is available
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
> Attachments: hadoop-2015.txt
>
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2015) Job tracker should report the number
of splits that are local to some task trackers
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Runping Qi updated HADOOP-2015:
-------------------------------
Attachment: hadoop-2015.txt
a better one
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
> Attachments: hadoop-2015.txt
>
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2015) Job tracker should report the
number of splits that are local to some task trackers
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533871 ]
Hadoop QA commented on HADOOP-2015:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12367515/hadoop-2015.txt
against trunk revision r583424.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests -1. The patch failed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/916/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/916/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/916/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/916/console
This message is automatically generated.
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
> Attachments: hadoop-2015.txt
>
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2015) Job tracker should report the
number of splits that are local to some task trackers
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533969 ]
Hadoop QA commented on HADOOP-2015:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12367530/hadoop-2015.txt
against trunk revision r583424.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/919/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/919/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/919/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/919/console
This message is automatically generated.
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
> Attachments: hadoop-2015.txt
>
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2015) Job tracker should report the number
of splits that are local to some task trackers
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Runping Qi updated HADOOP-2015:
-------------------------------
Attachment: hadoop-2015.txt
Fix a bug in the patch
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
> Attachments: hadoop-2015.txt
>
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-2015) Job tracker should report the number
of splits that are local to some task trackers
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Runping Qi updated HADOOP-2015:
-------------------------------
Status: Patch Available (was: Open)
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
> Attachments: hadoop-2015.txt
>
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-2015) Job tracker should report the
number of splits that are local to some task trackers
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533561 ]
Hadoop QA commented on HADOOP-2015:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12367404/hadoop-2015.txt
against trunk revision r583309.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/913/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/913/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/913/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/913/console
This message is automatically generated.
> Job tracker should report the number of splits that are local to some task trackers
> -----------------------------------------------------------------------------------
>
> Key: HADOOP-2015
> URL: https://issues.apache.org/jira/browse/HADOOP-2015
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Reporter: Runping Qi
> Assignee: Runping Qi
> Attachments: hadoop-2015.txt
>
>
> Right now, jon tracker keeps track the number of launched mappers with local data.
> However, it is not clear how many mappers that are potentially be launched with data locality.
> This information is readily available in Job Tracker. It is just a matter to create a separate global counter
> and set it at the Job Tracker initialization time.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.