You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2009/12/14 16:46:18 UTC
[jira] Created: (CHUKWA-430) Narrow down input for FSM mapreduce
job
Narrow down input for FSM mapreduce job
---------------------------------------
Key: CHUKWA-430
URL: https://issues.apache.org/jira/browse/CHUKWA-430
Project: Hadoop Chukwa
Issue Type: Bug
Components: MR Data Processors
Affects Versions: 0.4.0
Reporter: Eric Yang
Assignee: Eric Yang
Fix For: 0.4.0
FSMDataloader supplies all demux output data to FSM state machine. This is not efficient because most of the data type do not contribute to state generation. According to Jiaqi, the state machine requires the following types:
JobHistoryTaskDataMapper:
/chukwa/repos/chukwa/JobData
/chukwa/repos/chukwa/TaskData
TaskTrackerClientTraceMapper:
/chukwa/repos/chukwa/ClientTraceDetailed
DataNodeClientTraceMapper:
/chukwa/repos/chukwa/ClientTraceDetailed
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-430) Narrow down input for FSM mapreduce
job
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-430:
-----------------------------
Status: Patch Available (was: Open)
> Narrow down input for FSM mapreduce job
> ---------------------------------------
>
> Key: CHUKWA-430
> URL: https://issues.apache.org/jira/browse/CHUKWA-430
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: MR Data Processors
> Affects Versions: 0.4.0
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.4.0
>
> Attachments: CHUKWA-430.patch
>
>
> FSMDataloader supplies all demux output data to FSM state machine. This is not efficient because most of the data type do not contribute to state generation. According to Jiaqi, the state machine requires the following types:
> JobHistoryTaskDataMapper:
> /chukwa/repos/chukwa/JobData
> /chukwa/repos/chukwa/TaskData
> TaskTrackerClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> DataNodeClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> This jira is to optimize the data loader supplied input, and narrow down the required input type.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-430) Narrow down input for FSM mapreduce
job
Posted by "Jiaqi Tan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790483#action_12790483 ]
Jiaqi Tan commented on CHUKWA-430:
----------------------------------
+1 to patch, looks good to me
> Narrow down input for FSM mapreduce job
> ---------------------------------------
>
> Key: CHUKWA-430
> URL: https://issues.apache.org/jira/browse/CHUKWA-430
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: MR Data Processors
> Affects Versions: 0.4.0
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.4.0
>
> Attachments: CHUKWA-430.patch
>
>
> FSMDataloader supplies all demux output data to FSM state machine. This is not efficient because most of the data type do not contribute to state generation. According to Jiaqi, the state machine requires the following types:
> JobHistoryTaskDataMapper:
> /chukwa/repos/chukwa/JobData
> /chukwa/repos/chukwa/TaskData
> TaskTrackerClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> DataNodeClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> This jira is to optimize the data loader supplied input, and narrow down the required input type.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-430) Narrow down input for FSM mapreduce
job
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-430:
-----------------------------
Attachment: CHUKWA-430.patch
Narrow down the input type from 70 to 4.
> Narrow down input for FSM mapreduce job
> ---------------------------------------
>
> Key: CHUKWA-430
> URL: https://issues.apache.org/jira/browse/CHUKWA-430
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: MR Data Processors
> Affects Versions: 0.4.0
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.4.0
>
> Attachments: CHUKWA-430.patch
>
>
> FSMDataloader supplies all demux output data to FSM state machine. This is not efficient because most of the data type do not contribute to state generation. According to Jiaqi, the state machine requires the following types:
> JobHistoryTaskDataMapper:
> /chukwa/repos/chukwa/JobData
> /chukwa/repos/chukwa/TaskData
> TaskTrackerClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> DataNodeClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> This jira is to optimize the data loader supplied input, and narrow down the required input type.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-430) Narrow down input for FSM mapreduce
job
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12791416#action_12791416 ]
Hudson commented on CHUKWA-430:
-------------------------------
Integrated in Chukwa-trunk #229 (See [http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/229/])
. Narrow down the list of demux output for FSM to improve processing time. (Eric Yang)
> Narrow down input for FSM mapreduce job
> ---------------------------------------
>
> Key: CHUKWA-430
> URL: https://issues.apache.org/jira/browse/CHUKWA-430
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: MR Data Processors
> Affects Versions: 0.4.0
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.4.0
>
> Attachments: CHUKWA-430.patch
>
>
> FSMDataloader supplies all demux output data to FSM state machine. This is not efficient because most of the data type do not contribute to state generation. According to Jiaqi, the state machine requires the following types:
> JobHistoryTaskDataMapper:
> /chukwa/repos/chukwa/JobData
> /chukwa/repos/chukwa/TaskData
> TaskTrackerClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> DataNodeClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> This jira is to optimize the data loader supplied input, and narrow down the required input type.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-430) Narrow down input for FSM mapreduce
job
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-430:
-----------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this, thanks Ari and Jiaqi.
> Narrow down input for FSM mapreduce job
> ---------------------------------------
>
> Key: CHUKWA-430
> URL: https://issues.apache.org/jira/browse/CHUKWA-430
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: MR Data Processors
> Affects Versions: 0.4.0
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.4.0
>
> Attachments: CHUKWA-430.patch
>
>
> FSMDataloader supplies all demux output data to FSM state machine. This is not efficient because most of the data type do not contribute to state generation. According to Jiaqi, the state machine requires the following types:
> JobHistoryTaskDataMapper:
> /chukwa/repos/chukwa/JobData
> /chukwa/repos/chukwa/TaskData
> TaskTrackerClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> DataNodeClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> This jira is to optimize the data loader supplied input, and narrow down the required input type.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-430) Narrow down input for FSM mapreduce
job
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12790427#action_12790427 ]
Ari Rabkin commented on CHUKWA-430:
-----------------------------------
+1 to patch
> Narrow down input for FSM mapreduce job
> ---------------------------------------
>
> Key: CHUKWA-430
> URL: https://issues.apache.org/jira/browse/CHUKWA-430
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: MR Data Processors
> Affects Versions: 0.4.0
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.4.0
>
> Attachments: CHUKWA-430.patch
>
>
> FSMDataloader supplies all demux output data to FSM state machine. This is not efficient because most of the data type do not contribute to state generation. According to Jiaqi, the state machine requires the following types:
> JobHistoryTaskDataMapper:
> /chukwa/repos/chukwa/JobData
> /chukwa/repos/chukwa/TaskData
> TaskTrackerClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> DataNodeClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> This jira is to optimize the data loader supplied input, and narrow down the required input type.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-430) Narrow down input for FSM mapreduce
job
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-430:
-----------------------------
Description:
FSMDataloader supplies all demux output data to FSM state machine. This is not efficient because most of the data type do not contribute to state generation. According to Jiaqi, the state machine requires the following types:
JobHistoryTaskDataMapper:
/chukwa/repos/chukwa/JobData
/chukwa/repos/chukwa/TaskData
TaskTrackerClientTraceMapper:
/chukwa/repos/chukwa/ClientTraceDetailed
DataNodeClientTraceMapper:
/chukwa/repos/chukwa/ClientTraceDetailed
This jira is to optimize the data loader supplied input, and narrow down the required input type.
was:
FSMDataloader supplies all demux output data to FSM state machine. This is not efficient because most of the data type do not contribute to state generation. According to Jiaqi, the state machine requires the following types:
JobHistoryTaskDataMapper:
/chukwa/repos/chukwa/JobData
/chukwa/repos/chukwa/TaskData
TaskTrackerClientTraceMapper:
/chukwa/repos/chukwa/ClientTraceDetailed
DataNodeClientTraceMapper:
/chukwa/repos/chukwa/ClientTraceDetailed
> Narrow down input for FSM mapreduce job
> ---------------------------------------
>
> Key: CHUKWA-430
> URL: https://issues.apache.org/jira/browse/CHUKWA-430
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: MR Data Processors
> Affects Versions: 0.4.0
> Reporter: Eric Yang
> Assignee: Eric Yang
> Fix For: 0.4.0
>
>
> FSMDataloader supplies all demux output data to FSM state machine. This is not efficient because most of the data type do not contribute to state generation. According to Jiaqi, the state machine requires the following types:
> JobHistoryTaskDataMapper:
> /chukwa/repos/chukwa/JobData
> /chukwa/repos/chukwa/TaskData
> TaskTrackerClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> DataNodeClientTraceMapper:
> /chukwa/repos/chukwa/ClientTraceDetailed
> This jira is to optimize the data loader supplied input, and narrow down the required input type.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.