You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Cheng (JIRA)" <ji...@apache.org> on 2009/06/09 00:14:07 UTC
[jira] Created: (CHUKWA-281) Create pig scripts to perform down
sampling
Create pig scripts to perform down sampling
-------------------------------------------
Key: CHUKWA-281
URL: https://issues.apache.org/jira/browse/CHUKWA-281
Project: Hadoop Chukwa
Issue Type: New Feature
Components: Data Processors
Reporter: Cheng
Assignee: Cheng
Use pig scripts instead of MySQL to perform down sampling.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-281) Create pig scripts to perform down
sampling
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-281:
-----------------------------
Status: Patch Available (was: Open)
+1 looks good.
> Create pig scripts to perform down sampling
> -------------------------------------------
>
> Key: CHUKWA-281
> URL: https://issues.apache.org/jira/browse/CHUKWA-281
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: Data Processors
> Reporter: Cheng
> Assignee: Cheng
> Attachments: chukwa-281.2.patch
>
>
> Use pig scripts instead of MySQL to perform down sampling.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-281) Create pig scripts to perform down
sampling
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-281:
-------------------------
Attachment: (was: chukwa-281-1.patch)
> Create pig scripts to perform down sampling
> -------------------------------------------
>
> Key: CHUKWA-281
> URL: https://issues.apache.org/jira/browse/CHUKWA-281
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: Data Processors
> Reporter: Cheng
> Assignee: Cheng
> Attachments: chukwa-281.2.patch
>
>
> Use pig scripts instead of MySQL to perform down sampling.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-281) Create pig scripts to perform down
sampling
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-281:
-----------------------------
Resolution: Fixed
Fix Version/s: 0.2.0
Status: Resolved (was: Patch Available)
I just committed this, thanks Cheng.
> Create pig scripts to perform down sampling
> -------------------------------------------
>
> Key: CHUKWA-281
> URL: https://issues.apache.org/jira/browse/CHUKWA-281
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: Data Processors
> Reporter: Cheng
> Assignee: Cheng
> Fix For: 0.2.0
>
> Attachments: chukwa-281.2.patch
>
>
> Use pig scripts instead of MySQL to perform down sampling.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-281) Create pig scripts to perform down
sampling
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717474#action_12717474 ]
Eric Yang commented on CHUKWA-281:
----------------------------------
This feature is fairly big. Please provide a test case for this feature. Thanks :)
> Create pig scripts to perform down sampling
> -------------------------------------------
>
> Key: CHUKWA-281
> URL: https://issues.apache.org/jira/browse/CHUKWA-281
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: Data Processors
> Reporter: Cheng
> Assignee: Cheng
> Attachments: chukwa-281.patch
>
>
> Use pig scripts instead of MySQL to perform down sampling.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-281) Create pig scripts to perform down
sampling
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-281:
-------------------------
Attachment: (was: chukwa-281.patch)
> Create pig scripts to perform down sampling
> -------------------------------------------
>
> Key: CHUKWA-281
> URL: https://issues.apache.org/jira/browse/CHUKWA-281
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: Data Processors
> Reporter: Cheng
> Assignee: Cheng
> Attachments: chukwa-281-1.patch
>
>
> Use pig scripts instead of MySQL to perform down sampling.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-281) Create pig scripts to perform down
sampling
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-281:
-------------------------
Attachment: chukwa-281-1.patch
New patch created. Pig scripts are now included in the package. Will provide unit test case later.
> Create pig scripts to perform down sampling
> -------------------------------------------
>
> Key: CHUKWA-281
> URL: https://issues.apache.org/jira/browse/CHUKWA-281
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: Data Processors
> Reporter: Cheng
> Assignee: Cheng
> Attachments: chukwa-281-1.patch
>
>
> Use pig scripts instead of MySQL to perform down sampling.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-281) Create pig scripts to perform down
sampling
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-281:
-------------------------
Attachment: chukwa-281.patch
Shell script downSampling.sh could be called from command line or cron job to generate the down sampling results.
command line options:
-c <space seperated cluster names>
optional. if no cluster specified, hadoop dfs command will be used to get all clusters from /chukwa/repos
-d <yyyyMMdd>
optional. if no date is given, yesterday will be used
-h help
-m <space seperated metrics list>
optional. Default value is 'SystemMetrics Hadoop_dfs_namenode Hadoop_dfs_FSDirectory Hadoop_dfs_datanode Hadoop_rpc_metrics Hadoop_mapred_jobtracker Hadoop_jvm_metrics Hadoop_dfs_FSNamesystem Df'
-t time
optional. default value is '5 30 180 720'
-n <add|remove>
Add/remove cron jobs.
optional.
start: add cron entry
remove: remove cron entry
> Create pig scripts to perform down sampling
> -------------------------------------------
>
> Key: CHUKWA-281
> URL: https://issues.apache.org/jira/browse/CHUKWA-281
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: Data Processors
> Reporter: Cheng
> Assignee: Cheng
> Attachments: chukwa-281.patch
>
>
> Use pig scripts instead of MySQL to perform down sampling.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-281) Create pig scripts to perform down
sampling
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12718500#action_12718500 ]
Hudson commented on CHUKWA-281:
-------------------------------
Integrated in Chukwa-trunk #49 (See [http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/49/])
. Created pig scripts to perform down sampling. (Cheng Zhang via Eric Yang)
> Create pig scripts to perform down sampling
> -------------------------------------------
>
> Key: CHUKWA-281
> URL: https://issues.apache.org/jira/browse/CHUKWA-281
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: Data Processors
> Reporter: Cheng
> Assignee: Cheng
> Fix For: 0.2.0
>
> Attachments: chukwa-281.2.patch
>
>
> Use pig scripts instead of MySQL to perform down sampling.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-281) Create pig scripts to perform down
sampling
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717952#action_12717952 ]
Eric Yang commented on CHUKWA-281:
----------------------------------
Looks like the patch contains non-related code like TaskLogAppender, and ChukwaJobInstrumentation.java. Please make sure only related code are in the patch.
> Create pig scripts to perform down sampling
> -------------------------------------------
>
> Key: CHUKWA-281
> URL: https://issues.apache.org/jira/browse/CHUKWA-281
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: Data Processors
> Reporter: Cheng
> Assignee: Cheng
> Attachments: chukwa-281-1.patch
>
>
> Use pig scripts instead of MySQL to perform down sampling.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-281) Create pig scripts to perform down
sampling
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717513#action_12717513 ]
Eric Yang commented on CHUKWA-281:
----------------------------------
Pig scripts are not bundled as part of the tarball. Please update the build procedure.
> Create pig scripts to perform down sampling
> -------------------------------------------
>
> Key: CHUKWA-281
> URL: https://issues.apache.org/jira/browse/CHUKWA-281
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: Data Processors
> Reporter: Cheng
> Assignee: Cheng
> Attachments: chukwa-281.patch
>
>
> Use pig scripts instead of MySQL to perform down sampling.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-281) Create pig scripts to perform down
sampling
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-281:
-------------------------
Attachment: chukwa-281.2.patch
Regenerated the patch.
> Create pig scripts to perform down sampling
> -------------------------------------------
>
> Key: CHUKWA-281
> URL: https://issues.apache.org/jira/browse/CHUKWA-281
> Project: Hadoop Chukwa
> Issue Type: New Feature
> Components: Data Processors
> Reporter: Cheng
> Assignee: Cheng
> Attachments: chukwa-281.2.patch
>
>
> Use pig scripts instead of MySQL to perform down sampling.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.