You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Eric Yang (JIRA)" <ji...@apache.org> on 2009/01/15 19:22:59 UTC
[jira] Created: (HADOOP-5054) Chukwa database partitions does not
get created automatically
Chukwa database partitions does not get created automatically
-------------------------------------------------------------
Key: HADOOP-5054
URL: https://issues.apache.org/jira/browse/HADOOP-5054
Project: Hadoop Core
Issue Type: Bug
Components: contrib/chukwa
Environment: Redhat EL 5.1, Java 6
Reporter: Eric Yang
dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5054) Chukwa database partitions does not
get created automatically
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-5054:
------------------------------
Status: Open (was: Patch Available)
Automatic partition expiration is also not working correctly.
> Chukwa database partitions does not get created automatically
> -------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5054) Chukwa database partitions does not
create/expire automatically
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-5054:
------------------------------
Attachment: HADOOP-5054.patch
- Fixed shell script with date generation.
- Fixed data expiration for partition smaller than positive number.
- Added debug statement for Table Creation.
> Chukwa database partitions does not create/expire automatically
> ---------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
> The database expiration script need to handle the case where partition number is invalid.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5054) Chukwa database partitions does not
create/expire automatically
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-5054:
------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this. Thanks guys.
> Chukwa database partitions does not create/expire automatically
> ---------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
> The database expiration script need to handle the case where partition number is invalid.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5054) Chukwa database partitions does not
create/expire automatically
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677481#action_12677481 ]
Ari Rabkin commented on HADOOP-5054:
------------------------------------
Okiedoke, I'm convinced.
> Chukwa database partitions does not create/expire automatically
> ---------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
> The database expiration script need to handle the case where partition number is invalid.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5054) Chukwa database partitions does not
create/expire automatically
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-5054:
------------------------------
Status: Patch Available (was: Open)
> Chukwa database partitions does not create/expire automatically
> ---------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
> The database expiration script need to handle the case where partition number is invalid.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5054) Chukwa database partitions does not
create/expire automatically
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672087#action_12672087 ]
Ari Rabkin commented on HADOOP-5054:
------------------------------------
Looks okay. I'm a bit nervous about the lack of tests, though I understand that it's hard to unit-test this sort of thing. Has it been tried out in production?
> Chukwa database partitions does not create/expire automatically
> ---------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
> The database expiration script need to handle the case where partition number is invalid.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5054) Chukwa database partitions does not
get created automatically
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-5054:
------------------------------
Release Note:
What is new in HADOOP-5054:
- Improved database partitioning by date.
Hadoop Flags: [Reviewed]
Status: Patch Available (was: Open)
> Chukwa database partitions does not get created automatically
> -------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5054) Chukwa database partitions does not
create/expire automatically
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677474#action_12677474 ]
Jerome Boulon commented on HADOOP-5054:
---------------------------------------
I put a +1 even if the number of partition is hardcoded since it fixes some critical bugs.
Adding the number of partitions to the config requires to revalidate the current logic so it's better to open a Jira request just for it if needed.
> Chukwa database partitions does not create/expire automatically
> ---------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
> The database expiration script need to handle the case where partition number is invalid.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5054) Chukwa database partitions does not
create/expire automatically
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12676852#action_12676852 ]
Ari Rabkin commented on HADOOP-5054:
------------------------------------
It looks like this patch hardcodes the number of partitions to keep. Can we push this into config? Should we?
> Chukwa database partitions does not create/expire automatically
> ---------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
> The database expiration script need to handle the case where partition number is invalid.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5054) Chukwa database partitions does not
create/expire automatically
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-5054:
------------------------------
Description:
dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
The database expiration script need to handle the case where partition number is invalid.
was:
dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
Summary: Chukwa database partitions does not create/expire automatically (was: Chukwa database partitions does not get created automatically)
> Chukwa database partitions does not create/expire automatically
> ---------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
> The database expiration script need to handle the case where partition number is invalid.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5054) Chukwa database partitions does not
create/expire automatically
Posted by "Mac Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672109#action_12672109 ]
Mac Yang commented on HADOOP-5054:
----------------------------------
Yes, have been running this in our deployment.
> Chukwa database partitions does not create/expire automatically
> ---------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
> The database expiration script need to handle the case where partition number is invalid.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5054) Chukwa database partitions does not
create/expire automatically
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-5054:
------------------------------
Attachment: (was: HADOOP-5054.patch)
> Chukwa database partitions does not create/expire automatically
> ---------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
> The database expiration script need to handle the case where partition number is invalid.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5054) Chukwa database partitions does not
create/expire automatically
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12677449#action_12677449 ]
Eric Yang commented on HADOOP-5054:
-----------------------------------
The number of partitions to keep is hard coded for now. If more people show interest that "the number of partitions to keep" should be configurable, then we move this to config file.
> Chukwa database partitions does not create/expire automatically
> ---------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
> The database expiration script need to handle the case where partition number is invalid.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-5054) Chukwa database partitions does not
create/expire automatically
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12681772#action_12681772 ]
Hudson commented on HADOOP-5054:
--------------------------------
Integrated in Hadoop-trunk #778 (See [http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/778/])
> Chukwa database partitions does not create/expire automatically
> ---------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
> The database expiration script need to handle the case where partition number is invalid.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-5054) Chukwa database partitions does not
get created automatically
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang reassigned HADOOP-5054:
---------------------------------
Assignee: Eric Yang
> Chukwa database partitions does not get created automatically
> -------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-5054) Chukwa database partitions does not
get created automatically
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-5054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated HADOOP-5054:
------------------------------
Attachment: HADOOP-5054.patch
Moved date generation into the shell script while loop.
> Chukwa database partitions does not get created automatically
> -------------------------------------------------------------
>
> Key: HADOOP-5054
> URL: https://issues.apache.org/jira/browse/HADOOP-5054
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/chukwa
> Environment: Redhat EL 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Eric Yang
> Attachments: HADOOP-5054.patch
>
>
> dbAdmin.sh should create database partitions base on time, but this is not working because the timestamp variable is not inside the loop for creating database partitions. When the script is running, it would create partitions corresponding to the execution time once only.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.