You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Cheng (JIRA)" <ji...@apache.org> on 2009/04/28 01:57:30 UTC
[jira] Created: (CHUKWA-193) Handling Meta in job log file
Handling Meta in job log file
------------------------------
Key: CHUKWA-193
URL: https://issues.apache.org/jira/browse/CHUKWA-193
Project: Hadoop Chukwa
Issue Type: Bug
Reporter: Cheng
Priority: Minor
Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-193) Handling Meta in job log file
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-193:
-------------------------
Attachment: (was: chukwa-193_1.patch)
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193.2.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-193) Handling Meta in job log file
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Yang updated CHUKWA-193:
-----------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I just committed this, thanks Cheng.
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193_3.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-193) Handling Meta in job log file
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12704339#action_12704339 ]
Hudson commented on CHUKWA-193:
-------------------------------
Integrated in Chukwa-trunk #12 (See [http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/12/])
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193_3.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-193) Handling Meta in job log file
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716990#action_12716990 ]
Hudson commented on CHUKWA-193:
-------------------------------
Integrated in Chukwa-trunk #45 (See [http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/45/])
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193_3.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-193) Handling Meta in job log file
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-193:
-------------------------
Attachment: chukwa-193.patch
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-193) Handling Meta in job log file
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703876#action_12703876 ]
Eric Yang commented on CHUKWA-193:
----------------------------------
+1 on discard unrecognized tag, and use time partition in the archive key.
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193_3.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-193) Handling Meta in job log file
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703454#action_12703454 ]
Cheng commented on CHUKWA-193:
------------------------------
We can get jobid from file name which is in the meta data of the chunk,
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-193) Handling Meta in job log file
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-193:
-------------------------
Attachment: chukwa-193.2.patch
new patch.
* if no time stamp, use the time partition in the archiveKey
* use pre-compiled patten
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193.2.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-193) Handling Meta in job log file
Posted by "Terence Kwan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703712#action_12703712 ]
Terence Kwan commented on CHUKWA-193:
-------------------------------------
from the syntax point of view, it is looking good. +1
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193_1.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-193) Handling Meta in job log file
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-193:
-------------------------
Status: Patch Available (was: Open)
Submitted patch. Changes:
1. If line starts with "Meta", empty the savedLine for multiple lines exception handling
2. If log type is "Meta", discard it
3. Set time stamp to current if it's 0
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-193) Handling Meta in job log file
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703692#action_12703692 ]
Jerome Boulon commented on CHUKWA-193:
--------------------------------------
Cheng,
When the date is not available could you use the timePartition set at the Chunk level.
You can get it using: archiveKey.getTimePartition();
Could you also compile your pattern once and reused it.
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193_1.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-193) Handling Meta in job log file
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-193:
-------------------------
Attachment: (was: chukwa-193.2.patch)
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193_3.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-193) Handling Meta in job log file
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-193:
-------------------------
Attachment: chukwa-193_1.patch
parse jobid from the stream name of chunk
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193_1.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-193) Handling Meta in job log file
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-193:
-------------------------
Attachment: chukwa-193_3.patch
ignore unrecognized tag.
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193_3.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-193) Handling Meta in job log file
Posted by "Cheng (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheng updated CHUKWA-193:
-------------------------
Attachment: (was: chukwa-193.patch)
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193_1.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-193) Handling Meta in job log file
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703720#action_12703720 ]
Jerome Boulon commented on CHUKWA-193:
--------------------------------------
-1 see my comments
> Handling Meta in job log file
> ------------------------------
>
> Key: CHUKWA-193
> URL: https://issues.apache.org/jira/browse/CHUKWA-193
> Project: Hadoop Chukwa
> Issue Type: Bug
> Reporter: Cheng
> Priority: Minor
> Attachments: chukwa-193_1.patch
>
>
> Since Meta has no timestamp and jobid, currently all metas have the same key /0/null/0 and they are put into the same dir.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.