You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Ari Rabkin (JIRA)" <ji...@apache.org> on 2009/05/14 08:06:45 UTC
[jira] Created: (CHUKWA-238) race condition with archive manager
race condition with archive manager
-----------------------------------
Key: CHUKWA-238
URL: https://issues.apache.org/jira/browse/CHUKWA-238
Project: Hadoop Chukwa
Issue Type: Bug
Components: Data Processors
Reporter: Ari Rabkin
Priority: Blocker
If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop:
Symtom:
a log file full of statements like:
2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-238) race condition with archive manager
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-238:
------------------------------
Attachment: chukwa-238.patch
Revised as per Jerome's comment; except that the info() log has been changed to debug().
> race condition with archive manager
> -----------------------------------
>
> Key: CHUKWA-238
> URL: https://issues.apache.org/jira/browse/CHUKWA-238
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Reporter: Ari Rabkin
> Priority: Blocker
> Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-238) race condition with archive manager
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-238:
------------------------------
Status: Patch Available (was: Open)
> race condition with archive manager
> -----------------------------------
>
> Key: CHUKWA-238
> URL: https://issues.apache.org/jira/browse/CHUKWA-238
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Reporter: Ari Rabkin
> Priority: Blocker
> Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop:
> Symtom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-238) race condition with archive manager
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710493#action_12710493 ]
Jerome Boulon commented on CHUKWA-238:
--------------------------------------
+1 looks good
> race condition with archive manager
> -----------------------------------
>
> Key: CHUKWA-238
> URL: https://issues.apache.org/jira/browse/CHUKWA-238
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Reporter: Ari Rabkin
> Priority: Blocker
> Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-238) race condition with archive manager
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710780#action_12710780 ]
Hudson commented on CHUKWA-238:
-------------------------------
Integrated in Chukwa-trunk #28 (See [http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/28/])
. Resolve race condition in archiving.
> race condition with archive manager
> -----------------------------------
>
> Key: CHUKWA-238
> URL: https://issues.apache.org/jira/browse/CHUKWA-238
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Priority: Blocker
> Fix For: 0.1.2, 0.2.0
>
> Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-238) race condition with archive manager
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-238:
------------------------------
Description:
If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
Symptom:
a log file full of statements like:
2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
was:
If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop:
Symtom:
a log file full of statements like:
2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
> race condition with archive manager
> -----------------------------------
>
> Key: CHUKWA-238
> URL: https://issues.apache.org/jira/browse/CHUKWA-238
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Reporter: Ari Rabkin
> Priority: Blocker
> Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-238) race condition with archive manager
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-238:
------------------------------
Attachment: chukwa-238.patch
> race condition with archive manager
> -----------------------------------
>
> Key: CHUKWA-238
> URL: https://issues.apache.org/jira/browse/CHUKWA-238
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Reporter: Ari Rabkin
> Priority: Blocker
> Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop:
> Symtom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CHUKWA-238) race condition with archive manager
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin reassigned CHUKWA-238:
---------------------------------
Assignee: Ari Rabkin
> race condition with archive manager
> -----------------------------------
>
> Key: CHUKWA-238
> URL: https://issues.apache.org/jira/browse/CHUKWA-238
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Priority: Blocker
> Fix For: 0.1.2, 0.2.0
>
> Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-238) race condition with archive manager
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716971#action_12716971 ]
Hudson commented on CHUKWA-238:
-------------------------------
Integrated in Chukwa-trunk #45 (See [http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/45/])
> race condition with archive manager
> -----------------------------------
>
> Key: CHUKWA-238
> URL: https://issues.apache.org/jira/browse/CHUKWA-238
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Reporter: Ari Rabkin
> Assignee: Ari Rabkin
> Priority: Blocker
> Fix For: 0.1.2, 0.2.0
>
> Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CHUKWA-238) race condition with archive manager
Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709978#action_12709978 ]
Jerome Boulon commented on CHUKWA-238:
--------------------------------------
Ari,
We cannot do that since the code checks for daysInRawArchiveDir[0] and also we don't want to sleep for 30 minutes
We need to add a specific block right after FileStatus[] daysInRawArchiveDir = fs.listStatus(pDailyRawArchivesInput);
if (daysInRawArchiveDir.length == 0 ) {
log.info( pDailyRawArchivesInput + " is empty, going to sleep for 1 minute");
Thread.sleep(1 * 60 * 1000);
continue;
}
> race condition with archive manager
> -----------------------------------
>
> Key: CHUKWA-238
> URL: https://issues.apache.org/jira/browse/CHUKWA-238
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Reporter: Ari Rabkin
> Priority: Blocker
> Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-238) race condition with archive manager
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-238:
------------------------------
Resolution: Fixed
Fix Version/s: 0.2.0
0.1.2
Status: Resolved (was: Patch Available)
I just committed this.
> race condition with archive manager
> -----------------------------------
>
> Key: CHUKWA-238
> URL: https://issues.apache.org/jira/browse/CHUKWA-238
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Reporter: Ari Rabkin
> Priority: Blocker
> Fix For: 0.1.2, 0.2.0
>
> Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CHUKWA-238) race condition with archive manager
Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ari Rabkin updated CHUKWA-238:
------------------------------
Attachment: (was: chukwa-238.patch)
> race condition with archive manager
> -----------------------------------
>
> Key: CHUKWA-238
> URL: https://issues.apache.org/jira/browse/CHUKWA-238
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: Data Processors
> Reporter: Ari Rabkin
> Priority: Blocker
> Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.