You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@chukwa.apache.org by "Ari Rabkin (JIRA)" <ji...@apache.org> on 2009/05/14 08:06:45 UTC

[jira] Created: (CHUKWA-238) race condition with archive manager

race condition with archive manager
-----------------------------------

                 Key: CHUKWA-238
                 URL: https://issues.apache.org/jira/browse/CHUKWA-238
             Project: Hadoop Chukwa
          Issue Type: Bug
          Components: Data Processors
            Reporter: Ari Rabkin
            Priority: Blocker


If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop:
Symtom:
a log file full of statements like:
2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (CHUKWA-238) race condition with archive manager

Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ari Rabkin updated CHUKWA-238:
------------------------------

    Attachment: chukwa-238.patch

Revised as per Jerome's comment; except that the info() log has been changed to debug().

> race condition with archive manager
> -----------------------------------
>
>                 Key: CHUKWA-238
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-238
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: Data Processors
>            Reporter: Ari Rabkin
>            Priority: Blocker
>         Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (CHUKWA-238) race condition with archive manager

Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ari Rabkin updated CHUKWA-238:
------------------------------

    Status: Patch Available  (was: Open)

> race condition with archive manager
> -----------------------------------
>
>                 Key: CHUKWA-238
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-238
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: Data Processors
>            Reporter: Ari Rabkin
>            Priority: Blocker
>         Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop:
> Symtom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-238) race condition with archive manager

Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710493#action_12710493 ] 

Jerome Boulon commented on CHUKWA-238:
--------------------------------------

+1 looks good

> race condition with archive manager
> -----------------------------------
>
>                 Key: CHUKWA-238
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-238
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: Data Processors
>            Reporter: Ari Rabkin
>            Priority: Blocker
>         Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-238) race condition with archive manager

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710780#action_12710780 ] 

Hudson commented on CHUKWA-238:
-------------------------------

Integrated in Chukwa-trunk #28 (See [http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/28/])
    . Resolve race condition in archiving.


> race condition with archive manager
> -----------------------------------
>
>                 Key: CHUKWA-238
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-238
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: Data Processors
>            Reporter: Ari Rabkin
>            Assignee: Ari Rabkin
>            Priority: Blocker
>             Fix For: 0.1.2, 0.2.0
>
>         Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (CHUKWA-238) race condition with archive manager

Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ari Rabkin updated CHUKWA-238:
------------------------------

    Description: 
If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
Symptom:
a log file full of statements like:
2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives


  was:
If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop:
Symtom:
a log file full of statements like:
2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives



> race condition with archive manager
> -----------------------------------
>
>                 Key: CHUKWA-238
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-238
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: Data Processors
>            Reporter: Ari Rabkin
>            Priority: Blocker
>         Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (CHUKWA-238) race condition with archive manager

Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ari Rabkin updated CHUKWA-238:
------------------------------

    Attachment: chukwa-238.patch

> race condition with archive manager
> -----------------------------------
>
>                 Key: CHUKWA-238
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-238
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: Data Processors
>            Reporter: Ari Rabkin
>            Priority: Blocker
>         Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop:
> Symtom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Assigned: (CHUKWA-238) race condition with archive manager

Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ari Rabkin reassigned CHUKWA-238:
---------------------------------

    Assignee: Ari Rabkin

> race condition with archive manager
> -----------------------------------
>
>                 Key: CHUKWA-238
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-238
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: Data Processors
>            Reporter: Ari Rabkin
>            Assignee: Ari Rabkin
>            Priority: Blocker
>             Fix For: 0.1.2, 0.2.0
>
>         Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-238) race condition with archive manager

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12716971#action_12716971 ] 

Hudson commented on CHUKWA-238:
-------------------------------

Integrated in Chukwa-trunk #45 (See [http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/45/])
    

> race condition with archive manager
> -----------------------------------
>
>                 Key: CHUKWA-238
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-238
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: Data Processors
>            Reporter: Ari Rabkin
>            Assignee: Ari Rabkin
>            Priority: Blocker
>             Fix For: 0.1.2, 0.2.0
>
>         Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (CHUKWA-238) race condition with archive manager

Posted by "Jerome Boulon (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709978#action_12709978 ] 

Jerome Boulon commented on CHUKWA-238:
--------------------------------------

Ari,
We cannot do that since the code checks for daysInRawArchiveDir[0] and also we don't want to sleep for 30  minutes 
We need to add a specific block right after FileStatus[] daysInRawArchiveDir = fs.listStatus(pDailyRawArchivesInput);

if (daysInRawArchiveDir.length == 0 )  {
 log.info( pDailyRawArchivesInput + " is empty, going to sleep for 1 minute");
 Thread.sleep(1 * 60 * 1000);
  continue;
}



> race condition with archive manager
> -----------------------------------
>
>                 Key: CHUKWA-238
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-238
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: Data Processors
>            Reporter: Ari Rabkin
>            Priority: Blocker
>         Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (CHUKWA-238) race condition with archive manager

Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ari Rabkin updated CHUKWA-238:
------------------------------

       Resolution: Fixed
    Fix Version/s: 0.2.0
                   0.1.2
           Status: Resolved  (was: Patch Available)

I just committed this.

> race condition with archive manager
> -----------------------------------
>
>                 Key: CHUKWA-238
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-238
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: Data Processors
>            Reporter: Ari Rabkin
>            Priority: Blocker
>             Fix For: 0.1.2, 0.2.0
>
>         Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (CHUKWA-238) race condition with archive manager

Posted by "Ari Rabkin (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/CHUKWA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ari Rabkin updated CHUKWA-238:
------------------------------

    Attachment:     (was: chukwa-238.patch)

> race condition with archive manager
> -----------------------------------
>
>                 Key: CHUKWA-238
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-238
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: Data Processors
>            Reporter: Ari Rabkin
>            Priority: Blocker
>         Attachments: chukwa-238.patch
>
>
> If the archive manager starts before any data has been loaded into the repository, it'll buzz through its main loop, filling logs files with garbage at maximum speed:
> Symptom:
> a log file full of statements like:
> 2009-05-13 22:59:57,233 INFO main ChukwaArchiveManager - Raw Archive dir:/chukwa/dataSinkArchives

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.