You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org> on 2012/05/03 14:54:51 UTC

[jira] [Created] (BOOKKEEPER-234) EntryLogger will throw NPE, if dirs does not exist or IO Errors.

Uma Maheswara Rao G created BOOKKEEPER-234:
----------------------------------------------

             Summary: EntryLogger will throw NPE, if dirs does not exist or IO Errors.
                 Key: BOOKKEEPER-234
                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-234
             Project: Bookkeeper
          Issue Type: Bug
          Components: bookkeeper-server
    Affects Versions: 4.1.0
            Reporter: Uma Maheswara Rao G
            Assignee: Uma Maheswara Rao G


I think, Entry log should check the exitance of directories, before going to get the last Log entries. Because, listFiles will retunr null on non existant directories or any IO Errors. We have to add the check for directory existnace check and null checks in side getLastLogID api in EntryLogger.

We may need to handle in LedgerCacheImpl also.

Do we need to add them in bad disks list? and others will refer this list before they use the dirs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (BOOKKEEPER-234) EntryLogger will throw NPE, if any dir does not exist or IO Errors.

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/BOOKKEEPER-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271218#comment-13271218 ] 

Ivan Kelly commented on BOOKKEEPER-234:
---------------------------------------

Committed as r1335996. Thanks Uma.
                
> EntryLogger will throw NPE, if any dir does not exist or IO Errors.
> -------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-234
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-234
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: bookkeeper-server
>    Affects Versions: 4.1.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>         Attachments: BOOKKEEPER-234.patch, BOOKKEEPER-234.patch
>
>
> I think, Entry log should check the exitance of directories, before going to get the last Log entries. Because, listFiles will retunr null on non existant directories or any IO Errors. We have to add the check for directory existnace check and null checks in side getLastLogID api in EntryLogger.
> We may need to handle in LedgerCacheImpl also.
> Do we need to add them in bad disks list? and others will refer this list before they use the dirs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (BOOKKEEPER-234) EntryLogger will throw NPE, if any dir does not exist or IO Errors.

Posted by "Sijie Guo (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/BOOKKEEPER-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270294#comment-13270294 ] 

Sijie Guo commented on BOOKKEEPER-234:
--------------------------------------

lgtm +1
                
> EntryLogger will throw NPE, if any dir does not exist or IO Errors.
> -------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-234
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-234
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: bookkeeper-server
>    Affects Versions: 4.1.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>         Attachments: BOOKKEEPER-234.patch
>
>
> I think, Entry log should check the exitance of directories, before going to get the last Log entries. Because, listFiles will retunr null on non existant directories or any IO Errors. We have to add the check for directory existnace check and null checks in side getLastLogID api in EntryLogger.
> We may need to handle in LedgerCacheImpl also.
> Do we need to add them in bad disks list? and others will refer this list before they use the dirs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (BOOKKEEPER-234) EntryLogger will throw NPE, if any dir does not exist or IO Errors.

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/BOOKKEEPER-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13269938#comment-13269938 ] 

Uma Maheswara Rao G commented on BOOKKEEPER-234:
------------------------------------------------

Attached the patch, which addresses the issue in EntryLogger.
                
> EntryLogger will throw NPE, if any dir does not exist or IO Errors.
> -------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-234
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-234
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: bookkeeper-server
>    Affects Versions: 4.1.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>         Attachments: BOOKKEEPER-234.patch
>
>
> I think, Entry log should check the exitance of directories, before going to get the last Log entries. Because, listFiles will retunr null on non existant directories or any IO Errors. We have to add the check for directory existnace check and null checks in side getLastLogID api in EntryLogger.
> We may need to handle in LedgerCacheImpl also.
> Do we need to add them in bad disks list? and others will refer this list before they use the dirs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (BOOKKEEPER-234) EntryLogger will throw NPE, if any dir does not exist or IO Errors.

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/BOOKKEEPER-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270529#comment-13270529 ] 

Uma Maheswara Rao G commented on BOOKKEEPER-234:
------------------------------------------------

Yep, Ivan, I have seen it in checkEnvironment call. If directory does not exist then it is creating it.

Yes, In my  testing environment it happened on the the startup of BK, unfortunately dirs have been cleaned(through some test scripts) . I am assuming that, after this step(checkEnvironment) only that might have happend. So, I directly verified that EntryLogger code and found that peice for NPE possibility. Do we need to throw the exception if we meet that situation, as we may not have Version files and all right? Now I can see this is very very rare case. Do we need to handle it?
                
> EntryLogger will throw NPE, if any dir does not exist or IO Errors.
> -------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-234
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-234
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: bookkeeper-server
>    Affects Versions: 4.1.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>         Attachments: BOOKKEEPER-234.patch
>
>
> I think, Entry log should check the exitance of directories, before going to get the last Log entries. Because, listFiles will retunr null on non existant directories or any IO Errors. We have to add the check for directory existnace check and null checks in side getLastLogID api in EntryLogger.
> We may need to handle in LedgerCacheImpl also.
> Do we need to add them in bad disks list? and others will refer this list before they use the dirs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (BOOKKEEPER-234) EntryLogger will throw NPE, if any dir does not exist or IO Errors.

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/BOOKKEEPER-234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uma Maheswara Rao G updated BOOKKEEPER-234:
-------------------------------------------

    Summary: EntryLogger will throw NPE, if any dir does not exist or IO Errors.  (was: EntryLogger will throw NPE, if dirs does not exist or IO Errors.)
    
> EntryLogger will throw NPE, if any dir does not exist or IO Errors.
> -------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-234
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-234
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: bookkeeper-server
>    Affects Versions: 4.1.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>
> I think, Entry log should check the exitance of directories, before going to get the last Log entries. Because, listFiles will retunr null on non existant directories or any IO Errors. We have to add the check for directory existnace check and null checks in side getLastLogID api in EntryLogger.
> We may need to handle in LedgerCacheImpl also.
> Do we need to add them in bad disks list? and others will refer this list before they use the dirs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (BOOKKEEPER-234) EntryLogger will throw NPE, if any dir does not exist or IO Errors.

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/BOOKKEEPER-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270483#comment-13270483 ] 

Ivan Kelly commented on BOOKKEEPER-234:
---------------------------------------

Where are you seeing the error occurring? There's a check when Bookie is created already to ensure that the directories exist.
                
> EntryLogger will throw NPE, if any dir does not exist or IO Errors.
> -------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-234
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-234
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: bookkeeper-server
>    Affects Versions: 4.1.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>         Attachments: BOOKKEEPER-234.patch
>
>
> I think, Entry log should check the exitance of directories, before going to get the last Log entries. Because, listFiles will retunr null on non existant directories or any IO Errors. We have to add the check for directory existnace check and null checks in side getLastLogID api in EntryLogger.
> We may need to handle in LedgerCacheImpl also.
> Do we need to add them in bad disks list? and others will refer this list before they use the dirs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (BOOKKEEPER-234) EntryLogger will throw NPE, if any dir does not exist or IO Errors.

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/BOOKKEEPER-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270556#comment-13270556 ] 

Uma Maheswara Rao G commented on BOOKKEEPER-234:
------------------------------------------------

true...Let me update the patch with this change in some time.
                
> EntryLogger will throw NPE, if any dir does not exist or IO Errors.
> -------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-234
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-234
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: bookkeeper-server
>    Affects Versions: 4.1.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>         Attachments: BOOKKEEPER-234.patch
>
>
> I think, Entry log should check the exitance of directories, before going to get the last Log entries. Because, listFiles will retunr null on non existant directories or any IO Errors. We have to add the check for directory existnace check and null checks in side getLastLogID api in EntryLogger.
> We may need to handle in LedgerCacheImpl also.
> Do we need to add them in bad disks list? and others will refer this list before they use the dirs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (BOOKKEEPER-234) EntryLogger will throw NPE, if any dir does not exist or IO Errors.

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/BOOKKEEPER-234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uma Maheswara Rao G updated BOOKKEEPER-234:
-------------------------------------------

    Attachment: BOOKKEEPER-234.patch

Attached the patch as discussed.
                
> EntryLogger will throw NPE, if any dir does not exist or IO Errors.
> -------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-234
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-234
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: bookkeeper-server
>    Affects Versions: 4.1.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>         Attachments: BOOKKEEPER-234.patch, BOOKKEEPER-234.patch
>
>
> I think, Entry log should check the exitance of directories, before going to get the last Log entries. Because, listFiles will retunr null on non existant directories or any IO Errors. We have to add the check for directory existnace check and null checks in side getLastLogID api in EntryLogger.
> We may need to handle in LedgerCacheImpl also.
> Do we need to add them in bad disks list? and others will refer this list before they use the dirs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (BOOKKEEPER-234) EntryLogger will throw NPE, if any dir does not exist or IO Errors.

Posted by "Ivan Kelly (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/BOOKKEEPER-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270536#comment-13270536 ] 

Ivan Kelly commented on BOOKKEEPER-234:
---------------------------------------

I think this is a very exceptional scenario (where the directory is deleted, after checkEnvironment and while the bookie is running). We shouldn't allow the bookie to continue as normal if an admin is doing this. That said NPEs are ugly. It would be better if the entry logger checked for the existence of the directory, and if it doesn't exist, throw a FileNotFoundException. 
                
> EntryLogger will throw NPE, if any dir does not exist or IO Errors.
> -------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-234
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-234
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: bookkeeper-server
>    Affects Versions: 4.1.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>         Attachments: BOOKKEEPER-234.patch
>
>
> I think, Entry log should check the exitance of directories, before going to get the last Log entries. Because, listFiles will retunr null on non existant directories or any IO Errors. We have to add the check for directory existnace check and null checks in side getLastLogID api in EntryLogger.
> We may need to handle in LedgerCacheImpl also.
> Do we need to add them in bad disks list? and others will refer this list before they use the dirs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Updated] (BOOKKEEPER-234) EntryLogger will throw NPE, if any dir does not exist or IO Errors.

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/BOOKKEEPER-234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uma Maheswara Rao G updated BOOKKEEPER-234:
-------------------------------------------

    Attachment: BOOKKEEPER-234.patch
    
> EntryLogger will throw NPE, if any dir does not exist or IO Errors.
> -------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-234
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-234
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: bookkeeper-server
>    Affects Versions: 4.1.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>         Attachments: BOOKKEEPER-234.patch
>
>
> I think, Entry log should check the exitance of directories, before going to get the last Log entries. Because, listFiles will retunr null on non existant directories or any IO Errors. We have to add the check for directory existnace check and null checks in side getLastLogID api in EntryLogger.
> We may need to handle in LedgerCacheImpl also.
> Do we need to add them in bad disks list? and others will refer this list before they use the dirs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (BOOKKEEPER-234) EntryLogger will throw NPE, if any dir does not exist or IO Errors.

Posted by "Hudson (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/BOOKKEEPER-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271222#comment-13271222 ] 

Hudson commented on BOOKKEEPER-234:
-----------------------------------

Integrated in bookkeeper-trunk #498 (See [https://builds.apache.org/job/bookkeeper-trunk/498/])
    BOOKKEEPER-234: EntryLogger will throw NPE, if any dir does not exist or IO Errors. (umamaheswararao via ivank) (Revision 1335996)

     Result = UNSTABLE
ivank : 
Files : 
* /zookeeper/bookkeeper/trunk/CHANGES.txt
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java
* /zookeeper/bookkeeper/trunk/bookkeeper-server/src/test/java/org/apache/bookkeeper/bookie/EntryLogTest.java

                
> EntryLogger will throw NPE, if any dir does not exist or IO Errors.
> -------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-234
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-234
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: bookkeeper-server
>    Affects Versions: 4.1.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>             Fix For: 4.1.0
>
>         Attachments: BOOKKEEPER-234.patch, BOOKKEEPER-234.patch
>
>
> I think, Entry log should check the exitance of directories, before going to get the last Log entries. Because, listFiles will retunr null on non existant directories or any IO Errors. We have to add the check for directory existnace check and null checks in side getLastLogID api in EntryLogger.
> We may need to handle in LedgerCacheImpl also.
> Do we need to add them in bad disks list? and others will refer this list before they use the dirs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

[jira] [Commented] (BOOKKEEPER-234) EntryLogger will throw NPE, if any dir does not exist or IO Errors.

Posted by "Uma Maheswara Rao G (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/BOOKKEEPER-234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13270320#comment-13270320 ] 

Uma Maheswara Rao G commented on BOOKKEEPER-234:
------------------------------------------------

Thanks a lot, Sijie for the review!
                
> EntryLogger will throw NPE, if any dir does not exist or IO Errors.
> -------------------------------------------------------------------
>
>                 Key: BOOKKEEPER-234
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-234
>             Project: Bookkeeper
>          Issue Type: Bug
>          Components: bookkeeper-server
>    Affects Versions: 4.1.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>         Attachments: BOOKKEEPER-234.patch
>
>
> I think, Entry log should check the exitance of directories, before going to get the last Log entries. Because, listFiles will retunr null on non existant directories or any IO Errors. We have to add the check for directory existnace check and null checks in side getLastLogID api in EntryLogger.
> We may need to handle in LedgerCacheImpl also.
> Do we need to add them in bad disks list? and others will refer this list before they use the dirs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira