You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Sijie Guo (Created) (JIRA)" <ji...@apache.org> on 2012/01/16 08:05:40 UTC
[jira] [Created] (BOOKKEEPER-153) Ledger can't be opened or closed
due to zero-length metadata
Ledger can't be opened or closed due to zero-length metadata
------------------------------------------------------------
Key: BOOKKEEPER-153
URL: https://issues.apache.org/jira/browse/BOOKKEEPER-153
Project: Bookkeeper
Issue Type: Bug
Components: bookkeeper-client
Affects Versions: 4.0.0
Reporter: Sijie Guo
Assignee: Sijie Guo
Fix For: 4.1.0
Currently creating ledger path and writing ledger metadata are not in a transaction. so if the bookkeeper client (hub server uses bookkeeper client) is crashed, we have a ledger existed in zookeeper with zero-length metadata. we can't open/close it.
we should create the ledger path with initial metadata to avoid such case. besides that, we need to add code in openLedgerOp to handle zero-length metadata for backward compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-153) Ledger can't be opened or closed
due to zero-length metadata
Posted by "Sijie Guo (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-153:
---------------------------------
Attachment: BK-153.patch_v2
remove codes change in LedgerOpenOp
> Ledger can't be opened or closed due to zero-length metadata
> ------------------------------------------------------------
>
> Key: BOOKKEEPER-153
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-153
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BK-153.patch, BK-153.patch_v2
>
>
> Currently creating ledger path and writing ledger metadata are not in a transaction. so if the bookkeeper client (hub server uses bookkeeper client) is crashed, we have a ledger existed in zookeeper with zero-length metadata. we can't open/close it.
> we should create the ledger path with initial metadata to avoid such case. besides that, we need to add code in openLedgerOp to handle zero-length metadata for backward compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-153) Ledger can't be opened or
closed due to zero-length metadata
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187686#comment-13187686 ]
Sijie Guo commented on BOOKKEEPER-153:
--------------------------------------
yes parseLedgerConfig did throw an IOException.
since the bug is found in hub server, topic can't be acquired due to hub server can't closing such kind ledger. if we don't add code in LedgerOpenOp, we have to find all such kind ledgers and handle them manually (maybe delete them). throwing IOException here is OK for me. but I am not sure do we need to keep backward compatibility to handle data/metadata produced by old version. what is your opinion?
> Ledger can't be opened or closed due to zero-length metadata
> ------------------------------------------------------------
>
> Key: BOOKKEEPER-153
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-153
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BK-153.patch
>
>
> Currently creating ledger path and writing ledger metadata are not in a transaction. so if the bookkeeper client (hub server uses bookkeeper client) is crashed, we have a ledger existed in zookeeper with zero-length metadata. we can't open/close it.
> we should create the ledger path with initial metadata to avoid such case. besides that, we need to add code in openLedgerOp to handle zero-length metadata for backward compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-153) Ledger can't be opened or
closed due to zero-length metadata
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186880#comment-13186880 ]
Sijie Guo commented on BOOKKEEPER-153:
--------------------------------------
currently flow of creation is
1) create a zero-length bytes znode in zookeeper
2) allocate an ensemble to add it to ledger metadata, write it to the created znode
3) return the created ledger handle
why not
1) allocate an ensemble first
2) create a znode with the initial metadata in zookeeper (this creation could make adding metadata and creating znode in a same transaction)
for backward compatibility, there exists zero-length metadata before. we should let regular operations (such as open / openNoRecovery ) can execute over such ledgers.
> Ledger can't be opened or closed due to zero-length metadata
> ------------------------------------------------------------
>
> Key: BOOKKEEPER-153
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-153
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
>
> Currently creating ledger path and writing ledger metadata are not in a transaction. so if the bookkeeper client (hub server uses bookkeeper client) is crashed, we have a ledger existed in zookeeper with zero-length metadata. we can't open/close it.
> we should create the ledger path with initial metadata to avoid such case. besides that, we need to add code in openLedgerOp to handle zero-length metadata for backward compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-153) Ledger can't be opened or
closed due to zero-length metadata
Posted by "Flavio Junqueira (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186785#comment-13186785 ]
Flavio Junqueira commented on BOOKKEEPER-153:
---------------------------------------------
Hi Sijie, I'm not sure how adding the metadata when creating the znode avoids such cases. If a bookkeeper client creates a ledger and crashes before writing anything, then that ledger is clearly useless, so we just need to be able to identify and get rid of it.
Perhaps the idea you're trying to convey is that if the metadata is written there, then we can at execute the regular operations over the ledger, like open and delete. Is it the case?
> Ledger can't be opened or closed due to zero-length metadata
> ------------------------------------------------------------
>
> Key: BOOKKEEPER-153
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-153
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
>
> Currently creating ledger path and writing ledger metadata are not in a transaction. so if the bookkeeper client (hub server uses bookkeeper client) is crashed, we have a ledger existed in zookeeper with zero-length metadata. we can't open/close it.
> we should create the ledger path with initial metadata to avoid such case. besides that, we need to add code in openLedgerOp to handle zero-length metadata for backward compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-153) Ledger can't be opened or
closed due to zero-length metadata
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186951#comment-13186951 ]
Sijie Guo commented on BOOKKEEPER-153:
--------------------------------------
we don't need to modify writeLedgerConfig. we need to change the flow of LedgerCreateOp to pass initial metadata when creating ledger path.
besides that, we need to change LedgerOpenOp to handle zero-length metadata when parsing ledger metadata.
I created a patch to fix it.
> Ledger can't be opened or closed due to zero-length metadata
> ------------------------------------------------------------
>
> Key: BOOKKEEPER-153
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-153
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BK-153.patch
>
>
> Currently creating ledger path and writing ledger metadata are not in a transaction. so if the bookkeeper client (hub server uses bookkeeper client) is crashed, we have a ledger existed in zookeeper with zero-length metadata. we can't open/close it.
> we should create the ledger path with initial metadata to avoid such case. besides that, we need to add code in openLedgerOp to handle zero-length metadata for backward compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BOOKKEEPER-153) Ledger can't be opened or closed
due to zero-length metadata
Posted by "Sijie Guo (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sijie Guo updated BOOKKEEPER-153:
---------------------------------
Attachment: BK-153.patch
> Ledger can't be opened or closed due to zero-length metadata
> ------------------------------------------------------------
>
> Key: BOOKKEEPER-153
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-153
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BK-153.patch
>
>
> Currently creating ledger path and writing ledger metadata are not in a transaction. so if the bookkeeper client (hub server uses bookkeeper client) is crashed, we have a ledger existed in zookeeper with zero-length metadata. we can't open/close it.
> we should create the ledger path with initial metadata to avoid such case. besides that, we need to add code in openLedgerOp to handle zero-length metadata for backward compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-153) Ledger can't be opened or
closed due to zero-length metadata
Posted by "Flavio Junqueira (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13186882#comment-13186882 ]
Flavio Junqueira commented on BOOKKEEPER-153:
---------------------------------------------
I think we have done it for simplicity, since we rely upon writeLedgerConfig to write ledger metadata, and writeLedgerConfig only invokes setData. I agree that it has the issue you're raising, though.
If we keep relying upon writeLedgerConfig for writing the ledger metadata, then we will have a separation between creating it initially and writing the metadata (non-atomic). To fix the problem you're raising, it sounds like we have to either modify writeLedgerConfig or have a different code path for the initial write.
> Ledger can't be opened or closed due to zero-length metadata
> ------------------------------------------------------------
>
> Key: BOOKKEEPER-153
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-153
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
>
> Currently creating ledger path and writing ledger metadata are not in a transaction. so if the bookkeeper client (hub server uses bookkeeper client) is crashed, we have a ledger existed in zookeeper with zero-length metadata. we can't open/close it.
> we should create the ledger path with initial metadata to avoid such case. besides that, we need to add code in openLedgerOp to handle zero-length metadata for backward compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-153) Ledger can't be opened or
closed due to zero-length metadata
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188403#comment-13188403 ]
Sijie Guo commented on BOOKKEEPER-153:
--------------------------------------
discussed with Ivan offline, these ledgers are orphan ledgers (failed creation), which only affect recovery tool. It would be better to handle such kind of ledgers in recovery tool. so I would remove code changes in LedgerOpenOp and create another jira to handle it in recovery tool.
> Ledger can't be opened or closed due to zero-length metadata
> ------------------------------------------------------------
>
> Key: BOOKKEEPER-153
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-153
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BK-153.patch
>
>
> Currently creating ledger path and writing ledger metadata are not in a transaction. so if the bookkeeper client (hub server uses bookkeeper client) is crashed, we have a ledger existed in zookeeper with zero-length metadata. we can't open/close it.
> we should create the ledger path with initial metadata to avoid such case. besides that, we need to add code in openLedgerOp to handle zero-length metadata for backward compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-153) Ledger can't be opened or
closed due to zero-length metadata
Posted by "Sijie Guo (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13188138#comment-13188138 ]
Sijie Guo commented on BOOKKEEPER-153:
--------------------------------------
I don't like the changes in LedgerOpenOp either.
The ledgers created using this patch would never encounter such problem. My concern is that the data/metadata generated by old code, do we need to handle them in new code? The zero-length-metadata ledgers did exist in the system using BOOKKEEPER-4.0.0. How we handle them when user upgrade it from 4.0.0 to 4.1.0?
> Ledger can't be opened or closed due to zero-length metadata
> ------------------------------------------------------------
>
> Key: BOOKKEEPER-153
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-153
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BK-153.patch
>
>
> Currently creating ledger path and writing ledger metadata are not in a transaction. so if the bookkeeper client (hub server uses bookkeeper client) is crashed, we have a ledger existed in zookeeper with zero-length metadata. we can't open/close it.
> we should create the ledger path with initial metadata to avoid such case. besides that, we need to add code in openLedgerOp to handle zero-length metadata for backward compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-153) Ledger can't be opened or
closed due to zero-length metadata
Posted by "Ivan Kelly (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187752#comment-13187752 ]
Ivan Kelly commented on BOOKKEEPER-153:
---------------------------------------
but the case of the topic being mapped to a ledger whose metadata is empty shouldn't exist anymore due to the fix in LedgerCreateOp. Of course, something else could break the metadata, but it won't be our code.
My problem with the LedgerOpenOp changes are that they add a special case, and a fair bit of code, for a condition that should never happen.
> Ledger can't be opened or closed due to zero-length metadata
> ------------------------------------------------------------
>
> Key: BOOKKEEPER-153
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-153
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BK-153.patch
>
>
> Currently creating ledger path and writing ledger metadata are not in a transaction. so if the bookkeeper client (hub server uses bookkeeper client) is crashed, we have a ledger existed in zookeeper with zero-length metadata. we can't open/close it.
> we should create the ledger path with initial metadata to avoid such case. besides that, we need to add code in openLedgerOp to handle zero-length metadata for backward compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BOOKKEEPER-153) Ledger can't be opened or
closed due to zero-length metadata
Posted by "Ivan Kelly (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BOOKKEEPER-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187599#comment-13187599 ]
Ivan Kelly commented on BOOKKEEPER-153:
---------------------------------------
The changes to LedgerCreateOp are nice. They simplify it a lot. I don't like the changes to LedgerOpenOp though. The LedgerCreateOp should stop this from happening, so now it should only happen in some sort of rare corruption case. I think it's better just to throw the IOException in this case rather than handling a special case of data.length==0 in numerous places. I assume parseLedgerConfig does throw a IOException when the data is empty.
> Ledger can't be opened or closed due to zero-length metadata
> ------------------------------------------------------------
>
> Key: BOOKKEEPER-153
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-153
> Project: Bookkeeper
> Issue Type: Bug
> Components: bookkeeper-client
> Affects Versions: 4.0.0
> Reporter: Sijie Guo
> Assignee: Sijie Guo
> Fix For: 4.1.0
>
> Attachments: BK-153.patch
>
>
> Currently creating ledger path and writing ledger metadata are not in a transaction. so if the bookkeeper client (hub server uses bookkeeper client) is crashed, we have a ledger existed in zookeeper with zero-length metadata. we can't open/close it.
> we should create the ledger path with initial metadata to avoid such case. besides that, we need to add code in openLedgerOp to handle zero-length metadata for backward compatibility.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira