You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bookkeeper.apache.org by "TakaHiR07 (via GitHub)" <gi...@apache.org> on 2023/03/17 10:21:39 UTC
[GitHub] [bookkeeper] TakaHiR07 opened a new pull request, #3874: [bookie-shell] fix readLedger with parameter -bookie and -ledgerid can not read any entries in ledger
TakaHiR07 opened a new pull request, #3874:
URL: https://github.com/apache/bookkeeper/pull/3874
### Motivation
If we use bookieshell readLedger only with parameter -ledgerid, we can read all entries in this ledger, but if use with parameter -ledgerid and --bookie, we cant read any entries.
This is because it try to reverse entries from firstentry(default 0) to lastEntry(default -1). Therefore it would directly finish without read any entries, misleading us that ledgerId in our selected bookie has no entry.
### Changes
when set -ledgerid and -bookie and not set -entry, continue to read next entry until get BKNoSuchEntryException
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] TakaHiR07 commented on a diff in pull request #3874: [bookie-shell] fix readLedger with parameter -bookie and -ledgerid can not read any entries in ledger
Posted by "TakaHiR07 (via GitHub)" <gi...@apache.org>.
TakaHiR07 commented on code in PR #3874:
URL: https://github.com/apache/bookkeeper/pull/3874#discussion_r1141730774
##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/tools/cli/commands/bookie/ReadLedgerCommand.java:
##########
@@ -189,33 +188,39 @@ private boolean readledger(ServerConfiguration serverConf, ReadLedgerFlags flags
executor, scheduler, NullStatsLogger.INSTANCE,
bk.getBookieAddressResolver());
- LongStream.range(flags.firstEntryId, lastEntry).forEach(entryId -> {
+ long nextEntryId = flags.firstEntryId;
+ while (lastEntry == -1 || nextEntryId <= lastEntry) {
CompletableFuture<Void> future = new CompletableFuture<>();
- bookieClient.readEntry(bookie, flags.ledgerId, entryId,
- (rc, ledgerId1, entryId1, buffer, ctx) -> {
- if (rc != BKException.Code.OK) {
- LOG.error("Failed to read entry {} -- {}", entryId1,
- BKException.getMessage(rc));
- future.completeExceptionally(BKException.create(rc));
- return;
- }
-
- LOG.info("--------- Lid={}, Eid={} ---------",
- ledgerIdFormatter.formatLedgerId(flags.ledgerId), entryId);
- if (flags.msg) {
- LOG.info("Data: " + ByteBufUtil.prettyHexDump(buffer));
- }
-
- future.complete(null);
- }, null, BookieProtocol.FLAG_NONE);
+ long entryId = nextEntryId;
+ bookieClient.readEntry(bookie, flags.ledgerId, nextEntryId,
+ (rc, ledgerId1, entryId1, buffer, ctx) -> {
+ if (rc != BKException.Code.OK) {
+ LOG.error("Failed to read entry {} -- {}", entryId1,
+ BKException.getMessage(rc));
+ future.completeExceptionally(BKException.create(rc));
+ return;
+ }
+
+ LOG.info("--------- Lid={}, Eid={} ---------",
+ ledgerIdFormatter.formatLedgerId(flags.ledgerId), entryId);
+ if (flags.msg) {
+ LOG.info("Data: " + ByteBufUtil.prettyHexDump(buffer));
+ }
+
+ future.complete(null);
+ }, null, BookieProtocol.FLAG_NONE);
try {
future.get();
} catch (Exception e) {
LOG.error("Error future.get while reading entries from ledger {}", flags.ledgerId, e);
+ if (e.getCause() instanceof BKException.BKNoSuchEntryException && lastEntry == -1) {
Review Comment:
Second way is more reasonable, I will implement it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [bookie-shell] fix readLedger with parameter -bookie and -ledgerid can not read any entries in ledger [bookkeeper]
Posted by "shoothzj (via GitHub)" <gi...@apache.org>.
shoothzj commented on PR #3874:
URL: https://github.com/apache/bookkeeper/pull/3874#issuecomment-2081406589
@TakaHiR07 Are you still working on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] horizonzy commented on a diff in pull request #3874: [bookie-shell] fix readLedger with parameter -bookie and -ledgerid can not read any entries in ledger
Posted by "horizonzy (via GitHub)" <gi...@apache.org>.
horizonzy commented on code in PR #3874:
URL: https://github.com/apache/bookkeeper/pull/3874#discussion_r1141404364
##########
bookkeeper-server/src/main/java/org/apache/bookkeeper/tools/cli/commands/bookie/ReadLedgerCommand.java:
##########
@@ -189,33 +188,39 @@ private boolean readledger(ServerConfiguration serverConf, ReadLedgerFlags flags
executor, scheduler, NullStatsLogger.INSTANCE,
bk.getBookieAddressResolver());
- LongStream.range(flags.firstEntryId, lastEntry).forEach(entryId -> {
+ long nextEntryId = flags.firstEntryId;
+ while (lastEntry == -1 || nextEntryId <= lastEntry) {
CompletableFuture<Void> future = new CompletableFuture<>();
- bookieClient.readEntry(bookie, flags.ledgerId, entryId,
- (rc, ledgerId1, entryId1, buffer, ctx) -> {
- if (rc != BKException.Code.OK) {
- LOG.error("Failed to read entry {} -- {}", entryId1,
- BKException.getMessage(rc));
- future.completeExceptionally(BKException.create(rc));
- return;
- }
-
- LOG.info("--------- Lid={}, Eid={} ---------",
- ledgerIdFormatter.formatLedgerId(flags.ledgerId), entryId);
- if (flags.msg) {
- LOG.info("Data: " + ByteBufUtil.prettyHexDump(buffer));
- }
-
- future.complete(null);
- }, null, BookieProtocol.FLAG_NONE);
+ long entryId = nextEntryId;
+ bookieClient.readEntry(bookie, flags.ledgerId, nextEntryId,
+ (rc, ledgerId1, entryId1, buffer, ctx) -> {
+ if (rc != BKException.Code.OK) {
+ LOG.error("Failed to read entry {} -- {}", entryId1,
+ BKException.getMessage(rc));
+ future.completeExceptionally(BKException.create(rc));
+ return;
+ }
+
+ LOG.info("--------- Lid={}, Eid={} ---------",
+ ledgerIdFormatter.formatLedgerId(flags.ledgerId), entryId);
+ if (flags.msg) {
+ LOG.info("Data: " + ByteBufUtil.prettyHexDump(buffer));
+ }
+
+ future.complete(null);
+ }, null, BookieProtocol.FLAG_NONE);
try {
future.get();
} catch (Exception e) {
LOG.error("Error future.get while reading entries from ledger {}", flags.ledgerId, e);
+ if (e.getCause() instanceof BKException.BKNoSuchEntryException && lastEntry == -1) {
Review Comment:
If the ledger only has 100 entries, but the input param `lastEntry` is 10000, it will still read 100-10000 entries.
There are two ways to solve it.
1. remove the `lastEntry == -1` judgment, once meet BKNoSuchEntryException, end the loop.
2. Add more limitations for the loop. (lastEntry == -1 || nextEntryId <= lastEntry && nextEntryId <= LAC), we can use bookieClient.readLac() to get the LAC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org