You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bookkeeper.apache.org by GitBox <gi...@apache.org> on 2020/05/27 12:15:27 UTC
[GitHub] [bookkeeper] hangc0276 opened a new pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
hangc0276 opened a new pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349
### Motivation
When i use `bookkeeper shell readlogmetadata` to get an entrylog file's metadata info, i encountered the following exception:
```
19:51:11.282 [main] INFO org.apache.bookkeeper.bookie.EntryLogger - Failed to get ledgers map index from: 0.log : No file for log 0
19:51:11.284 [main] WARN org.apache.bookkeeper.bookie.EntryLogger - Failed to get channel to scan entry log: 0.log
Exception in thread "main" com.google.common.util.concurrent.UncheckedExecutionException: No file for log 0
at org.apache.bookkeeper.tools.cli.commands.bookie.ReadLogMetadataCommand.apply(ReadLogMetadataCommand.java:106)
at org.apache.bookkeeper.bookie.BookieShell$ReadLogMetadataCmd.runCmd(BookieShell.java:988)
at org.apache.bookkeeper.bookie.BookieShell$MyCommand.runCmd(BookieShell.java:223)
at org.apache.bookkeeper.bookie.BookieShell.run(BookieShell.java:1976)
at org.apache.bookkeeper.bookie.BookieShell.main(BookieShell.java:2067)
Caused by: java.io.FileNotFoundException: No file for log 0
at org.apache.bookkeeper.bookie.EntryLogger.findFile(EntryLogger.java:960)
at org.apache.bookkeeper.bookie.EntryLogger.getChannelForLogId(EntryLogger.java:895)
at org.apache.bookkeeper.bookie.EntryLogger.scanEntryLog(EntryLogger.java:976)
at org.apache.bookkeeper.bookie.EntryLogger.extractEntryLogMetadataByScanning(EntryLogger.java:1132)
at org.apache.bookkeeper.bookie.EntryLogger.getEntryLogMetadata(EntryLogger.java:1041)
at org.apache.bookkeeper.tools.cli.commands.bookie.ReadLogMetadataCommand.printEntryLogMetadata(ReadLogMetadataCommand.java:132)
at org.apache.bookkeeper.tools.cli.commands.bookie.ReadLogMetadataCommand.readLogMetadata(ReadLogMetadataCommand.java:125)
at org.apache.bookkeeper.tools.cli.commands.bookie.ReadLogMetadataCommand.apply(ReadLogMetadataCommand.java:104)
```
In the source code, it parse the entry log id with 10 radix, however, the entry log file name is 16 radix entry log id and parse failed. Then parsing it with entry log file name, and set the log file name into flags. When parse the flags in `ReadLogMetadataCommand` class, it first check
```
private static final long DEFAULT_LOGID = -1L
if (flags.logId != DEFAULT_LOGID) {
logid = flags.logId;
}
```
the default flag.logid is 0, so the `logid` will be set to 0.
### Changes
1. using `16 radix` to parse entry log id
2. when using entry log file name to get entry log metatdata, set the flags.logid to `-1`
3. export entry log usage to the output
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] hangc0276 commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-635147846
rerun failure tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] eolivelli commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-636489665
Maybe you are missing some **synchronized** block
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] eolivelli commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-772301389
@hangc0276 do you mind merging this patch to current master ?
I would like to push this patch as soon as CI is green
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] lamber-ken commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
lamber-ken commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-635484434
Hi @hangc0276, it seems the codebase do not need to chang. Your stackstrace didn't caused by parse issuse, please take a look at data dir, you can use following command
```
ls -al /tmp/bk-data/current
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] hangc0276 closed pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
hangc0276 closed pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] Ghatage commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
Ghatage commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-741562153
rerun failure checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] eolivelli commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-741756582
I am going to merge this patch as soon as CI passes
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] eolivelli commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-636489498
It is a spotbugs error:
[ERROR] Inconsistent synchronization of org.apache.bookkeeper.tools.cli.commands.bookie.ReadLogMetadataCommand.entryLogger; locked 66% of time [org.apache.bookkeeper.tools.cli.commands.bookie.ReadLogMetadataCommand, org.apache.bookkeeper.tools.cli.commands.bookie.ReadLogMetadataCommand, org.apache.bookkeeper.tools.cli.commands.bookie.ReadLogMetadataCommand] Unsynchronized access at ReadLogMetadataCommand.java:[line 132]Synchronized access at ReadLogMetadataCommand.java:[line 144]Synchronized access at ReadLogMetadataCommand.java:[line 145] IS2_INCONSISTENT_SYNC
[ERROR] Failed to execute goal com.github.spotbugs:spotbugs-maven-plugin:3.1.8:check (default-cli) on project bookkeeper-server: failed with 1 bugs and 0 errors -> [Help 1]
Post
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] hangc0276 commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-978735799
@eolivelli The tests have been passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] zymap merged pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
zymap merged pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] eolivelli commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-768110912
rerun-failure-tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] hangc0276 commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-739637458
> LGTM
>
> @hangc0276 can you find the spotbug issue?
OK, I will fix it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] eolivelli closed pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
eolivelli closed pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] eolivelli commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
eolivelli commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-741755839
rerun failure checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] hangc0276 commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-636478817
> For example, the log file is `c.log`, if apply your patch, it will read `18.log`, it's wrong.
>
> ```
> Long.parseLong("12", 16) = 18
> ```
@lamber-ken thanks for your reply. The entry log file name is HexString in EntryLoggerAllocator.java
```
logFileName = Long.toHexString(preallocatedLogId) + suffix;
```
For the `c.log`, in my patch, it will parse entry log id
```
Long.parseLong("c", 16) = 12
```
For the command: `ls -al /tmp/bk-data/current`
```
-rw-r--r-- 1 root root 1074482107 Apr 16 16:38 3dd.log
-rw-r--r-- 1 root root 1074958616 Apr 16 16:38 3de.log
-rw-r--r-- 1 root root 1074669756 Apr 16 16:39 3df.log
-rw-r--r-- 1 root root 1074372627 Apr 16 16:39 3e0.log
-rw-r--r-- 1 root root 1076423506 Apr 16 16:58 3e8.log
-rw-r--r-- 1 root root 1075989200 Apr 16 16:58 3e9.log
-rw-r--r-- 1 root root 1077057693 Apr 16 16:59 3ea.log
-rw-r--r-- 1 root root 1075034201 Apr 16 17:00 3eb.log
-rw-r--r-- 1 root root 1073316319 Apr 16 21:23 3ec.log
-rw-r--r-- 1 root root 10316071 Apr 16 21:23 3ed.log
-rw-r--r-- 1 root root 1112640702 Apr 20 18:16 3ee.log
-rw-r--r-- 1 root root 1074654208 Apr 20 18:17 3ef.log
```
I am confused why `Compatibility Check Java8 / check` run failed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] hangc0276 commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-634695020
rerun failure checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] lamber-ken commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
lamber-ken commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-635485911
For example, the log file is `c.log`, if apply your patch, it will read `18.log`, it's wrong.
```
Long.parseLong("12", 16) = 18
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] hangc0276 commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
hangc0276 commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-977996896
rerun-failure-tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@bookkeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [bookkeeper] lamber-ken commented on pull request #2349: fix readlogmetadata failed bug and export entrylog file usage to output
Posted by GitBox <gi...@apache.org>.
lamber-ken commented on pull request #2349:
URL: https://github.com/apache/bookkeeper/pull/2349#issuecomment-634876786
rerun failure tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org