You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "dlmarion (via GitHub)" <gi...@apache.org> on 2023/12/01 15:50:26 UTC
[I] TabletManagementIteratorIT failing in elasticity [accumulo]
dlmarion opened a new issue, #4004:
URL: https://github.com/apache/accumulo/issues/4004
TabletManagementIteratorIT is expecting one tablet to be returned when testing the volume replacement code, but two tablets are being returned. One tablet is being returned because there is an error on the TabletServer in the TabletManagementIterator. The error is:
```
2023-12-01T15:28:58,442 [state.TabletManagementIterator] ERROR: Error computing tablet management actions for extent: 4;some split<
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: default:8080/d4441065-72a7-4536-9f4c-d649ecfcec8e
at org.apache.hadoop.fs.Path.initialize(Path.java:263) ~[hadoop-client-api-3.3.6.jar:?]
at org.apache.hadoop.fs.Path.<init>(Path.java:221) ~[hadoop-client-api-3.3.6.jar:?]
at org.apache.accumulo.server.fs.VolumeUtil.switchVolumes(VolumeUtil.java:93) ~[classes/:?]
at org.apache.accumulo.server.fs.VolumeUtil.volumeReplacementEvaluation(VolumeUtil.java:180) ~[classes/:?]
at org.apache.accumulo.server.fs.VolumeUtil.needsVolumeReplacement(VolumeUtil.java:143) ~[classes/:?]
at org.apache.accumulo.server.manager.state.TabletManagementIterator.computeTabletManagementActions(TabletManagementIterator.java:254) ~[classes/:?]
at org.apache.accumulo.server.manager.state.TabletManagementIterator.consume(TabletManagementIterator.java:210) [classes/:?]
```
Looking at VolumeUtil.switchVolumes, it's trying to create a Hadoop Path object from `default:8080/d4441065-72a7-4536-9f4c-d649ecfcec8e`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Changes to LogEntry may have broken VolumeUtil.switchVolumes [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on issue #4004:
URL: https://github.com/apache/accumulo/issues/4004#issuecomment-1836775527
Added #4007 as a test in 2.1 for VolumeUtil.switchVolumes. I'm curious if this test fails on the merge to `main`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] TabletManagementIteratorIT failing in elasticity [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on issue #4004:
URL: https://github.com/apache/accumulo/issues/4004#issuecomment-1836355161
I'm curious if this is broken in `main` as well. Looking at VolumeUtilTest, there are no `WAL` FileType objects being tested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Changes to LogEntry may have broken VolumeUtil.switchVolumes [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion closed issue #4004: Changes to LogEntry may have broken VolumeUtil.switchVolumes
URL: https://github.com/apache/accumulo/issues/4004
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Changes to LogEntry may have broken VolumeUtil.switchVolumes [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on issue #4004:
URL: https://github.com/apache/accumulo/issues/4004#issuecomment-1836857241
Merged #4007 from 2.1 -> elasticity with no real issues. I *think* what happened is that even though #3939 was committed before the changes to the TabletManagementIteratorIT code, it was merged to `elasticity` **after** #3893. #3893 introduced some code that tested the volume replacements, and used a relative path with host and port. This test succeeded at the time, but began failing with #3939 was merged from `main` to `elasticity`.
I created #4010 to fix the test and resolve this issue.
I still think that the comment at https://github.com/apache/accumulo/blob/main/core/src/main/java/org/apache/accumulo/core/tabletserver/log/LogEntry.java#L45 is wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Changes to LogEntry may have broken VolumeUtil.switchVolumes [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on issue #4004:
URL: https://github.com/apache/accumulo/issues/4004#issuecomment-1836411572
This may be related to the changes in #3939
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Changes to LogEntry may have broken VolumeUtil.switchVolumes [accumulo]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on issue #4004:
URL: https://github.com/apache/accumulo/issues/4004#issuecomment-1836714829
Looking at the 2.1 code, I'm able to determine the following:
1. A TabletServer allocates write-ahead log files and registers them in ZooKeeper. Example:
```
[zk: localhost:2181(CONNECTED) 10] ls /accumulo/01af27af-ad10-416e-96e2-5fc35217022d/wals/localhost:9997[10000a6d8260003]
[2df26684-6e96-4b88-8df4-75a19a20b779, 761f4d9b-e2dd-4c3e-96d7-2aac6b695dde]
```
2. The data in the znode for the WAL contains the state and path. Example:
```
[zk: localhost:2181(CONNECTED) 8] get /accumulo/01af27af-ad10-416e-96e2-5fc35217022d/wals/localhost:9997[10000a6d8260003]/2df26684-6e96-4b88-8df4-75a19a20b779
OPEN,hdfs://localhost:9000/accumulo4/wal/localhost+9997/2df26684-6e96-4b88-8df4-75a19a20b779
```
3. When a TabletServer dies, the `TabletGroupWatcher.handleDeadTablets` method calls `TabletStateStore.unassign` which writes a `wal` reference to the tablet metadata. This occurs so that when the Tablet is hosted again it can perform recovery using those write-ahead log files.
4. To create the map of TabletServers and write-ahead log files that are used in step 3 above, the TabletGroupWatcher first calls `TabletGroupWatcher.unassignDeadTablet`, which in turn calls `WalStateManager.getWalsInUse`. If you look at that method it parses the ZNode (from step1 above) and returns the full path. Thus, the logs that are written to the metadata table in `MetadataStateStore.unassign` line 127 are the full path.
Thus, in 2.1, when `VolumeUtil.switchVolumes` is called to switch the volume of the `LogEntry`, the filename is the full path.
Did this change for 3.0 / 3.1? Or is the comment https://github.com/apache/accumulo/blob/main/core/src/main/java/org/apache/accumulo/core/tabletserver/log/LogEntry.java#L45 incorrect?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org