You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2021/02/17 16:56:54 UTC
[GitHub] [accumulo] milleruntime opened a new issue #1935: Invalid ZooLock found when running FateCommand
milleruntime opened a new issue #1935:
URL: https://github.com/apache/accumulo/issues/1935
While running 2.1 in Uno, I issued the `fate print` command in the shell and got the following warning:
<pre>
2021-02-17T11:49:53,986 [zookeeper.ZooLock] TRACE: validating and sorting children at path /accumulo/d4197716-1810-44e9-bda9-2f4f8b1afd1a/table_locks/35
2021-02-17T11:49:53,986 [zookeeper.ZooLock] TRACE: Validating lock-0000000000
2021-02-17T11:49:53,986 [zookeeper.ZooLock] WARN : Child found with invalid format: lock-0000000000 (does not start with zlock#)
</pre>
This seems to only happen where there is at least 1 active FATE transaction. It is not clear where in the FATE code the invalid lock is being used. It could be by the underlying FATE utils or by `FateCommand` itself.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] dlmarion commented on issue #1935: Invalid ZooLock found when running FateCommand
Posted by GitBox <gi...@apache.org>.
dlmarion commented on issue #1935:
URL: https://github.com/apache/accumulo/issues/1935#issuecomment-780840341
I believe what's happening here is something other than ZooLock is creating `/accumulo/d4197716-1810-44e9-bda9-2f4f8b1afd1a/table_locks/35/lock-0000000000` and then ZooLock later see's it and complains about it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] Manno15 closed issue #1935: Invalid ZooLock found when running FateCommand
Posted by GitBox <gi...@apache.org>.
Manno15 closed issue #1935:
URL: https://github.com/apache/accumulo/issues/1935
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] EdColeman commented on issue #1935: Invalid ZooLock found when running FateCommand
Posted by GitBox <gi...@apache.org>.
EdColeman commented on issue #1935:
URL: https://github.com/apache/accumulo/issues/1935#issuecomment-790840448
My preference would be to use ZooLock so that there is one place in the code that is managing a lock in zookeeper - otherwise - the tableOps utils would need to be verified that it handles the ZooKeeper connection edge cases that ZooLock was recently modified to handle.
However, that may not be straight forward because ZooQueueLock seems to be either doing something different - or at least has extended the lock concept - for example it seems to have both persistent and ephemeral nodes, and it may be writing data to the node. It may be that the epidermal locks can delegate to ZooLock and the persistent nodes handled separately? Maybe using lock here is overloaded and maybe a this represents something with a different / more descriptive name?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] Manno15 commented on issue #1935: Invalid ZooLock found when running FateCommand
Posted by GitBox <gi...@apache.org>.
Manno15 commented on issue #1935:
URL: https://github.com/apache/accumulo/issues/1935#issuecomment-789969159
I went back and tested this with `2.0.1` and it appears that the issue is still there. I still noticed the Lock that was held was `lock-000000000`. We just didn't log it as a warning then. Which I believe was added in #1866. I plan to investigate this further if neither of you two is doing so already.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] Manno15 edited a comment on issue #1935: Invalid ZooLock found when running FateCommand
Posted by GitBox <gi...@apache.org>.
Manno15 edited a comment on issue #1935:
URL: https://github.com/apache/accumulo/issues/1935#issuecomment-790789936
It appears the issue is because tableOps has its own Util function that utilizes `ZooQueueLock.java` which uses a different prefix (which is just "Lock-"). By changing that prefix to include "zlock#", it got through that first check but now fails because it does not end with a "#". I am working on getting past that now, but before I get too far I wanted to get ideas on what the best solution would be. Should we adapt `ZooQueueLock` to use the naming conventions we expect, adapt `tableOps/Utils` to use `ZooLock` itself so it will also use a UUID, or alter the if statements `ZooLock.validateAndSortChildrenByLockPrefix` to account for `tableOps/Utils` using a different naming convention when we call things like `Fate Print`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on issue #1935: Invalid ZooLock found when running FateCommand
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on issue #1935:
URL: https://github.com/apache/accumulo/issues/1935#issuecomment-790873402
At the very least, if it's using a different implementation, it shouldn't be using the same path in ZK as ZooLock.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] Manno15 commented on issue #1935: Invalid ZooLock found when running FateCommand
Posted by GitBox <gi...@apache.org>.
Manno15 commented on issue #1935:
URL: https://github.com/apache/accumulo/issues/1935#issuecomment-790789936
It appears the issue is because tableOps has its own Util function that utilizes `ZooQueueLock.java` which uses a different prefix (which is just "Lock-"). By changing that prefix to include "zlock#", it got through that first check but now fails because it does not end with a "#". I am working on getting past that now, but before I get too far I wanted to get ideas on what the best solution would be. Should we adapt `ZooQueueLock` to use the naming conventions we expect, adapt `tableOps/Utils` to use `ZooLock` itself so it will also use a UUID, or alter the check statements if `ZooLock.validateAndSortChildrenByLockPrefix` to account for `tableOps/Utils` using a different naming convention when we call things like `Fate Print`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] Manno15 commented on issue #1935: Invalid ZooLock found when running FateCommand
Posted by GitBox <gi...@apache.org>.
Manno15 commented on issue #1935:
URL: https://github.com/apache/accumulo/issues/1935#issuecomment-791522032
I created a draft PR with a simple solution of altering the prefix in `ZooQueueLock` to match the expected prefix from `ZooLock.validateAndSortChildrenByLockPrefix`. I am still investigating the possible solutions mentioned in the comments above.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] Manno15 edited a comment on issue #1935: Invalid ZooLock found when running FateCommand
Posted by GitBox <gi...@apache.org>.
Manno15 edited a comment on issue #1935:
URL: https://github.com/apache/accumulo/issues/1935#issuecomment-789969159
I went back and tested this with `2.0.1` and it appears that the issue is still there. I still noticed the Lock that was held was `lock-000000000`, without a lock prefix. We just didn't log it as a warning then. Which I believe was added in #1866. I plan to investigate this further if neither of you two is doing so already.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] DomGarguilo commented on issue #1935: Invalid ZooLock found when running FateCommand
Posted by GitBox <gi...@apache.org>.
DomGarguilo commented on issue #1935:
URL: https://github.com/apache/accumulo/issues/1935#issuecomment-790026928
I have observed this as well while running `ZooLockTest`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org