You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by Jeff Kubina <je...@gmail.com> on 2017/04/05 13:40:45 UTC

Does a hard shutdown of accumulo clear all fates in zookeeper

I had three fates in the accumulo fate queue that had no tasks assigned to
them and could not be deleted or failed. All create/delete/clone etc
commands were timing out. I did a hard shutdown of all services. I had
planned to do "accumulo org.apache.accumulo.server.fate.Admin kill <txid>"
but that command is deprecated and replaced with the fate shell command.
Since the fate shell command failed does a hard shutdown ensure that all
the fates in zookeeper are removed or do I need to delete them manually?

Re: Does a hard shutdown of accumulo clear all fates in zookeeper

Posted by Josh Elser <jo...@gmail.com>.
No, by design, FATE operations wouldn't be removed on shutdown as it 
would leave the system in an inconsistent state.

Think back to the common problem we're trying to solve with FATE: what 
happens to a partial table creation during unexpected shutdown. We the 
FATE runner comes back up (the master), it needs to be able to rollback 
to the last known "good" state of the FATE op, and then proceed. The 
FATE op _should_ just proceed on its own after a restart.

However, it sounds like you might have run into a bug with FATE which is 
causing this situation to happen. Have you done any more digging into 
why these operations were "stuck" and timing out? If you are confident 
that these are orphaned and the system isn't properly managing them, you 
would have to kill them yourself.

Jeff Kubina wrote:
> I had three fates in the accumulo fate queue that had no tasks assigned
> to them and could not be deleted or failed. All create/delete/clone etc
> commands were timing out. I did a hard shutdown of all services. I had
> planned to do "accumulo org.apache.accumulo.server.fate.Admin kill
> <txid>" but that command is deprecated and replaced with the fate shell
> command. Since the fate shell command failed does a hard shutdown ensure
> that all the fates in zookeeper are removed or do I need to delete them
> manually?