You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2021/04/12 19:46:41 UTC
[GitHub] [accumulo] keith-turner opened a new issue #2013: Fix race condition in external compaction commit
keith-turner opened a new issue #2013:
URL: https://github.com/apache/accumulo/issues/2013
When an external compaction commits and the tablet is not present, it [deletes the marker](https://github.com/apache/accumulo/blob/75d7f67fef54771061b0fff45fdea507ebbf4ca9/server/tserver/src/main/java/org/apache/accumulo/tserver/compactions/CompactionManager.java#L458-L460). There is a race condition where if a tablet was recently unloaded, this could be deleted resulting in the work of an external compaction being for lost and probably redone later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on issue #2013: Fix race condition in external compaction commit
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #2013:
URL: https://github.com/apache/accumulo/issues/2013#issuecomment-818112876
I think tablet servers should not delete the external compaction final state markers when they do not know about the external compaction. They should only delete these markers when they are hosting the tablet related to the compaction and the tablet knows about the compaction.
For final state markers that point reference external compactions that no longer exists or tablets that no longer exist, the coordinator could periodically look for these and clean them up. This periodic detection would be similar to what the dead compaction detection is currently doing, but would be looking for the inverse. The periodic algorithm could do the following.
1. Scan the final states markers in metadata table (~ecomp section) and store them in a set w/ UUID (compaction id) as key.
2. Scan the metadata tablets section looking for compactions tablets think are running and remove them from the set.
3. Delete any final state markers in the set since these reference tablets and/or external compactions that no longer exists.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner edited a comment on issue #2013: Fix race condition in external compaction commit
Posted by GitBox <gi...@apache.org>.
keith-turner edited a comment on issue #2013:
URL: https://github.com/apache/accumulo/issues/2013#issuecomment-818112876
I think tablet servers should not delete the external compaction final state markers when they do not know about the external compaction. They should only delete these markers when they are hosting the tablet related to the compaction and the tablet knows about the compaction.
For final state markers that reference external compactions that no longer exists or tablets that no longer exist, the coordinator could periodically look for these and clean them up. This periodic detection would be similar to what the dead compaction detection is currently doing, but would be looking for the inverse. The periodic algorithm could do the following.
1. Scan the final states markers in metadata table (~ecomp section) and store them in a set w/ UUID (compaction id) as key.
2. Scan the metadata tablets section looking for compactions tablets think are running and remove them from the set.
3. Delete any final state markers in the set since these reference tablets and/or external compactions that no longer exists.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner closed issue #2013: Fix race condition in external compaction commit
Posted by GitBox <gi...@apache.org>.
keith-turner closed issue #2013:
URL: https://github.com/apache/accumulo/issues/2013
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] dlmarion commented on issue #2013: Fix race condition in external compaction commit
Posted by GitBox <gi...@apache.org>.
dlmarion commented on issue #2013:
URL: https://github.com/apache/accumulo/issues/2013#issuecomment-827730164
@keith-turner - You moved this to Done, but it's still open. Is this good to close?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org