You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2020/07/01 21:17:01 UTC
[GitHub] [accumulo] keith-turner opened a new issue #1648: Generator compactions are failing
keith-turner opened a new issue #1648:
URL: https://github.com/apache/accumulo/issues/1648
After the changes in #1605 TableOperationsIT.testCompactEmptyTableWithGeneratorIterator started to fail. This is because this test initiates a compaction with user iterators on an empty table. Before the changes in #1605 the compaction would run even though there was no data and the iterators generated data. After the changes in #1605 a tablet with no data will not run a compaction.
Should this behavior be implemented, it would require multiple special cases. While starting to look into implementing empty compactions, I found the following cases. There may be more.
1. Only allow a single concurrent empty compaction to run. The changes in #1605 enable multiple concurrent compactions to run against a single tablet as long they are compacting a disjoint set of files. Two empty compactions have a disjoint set of files, so the current per tablet tracking of compacting files would not be sufficient to prevent concurrent empty compactions. Would need special handling just for the empty set.
2. Only commit empty compaction when there are no files in tablet. Its possible that while an empty compaction is running, that a new files arrives for the tablet. Given that empty compactions are special cases, should this compaction be allowed to complete in this case? The only reason the special empty compaction was started was because there were no files, so it seems it should only complete successfully if there are no files when the compaction finishes. I don't think the old code handled this case in anyway, but it could have happened in the old code. The old code would have committed an empty compaction where a file arrived after it started.
3. User compactions can select a subset of files to compact. When a tablet has files, but the user selects none for compaction should an empty compaction run? I don't think so.
These special cases will require specialized code, increasing the complexity of the compaction code. I am not sure the complexity is worth it. I was unaware of this capability until I ran into the test. I have not yet found any user facing documentation that advertises this feature.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner commented on issue #1648: Generator compactions are failing
Posted by GitBox <gi...@apache.org>.
keith-turner commented on issue #1648:
URL: https://github.com/apache/accumulo/issues/1648#issuecomment-652664735
Duplicate of #1647.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] keith-turner closed issue #1648: Generator compactions are failing
Posted by GitBox <gi...@apache.org>.
keith-turner closed issue #1648:
URL: https://github.com/apache/accumulo/issues/1648
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org