You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "DomGarguilo (via GitHub)" <gi...@apache.org> on 2025/06/17 17:25:20 UTC
[PR] Schedule cleanup of interrupted scan refs [accumulo]
DomGarguilo opened a new pull request, #5652:
URL: https://github.com/apache/accumulo/pull/5652
Fixes #5650
Adds a scheduled task to the tablet server to attempt to remove any scan refs that may not have been properly cleaned up due to an interruption in the scan or scan cleanup process.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Schedule cleanup of interrupted scan refs [accumulo]
Posted by "DomGarguilo (via GitHub)" <gi...@apache.org>.
DomGarguilo commented on PR #5652:
URL: https://github.com/apache/accumulo/pull/5652#issuecomment-2997141342
@keith-turner, I merged this in and will begin working on your suggestions in a follow on PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Schedule cleanup of interrupted scan refs [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5652:
URL: https://github.com/apache/accumulo/pull/5652#discussion_r2155382782
##########
server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/Tablet.java:
##########
@@ -2220,6 +2220,10 @@ public MetadataUpdateCount getUpdateCount() {
return getDatafileManager().getUpdateCount();
}
+ public void removeOrphanedScanRefs() {
+ getDatafileManager().removeOrphanedScanRefs();
+ }
Review Comment:
We want to avoid writing to the metadata table after a tablet is closed because the rest of the system assumes that once a tablet is unloaded it will not write. The following change does two things it checks if the tablet is closed and if its not closed it increments writesInProgress to make Tablet.close() wait until this completes (the bulk import code also may modify the metadata table and does the same thing).
```suggestion
public void removeOrphanedScanRefs() {
synchronized (this){
if(isClosed() || isClosing()) {
return;
}
incrementWritesInProgress();
}
try {
getDatafileManager().removeOrphanedScanRefs();
}finally {
decrementWritesInProgress();
}
}
```
Overall this code and the code in DataFileManager is unlocking and lock the tablet object a lot, but not sure if that could be improved w/o large changes to the code (which could introduce bugs).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Schedule cleanup of interrupted scan refs [accumulo]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #5652:
URL: https://github.com/apache/accumulo/pull/5652#discussion_r2155382782
##########
server/tserver/src/main/java/org/apache/accumulo/tserver/tablet/Tablet.java:
##########
@@ -2220,6 +2220,10 @@ public MetadataUpdateCount getUpdateCount() {
return getDatafileManager().getUpdateCount();
}
+ public void removeOrphanedScanRefs() {
+ getDatafileManager().removeOrphanedScanRefs();
+ }
Review Comment:
We want to avoid writing to the metadata table after a tablet is closed because the rest of the system assumes that once a tablet is unloaded it will not write. The following change does two things it checks if the tablet is closed and if its not closed it increments writesInProgress to make Tablet.close() wait until this completes (the bulk import code also may modify the metadata table and does the same thing).
```suggestion
public void removeOrphanedScanRefs() {
synchronized (this){
if(isClosed() || isClosing()) {
return;
}
// TODO if a check is done here to see if there are orphaned scans and none are found could return and not do the increment.
// this would cut down on the number of times the lock is acquired, but not sure how clean this would be.
incrementWritesInProgress();
}
try {
getDatafileManager().removeOrphanedScanRefs();
}finally {
decrementWritesInProgress();
}
}
```
Overall this code and the code in DataFileManager is unlocking and lock the tablet object a lot, but not sure if that could be improved w/o large changes to the code (which could introduce bugs).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Schedule cleanup of interrupted scan refs [accumulo]
Posted by "DomGarguilo (via GitHub)" <gi...@apache.org>.
DomGarguilo merged PR #5652:
URL: https://github.com/apache/accumulo/pull/5652
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org