You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by GitBox <gi...@apache.org> on 2020/10/15 17:22:53 UTC
[GitHub] [accumulo] ctubbsii opened a new issue #1739: CreateTable FaTE operation may not clean up splits files if operation is killed
ctubbsii opened a new issue #1739:
URL: https://github.com/apache/accumulo/issues/1739
The CreateTable FaTE operation creates files in DFS for storing split points, if the user is creating a pre-split table. If the task completes, it should delete these files. However, if the job fails, it may not delete one or both. The `undo` method for the `ChooseDir` step should delete one of the files, but nothing seems like it will delete the other one, unless the operation completes successfully.
Found on main/2.1.0-SNAPSHOT branch while working on PR #1738 ; it may affect other versions as well
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on issue #1739: CreateTable FaTE operation may not clean up splits files if operation is killed
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on issue #1739:
URL: https://github.com/apache/accumulo/issues/1739#issuecomment-710354369
@Manno15 The files should not exist after the table is created. There is already a cleanup step at the end of creating a table. The current cleanup step might be broken, but it should work after my changes in #1738 are complete and merged. The issue identified in this ticket is only to address cleanup of these files if the table creation fails... not if it finishes, because that's already handled.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] Manno15 edited a comment on issue #1739: CreateTable FaTE operation may not clean up splits files if operation is killed
Posted by GitBox <gi...@apache.org>.
Manno15 edited a comment on issue #1739:
URL: https://github.com/apache/accumulo/issues/1739#issuecomment-710391364
I mistyped in my previous comment. I call my function after the create table one. Regardless if it was successful or not. So if it fails, the files are still cleaned up. This was more a proof of concept, it would probably be better to alter our current cleanup method so it works even if the create table operation fails.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] Manno15 commented on issue #1739: CreateTable FaTE operation may not clean up splits files if operation is killed
Posted by GitBox <gi...@apache.org>.
Manno15 commented on issue #1739:
URL: https://github.com/apache/accumulo/issues/1739#issuecomment-710090234
Okay, I will test that out. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii commented on issue #1739: CreateTable FaTE operation may not clean up splits files if operation is killed
Posted by GitBox <gi...@apache.org>.
ctubbsii commented on issue #1739:
URL: https://github.com/apache/accumulo/issues/1739#issuecomment-710082494
I can't think of a great way to do it. However, I'm not that familiar with the FaTE code right now. I was thinking that you might have to introduce a delay to test it (insert a long Thread.sleep in the CreateTable.call method, just to force it to take a long time... and then remove it when you're done testing). While it is sleeping, you could fail the fate operation (I think you can do that in the shell, but I'd have to check), and then restart the master service so the operation won't complete.
However, I don't actually know that this would work, and there may be different cleanup behavior if the fate operation is canceled and if it fails on its own. I'm not sure how that works.
I do know that these temporary files names are different in #1738, so whatever you do for this issue, you may need to adjust slightly after that PR is merged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] ctubbsii closed issue #1739: CreateTable FaTE operation may not clean up splits files if operation is killed
Posted by GitBox <gi...@apache.org>.
ctubbsii closed issue #1739:
URL: https://github.com/apache/accumulo/issues/1739
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] Manno15 edited a comment on issue #1739: CreateTable FaTE operation may not clean up splits files if operation is killed
Posted by GitBox <gi...@apache.org>.
Manno15 edited a comment on issue #1739:
URL: https://github.com/apache/accumulo/issues/1739#issuecomment-710238909
I did confirm that your suggestion works. I was able to fail the fate operation in the shell while master service was down and the split files were still in DFS. I created a simple delete function that's called after table is created to check if those files still exist and then delete them if they do. I still need to do more testing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] Manno15 commented on issue #1739: CreateTable FaTE operation may not clean up splits files if operation is killed
Posted by GitBox <gi...@apache.org>.
Manno15 commented on issue #1739:
URL: https://github.com/apache/accumulo/issues/1739#issuecomment-710391364
I misspoke in my previous comment. It is called after the create table call. Regardless if it completed or not. So if it fails, the files are still cleaned up.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] Manno15 commented on issue #1739: CreateTable FaTE operation may not clean up splits files if operation is killed
Posted by GitBox <gi...@apache.org>.
Manno15 commented on issue #1739:
URL: https://github.com/apache/accumulo/issues/1739#issuecomment-710238909
I did confirm that your suggestion works. I was able to fail the fate operation in the shell while master service was down and the split files were still in DFS. I created a simple delete function that's called after table is created to check if those files still exist and then delete them if they do.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo] Manno15 commented on issue #1739: CreateTable FaTE operation may not clean up splits files if operation is killed
Posted by GitBox <gi...@apache.org>.
Manno15 commented on issue #1739:
URL: https://github.com/apache/accumulo/issues/1739#issuecomment-710076020
Is there a good way to force a pre-split to fail? I want to confirm things on my end so I have a better idea on what to look for.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org