[jira] [Assigned] (HDDS-13242) getTrashRoots is limited by the max list page size

["Ivan Andika (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDDS-13242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ivan Andika reassigned HDDS-13242:
----------------------------------

    Assignee: Ivan Andika

> getTrashRoots is limited by the max list page size
> --------------------------------------------------
>
>                 Key: HDDS-13242
>                 URL: https://issues.apache.org/jira/browse/HDDS-13242
>             Project: Apache Ozone
>          Issue Type: Bug
>            Reporter: Ivan Andika
>            Assignee: Ivan Andika
>            Priority: Minor
>
> TrashOzoneFileSystem#listStatus (used by TrashOzoneFileSystem#getTrashRoots) uses Integer.MAX_VALUE as the number of list entries. However, since HDDS-7290, a single listStatus is limited to ozone.fs.listing.page.size.max configuration, which defaults to 1024. 
> Although it is not common to have more than 1024 trash roots (e.g. which implies 1024 unique users deletes to trash in the same bucket). This might cause some trash roots to not be processed. We can first write a test to validate the issue.
> One solution is to use a listing iterator to send multiple smaller listStatus API, similar to OzoneFileStatusIterator. We can improve this iterator by persistently storing the current RocksDB iterator across listStatus calls instead of creating one for every listStatus call.
> Note: The FS client implementation BasicRootedOzoneFileSystem#listStatusAdapter (used by BasicRootedOzoneFileSystem#getTrashRoots) also default to 1024, so it will also have the same issues. However, since getTrashRoots are only used for server-side (OM) trash cleanup, the client-side getTrashRoots are rarely used.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org