You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by GitBox <gi...@apache.org> on 2022/10/06 16:51:14 UTC
[GitHub] [ozone] adoroszlai commented on a diff in pull request #3803: HDDS-7290. provide a config to increase the list batch size in OzoneFileSystem
adoroszlai commented on code in PR #3803:
URL: https://github.com/apache/ozone/pull/3803#discussion_r989271276
##########
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/BasicOzoneFileSystem.java:
##########
@@ -91,6 +94,8 @@ public class BasicOzoneFileSystem extends FileSystem {
static final Logger LOG =
LoggerFactory.getLogger(BasicOzoneFileSystem.class);
+ private static int listingPageSize = 1024;
Review Comment:
I recommend setting it to `OZONE_CLIENT_FS_LISTING_PAGE_SIZE_DEFAULT` to avoid duplication.
##########
hadoop-ozone/common/src/main/java/org/apache/hadoop/ozone/om/OMConfigKeys.java:
##########
@@ -405,4 +405,17 @@ private OMConfigKeys() {
public static final TimeDuration OZONE_OM_CONTAINER_LOCATION_CACHE_TTL_DEFAULT
= TimeDuration.valueOf(360, TimeUnit.MINUTES);
+
+ // Items listing page size for fs client sub-commands output
+ public static final String OZONE_CLIENT_FS_MAX_LISTING_PAGE_SIZE =
+ "ozone.client.fs.max.listing.page.size";
+
+ public static final String
+ OZONE_CLIENT_FS_LISTING_PAGE_SIZE =
+ "ozone.client.fs.default.listing.page.size";
+ public static final int
+ OZONE_CLIENT_FS_LISTING_PAGE_SIZE_DEFAULT = 1024;
Review Comment:
`OMConfigKeys` is for OM server-side settings. Since this setting is for the client, I think `OzoneConfigKeys` would be a better place for it.
##########
hadoop-hdds/common/src/main/resources/ozone-default.xml:
##########
@@ -3358,5 +3358,22 @@
</description>
</property>
+ <property>
+ <name>ozone.client.fs.default.listing.page.size</name>
+ <value>1024</value>
+ <tag>OZONE, CLIENT</tag>
+ <description>
+ Default listing page size value used for listing items on fs related sub-commands.
+ </description>
+ </property>
+
+ <property>
+ <name>ozone.client.fs.max.listing.page.size</name>
Review Comment:
I think this config property is unnecessary. Consider the case where a user wants to use page size P, which is considered too large. They can just set `ozone.client.fs.max.listing.page.size` to P+1 and bypass the limit.
##########
hadoop-hdds/common/src/main/resources/ozone-default.xml:
##########
@@ -3358,5 +3358,22 @@
</description>
</property>
+ <property>
+ <name>ozone.client.fs.default.listing.page.size</name>
+ <value>1024</value>
+ <tag>OZONE, CLIENT</tag>
+ <description>
+ Default listing page size value used for listing items on fs related sub-commands.
Review Comment:
Can users override this config any way (e.g. by some command-line option)? If not, then I think "default" is unnecessary in this name/description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org