You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2022/09/23 17:10:04 UTC
[GitHub] [iceberg] danielcweeks commented on pull request #4518: core: Provide mechanism to cache manifest file content
danielcweeks commented on PR #4518:
URL: https://github.com/apache/iceberg/pull/4518#issuecomment-1256455762
Hey @rizaon I think this is getting real close now (thanks for sticking with us on this). Now that we've gotten this far, I see there's a little more we can do to consolidate the logic in ContentCache. For example, if you pull the property configuration into ContentCache constructor, we can remove a lot from ManifestFiles. For example:
```java
public ContentCache(FileIO fileio) {
this.fileio = fileio;
long expireAfterAccessMs = PropertyUtil.propertyAsLong(
fileio.properties(),
CatalogProperties.IO_MANIFEST_CACHE_EXPIRATION_INTERVAL_MS,
CatalogProperties.IO_MANIFEST_CACHE_EXPIRATION_INTERVAL_MS_DEFAULT);
long maxTotalBytes = PropertyUtil.propertyAsLong(
fileio.properties(),
CatalogProperties.IO_MANIFEST_CACHE_MAX_TOTAL_BYTES,
CatalogProperties.IO_MANIFEST_CACHE_MAX_TOTAL_BYTES_DEFAULT);
long maxContentLength = PropertyUtil.propertyAsLong(
fileio.properties(),
CatalogProperties.IO_MANIFEST_CACHE_MAX_CONTENT_LENGTH,
CatalogProperties.IO_MANIFEST_CACHE_MAX_CONTENT_LENGTH_DEFAULT);
ValidationException.check(expireAfterAccessMs >= 0, "expireAfterAccessMs is less than 0");
ValidationException.check(maxTotalBytes > 0, "maxTotalBytes is equal or less than 0");
ValidationException.check(maxContentLength > 0, "maxContentLength is equal or less than 0");
this.expireAfterAccessMs = expireAfterAccessMs;
this.maxTotalBytes = maxTotalBytes;
this.maxContentLength = maxContentLength;
```
You can also pull `CONTENT_CACHES`, `contentCache(FileIO io)`, and `dropCache(FileIO fileIO)` into `ContentCache` as well.
I played around with moving more of the logic into `ContentCache` and was able to move everything (minus the `newInputFile`) logic into ContentCache, which I feel is a cleaner separation of concerns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org