You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/07/28 08:43:31 UTC
[GitHub] [iceberg] rymurr commented on a change in pull request #2878: API: extend CloseableGroup and close resources when closing aws catalogs
rymurr commented on a change in pull request #2878:
URL: https://github.com/apache/iceberg/pull/2878#discussion_r678092705
##########
File path: api/src/main/java/org/apache/iceberg/io/CloseableGroup.java
##########
@@ -23,20 +23,47 @@
import java.io.IOException;
import java.util.Deque;
import org.apache.iceberg.relocated.com.google.common.collect.Lists;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
public abstract class CloseableGroup implements Closeable {
- private final Deque<Closeable> closeables = Lists.newLinkedList();
+ private static final Logger LOG = LoggerFactory.getLogger(CloseableGroup.class);
- protected void addCloseable(Closeable closeable) {
+ private final Deque<AutoCloseable> closeables = Lists.newLinkedList();
+ private boolean suppressCloseFailure = false;
+
+ protected void addCloseable(AutoCloseable closeable) {
closeables.add(closeable);
}
+ protected void setSuppressCloseFailure(boolean shouldSuppress) {
+ this.suppressCloseFailure = shouldSuppress;
+ }
+
@Override
public void close() throws IOException {
Review comment:
Is the close method idempotent? What happens if it is called twice?
##########
File path: api/src/main/java/org/apache/iceberg/io/CloseableGroup.java
##########
@@ -23,20 +23,47 @@
import java.io.IOException;
import java.util.Deque;
import org.apache.iceberg.relocated.com.google.common.collect.Lists;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
public abstract class CloseableGroup implements Closeable {
- private final Deque<Closeable> closeables = Lists.newLinkedList();
+ private static final Logger LOG = LoggerFactory.getLogger(CloseableGroup.class);
- protected void addCloseable(Closeable closeable) {
+ private final Deque<AutoCloseable> closeables = Lists.newLinkedList();
+ private boolean suppressCloseFailure = false;
+
+ protected void addCloseable(AutoCloseable closeable) {
closeables.add(closeable);
}
+ protected void setSuppressCloseFailure(boolean shouldSuppress) {
+ this.suppressCloseFailure = shouldSuppress;
+ }
+
@Override
public void close() throws IOException {
while (!closeables.isEmpty()) {
- Closeable toClose = closeables.removeFirst();
+ AutoCloseable toClose = closeables.removeFirst();
if (toClose != null) {
- toClose.close();
+ if (suppressCloseFailure) {
Review comment:
Personally I think the caller should be handling errors. Why suppress this at the source?
##########
File path: core/src/main/java/org/apache/iceberg/BaseMetastoreCatalog.java
##########
@@ -25,13 +25,14 @@
import org.apache.iceberg.exceptions.AlreadyExistsException;
import org.apache.iceberg.exceptions.CommitFailedException;
import org.apache.iceberg.exceptions.NoSuchTableException;
+import org.apache.iceberg.io.CloseableGroup;
import org.apache.iceberg.relocated.com.google.common.base.MoreObjects;
import org.apache.iceberg.relocated.com.google.common.base.Preconditions;
import org.apache.iceberg.relocated.com.google.common.collect.ImmutableMap;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-public abstract class BaseMetastoreCatalog implements Catalog {
+public abstract class BaseMetastoreCatalog extends CloseableGroup implements Catalog {
Review comment:
I think closeable should be handled by the concrete impl of a catalog and not by the base class, I don't think all catalogs must implement closeable by design. This should be done via composition rather than inheritance. Additionally, I don't know what the effect of this change is on other catalogs so this feels a bit on the risky side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org