You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2020/10/06 17:06:46 UTC
[GitHub] [iceberg] stevenzwu opened a new issue #1554: Introduce a serializable table class
stevenzwu opened a new issue #1554:
URL: https://github.com/apache/iceberg/issues/1554
Iceberg's `Table` interface is not serializable. There are benefits of holding some of the `Table` fields inside a serializable holder object, as it can simplify the writer like Flink `RowDataTaskWriterFactory`. We can consolidate a lot of constructor args into one.
```
public RowDataTaskWriterFactory(Schema schema,
RowType flinkSchema,
PartitionSpec spec,
LocationProvider locations,
FileIO io,
EncryptionManager encryptionManager,
long targetFileSizeBytes,
FileFormat format,
Map<String, String> tableProperties) {
```
Here is what we have internally.
```
public class TableInfo implements Serializable {
private final String location;
private final Schema schema;
private final PartitionSpec spec;
private final Map<String, String> properties;
private final FileIO fileIO;
private final EncryptionManager encryptionManager;
public static TableInfo fromTable(Table table) {
return TableInfo.builder()
.location(table.location())
.schema(table.schema())
.spec(table.spec())
.properties(table.properties())
.fileIO(table.io())
.encryptionManager(table.encryption())
.build();
}
}
```
@openinx @JingsongLi @rdblue what do you think? if you think this is useful, I can push the change upstream. We can discuss the name `TableInfo` vs `SerializableTable` vs `TableSerializableParts`.
`TableMetadata` is for a similar purpose. But it seems to be designed for a different purpose on how to update metadata.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] stevenzwu commented on issue #1554: Introduce a serializable table class
Posted by GitBox <gi...@apache.org>.
stevenzwu commented on issue #1554:
URL: https://github.com/apache/iceberg/issues/1554#issuecomment-891494467
resolve by PR #2923
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] stevenzwu closed issue #1554: Introduce a serializable table class
Posted by GitBox <gi...@apache.org>.
stevenzwu closed issue #1554:
URL: https://github.com/apache/iceberg/issues/1554
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org