You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "hamilton-earthscope (via GitHub)" <gi...@apache.org> on 2025/10/24 23:03:45 UTC
[PR] feat(table): support writing gzipped metadata json [iceberg-go]
hamilton-earthscope opened a new pull request, #612:
URL: https://github.com/apache/iceberg-go/pull/612
Followup to #610 ; add write support for gzipped table metadata.
Behavior is optional and enabled when table property `write.metadata.compression-codec` is set to `gzip`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(table): support writing gzipped metadata json [iceberg-go]
Posted by "hamilton-earthscope (via GitHub)" <gi...@apache.org>.
hamilton-earthscope commented on code in PR #612:
URL: https://github.com/apache/iceberg-go/pull/612#discussion_r2466323845
##########
catalog/internal/utils.go:
##########
@@ -31,40 +32,48 @@ import (
"github.com/apache/iceberg-go"
"github.com/apache/iceberg-go/catalog"
- "github.com/apache/iceberg-go/io"
+ icebergio "github.com/apache/iceberg-go/io"
"github.com/apache/iceberg-go/table"
"github.com/google/uuid"
)
-func GetMetadataLoc(location string, newVersion uint) string {
- return fmt.Sprintf("%s/metadata/%05d-%s.metadata.json",
- location, newVersion, uuid.New().String())
-}
Review Comment:
t was not.
i tried to use it but got a circular reference between `table/` and `internal/` so i just removed the dead code instead.
##########
catalog/internal/utils.go:
##########
@@ -31,40 +32,48 @@ import (
"github.com/apache/iceberg-go"
"github.com/apache/iceberg-go/catalog"
- "github.com/apache/iceberg-go/io"
+ icebergio "github.com/apache/iceberg-go/io"
"github.com/apache/iceberg-go/table"
"github.com/google/uuid"
)
-func GetMetadataLoc(location string, newVersion uint) string {
- return fmt.Sprintf("%s/metadata/%05d-%s.metadata.json",
- location, newVersion, uuid.New().String())
-}
Review Comment:
it was not.
i tried to use it but got a circular reference between `table/` and `internal/` so i just removed the dead code instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(table): support writing gzipped metadata json [iceberg-go]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade merged PR #612:
URL: https://github.com/apache/iceberg-go/pull/612
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(table): support writing gzipped metadata json [iceberg-go]
Posted by "twuebi (via GitHub)" <gi...@apache.org>.
twuebi commented on code in PR #612:
URL: https://github.com/apache/iceberg-go/pull/612#discussion_r2465200632
##########
table/properties.go:
##########
@@ -73,6 +73,9 @@ const (
MetadataPreviousVersionsMaxKey = "write.metadata.previous-versions-max"
MetadataPreviousVersionsMaxDefault = 100
+ MetadataCompressionKey = "write.metadata.compression-codec"
+ MetadataCompressionDefault = ""
Review Comment:
I think the default is "none":
`org.apache.iceberg.TableProperties#METADATA_COMPRESSION_DEFAULT`
```java
public static final String METADATA_COMPRESSION = "write.metadata.compression-codec";
public static final String METADATA_COMPRESSION_DEFAULT = "none";
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(table): support writing gzipped metadata json [iceberg-go]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #612:
URL: https://github.com/apache/iceberg-go/pull/612#discussion_r2466314252
##########
catalog/internal/utils.go:
##########
@@ -31,40 +32,48 @@ import (
"github.com/apache/iceberg-go"
"github.com/apache/iceberg-go/catalog"
- "github.com/apache/iceberg-go/io"
+ icebergio "github.com/apache/iceberg-go/io"
"github.com/apache/iceberg-go/table"
"github.com/google/uuid"
)
-func GetMetadataLoc(location string, newVersion uint) string {
- return fmt.Sprintf("%s/metadata/%05d-%s.metadata.json",
- location, newVersion, uuid.New().String())
-}
Review Comment:
was this function not being used at all?
##########
catalog/internal/utils.go:
##########
@@ -31,40 +32,48 @@ import (
"github.com/apache/iceberg-go"
"github.com/apache/iceberg-go/catalog"
- "github.com/apache/iceberg-go/io"
+ icebergio "github.com/apache/iceberg-go/io"
"github.com/apache/iceberg-go/table"
"github.com/google/uuid"
)
-func GetMetadataLoc(location string, newVersion uint) string {
- return fmt.Sprintf("%s/metadata/%05d-%s.metadata.json",
- location, newVersion, uuid.New().String())
-}
-
-func WriteTableMetadata(metadata table.Metadata, fs io.WriteFileIO, loc string) error {
+func WriteTableMetadata(metadata table.Metadata, fs icebergio.WriteFileIO, loc string, compression string) error {
Review Comment:
compression should probably be an enum and we should error if it's not an empty string or one we recognize
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org