You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "tanmayrauth (via GitHub)" <gi...@apache.org> on 2026/04/21 05:51:22 UTC
[PR] feat(cli): add --schema-from-file flag to infer table schema from parquet files [iceberg-go]
tanmayrauth opened a new pull request, #927:
URL: https://github.com/apache/iceberg-go/pull/927
Added a new --schema-from-file flag to `iceberg create table` that infers the table schema from a local data file. Currently supports Parquet (.parquet, .parq), with the dispatch designed to support additional formats (e.g. Avro, ORC) in the future.
When a Parquet file contains Iceberg field IDs, they are preserved. Files without field IDs are assigned fresh sequential IDs. The flag is mutually exclusive with --schema, and the inferred schema is printed before table creation for user verification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(cli): add --schema-from-file flag to infer table schema from parquet files [iceberg-go]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #927:
URL: https://github.com/apache/iceberg-go/pull/927#discussion_r3126716319
##########
cmd/iceberg/main.go:
##########
@@ -552,3 +580,53 @@ func mergeConf(fileConf *config.CatalogConfig, resConfig *Config, explicitFlags
resConfig.Warehouse = fileConf.Warehouse
}
}
+
+func schemaFromFile(path string) (*iceberg.Schema, error) {
+ ext := strings.ToLower(filepath.Ext(path))
+
+ switch ext {
+ case ".parquet", ".parq":
+ return schemaFromParquetFile(path)
+ default:
+ return nil, fmt.Errorf("unsupported file format %q for %q: only .parquet files are currently supported", ext, path)
Review Comment:
`%q` is for a single quoted character literal, not for strings. This should probably be `%s` right? Also, if you're going to use the extension to validate whether or not it's the right format you should include both `.parquet` and `.parq` in the error message if those are the two extensions supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(cli): add --schema-from-file flag to infer table schema from parquet files [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on PR #927:
URL: https://github.com/apache/iceberg-go/pull/927#issuecomment-4291763984
Updated it with --infer-schema
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(cli): add --schema-from-file flag to infer table schema from parquet files [iceberg-go]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #927:
URL: https://github.com/apache/iceberg-go/pull/927#issuecomment-4290678364
My vote would be for `--infer-schema`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(cli): add --schema-from-file flag to infer table schema from parquet files [iceberg-go]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade merged PR #927:
URL: https://github.com/apache/iceberg-go/pull/927
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(cli): add --schema-from-file flag to infer table schema from parquet files [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on PR #927:
URL: https://github.com/apache/iceberg-go/pull/927#issuecomment-4298814852
@zeroshade if the changes are fine, shall we merge this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(cli): add --schema-from-file flag to infer table schema from parquet files [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on code in PR #927:
URL: https://github.com/apache/iceberg-go/pull/927#discussion_r3127366557
##########
cmd/iceberg/main.go:
##########
@@ -552,3 +580,53 @@ func mergeConf(fileConf *config.CatalogConfig, resConfig *Config, explicitFlags
resConfig.Warehouse = fileConf.Warehouse
}
}
+
+func schemaFromFile(path string) (*iceberg.Schema, error) {
+ ext := strings.ToLower(filepath.Ext(path))
+
+ switch ext {
+ case ".parquet", ".parq":
+ return schemaFromParquetFile(path)
+ default:
+ return nil, fmt.Errorf("unsupported file format %q for %q: only .parquet files are currently supported", ext, path)
Review Comment:
Done ✅
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org