You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "tanmayrauth (via GitHub)" <gi...@apache.org> on 2026/05/01 22:08:30 UTC
[PR] feat(catalog): hadoop table and namespace CRUD operations [iceberg-go]
tanmayrauth opened a new pull request, #969:
URL: https://github.com/apache/iceberg-go/pull/969
[4: CreateTable + LoadTable + CheckTableExists](https://github.com/apache/iceberg-go/issues/798#issuecomment-4320784323)
Implement the three core table operations. CreateTable validates the namespace exists, rejects custom locations, builds metadata via table.NewMetadata, writes v1.metadata.json through a temp-file-plus-rename pattern, and does a best-effort version-hint write. LoadTable calls findVersion to get the current version, builds the metadata path, and delegates to table.NewFromLocation. CheckTableExists delegates to isTableDir. Tests cover create-and-load round-trip, create with partition spec / sort order / properties, reject custom location, create in non-existent namespace, create duplicate, load non-existent, load with stale hint, and check exists true/false.
Relates to #768
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(catalog): hadoop table and namespace CRUD operations [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on PR #969:
URL: https://github.com/apache/iceberg-go/pull/969#issuecomment-4375429780
Updated the docstring.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(catalog): hadoop table and namespace CRUD operations [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on PR #969:
URL: https://github.com/apache/iceberg-go/pull/969#issuecomment-4400224086
Resolved the conflicts again 🥲
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(catalog): hadoop table and namespace CRUD operations [iceberg-go]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on PR #969:
URL: https://github.com/apache/iceberg-go/pull/969#issuecomment-4380582108
looks good, just need to resolve the conflicts!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(catalog): hadoop table and namespace CRUD operations [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on PR #969:
URL: https://github.com/apache/iceberg-go/pull/969#issuecomment-4365378339
@laskoviymishka @zeroshade can you please review this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(catalog): hadoop table and namespace CRUD operations [iceberg-go]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade merged PR #969:
URL: https://github.com/apache/iceberg-go/pull/969
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(catalog): hadoop table and namespace CRUD operations [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on PR #969:
URL: https://github.com/apache/iceberg-go/pull/969#issuecomment-4401600767
@zeroshade can you please merge this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(catalog): hadoop table and namespace CRUD operations [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on PR #969:
URL: https://github.com/apache/iceberg-go/pull/969#issuecomment-4399756224
@zeroshade I've rebased the branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(catalog): hadoop table and namespace CRUD operations [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on code in PR #969:
URL: https://github.com/apache/iceberg-go/pull/969#discussion_r3183791682
##########
catalog/hadoop/hadoop.go:
##########
@@ -235,8 +237,104 @@ func (c *Catalog) findVersion(ident table.Identifier) (int, error) {
return c.scanForward(ident, maxVer), nil
}
-func (c *Catalog) CreateTable(_ context.Context, _ table.Identifier, _ *iceberg.Schema, _ ...catalog.CreateTableOpt) (*table.Table, error) {
- return nil, errors.New("hadoop catalog: CreateTable not yet implemented")
+func (c *Catalog) CreateTable(ctx context.Context, ident table.Identifier, sc *iceberg.Schema, opts ...catalog.CreateTableOpt) (*table.Table, error) {
+ var cfg catalog.CreateTableCfg
+ for _, opt := range opts {
+ opt(&cfg)
+ }
+
+ if len(ident) < 2 {
+ return nil, errors.New("hadoop catalog: table identifier must have at least a namespace and table name")
+ }
+
+ ns := catalog.NamespaceFromIdent(ident)
+ nsPath := c.namespaceToPath(ns)
+
+ info, err := os.Stat(nsPath)
+ if os.IsNotExist(err) || (err == nil && !info.IsDir()) {
+ return nil, fmt.Errorf("%w: %s", catalog.ErrNoSuchNamespace, strings.Join(ns, "."))
+ }
Review Comment:
This is intentionally local-only for now to match the scoped plan (local parity with Spark's Java HadoopCatalog first). The io.IO interface doesn't currently have Stat or MkdirAll equivalents needed for directory-based namespace operations, so switching to it would require extending the interface. I'll open a follow-up issue to add something like StatableIO and refactor to use icebergio.IO throughout for HDFS/cloud support.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(catalog): hadoop table and namespace CRUD operations [iceberg-go]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on code in PR #969:
URL: https://github.com/apache/iceberg-go/pull/969#discussion_r3183805197
##########
catalog/hadoop/hadoop.go:
##########
@@ -235,8 +237,104 @@ func (c *Catalog) findVersion(ident table.Identifier) (int, error) {
return c.scanForward(ident, maxVer), nil
}
-func (c *Catalog) CreateTable(_ context.Context, _ table.Identifier, _ *iceberg.Schema, _ ...catalog.CreateTableOpt) (*table.Table, error) {
- return nil, errors.New("hadoop catalog: CreateTable not yet implemented")
+func (c *Catalog) CreateTable(ctx context.Context, ident table.Identifier, sc *iceberg.Schema, opts ...catalog.CreateTableOpt) (*table.Table, error) {
+ var cfg catalog.CreateTableCfg
+ for _, opt := range opts {
+ opt(&cfg)
+ }
+
+ if len(ident) < 2 {
+ return nil, errors.New("hadoop catalog: table identifier must have at least a namespace and table name")
+ }
+
+ ns := catalog.NamespaceFromIdent(ident)
+ nsPath := c.namespaceToPath(ns)
+
+ info, err := os.Stat(nsPath)
+ if os.IsNotExist(err) || (err == nil && !info.IsDir()) {
+ return nil, fmt.Errorf("%w: %s", catalog.ErrNoSuchNamespace, strings.Join(ns, "."))
+ }
Review Comment:
Fair enough. Let's continue to use `pkg.go.dev/io/fs` as inspiration for any changes we make to the IO package.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] feat(catalog): hadoop table and namespace CRUD operations [iceberg-go]
Posted by "tanmayrauth (via GitHub)" <gi...@apache.org>.
tanmayrauth commented on PR #969:
URL: https://github.com/apache/iceberg-go/pull/969#issuecomment-4392589331
Resolved the conflicts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org