[I] feat(catalog): RenameTable feature in hadoop catalog [iceberg-go]

["tanmayrauth (via GitHub)" <gi...@apache.org>]

tanmayrauth opened a new issue, #1097:
URL: https://github.com/apache/iceberg-go/issues/1097

   ### Feature Request / Improvement
   
   Currently returns an unsupported error, matching Java's HadoopCatalog behavior.   If we decide to support this in the future, it would involve moving the table  directory to the new location — but Java chose not to implement this due to lack  of atomic rename guarantees across all Hadoop-compatible filesystems. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

Re: [I] feat(catalog): RenameTable feature in hadoop catalog [iceberg-go]

["tanmayrauth (via GitHub)" <gi...@apache.org>]

tanmayrauth commented on issue #1097:
URL: https://github.com/apache/iceberg-go/issues/1097#issuecomment-4927664806

   Good question, and it ties directly back to what we worked through in #1096.
   
   First, the cross-scheme part (s3://→file://): that's not something RenameTable can express. A Hadoop catalog instance is bound to a single warehouse root and a single FileIO resolved from that root's scheme (same as we set up in #1096). The from/to args are namespace+name path segments resolved against that one warehouse — they're not full URIs. So both sides always share the catalog's scheme; there's no seam to rename from an s3 table to a file table. Moving a table between providers is two catalog instances plus an actual byte copy, RenameTable neither crosses schemes nor transfers files, so it wouldn't help "in  advance of" the transfer either.
   
   Second, unlike every other catalog, rename here isn't a pointer swap. In REST/SQL/Glue/Hive, RenameTable just updates the catalog's index entry and no data moves. The Hadoop catalog has no index, a table's identity is its path under the warehouse (and CreateTable even rejects custom locations). So "rename"  necessarily means physically relocating the table directory. That's precisely why Java punts.
   
   On guarding it behind allow-unsafe-commits: directionally that's the consistent choice given the precedent we set. But note what it would be gating. The load-bearing op is Rename, the same primitive we already flagged as non-atomic on blob stores (copy+delete, crash mid-op leaves a half state). A table rename  is a directory-level move across many objects, which is strictly worse than the single-file rename in CommitTable. So it inherits — and amplifies — the exact correctness gap we deferred to the concurrent-write follow-up, rather than being covered by that flag.
   
   There's also a metadata trap: Iceberg metadata references manifests and data files by absolute path. Moving the directory without rewriting the metadata tree leaves those pointers dangling, so a correct rename is effectively a copy + full metadata rewrite, not a cheap move. (Interestingly this is exactly where your downstream relative-path work would help — with relative paths a same-warehouse move stops breaking internal references.)
   
   So my take: if we add RenameTable to the Hadoop catalog, scope it narrowly to same-warehouse, same-scheme moves, gate it behind allow-unsafe-commits, and treat it as blocked on the same atomicity work as the concurrent-write follow-up, not shipped ahead of it. For your actual goals (cross-provider moves,  subsetting for local debugging), a dedicated copy/migrate flow across two catalogs that rewrites metadata paths is the right shape; RenameTable isn't the vehicle for that.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

Re: [I] feat(catalog): RenameTable feature in hadoop catalog [iceberg-go]

["C-Loftus (via GitHub)" <gi...@apache.org>]

C-Loftus commented on issue #1097:
URL: https://github.com/apache/iceberg-go/issues/1097#issuecomment-4918662343

   Question on this @tanmayrauth , would it be reasonable to implement this and guard it behind `allow-unsafe-commits` similar to the logic in #1096 ? 
   
   As context, I was looking for ways in iceberg-go to move tables from one cloud storage provider to another or subsetting a part of a table for local debugging, and I was wondering if this functionality would be the right way to accomplish that in advance of the operation to actual transfer the files? i.e. would `RenameTable` allow for renaming a table with an `s3://` scheme to a table with `file://` in advance of a transfer from one filesystem to another, or would the schemes need to match? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org