You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "saitharun15 (via GitHub)" <gi...@apache.org> on 2025/08/21 17:26:06 UTC
[I] Enhance Spark's Register Table procedure to accept metadata folder as input [iceberg]
saitharun15 opened a new issue, #13889:
URL: https://github.com/apache/iceberg/issues/13889
### Feature Request / Improvement
Currently Spark's Register Table Procedure requires a full, direct path to a specific metadata file as input to register a table in a catalog.
This can be improved by allowing a metadata folder path as input, where the procedure automatically identifies and uses the **latest metadata.json** file within that directory to register the table, making the process more user-friendly.
This behavior is already implemented in Presto Iceberg ([PR](https://github.com/prestodb/presto/pull/21335) / [Implementation](https://github.com/kiersten-stokes/presto/blob/master/presto-iceberg/src/main/java/com/facebook/presto/iceberg/RegisterTableProcedure.java)]), and adding the same support in Spark would simplify table registration.
### Query engine
Spark
### Willingness to contribute
- [ ] I can contribute this improvement/feature independently
- [x] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Enhance Spark's Register Table procedure to accept metadata folder as input [iceberg]
Posted by "saitharun15 (via GitHub)" <gi...@apache.org>.
saitharun15 commented on issue #13889:
URL: https://github.com/apache/iceberg/issues/13889#issuecomment-3211697670
Hi @RussellSpitzer, thanks for raising this, that’s a really good point. I agree that “latest” isn’t obvious, and it can lead to ambiguity.
I’ve raised a PR and updated the documentation to clarify the behavior. Could you please review and let me know your thoughts. I’m happy to get feedback or suggestions on the PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Enhance Spark's Register Table procedure to accept metadata folder as input [iceberg]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on issue #13889:
URL: https://github.com/apache/iceberg/issues/13889#issuecomment-3211511716
I'm a little worried about this because I feel like the behavior is not quite as simple as "latest_metadata_version". From my scan of the Trino Code the behavior is
1. Highest Versioned File (allowing either metastore file names or hadoop names)
2. On timestamp tie between multiple versioned files, choose the one that was written last
Which I think may be debatable ?
After all if I have
V4 written today and
V10 written last year, should I take v10?
I don't have a problem adding this functionaltiy but we should probably be pretty clear in exactly what behavior we are implementing in the docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Enhance Spark's Register Table procedure to accept metadata folder as input [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #13889:
URL: https://github.com/apache/iceberg/issues/13889#issuecomment-3917779242
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Enhance Spark's Register Table procedure to accept metadata folder as input [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #13889: Enhance Spark's Register Table procedure to accept metadata folder as input
URL: https://github.com/apache/iceberg/issues/13889
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Enhance Spark's Register Table procedure to accept metadata folder as input [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #13889:
URL: https://github.com/apache/iceberg/issues/13889#issuecomment-4001168983
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org