You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "manuzhang (via GitHub)" <gi...@apache.org> on 2025/08/05 10:00:39 UTC
[PR] Spark 4.0: Add `spark.app.id` to snapshot summary of BaseSnapshotUpdateSparkAction [iceberg]
manuzhang opened a new pull request, #13740:
URL: https://github.com/apache/iceberg/pull/13740
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark 4.0: Add `spark.app.id` to snapshot summary of BaseSnapshotUpdateSparkAction [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13740:
URL: https://github.com/apache/iceberg/pull/13740#issuecomment-3316377177
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark 4.0: Add `spark.app.id` to snapshot summary of BaseSnapshotUpdateSparkAction [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #13740:
URL: https://github.com/apache/iceberg/pull/13740#discussion_r2278446194
##########
spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/BaseSnapshotUpdateSparkAction.java:
##########
@@ -38,10 +39,12 @@ public ThisT snapshotProperty(String property, String value) {
protected void commit(org.apache.iceberg.SnapshotUpdate<?> update) {
summary.forEach(update::set);
+ update.set(SparkWriteUtil.SPARK_APP_ID, spark().sparkContext().applicationId());
Review Comment:
even then I don't think we need to update the snapshot summary manually to pass the application ID when we have already a central place where we could do that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark 4.0: Add `spark.app.id` to snapshot summary of BaseSnapshotUpdateSparkAction [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #13740: Spark 4.0: Add `spark.app.id` to snapshot summary of BaseSnapshotUpdateSparkAction
URL: https://github.com/apache/iceberg/pull/13740
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark 4.0: Add `spark.app.id` to snapshot summary of BaseSnapshotUpdateSparkAction [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #13740:
URL: https://github.com/apache/iceberg/pull/13740#issuecomment-3290081391
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark 4.0: Add `spark.app.id` to snapshot summary of BaseSnapshotUpdateSparkAction [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on code in PR #13740:
URL: https://github.com/apache/iceberg/pull/13740#discussion_r2256567456
##########
spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/BaseSnapshotUpdateSparkAction.java:
##########
@@ -38,10 +39,12 @@ public ThisT snapshotProperty(String property, String value) {
protected void commit(org.apache.iceberg.SnapshotUpdate<?> update) {
summary.forEach(update::set);
+ update.set(SparkWriteUtil.SPARK_APP_ID, spark().sparkContext().applicationId());
Review Comment:
all of this is already automatically handled via `EnvironmentContext`. It's handled at https://github.com/apache/iceberg/blob/3e0f7a1bfb167c4d18ebdc9d0f69afe0f4754512/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/SparkCatalog.java#L778 and then added to the summary in https://github.com/apache/iceberg/blob/be577eeac631d77243beb57409e476bf197f79d7/core/src/main/java/org/apache/iceberg/SnapshotProducer.java#L409
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Spark 4.0: Add `spark.app.id` to snapshot summary of BaseSnapshotUpdateSparkAction [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on code in PR #13740:
URL: https://github.com/apache/iceberg/pull/13740#discussion_r2278175908
##########
spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/BaseSnapshotUpdateSparkAction.java:
##########
@@ -38,10 +39,12 @@ public ThisT snapshotProperty(String property, String value) {
protected void commit(org.apache.iceberg.SnapshotUpdate<?> update) {
summary.forEach(update::set);
+ update.set(SparkWriteUtil.SPARK_APP_ID, spark().sparkContext().applicationId());
Review Comment:
Yes, but it's in a different field `app-id`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org