[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #8324: Spark 3.4, Docs: Add RemoveOrphanFiles time-interval specification and testing option to the exception message

["aokolnychyi (via GitHub)" <gi...@apache.org>]

aokolnychyi commented on code in PR #8324:
URL: https://github.com/apache/iceberg/pull/8324#discussion_r1294781145


##########
docs/spark-procedures.md:
##########
@@ -277,6 +277,10 @@ Used to remove files which are not referenced in any metadata files of an Iceber
 | `dry_run`     |    | boolean   | When true, don't actually remove files (defaults to false) |
 | `max_concurrent_deletes` |    | int       | Size of the thread pool used for delete file actions (by default, no thread pool is used) |
 
+{{< hint warning >}}
+The timestamp within 24 hours cannot be set to `older_than`. For testing `remove_orphan_files`, configure `spark.testing` to true in the SparkSession object.

Review Comment:
   I see value in explaining the 24 hour limit and even adding more context from the error message to the docs. However, I am not sure it is a good idea to mention our current internal hack for testing. I am concerned this would promote the misuse of the API. It also would prevent us from using another approach in the future.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org