[GitHub] [iceberg] flyrain commented on pull request #5742: Spark: Add read conf for setting threshold to use streaming delete filter

[GitBox <gi...@apache.org>]

flyrain commented on PR #5742:
URL: https://github.com/apache/iceberg/pull/5742#issuecomment-1253006072

   Hi @wypoon, thanks for the PR. I don't see a strong reason to expose the threshold to users. Instead, it's better to hide it from users. Here are reasons:
   1. It is an internal threshold that user doesn't have to understand, and probably don't want to understand.
   2. We can potentially remove it in the future if possible. We use to discuss that here https://github.com/apache/iceberg/pull/3535#issuecomment-996355892, it is not valid at that time though.
   3. We can adjust the value according to internal implementation. For example, we can increase the threshold when we use more efficient data structure to store pos delete rows.
   
   What do you think?
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org