[GitHub] [iceberg] kbendick commented on pull request #5254: log warning when filter conversion/bind expression fail

[GitBox <gi...@apache.org>]

kbendick commented on PR #5254:
URL: https://github.com/apache/iceberg/pull/5254#issuecomment-1183834641

   > OK, I'm fine with INFO, i think it's if the user selects a column without stats like struct or metadata column, I'm betting it shouldn't be every query, but we can change it if it is too much. I think we can keep logging the exception toString too, just not the whole stack. Thanks guys !
   
   I think INFO, but without the whole stack trace (just the toString or the message), is best.
   
   Not having a filter pushed down when you expect it to be pushed down can result in scanning huge amounts of data, so relative to some of the INFO level logs that come out of Spark, so I think it’s fine even as WARN (though if we expect it often enough with metadata columns than INFO is probably better).
   
   So +1 to dealing with the big stack trace but keeping the message. Good catch @szehon-ho!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org