[GitHub] [iceberg] RussellSpitzer edited a comment on issue #2934: IllegarlArgumentException when filtering on BinaryType column.

[GitBox <gi...@apache.org>]

RussellSpitzer edited a comment on issue #2934:
URL: https://github.com/apache/iceberg/issues/2934#issuecomment-895293363


   This is something that needs to be fixed in Iceberg, we haven't run into this yet but @manishmalhotrawork and I were talking about possible fixes over the weekend.
   
   I'm pretty sure the issue here is in predicate translation, we have a function sparkSql() which doesn't handle ByteBuffers even though those are valid predicates in evaluators. See
   
   https://github.com/apache/iceberg/blob/40e626a168a3137b42d023fb8f8f3835f3cd0a78/spark3/src/main/java/org/apache/iceberg/spark/Spark3Util.java#L718-L721
   
    I think we just need to convert incoming byteBuffers to byteArrays here
   
   https://github.com/apache/iceberg/blob/40e626a168a3137b42d023fb8f8f3835f3cd0a78/spark3/src/main/java/org/apache/iceberg/spark/SparkFilters.java#L198-L209
   
   So that the rest of our SparkUtil code will work with byte buffer literals generated by Spark


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org