[GitHub] [iceberg] openinx commented on pull request #3857: Flink 1.13: Add SerializableTable test cases.

[GitBox <gi...@apache.org>]

openinx commented on pull request #3857:
URL: https://github.com/apache/iceberg/pull/3857#issuecomment-1007243161


   > Is this due to Flink 1.13 only (e.g. fixed / not known to occur in 1.14 or 1.12) or did you just choose to implement in 1.13 for another reason?
   
   The fix for `SerializableTable`  should be okay for all the flink versions.  I'm trying to add this test casest flink 1.13 firstly, it should also be applied to other flink versions. I think I will port those to flink 1.12  & flink 1.13 then.
   
   > The SerializableSupplier is written as is to avoid calls to get on the user-side. 
   
   I think the `SerializableSupplier` is mainly designed for resolving the hadoop `Configuration` serialization & deserialization issues.  We maintain all the key-value pairs inside the in-memory HashMap from the hadoop `Configuration` and then restore to the hadoop `Configuration`  once we've transferred those bytes to engines's DAG operators ( Saying `SerializableSupplier#get`).
   
   The current patch don't change the previous intentional design, instead it is trying to implement it in another way to fix the  flink kyro serialization issues. 
   
   Yes, I agree it's worth to port those test cases to other flink versions.
   
   Thanks.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org