Re: [I] Comet native broadcast fails under spark.kryo.registrationRequired=true [datafusion-comet]

["dwsmith1983 (via GitHub)" <gi...@apache.org>]

dwsmith1983 commented on issue #5510:
URL: https://github.com/apache/datafusion-comet/issues/5510#issuecomment-5636864414

   #5051 (492dd6f2a) landed the fix this issue was split out to track. `CometKryoRegistrator` in spark/src/main/scala/org/apache/comet/CometKryoRegistrator.scala registers the `Array[ChunkedByteBuffer]` a native broadcast serializes, `CometDriverPlugin` logs a startup warning when Kryo registration is required but the registrator is not configured (spark/src/main/scala/org/apache/spark/Plugins.scala, `warnIfKryoRegistratorMissing`), and `CometInMemoryCacheKryoSuite` has a dedicated case, "Comet broadcast exchange survives Kryo with registration required", that runs with the in-memory cache disabled so it exercises only the broadcast path. Suggest closing.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org