[jira] [Commented] (SPARK-34658) Kryo serializer not working with Spark SQL module

["Hyukjin Kwon (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-34658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17298768#comment-17298768 ] 

Hyukjin Kwon commented on SPARK-34658:
--------------------------------------

Can you share how you reproduced this? IIRC, you should register the class to use Kyro. Let's interact with mailing list before filing it as an issue. I think you could have a better answer there.

> Kryo serializer not working with Spark SQL module
> -------------------------------------------------
>
>                 Key: SPARK-34658
>                 URL: https://issues.apache.org/jira/browse/SPARK-34658
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 3.0.1, 3.0.2, 3.1.1
>            Reporter: Nafis Sadat
>            Priority: Minor
>
> Hey all,
> I apologize if this has already been explored, but I keep getting this error whenever I have the Kryo registration required in my Spark app. The error is below; I have tried many a different types of combination to register this class but I just can't get anything to work, and so any help will be highly appreciated!
>  
> {code:java}
> Caused by: com.esotericsoftware.kryo.KryoException: java.lang.IllegalArgumentException: Class is not registered: org.apache.spark.sql.execution.datasources.InMemoryFileIndex$Seriali
> zableBlockLocation[]
> Note: To register this class use: kryo.register(org.apache.spark.sql.execution.datasources.InMemoryFileIndex$SerializableBlockLocation[].class);
> Serialization trace:
> blockLocations (org.apache.spark.sql.execution.datasources.InMemoryFileIndex$SerializableFileStatus)
> array (scala.collection.mutable.ArrayBuffer)
> {code}
>  
> I've tried all kinds of things to get these classes to be registered, but it's just not working for me..



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org