You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@spark.apache.org by Justin Uang <ju...@gmail.com> on 2015/05/22 17:14:41 UTC

UDTs and StringType upgrade issue for Spark 1.4.0

We ran into an issue regarding Strings in UDTs when upgrading to Spark
1.4.0-rc. I understand that it's a non-public APIs, so it's expected, but I
just wanted to bring it up for awareness and so we can maybe change the
release notes to mention them =)

Our UDT was serializing to a StringType, but now strings are represented
internally as UTF8String, so we had to change our UDT to use
UTF8String.apply() and UTF8String.toString() to convert back to String.