optional binary element (UTF8) is not a group

["Rajan, Naveen" <Na...@sony.com>]

Hi  All,
    My reports were working fine with JSON data formats. Even with the Parquet formats I am able to see the schema, counts & even the data via, show(). However, reports are crashing with the below stack-trace, especially when I have data in languages like Japanese.

org.apache.spark.SparkException: Job aborted due to stage failure: Task 10 in stage 2023.0 failed 4 times, most recent failure: Lost task 10.3 in stage 2023.0 (TID 17158, ip-172-31-12-157.us-west-2.compute.internal): java.lang.ClassCastException: optional binary element (UTF8) is not a group
    at org.apache.parquet.schema.Type.asGroupType(Type.java:202)
    at org.apache.spark.sql.execution.datasources.parquet.ParquetReadSupport$.org$apache$spark$sql$execution$datasources$parquet$ParquetReadSupport$$clipParquetType(ParquetReadSupport.scala:131)


Regards,
Naveen


________________________________

This email is confidential and intended only for the use of the individual or entity named above and may contain information that is privileged. If you are not the intended recipient, you are notified that any dissemination, distribution or copying of this email is strictly prohibited. If you have received this email in error, please notify us immediately by return email or telephone and destroy the original message. - This mail is sent via Sony Asia Pacific Mail Gateway..