You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "BBency (via GitHub)" <gi...@apache.org> on 2023/09/04 12:59:54 UTC
[GitHub] [hudi] BBency commented on issue #9094: Async Clustering failing with errors for MOR table
BBency commented on issue #9094:
URL: https://github.com/apache/hudi/issues/9094#issuecomment-1705233428
This issue is resolved now with help from AWS support team.
In my case, exception occurred because one or more of the parquet files to be clustered contained date type value before 1582-10-15. And so, in such case, Spark 3.0+ throws the error to avoid potential incompatibilities.
Setting the config "spark.sql.parquet.datetimeRebaseModeInWrite" to "CORRECTED" helped in resolving the issue.
I will go ahead and close the ticket
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org