You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yuemeng (Jira)" <ji...@apache.org> on 2022/11/11 02:52:00 UTC
[jira] [Updated] (HUDI-5195) FileIOUtils readDataFromPath will cut off data if the commit metadata is too huge
[ https://issues.apache.org/jira/browse/HUDI-5195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
yuemeng updated HUDI-5195:
--------------------------
Summary: FileIOUtils readDataFromPath will cut off data if the commit metadata is too huge (was: FileIOUtils readDataFromPath will cut off data if the commit data in hdfs is too huge)
> FileIOUtils readDataFromPath will cut off data if the commit metadata is too huge
> ---------------------------------------------------------------------------------
>
> Key: HUDI-5195
> URL: https://issues.apache.org/jira/browse/HUDI-5195
> Project: Apache Hudi
> Issue Type: Bug
> Components: core
> Reporter: yuemeng
> Assignee: yuemeng
> Priority: Major
>
> Currently, FileIOUtils readDataFromPath will cut off data if the commit data in HDFS is too huge, so it will return an invalid byte array that can't covert to hoodie commit metadata with exception such as follow
> {code}
> org.apache.hudi.com.fasterxml.jackson.databind.JsonMappingException: Unexpected end-of-input: was expecting closing '"' for name
> at [Source: java.io.StringReader@4c339eb1; line: 5752, column: 1546] (through reference chain: org.apache.hudi.common.model.HoodieCommitMetadata["writeStats"]->java.util.ArrayList[168])
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)