[GitHub] [hudi] hbgstc123 commented on issue #8674: [SUPPORT]xxx.parquet is not a Parquet file (length is too low: 0)

["hbgstc123 (via GitHub)" <gi...@apache.org>]

hbgstc123 commented on issue #8674:
URL: https://github.com/apache/hudi/issues/8674#issuecomment-1539795361

   ### Here is what I find from log of JM and TM log
   
   trigger checkpoint 3574 -> Completed checkpoint 3574 -> trigger checkpoint 3575
   
   -> TM AppendWriteFunction fail due to `Timeout(601000ms) while waiting for instant initialize`
   
   -> checkpoint 3575 failed due to TM fail
   
   -> TM restart, send bootstrap event to `StreamWriteOperatorCoordinator`
   
   -> `StreamWriteOperatorCoordinator` generate new commit 20230504102505338 (trigger by checkpoint 3574 complete)
   
   -> `StreamWriteOperatorCoordinator` receive bootstrap event from TM restart, start to rollback commit 20230504102505338 and start a new instant
   
   -> **`AppendWriteFunction` start to write with instant 20230504102505338(which is wrong, should write with the new instant after 20230504102505338 is rollbacked)**, and these new written data files of 20230504102505338 escape from rollback
   
   -> instant 20230504102505338 rollback complete, start a new instant 20230504102509969
   
   -> trigger checkpoint 3576 -> Completed checkpoint 3576
   
   -> start to commit 20230504102509969
   
   -> `AppendWriteFunction` start to write with instant 20230504102509969(which is wrong)
   
   -> delete invalid data files of 20230504102509969, part of data files wrong writing is deleted, cause write conflict error `File does not exist: ...... [Lease. Holder: DFSClient_NONMAPREDUCE_-536912522_86, pending creates: 5]`
   
   -> marker dir .temp/20230504102509969 removed leaving some 20230504102509969 data file of length 0 on hdfs
   
   -> schedule clustering c1, include those escaped data files of 20230504102509969.
   
   -> next time clustering run instant c1, will report this error.
   
   So the the cause is from when TM get the wrong instant 20230504102505338 to write, so I submit a pr to fix this: https://github.com/apache/hudi/pull/8673


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org