You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "RushabhK (via GitHub)" <gi...@apache.org> on 2025/06/02 06:37:11 UTC
Re: [I] [VL] Native parquet writer writing a corrupt / invalid parquet file on spark task failure [incubator-gluten]
RushabhK commented on issue #9801:
URL: https://github.com/apache/incubator-gluten/issues/9801#issuecomment-2929038203
Hello @JkSelf, I did try running with this fix in my environment. Although now there are job failures with FileNotFoundException. Please find the error logs below.
`Caused by: java.io.FileNotFoundException: listStatus(hadoopPath: gs://<some_path>/.spark-staging-6815c37c-acfc-4167-95ba-cd9fefaf325a/_temporary/a6b6b0b2-bc38-4620-8ca2-18be8938fbc4/00/tasks/attempt_202505301505413719290422332255928_0002_m_000298_1958): 'gs://<some_path>/.spark-staging-6815c37c-acfc-4167-95ba-cd9fefaf325a/_temporary/a6b6b0b2-bc38-4620-8ca2-18be8938fbc4/00/tasks/attempt_202505301505413719290422332255928_0002_m_000298_1958' does not exist.
at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.listStatus(GoogleHadoopFileSystemBase.java:937)
at org.apache.hadoop.fs.FileSystem.listStatusBatch(FileSystem.java:2066)
at org.apache.hadoop.fs.FileSystem$DirListingIterator.<init>(FileSystem.java:2317)
at org.apache.hadoop.fs.FileSystem.listStatusIterator(FileSystem.java:2361)
at org.apache.hadoop.mapreduce.lib.output.committer.manifest.impl.ManifestStoreOperationsThroughFileSystem.listStatusIterator(ManifestStoreOperationsThroughFileSystem.java:126)
at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.AbstractJobOrTaskStage.lambda$listStatusIterator$5(AbstractJobOrTaskStage.java:488)
at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.invokeTrackingDuration(IOStatisticsBinding.java:547)
at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.lambda$trackDurationOfOperation$5(IOStatisticsBinding.java:528)
at org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.trackDuration(IOStatisticsBinding.java:449)
at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.AbstractJobOrTaskStage.listStatusIterator(AbstractJobOrTaskStage.java:487)
at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.TaskAttemptScanDirectoryStage.scanDirectoryTree(TaskAttemptScanDirectoryStage.java:148)
at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.TaskAttemptScanDirectoryStage.executeStage(TaskAttemptScanDirectoryStage.java:77)
at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.TaskAttemptScanDirectoryStage.executeStage(TaskAttemptScanDirectoryStage.java:51)
at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.AbstractJobOrTaskStage.apply(AbstractJobOrTaskStage.java:210)
at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.CommitTaskStage.executeStage(CommitTaskStage.java:66)
at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.CommitTaskStage.executeStage(CommitTaskStage.java:41)
at org.apache.hadoop.mapreduce.lib.output.committer.manifest.stages.AbstractJobOrTaskStage.apply(AbstractJobOrTaskStage.java:210)
at org.apache.hadoop.mapreduce.lib.output.committer.manifest.ManifestCommitter.commitTask(ManifestCommitter.java:310)
at org.apache.spark.mapred.SparkHadoopMapRedUtil$.$anonfun$commitTask$1(SparkHadoopMapRedUtil.scala:51)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.util.Utils$.timeTakenMs(Utils.scala:552)
at org.apache.spark.mapred.SparkHadoopMapRedUtil$.performCommit$1(SparkHadoopMapRedUtil.scala:51)
at org.apache.spark.mapred.SparkHadoopMapRedUtil$.commitTask(SparkHadoopMapRedUtil.scala:78)
at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.commitTask(HadoopMapReduceCommitProtocol.scala:401)
at org.apache.spark.sql.execution.SparkWriteFilesCommitProtocol.$anonfun$commitTask$1(SparkWriteFilesCommitProtocol.scala:91)
at org.apache.spark.util.Utils$.timeTakenMs(Utils.scala:552)
at org.apache.spark.sql.execution.SparkWriteFilesCommitProtocol.commitTask(SparkWriteFilesCommitProtocol.scala:91)
at org.apache.spark.sql.execution.VeloxColumnarWriteFilesRDD.$anonfun$compute$2(VeloxColumnarWriteFilesExec.scala:218)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1397)
at org.apache.spark.sql.execution.VeloxColumnarWriteFilesRDD.compute(VeloxColumnarWriteFilesExec.scala:201)
... 13 more
Caused by: java.io.FileNotFoundException: Item not found: gs://<some_path>/.spark-staging-6815c37c-acfc-4167-95ba-cd9fefaf325a/_temporary/a6b6b0b2-bc38-4620-8ca2-18be8938fbc4/00/tasks/attempt_202505301505413719290422332255928_0002_m_000298_1958
at com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.gcsio.GoogleCloudStorageFileSystem.listFileInfo(GoogleCloudStorageFileSystem.java:1052)
at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.listStatus(GoogleHadoopFileSystemBase.java:928)
... 43 more
Caused by: org.apache.spark.SparkException: Task failed while writing rows to staging path: gs://<some_path>/.spark-staging-6815c37c-acfc-4167-95ba-cd9fefaf325a, output path: gs://<some_path>/.spark-staging-6815c37c-acfc-4167-95ba-cd9fefaf325a`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org