Re: [PR] AVRO-4170: Improve sync marker error message in DataFileStream [avro]

["wangyum (via GitHub)" <gi...@apache.org>]

wangyum commented on PR #3446:
URL: https://github.com/apache/avro/pull/3446#issuecomment-3142079889

   Current error message:
   ```
   25/07/29 01:07:53 INFO Executor: Executor killed task 178.0 in stage 1.0 (TID 535), reason: Stage cancelled: Job aborted due to stage failure: Task 52 in stage 1.0 failed 4 times, most recent failure: Lost task 52.3 in stage 1.0 (TID 1232) (my.executor.host.com executor 74): org.apache.avro.AvroRuntimeException: java.io.IOException: Invalid sync!
   	at org.apache.avro.file.DataFileStream.hasNext(DataFileStream.java:236)
   	at org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader.next(AvroGenericRecordReader.java:160)
   	at org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader.next(AvroGenericRecordReader.java:54)
   	at org.apache.spark.rdd.HadoopRDD$$anon$1.getNext(HadoopRDD.scala:324)
   	at org.apache.spark.rdd.HadoopRDD$$anon$1.getNext(HadoopRDD.scala:246)
   	at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:73)
   	at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
   	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
   	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
   	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
   	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.hashAgg_doAggregateWithKeys_0$(Unknown Source)
   	at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown Source)
   	at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:51)
   	at org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43)
   	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
   	at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:140)
   	at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
   	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:104)
   	at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:54)
   	at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:161)
   	at org.apache.spark.scheduler.Task.run(Task.scala:147)
   	at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:622)
   	at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
   	at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
   	at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@avro.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org