Re: [PR] [GLUTEN-7953][VL] Fetch and dump all inputs for micro benchmark on middle stage begin [incubator-gluten]

["marin-ma (via GitHub)" <gi...@apache.org>]

marin-ma commented on PR #7998:
URL: https://github.com/apache/incubator-gluten/pull/7998#issuecomment-2487179205

   > @marin-ma Curious what're the improvements here, comparing to #725? Thanks.
   
   @zhztheplayer Currently, the input files are being dumped during the middle stage of execution. This PR will fetch all the data and save it into file before the pipeline starts. The stage input will then be read from this dumped file. This will help to save a complete input file even if the task fails. In this way, we can have the full input data of the failed task, and reproduce the failure with microbenchmark.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org