[GitHub] [hudi] BalaMahesh commented on issue #7595: [SUPPORT] Hudi Clean and Delta commits taking ~50 mins to finish frequently

[GitBox <gi...@apache.org>]

BalaMahesh commented on issue #7595:
URL: https://github.com/apache/hudi/issues/7595#issuecomment-1370487901

   This is a non-partitioned table with minimum file size set to 1 MB and ~150 parquet files are created.  Below are the screenshots from spark web ui. 
   <img width="1440" alt="Screenshot 2023-01-04 at 9 53 03 AM" src="https://user-images.githubusercontent.com/25053668/210485678-01569009-8b3c-4b62-8d04-7d5149cbdb7b.png">
   <img width="1440" alt="Screenshot 2023-01-04 at 9 53 24 AM" src="https://user-images.githubusercontent.com/25053668/210485684-b5314c87-a794-4242-889f-3a7a6623e395.png">
   <img width="1435" alt="Screenshot 2023-01-04 at 9 53 50 AM" src="https://user-images.githubusercontent.com/25053668/210485691-d3112ff4-739e-429a-bca1-3abd44394ba6.png">
   <img width="1375" alt="Screenshot 2023-01-04 at 10 04 45 AM" src="https://user-images.githubusercontent.com/25053668/210485699-26aeede3-24fa-46ec-9e4f-b07d87b842c9.png">
   <img width="1440" alt="Screenshot 2023-01-04 at 10 02 19 AM" src="https://user-images.githubusercontent.com/25053668/210485707-bdd31536-0399-4a89-98b8-2816ed1fdff3.png">
   
   I am adding the logs between 23/01/04 03:48:51  and 23/01/04 04:41:25 - which took longer duration for delta commit.
    
   [hudi_logs.txt](https://github.com/apache/hudi/files/10341543/hudi_logs.txt)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org