You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Danny Chen (Jira)" <ji...@apache.org> on 2023/01/31 05:37:00 UTC
[jira] [Closed] (HUDI-5487) Reduce duplicate Logs in ExternalSpillableMap
[ https://issues.apache.org/jira/browse/HUDI-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Danny Chen closed HUDI-5487.
----------------------------
Reviewers: Danny Chen
Resolution: Fixed
Fixed via master branch: 1377143656a447ddd6f884380c93be6fb5ecf459
> Reduce duplicate Logs in ExternalSpillableMap
> ---------------------------------------------
>
> Key: HUDI-5487
> URL: https://issues.apache.org/jira/browse/HUDI-5487
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: dzcxzl
> Assignee: Danny Chen
> Priority: Minor
> Labels: pull-request-available
> Fix For: 0.13.1
>
>
> We see hundreds of thousands of duplicate logs in the executor log.
> {code:java}
> 22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
> 22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
> 22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
> 22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
> 22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
> 22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567
> 22/12/26 21:13:40,864 [Executor task launch worker for task 0.0 in stage 480.0 (TID 211376)] INFO ExternalSpillableMap: Update Estimated Payload size to => 4567 {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)