You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Jian Feng (Jira)" <ji...@apache.org> on 2021/09/10 18:04:00 UTC
[jira] [Created] (HUDI-2414) enable Hot and cold data separate when
ingest data
Jian Feng created HUDI-2414:
-------------------------------
Summary: enable Hot and cold data separate when ingest data
Key: HUDI-2414
URL: https://issues.apache.org/jira/browse/HUDI-2414
Project: Apache Hudi
Issue Type: Improvement
Components: Writer Core
Reporter: Jian Feng
when using Hudi to ingest e-commercial company's item data,there are massive update data into old partitions,if one record need update, then the whole file it belongs need rewrite, that result in every commit nearly rewrite the whole table.
I'm thinking if Hudi can provide a hot and cold data separate tool, work with specific column(such as create time and update time) to distinguish hot data and cold data, then rebuild table to separate them into different file groups, after recreate table, the performance will be much better
--
This message was sent by Atlassian Jira
(v8.3.4#803005)