You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "sivabalan narayanan (Jira)" <ji...@apache.org> on 2020/08/13 13:03:34 UTC
[jira] [Created] (HUDI-1187) Improvements/Follow up on Bulk Insert
V2
sivabalan narayanan created HUDI-1187:
-----------------------------------------
Summary: Improvements/Follow up on Bulk Insert V2
Key: HUDI-1187
URL: https://issues.apache.org/jira/browse/HUDI-1187
Project: Apache Hudi
Issue Type: Improvement
Components: Writer Core
Reporter: sivabalan narayanan
* Add java docs to KeyGeneratorInterface methods.
* validateRecordKeyFields() in CustomKeyGenerator could be moved up and used by other keyGens. Check and fix it.
* Unify usage of getters. For eg. in SimpleKeyGen we have something like
RowKeyGeneratorHelper.getRecordKeyFromRow(row, getRecordKeyFields(), recordKeyPositions, false);
for recordKeyFields, we use getRecordKeyFields(), where as for recordKeyPositions we use instance variable directly. Make it uniform in all key gen classes.
* Remove line 82 in [TestGlobalDeleteKeyGenerator.java|https://github.com/apache/hudi/commit/5dc8182ec308dba7ffd04ef159bd3041ede1b117#diff-4c306975590fe7bf2b27a6f5a9d9ff7e]
keyGenerator.buildFieldPositionMapIfNeeded(KeyGeneratorTestUtilities.structType);
* make buildFieldPositionMapIfNeeded(StructType structType) in BuildInKeyGen as protected.
* Introduce private method (and re-use) to generate positions for recordkeys and partition paths.
* boolean positionMapInitialized in buildMapPositionsIfNeeded
--
This message was sent by Atlassian Jira
(v8.3.4#803005)