You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Rajesh (Jira)" <ji...@apache.org> on 2022/04/01 22:07:00 UTC
[jira] (HUDI-3726) Switching from non-partitioned to partitioned key gen does not throw any exception
[ https://issues.apache.org/jira/browse/HUDI-3726 ]
Rajesh deleted comment on HUDI-3726:
------------------------------
was (Author: JIRAUSER285685):
scala> df.write.format("hudi").
| options(getQuickstartWriteConfigs).
| option(PRECOMBINE_FIELD_OPT_KEY, "ts").
| option(RECORDKEY_FIELD_OPT_KEY, "uuid").
| option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
| option(TABLE_NAME, tableName).
| option(HoodieWriteConfig.KEYGENERATOR_CLASS_NAME.key(), "org.apache.hudi.keygen.NonpartitionedKeyGenerator").
| mode(Overwrite).
| save(basePath)
warning: there was one deprecation warning; for details, enable `:setting -deprecation' or `:replay -deprecation'
22/03/31 17:32:20 WARN DFSPropertiesConfiguration: Cannot find HUDI_CONF_DIR, please set it as the dir of hudi-defaults.conf
22/03/31 17:32:20 WARN DFSPropertiesConfiguration: Properties file file:/etc/hudi/conf/hudi-defaults.conf not found. Ignoring to load props file
scala> df.write.format("hudi").
| options(getQuickstartWriteConfigs).
| option(PRECOMBINE_FIELD_OPT_KEY, "ts").
| option(RECORDKEY_FIELD_OPT_KEY, "uuid").
| option(PARTITIONPATH_FIELD_OPT_KEY, "partitionpath").
| option(TABLE_NAME, tableName).
| mode(Append).
| save(basePath)
[~shivnarayan] The key gen is staying as hoodie.table.keygenerator.class=org.apache.hudi.keygen.NonpartitionedKeyGenerator and not switching to SimpleKeyGenerator
Is that still an isssue.
> Switching from non-partitioned to partitioned key gen does not throw any exception
> ----------------------------------------------------------------------------------
>
> Key: HUDI-3726
> URL: https://issues.apache.org/jira/browse/HUDI-3726
> Project: Apache Hudi
> Issue Type: Bug
> Components: spark
> Reporter: sivabalan narayanan
> Assignee: Rajesh
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.11.0
>
> Attachments: Screen Shot 2022-04-01 at 12.28.12 PM.png
>
>
> in commit C1, if non-partitioned key gen is used and for commit C2, if key gen option is missed, we fallback to simple key gen and the write succeeds. I was expecting to throw exception if key gen class is changed.
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)