You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Raymond Xu (Jira)" <ji...@apache.org> on 2021/11/22 08:27:00 UTC
[jira] [Updated] (HUDI-2818) 2to3 upgrade only checks hoodie.table.keygenerator.class
[ https://issues.apache.org/jira/browse/HUDI-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Raymond Xu updated HUDI-2818:
-----------------------------
Description:
{noformat}
21/11/22 05:26:10 INFO UpgradeDowngrade: Attempting to move table from version ONE to THREE
21/11/22 05:26:10 ERROR HoodieDeltaStreamer: Shutting down delta-sync due to exception
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:460)
at java.util.Properties.setProperty(Properties.java:166)
at org.apache.hudi.common.config.HoodieConfig.setValue(HoodieConfig.java:58)
at org.apache.hudi.table.upgrade.UpgradeDowngrade.run(UpgradeDowngrade.java:132){noformat}
2to3 looks for config
{noformat}
hoodie.table.keygenerator.class{noformat}
and will result in putting null value to hashtable if user only sets
{noformat}
hoodie.datasource.write.keygenerator.class{noformat}
solution:
2to3 should check both props to set. And use {{hashtable}} in upgrade/downgrade handlers instead of hashmap. in case of getting null value, hashtable will error out earlier.
was:
21/11/22 05:26:10 INFO UpgradeDowngrade: Attempting to move table from version ONE to THREE
21/11/22 05:26:10 ERROR HoodieDeltaStreamer: Shutting down delta-sync due to exception
java.lang.NullPointerException
at java.util.Hashtable.put(Hashtable.java:460)
at java.util.Properties.setProperty(Properties.java:166)
at org.apache.hudi.common.config.HoodieConfig.setValue(HoodieConfig.java:58)
at org.apache.hudi.table.upgrade.UpgradeDowngrade.run(UpgradeDowngrade.java:132)
2to3 looks for config
hoodie.table.keygenerator.class
and will result in putting null value to hashtable if user only sets
hoodie.datasource.write.keygenerator.class
solution:
2to3 should check both props to set. And use {{hashtable}} in upgrade/downgrade handlers instead of hashmap. in case of getting null value, hashtable will error out earlier.
> 2to3 upgrade only checks hoodie.table.keygenerator.class
> --------------------------------------------------------
>
> Key: HUDI-2818
> URL: https://issues.apache.org/jira/browse/HUDI-2818
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Raymond Xu
> Assignee: Raymond Xu
> Priority: Blocker
> Fix For: 0.10.0
>
>
> {noformat}
> 21/11/22 05:26:10 INFO UpgradeDowngrade: Attempting to move table from version ONE to THREE
> 21/11/22 05:26:10 ERROR HoodieDeltaStreamer: Shutting down delta-sync due to exception
> java.lang.NullPointerException
> at java.util.Hashtable.put(Hashtable.java:460)
> at java.util.Properties.setProperty(Properties.java:166)
> at org.apache.hudi.common.config.HoodieConfig.setValue(HoodieConfig.java:58)
> at org.apache.hudi.table.upgrade.UpgradeDowngrade.run(UpgradeDowngrade.java:132){noformat}
>
> 2to3 looks for config
> {noformat}
> hoodie.table.keygenerator.class{noformat}
> and will result in putting null value to hashtable if user only sets
> {noformat}
> hoodie.datasource.write.keygenerator.class{noformat}
>
> solution:
> 2to3 should check both props to set. And use {{hashtable}} in upgrade/downgrade handlers instead of hashmap. in case of getting null value, hashtable will error out earlier.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)