You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/10/13 08:32:24 UTC
[GitHub] [hudi] matthiasdg opened a new issue, #6937: [SUPPORT] command line HiveSyncTool: parsing with JCommander broken
matthiasdg opened a new issue, #6937:
URL: https://github.com/apache/hudi/issues/6937
**Describe the problem you faced**
Upgraded to hudi 0.12 here (from 0.10) and it seems some things changed to the HiveSyncTool, more specifically to HiveSyncConfig.java
Where before there were defaults for some Boolean parameters, this is now no longer the case. As an example:
```
@Parameter(names = {"--spark-datasource"}, description = "Whether sync this table as spark data source table.")
public Boolean syncAsSparkDataSourceTable;
```
used to be
```
@Parameter(names = {"--spark-datasource"}, description = "Whether sync this table as spark data source table.")
public Boolean syncAsSparkDataSourceTable = true;
```
When I try to run the synctool commandline, I now get NPE's as soon as I include one of these Boolean parameters.
I found https://github.com/cbeust/jcommander/issues/447
This does not appear to be fixed in later JCommander releases at first sight (did a quick test overriding the dependencies).
Haven't tested yet whether switching to boolean as suggested in that issue fixes it. Reverting to default values as before also has the issue https://github.com/cbeust/jcommander/issues/378 (so it then just flips the set default which is weird). Specifying an arity 1 so the value has to be set is maybe another option (https://jcommander.org/).
**To Reproduce**
Steps to reproduce the behavior:
1. Run HiveSyncTool main and specify one of the boolean parameters; e.g. --spark-datasource, --ignore-exceptions...
2. a NullPointerException will be thrown
**Expected behavior**
Should run fine
**Environment Description**
* Hudi version : 0.12
**Stacktrace**
```
java.lang.NullPointerException:
[info] at org.apache.hudi.com.beust.jcommander.JCommander.parseValues(JCommander.java:718)
[info] at org.apache.hudi.com.beust.jcommander.JCommander.parse(JCommander.java:340)
[info] at org.apache.hudi.com.beust.jcommander.JCommander.parse(JCommander.java:319)
[info] at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:353)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on issue #6937: [SUPPORT] command line HiveSyncTool: parsing with JCommander broken
Posted by GitBox <gi...@apache.org>.
xicm commented on issue #6937:
URL: https://github.com/apache/hudi/issues/6937#issuecomment-1291598519
Hi, I can fix this bug.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan closed issue #6937: [SUPPORT] command line HiveSyncTool: parsing with JCommander broken
Posted by GitBox <gi...@apache.org>.
xushiyan closed issue #6937: [SUPPORT] command line HiveSyncTool: parsing with JCommander broken
URL: https://github.com/apache/hudi/issues/6937
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on issue #6937: [SUPPORT] command line HiveSyncTool: parsing with JCommander broken
Posted by GitBox <gi...@apache.org>.
yihua commented on issue #6937:
URL: https://github.com/apache/hudi/issues/6937#issuecomment-1278592001
@matthiasdg thanks for reporting the issue. We'll try to reproduce it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on issue #6937: [SUPPORT] command line HiveSyncTool: parsing with JCommander broken
Posted by GitBox <gi...@apache.org>.
xushiyan commented on issue #6937:
URL: https://github.com/apache/hudi/issues/6937#issuecomment-1293550491
> #7068 @yihua @xushiyan Have a review, if you are available.
Thanks for taking this up @xicm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on issue #6937: [SUPPORT] command line HiveSyncTool: parsing with JCommander broken
Posted by GitBox <gi...@apache.org>.
xicm commented on issue #6937:
URL: https://github.com/apache/hudi/issues/6937#issuecomment-1291642427
@yihua @xushiyan Have a review, if you are available.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org