You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "gamblewin (via GitHub)" <gi...@apache.org> on 2023/03/17 10:09:01 UTC
[GitHub] [hudi] gamblewin opened a new issue, #8216: [SUPPORT]Compile Error of hudi-0.13.0 "org.apache.hadoop.hive.common.type can not be converted to java.sql.Date"
gamblewin opened a new issue, #8216:
URL: https://github.com/apache/hudi/issues/8216
When compiling source code of hudi-0.13.0, report compile error:
"HiveAvroSerializer.java[302, 93]: org.apache.hadoop.hive.common.type can not be converted to java.sql.Date"
* Hudi version :0.13.0
* Hive version :3.1.3
* Hadoop version :3.0.0
* Running on Docker? (yes/no) :no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] gamblewin commented on issue #8216: [SUPPORT]Compile Error of hudi-0.13.0 "org.apache.hadoop.hive.common.type can not be converted to java.sql.Date"
Posted by "gamblewin (via GitHub)" <gi...@apache.org>.
gamblewin commented on issue #8216:
URL: https://github.com/apache/hudi/issues/8216#issuecomment-1486305373
> you can try [#7173](https://github.com/apache/hudi/pull/7173)
thx, this solved my problem. Btw, line 312 also needs to be changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] gamblewin closed issue #8216: [SUPPORT]Compile Error of hudi-0.13.0 "org.apache.hadoop.hive.common.type can not be converted to java.sql.Date"
Posted by "gamblewin (via GitHub)" <gi...@apache.org>.
gamblewin closed issue #8216: [SUPPORT]Compile Error of hudi-0.13.0 "org.apache.hadoop.hive.common.type can not be converted to java.sql.Date"
URL: https://github.com/apache/hudi/issues/8216
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xicm commented on issue #8216: [SUPPORT]Compile Error of hudi-0.13.0 "org.apache.hadoop.hive.common.type can not be converted to java.sql.Date"
Posted by "xicm (via GitHub)" <gi...@apache.org>.
xicm commented on issue #8216:
URL: https://github.com/apache/hudi/issues/8216#issuecomment-1477191650
you can try [#7173](https://github.com/apache/hudi/pull/7173)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] gamblewin commented on issue #8216: [SUPPORT]Compile Error of hudi-0.13.0 "org.apache.hadoop.hive.common.type can not be converted to java.sql.Date"
Posted by "gamblewin (via GitHub)" <gi...@apache.org>.
gamblewin commented on issue #8216:
URL: https://github.com/apache/hudi/issues/8216#issuecomment-1486310176
> @gamblewin I am able to reproduce the error . Any specific reason why you updating hive version to 3.1.3.
> When I tried to set hive version to 3.0.0, the hudi-hadoop-mr build got succeeded.
I've solved this problem according to #7173. Actually, it's because someone else has already installed hive 3.1.3 in the environment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] IntronsYanh commented on issue #8216: [SUPPORT]Compile Error of hudi-0.13.0 "org.apache.hadoop.hive.common.type can not be converted to java.sql.Date"
Posted by "IntronsYanh (via GitHub)" <gi...@apache.org>.
IntronsYanh commented on issue #8216:
URL: https://github.com/apache/hudi/issues/8216#issuecomment-1488144032
> > you can try [#7173](https://github.com/apache/hudi/pull/7173)
>
> thx, this solved my problem. Btw, line 312 also needs to be changed.
Hi gamblewin, may I have your modified for line 312?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] gamblewin commented on issue #8216: [SUPPORT]Compile Error of hudi-0.13.0 "org.apache.hadoop.hive.common.type can not be converted to java.sql.Date"
Posted by "gamblewin (via GitHub)" <gi...@apache.org>.
gamblewin commented on issue #8216:
URL: https://github.com/apache/hudi/issues/8216#issuecomment-1477291636
> @gamblewin Can you share the stacktrace of the error?
> Also share the maven version, java version and the command used to compile. Please also mention the branch you are building against.
@lokeshj1703 Thx for replying.
- Maven Version: 3.9.0
- Java Version: 1.8.0_331
- Hudi Branch: 0.13.0
- Change I did to `pom.xml`
`<hadoop.version>3.0.0</hadoop.version>`
`<hive.version>3.1.3</hive.version>`
- Compile Comand: `mvn clean package -DskipTests -Dflink1.13 -Dscala-2.12 -Dhadoop.version=3.0.0 -Pflink-bundle-shade-hive3`
- Stacktrace
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project hudi-hadoop-mr: Compilation failure: Compilation failure:
[ERROR] /Users/gamblewin/bigdata/hudi-0.13.0-test/hudi-0.13.0/hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HiveAvroSerializer.java:[302,93] 不兼容的类型: org.apache.hadoop.hive.common.type.Date无法转换为java.sql.Date
[ERROR] /Users/gamblewin/bigdata/hudi-0.13.0-test/hudi-0.13.0/hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HiveAvroSerializer.java:[305,72] 不兼容的类型: org.apache.hadoop.hive.common.type.Timestamp无法转换为java.sql.Timestamp
[ERROR] /Users/gamblewin/bigdata/hudi-0.13.0-test/hudi-0.13.0/hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/utils/HiveAvroSerializer.java:[309,98] 不兼容的类型: org.apache.hadoop.hive.common.type.Date无法转换为java.sql.Date
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project hudi-hadoop-mr: Compilation failure
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:347)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:330)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:213)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:175)
at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:76)
at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:163)
at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:160)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:260)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:172)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:100)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:821)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:270)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure
at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute (AbstractCompilerMojo.java:1215)
at org.apache.maven.plugin.compiler.CompilerMojo.execute (CompilerMojo.java:188)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:126)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:342)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:330)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:213)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:175)
at org.apache.maven.lifecycle.internal.MojoExecutor.access$000 (MojoExecutor.java:76)
at org.apache.maven.lifecycle.internal.MojoExecutor$1.run (MojoExecutor.java:163)
at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute (DefaultMojosExecutionStrategy.java:39)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:160)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:105)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:260)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:172)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:100)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:821)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:270)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <args> -rf :hudi-hadoop-mr
- Update
If I compile with command `mvn clean package -DskipTests`, everything works fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] lokeshj1703 commented on issue #8216: [SUPPORT]Compile Error of hudi-0.13.0 "org.apache.hadoop.hive.common.type can not be converted to java.sql.Date"
Posted by "lokeshj1703 (via GitHub)" <gi...@apache.org>.
lokeshj1703 commented on issue #8216:
URL: https://github.com/apache/hudi/issues/8216#issuecomment-1476004797
@gamblewin Can you share the stacktrace of the error?
Also share the maven version, java version and the command used to compile. Please also mention the branch you are building against.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] ad1happy2go commented on issue #8216: [SUPPORT]Compile Error of hudi-0.13.0 "org.apache.hadoop.hive.common.type can not be converted to java.sql.Date"
Posted by "ad1happy2go (via GitHub)" <gi...@apache.org>.
ad1happy2go commented on issue #8216:
URL: https://github.com/apache/hudi/issues/8216#issuecomment-1486258724
@gamblewin I am able to reproduce the error . Any specific reason why you updating hive version to 3.1.3.
When I tried to set hive version to 3.0.0, the hudi-hadoop-mr build got succeeded.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] gamblewin commented on issue #8216: [SUPPORT]Compile Error of hudi-0.13.0 "org.apache.hadoop.hive.common.type can not be converted to java.sql.Date"
Posted by "gamblewin (via GitHub)" <gi...@apache.org>.
gamblewin commented on issue #8216:
URL: https://github.com/apache/hudi/issues/8216#issuecomment-1486303249
> you can try [#7173](https://github.com/apache/hudi/pull/7173)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] gamblewin commented on issue #8216: [SUPPORT]Compile Error of hudi-0.13.0 "org.apache.hadoop.hive.common.type can not be converted to java.sql.Date"
Posted by "gamblewin (via GitHub)" <gi...@apache.org>.
gamblewin commented on issue #8216:
URL: https://github.com/apache/hudi/issues/8216#issuecomment-1488922874
> > > you can try [#7173](https://github.com/apache/hudi/pull/7173)
> >
> >
> > thx, this solved my problem. Btw, line 312 also needs to be changed.
>
> Hi gamblewin, may I have your modified for line 312?
```java
case INT:
if (schema.getLogicalType() != null && schema.getLogicalType().getName().equals("date")) {
// - return DateWritable.dateToDays(new WritableDateObjectInspector().getPrimitiveJavaObject(structFieldData));
return new DateWritable((DateWritable)structFieldData).getDays();
}
```
but I don't know if I did the right change, cuz after this alteration, compilation succeeded. But when I want to insert data into a hudi cdc COW table, it will report mistake.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org