You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/08/03 11:18:36 UTC
[GitHub] [hudi] springMoon opened a new issue #3392: [SUPPORT] Compile hudi master with hive version 2.2.1 error
springMoon opened a new issue #3392:
URL: https://github.com/apache/hudi/issues/3392
**_Tips before filing an issue_**
- Have you gone through our [FAQs](https://cwiki.apache.org/confluence/display/HUDI/FAQ)?
- Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.
- If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly.
**Describe the problem you faced**
Compile hudi master with hive version 2.2.1 error
**To Reproduce**
Steps to reproduce the behavior:
1. pull hudi master branch
2. compile it
command:
```
mvn clean package -DskipTests -Drat.skip=true -Pflink-bundle-shade-hive2,include-flink-sql-connector-hive -Dhadoop.version=3.0.0 -Dhive.version=2.1.1
```
**Expected behavior**
Compile hudi master with hive version 2.2.1 error
**Environment Description**
* Hudi version : master
* Spark version :
* Hive version : 2.1.1
* Hadoop version :3.0.0
* Storage (HDFS/S3/GCS..) : hdfs
* Running on Docker? (yes/no) : no
**Additional context**
use default hive version package, when sync hudi metadata to hive, will throw hive "Required field 'client_protocol' is unset!" exception, and sync fail
**Stacktrace**
```
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 25.462 s
[INFO] Finished at: 2021-08-03T19:15:30+08:00
[INFO] Final Memory: 119M/1420M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project hudi-hadoop-mr: Compilation failure
[ERROR] /home/wuxu/git/hudi/hudi-hadoop-mr/src/main/java/org/apache/hudi/hadoop/hive/HoodieCombineHiveInputFormat.java:[32,37] cannot find symbol
[ERROR] symbol: class StringInternUtils
[ERROR] location: package org.apache.hadoop.hive.common
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :hudi-hadoop-mr
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on issue #3392: [SUPPORT] Compile hudi master with hive version 2.1.1 error
Posted by GitBox <gi...@apache.org>.
xushiyan commented on issue #3392:
URL: https://github.com/apache/hudi/issues/3392#issuecomment-950421639
Close due to inactive
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan closed issue #3392: [SUPPORT] Compile hudi master with hive version 2.1.1 error
Posted by GitBox <gi...@apache.org>.
xushiyan closed issue #3392:
URL: https://github.com/apache/hudi/issues/3392
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jslorrma edited a comment on issue #3392: [SUPPORT] Compile hudi master with hive version 2.2.1 error
Posted by GitBox <gi...@apache.org>.
jslorrma edited a comment on issue #3392:
URL: https://github.com/apache/hudi/issues/3392#issuecomment-901289333
@springMoon Shouldn't the hive version in your title be `2.1.1`? I think a version `2.2.1` for hive does not exist and your compilation command as well as the environment description says version `2.1.1`.
Nevertheless I face the same issue when following [this comment](https://github.com/apache/hudi/issues/1376#issuecomment-595301096) and skipping the second step
of removing `HoodieCombineHiveInputFormat.java`. When I do remove `HoodieCombineHiveInputFormat.java` and try the compilation, it fails complaining about the missing class `HoodieCombineHiveInputFormat`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jslorrma commented on issue #3392: [SUPPORT] Compile hudi master with hive version 2.2.1 error
Posted by GitBox <gi...@apache.org>.
jslorrma commented on issue #3392:
URL: https://github.com/apache/hudi/issues/3392#issuecomment-901289333
@springMoon Shouldn't the hive version in your title be `2.1.1`? I think a version `2.2.1` for hive does not exist and your compilation command as well as the environment description says version `2.1.1`.
Nevertheless I face the same issue when following [this comment](https://github.com/apache/hudi/issues/1376#issuecomment-595301096) and skipping the second step (removing `HoodieCombineHiveInputFormat.java`), because when I remove `HoodieCombineHiveInputFormat.java` compilation fails complaining about missing class `HoodieCombineHiveInputFormat`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on issue #3392: [SUPPORT] Compile hudi master with hive version 2.1.1 error
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on issue #3392:
URL: https://github.com/apache/hudi/issues/3392#issuecomment-905188331
@danny0405 @leesf : Can you loop in someone who can look into this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] danny0405 commented on issue #3392: [SUPPORT] Compile hudi master with hive version 2.1.1 error
Posted by GitBox <gi...@apache.org>.
danny0405 commented on issue #3392:
URL: https://github.com/apache/hudi/issues/3392#issuecomment-905324993
You only need to package the flink bundle jar with hive 2.1.1, there is no need to package the mr-bundle first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on issue #3392: [SUPPORT] Compile hudi master with hive version 2.1.1 error
Posted by GitBox <gi...@apache.org>.
xushiyan commented on issue #3392:
URL: https://github.com/apache/hudi/issues/3392#issuecomment-926354023
> You only need to package the flink bundle jar with hive 2.1.1, there is no need to package the mr-bundle first.
@springMoon Did you get this resolved by the suggestion? Please close this if it worked for you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan edited a comment on issue #3392: [SUPPORT] Compile hudi master with hive version 2.1.1 error
Posted by GitBox <gi...@apache.org>.
nsivabalan edited a comment on issue #3392:
URL: https://github.com/apache/hudi/issues/3392#issuecomment-905188331
@danny0405 @leesf : Can you please loop in someone who can look into this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on issue #3392: [SUPPORT] Compile hudi master with hive version 2.1.1 error
Posted by GitBox <gi...@apache.org>.
codope commented on issue #3392:
URL: https://github.com/apache/hudi/issues/3392#issuecomment-905318597
I checked Hive repo. `StringInternUtils` was only added in 2.3. Will this Hive version work for you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org