You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Meghali Dhoble <dh...@us.ibm.com> on 2018/08/29 11:51:14 UTC
Hive build from source is failing on ppc64le
Hi Hive-Team,
I have been working to get the hive-jdbc built on the power platform.
However it's failing for me with an error which indicates that a particular
binary is not present at the maven-repository for ppc64le. I tried
replacing the binary at ~/.m2/repository location, however that didn't help
me.
I am unable to understand what binary it is looking for and is there a way
to manually build it and provide the path to it.
Could someone please help me understand this issue better?
OS - Linux (Ubuntu 18.04)
Architecture - ppc64le
The error log is as below:
[ERROR] Failed to execute goal
com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project
hive-standalone-metastore: Error resolving artifact:
com.google.protobuf:protoc:2.5.0: Failure to find
com.google.protobuf:protoc:exe:linux-ppcle_64:2.5.0 in
https://repo.maven.apache.org/maven2 was cached in the local repository,
resolution will not be reattempted until the update interval of central
has elapsed or updates are forced
[ERROR]
[ERROR] Try downloading the file manually from the project website.
[ERROR]
[ERROR] Then, install it using the command:
[ERROR] mvn install:install-file -DgroupId=com.google.protobuf
-DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-ppcle_64
-Dpackaging=exe -Dfile=/path/to/file
[ERROR]
[ERROR] Alternatively, if you host your own repository you can deploy
the file there:
[ERROR] mvn deploy:deploy-file -DgroupId=com.google.protobuf
-DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-ppcle_64
-Dpackaging=exe -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
[ERROR]
[ERROR]
[ERROR] com.google.protobuf:protoc:exe:2.5.0
Thanks,
Meghali Dhoble
Re: Hive build from source is failing on ppc64le
Posted by Meghali Dhoble <dh...@us.ibm.com>.
Hi Zoltan,
I gave a try to standalone-jdbc jar downloaded and tried building it and
its failing for me with different error.
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-antrun-plugin:1.7:run (setup-test-dirs)
on project hive-jdbc: An Ant BuildException has occured: /data/conf does
not exist.
[ERROR] around Ant part ...<copy
todir="/hive-standalone/target/testconf">... @ 10:50
in /hive-standalone/target/antrun/build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run
(setup-test-dirs) on project hive-jdbc: An Ant BuildException has
occured: /data/conf does not exist.
around Ant part ...<copy todir="/hive-standalone/target/testconf">... @
10:50 in /hive-standalone/target/antrun/build-main.xml
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:213)
Thanks,
Meghali
From: Zoltan Haindrich <ki...@rxd.hu>
To: dev@hive.apache.org, Meghali Dhoble <dh...@us.ibm.com>
Date: 08/29/2018 06:29 PM
Subject: Re: Hive build from source is failing on ppc64le
Hello,
it seems the metastore is also refering to it..
git grep 2.5.0|grep pom.xml
I would highly recommend to give a try to the standalone-jdbc driver before
trying to build Hive on a not yet supported platform.
cheers,
Zoltan
On 08/29/2018 02:26 PM, Meghali Dhoble wrote:
> Hi Zoltan,
> Thanks for the quick response.
> I did tried for version 3.6.1 and it still looks for version 2.5.0
binary, which is more confusing to me.
>
> However the download location which maven build is referring to is ,
https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.6.1/
, which doesn't have a ppc64le
> binary, in spite of the protobuf being available for power.
>
> Any idea how can we make the protobuf binary available at this location ?
>
> Thanks,
> Meghali Dhoble
>
> Inactive hide details for Zoltan Haindrich ---08/29/2018 05:36:22
PM---Hello, It seems ppc64le is added in
https://urldefense.pZoltan
Haindrich ---08/29/2018 05:36:22
> PM---Hello, It seems ppc64le is added in INVALID URI REMOVED
<INVALID%20URI%20REMOVED>
>
> From: Zoltan Haindrich <ki...@rxd.hu>
> To: dev@hive.apache.org, Meghali Dhoble <dh...@us.ibm.com>
> Date: 08/29/2018 05:36 PM
> Subject: Re: Hive build from source is failing on ppc64le
>
>
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>
>
> Hello,
>
> It seems ppc64le is added in
https://github.com/protocolbuffers/protobuf/pull/4411 to
protobuf;
> from here
https://github.com/protocolbuffers/protobuf/commit/7f92711085374322f416745e00e69b63c4fcbbf2
> I think you should try to use at least 3.6.1...
>
> Good luck building Hive on power...but I'm not sure; I think you should
first try a jdbc-standalone jar...from a different platforms build; because
I think hive's jdbc
> driver is entirely written in java; so it should work on power as well...
>
> or...use one from maven central:
>
http://central.maven.org/maven2/org/apache/hive/hive-jdbc/3.1.0/
>
>
> cheers,
> Zoltan
>
>
> On 08/29/2018 01:51 PM, Meghali Dhoble wrote:
> >
> >
> > Hi Hive-Team,
> > I have been working to get the hive-jdbc built on the power platform.
> > However it's failing for me with an error which indicates that a
particular
> > binary is not present at the maven-repository for ppc64le. I tried
> > replacing the binary at ~/.m2/repository location, however that didn't
help
> > me.
> >
> > I am unable to understand what binary it is looking for and is there a
way
> > to manually build it and provide the path to it.
> > Could someone please help me understand this issue better?
> >
> > OS - Linux (Ubuntu 18.04)
> > Architecture - ppc64le
> >
> > The error log is as below:
> >
> > [ERROR] Failed to execute goal
> > com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on
project
> > hive-standalone-metastore: Error resolving artifact:
> > com.google.protobuf:protoc:2.5.0: Failure to find
> > com.google.protobuf:protoc:exe:linux-ppcle_64:2.5.0 in
> >
https://repo.maven.apache.org/maven2 was
cached in the local repository,
> > resolution will not be reattempted until the update interval of
central
> > has elapsed or updates are forced
> > [ERROR]
> > [ERROR] Try downloading the file manually from the project
website.
> > [ERROR]
> > [ERROR] Then, install it using the command:
> > [ERROR] mvn install:install-file -DgroupId=com.google.protobuf
> > -DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-ppcle_64
> > -Dpackaging=exe -Dfile=/path/to/file
> > [ERROR]
> > [ERROR] Alternatively, if you host your own repository you can
deploy
> > the file there:
> > [ERROR] mvn deploy:deploy-file -DgroupId=com.google.protobuf
> > -DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-ppcle_64
> > -Dpackaging=exe -Dfile=/path/to/file -Durl=[url] -DrepositoryId=
[id]
> > [ERROR]
> > [ERROR]
> > [ERROR] com.google.protobuf:protoc:exe:2.5.0
> >
> >
> > Thanks,
> > Meghali Dhoble
> >
>
>
>
Re: Hive build from source is failing on ppc64le
Posted by Zoltan Haindrich <ki...@rxd.hu>.
Hello,
it seems the metastore is also refering to it..
git grep 2.5.0|grep pom.xml
I would highly recommend to give a try to the standalone-jdbc driver before trying to build Hive on a not yet supported platform.
cheers,
Zoltan
On 08/29/2018 02:26 PM, Meghali Dhoble wrote:
> Hi Zoltan,
> Thanks for the quick response.
> I did tried for version 3.6.1 and it still looks for version 2.5.0 binary, which is more confusing to me.
>
> However the download location which maven build is referring to is ,https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.6.1/, which doesn't have a ppc64le
> binary, in spite of the protobuf being available for power.
>
> Any idea how can we make the protobuf binary available at this location ?
>
> Thanks,
> Meghali Dhoble
>
> Inactive hide details for Zoltan Haindrich ---08/29/2018 05:36:22 PM---Hello, It seems ppc64le is added in https://urldefense.pZoltan Haindrich ---08/29/2018 05:36:22
> PM---Hello, It seems ppc64le is added in INVALID URI REMOVED <INVALID%20URI%20REMOVED>
>
> From: Zoltan Haindrich <ki...@rxd.hu>
> To: dev@hive.apache.org, Meghali Dhoble <dh...@us.ibm.com>
> Date: 08/29/2018 05:36 PM
> Subject: Re: Hive build from source is failing on ppc64le
>
> ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>
>
> Hello,
>
> It seems ppc64le is added in https://github.com/protocolbuffers/protobuf/pull/4411 to protobuf;
> from here https://github.com/protocolbuffers/protobuf/commit/7f92711085374322f416745e00e69b63c4fcbbf2
> I think you should try to use at least 3.6.1...
>
> Good luck building Hive on power...but I'm not sure; I think you should first try a jdbc-standalone jar...from a different platforms build; because I think hive's jdbc
> driver is entirely written in java; so it should work on power as well...
>
> or...use one from maven central:
> http://central.maven.org/maven2/org/apache/hive/hive-jdbc/3.1.0/
>
>
> cheers,
> Zoltan
>
>
> On 08/29/2018 01:51 PM, Meghali Dhoble wrote:
> >
> >
> > Hi Hive-Team,
> > I have been working to get the hive-jdbc built on the power platform.
> > However it's failing for me with an error which indicates that a particular
> > binary is not present at the maven-repository for ppc64le. I tried
> > replacing the binary at ~/.m2/repository location, however that didn't help
> > me.
> >
> > I am unable to understand what binary it is looking for and is there a way
> > to manually build it and provide the path to it.
> > Could someone please help me understand this issue better?
> >
> > OS - Linux (Ubuntu 18.04)
> > Architecture - ppc64le
> >
> > The error log is as below:
> >
> > [ERROR] Failed to execute goal
> > com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project
> > hive-standalone-metastore: Error resolving artifact:
> > com.google.protobuf:protoc:2.5.0: Failure to find
> > com.google.protobuf:protoc:exe:linux-ppcle_64:2.5.0 in
> > https://repo.maven.apache.org/maven2 was cached in the local repository,
> > resolution will not be reattempted until the update interval of central
> > has elapsed or updates are forced
> > [ERROR]
> > [ERROR] Try downloading the file manually from the project website.
> > [ERROR]
> > [ERROR] Then, install it using the command:
> > [ERROR] mvn install:install-file -DgroupId=com.google.protobuf
> > -DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-ppcle_64
> > -Dpackaging=exe -Dfile=/path/to/file
> > [ERROR]
> > [ERROR] Alternatively, if you host your own repository you can deploy
> > the file there:
> > [ERROR] mvn deploy:deploy-file -DgroupId=com.google.protobuf
> > -DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-ppcle_64
> > -Dpackaging=exe -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
> > [ERROR]
> > [ERROR]
> > [ERROR] com.google.protobuf:protoc:exe:2.5.0
> >
> >
> > Thanks,
> > Meghali Dhoble
> >
>
>
>
Re: Hive build from source is failing on ppc64le
Posted by Meghali Dhoble <dh...@us.ibm.com>.
Hi Zoltan,
Thanks for the quick response.
I did tried for version 3.6.1 and it still looks for version 2.5.0 binary,
which is more confusing to me.
However the download location which maven build is referring to is ,
https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/3.6.1/,
which doesn't have a ppc64le binary, in spite of the protobuf being
available for power.
Any idea how can we make the protobuf binary available at this location ?
Thanks,
Meghali Dhoble
From: Zoltan Haindrich <ki...@rxd.hu>
To: dev@hive.apache.org, Meghali Dhoble <dh...@us.ibm.com>
Date: 08/29/2018 05:36 PM
Subject: Re: Hive build from source is failing on ppc64le
Hello,
It seems ppc64le is added in
https://github.com/protocolbuffers/protobuf/pull/4411
to protobuf;
from here
https://github.com/protocolbuffers/protobuf/commit/7f92711085374322f416745e00e69b63c4fcbbf2
I think you should try to use at least 3.6.1...
Good luck building Hive on power...but I'm not sure; I think you should
first try a jdbc-standalone jar...from a different platforms build; because
I think hive's jdbc
driver is entirely written in java; so it should work on power as well...
or...use one from maven central:
http://central.maven.org/maven2/org/apache/hive/hive-jdbc/3.1.0/
cheers,
Zoltan
On 08/29/2018 01:51 PM, Meghali Dhoble wrote:
>
>
> Hi Hive-Team,
> I have been working to get the hive-jdbc built on the power platform.
> However it's failing for me with an error which indicates that a
particular
> binary is not present at the maven-repository for ppc64le. I tried
> replacing the binary at ~/.m2/repository location, however that didn't
help
> me.
>
> I am unable to understand what binary it is looking for and is there a
way
> to manually build it and provide the path to it.
> Could someone please help me understand this issue better?
>
> OS - Linux (Ubuntu 18.04)
> Architecture - ppc64le
>
> The error log is as below:
>
> [ERROR] Failed to execute goal
> com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on
project
> hive-standalone-metastore: Error resolving artifact:
> com.google.protobuf:protoc:2.5.0: Failure to find
> com.google.protobuf:protoc:exe:linux-ppcle_64:2.5.0 in
>
https://repo.maven.apache.org/maven2
was cached in the local repository,
> resolution will not be reattempted until the update interval of
central
> has elapsed or updates are forced
> [ERROR]
> [ERROR] Try downloading the file manually from the project website.
> [ERROR]
> [ERROR] Then, install it using the command:
> [ERROR] mvn install:install-file -DgroupId=com.google.protobuf
> -DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-ppcle_64
> -Dpackaging=exe -Dfile=/path/to/file
> [ERROR]
> [ERROR] Alternatively, if you host your own repository you can deploy
> the file there:
> [ERROR] mvn deploy:deploy-file -DgroupId=com.google.protobuf
> -DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-ppcle_64
> -Dpackaging=exe -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
> [ERROR]
> [ERROR]
> [ERROR] com.google.protobuf:protoc:exe:2.5.0
>
>
> Thanks,
> Meghali Dhoble
>
Re: Hive build from source is failing on ppc64le
Posted by Zoltan Haindrich <ki...@rxd.hu>.
Hello,
It seems ppc64le is added in https://github.com/protocolbuffers/protobuf/pull/4411 to protobuf;
from here https://github.com/protocolbuffers/protobuf/commit/7f92711085374322f416745e00e69b63c4fcbbf2
I think you should try to use at least 3.6.1...
Good luck building Hive on power...but I'm not sure; I think you should first try a jdbc-standalone jar...from a different platforms build; because I think hive's jdbc
driver is entirely written in java; so it should work on power as well...
or...use one from maven central:
http://central.maven.org/maven2/org/apache/hive/hive-jdbc/3.1.0/
cheers,
Zoltan
On 08/29/2018 01:51 PM, Meghali Dhoble wrote:
>
>
> Hi Hive-Team,
> I have been working to get the hive-jdbc built on the power platform.
> However it's failing for me with an error which indicates that a particular
> binary is not present at the maven-repository for ppc64le. I tried
> replacing the binary at ~/.m2/repository location, however that didn't help
> me.
>
> I am unable to understand what binary it is looking for and is there a way
> to manually build it and provide the path to it.
> Could someone please help me understand this issue better?
>
> OS - Linux (Ubuntu 18.04)
> Architecture - ppc64le
>
> The error log is as below:
>
> [ERROR] Failed to execute goal
> com.github.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project
> hive-standalone-metastore: Error resolving artifact:
> com.google.protobuf:protoc:2.5.0: Failure to find
> com.google.protobuf:protoc:exe:linux-ppcle_64:2.5.0 in
> https://repo.maven.apache.org/maven2 was cached in the local repository,
> resolution will not be reattempted until the update interval of central
> has elapsed or updates are forced
> [ERROR]
> [ERROR] Try downloading the file manually from the project website.
> [ERROR]
> [ERROR] Then, install it using the command:
> [ERROR] mvn install:install-file -DgroupId=com.google.protobuf
> -DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-ppcle_64
> -Dpackaging=exe -Dfile=/path/to/file
> [ERROR]
> [ERROR] Alternatively, if you host your own repository you can deploy
> the file there:
> [ERROR] mvn deploy:deploy-file -DgroupId=com.google.protobuf
> -DartifactId=protoc -Dversion=2.5.0 -Dclassifier=linux-ppcle_64
> -Dpackaging=exe -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
> [ERROR]
> [ERROR]
> [ERROR] com.google.protobuf:protoc:exe:2.5.0
>
>
> Thanks,
> Meghali Dhoble
>
Re: Hive build from source is failing on ppc64le
Posted by Meghali Dhoble <dh...@us.ibm.com>.
Hi Gopal,
I have these version changes are already present in the source, as I am
working on the origin/branch-3.1.
This is still not helping me, as its looking for
"com.google.protobuf:protoc:exe:linux-ppcle_64:2.5.0"; which I am unable to
figure out where this configuration is coming from.
[ERROR] Failed to execute goal com.g
ithub.os72:protoc-jar-maven-plugin:3.5.1.1:run (default) on project
hive-standalone-metastore: Error resolving artifact:
com.google.protobuf:protoc:2.5.0: Failure to find
com.google.protobuf:protoc:exe:linux-ppcle_64:2.5.0 in
https://repo.maven.apache.org/maven2 was cached in the local repository,
resolution will not be reattempted until the update interval of central has
elapsed or updates are forced
Thans,
Meghali
From: Gopal Vijayaraghavan <go...@apache.org>
To: "dev@hive.apache.org" <de...@hive.apache.org>
Cc: Meghali Dhoble <dh...@us.ibm.com>
Date: 08/30/2018 01:30 PM
Subject: Re: Hive build from source is failing on ppc64le
> I am unable to understand what binary it is looking for and is there a
way
> to manually build it and provide the path to it.
> Could someone please help me understand this issue better?
https://issues.apache.org/jira/browse/HIVE-18601
Does that patch help you?
Cheers,
Gopal
Re: Hive build from source is failing on ppc64le
Posted by Gopal Vijayaraghavan <go...@apache.org>.
> I am unable to understand what binary it is looking for and is there a way
> to manually build it and provide the path to it.
> Could someone please help me understand this issue better?
https://issues.apache.org/jira/browse/HIVE-18601
Does that patch help you?
Cheers,
Gopal