You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "Mikhail Epikhin (Jira)" <ji...@apache.org> on 2019/11/25 07:55:00 UTC
[jira] [Comment Edited] (BIGTOP-3274) Protocbuf build failed due to
git.savannah.gnu.org is not accessible
[ https://issues.apache.org/jira/browse/BIGTOP-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16981349#comment-16981349 ]
Mikhail Epikhin edited comment on BIGTOP-3274 at 11/25/19 7:54 AM:
-------------------------------------------------------------------
Why we are using protobuf-2.5.0?
I've found that only spark-components use it, but actually, spark doesn't need them. Also, I found that only spark 0.8.1 requires protobuf-2.5.0 [https://spark.apache.org/docs/0.8.1/building-with-maven.html]
Current version 2.2.3 doesn't need protobuf [https://spark.apache.org/docs/2.2.3/building-spark.html]
This is needed for patch _0001-Add-generic-GCC-support-for-atomic-operations.patch_ for supporting arm64 and ppc64el?
was (Author: epikhinm):
Why we are using protobuf-2.5.0?
I've found that only spark-components use it, but actually, spark doesn't need them. Also, I found that only spark 0.8.1 requires protobuf-2.5.0 [https://spark.apache.org/docs/0.8.1/building-with-maven.html]
Current version 2.2.3 doesn't need protobuf https://spark.apache.org/docs/2.2.3/building-spark.html
{code:java}
~/g/s/g/m/b/bigtop-packages ❯❯❯ ag "protobuf"
src/common/zeppelin/patch0-spark-2.2.3-compatibility.diff
11: <protobuf.version>2.5.0</protobuf.version>src/common/spark1/do-component-build
29: -Dprotobuf.version=2.5.0 \src/common/hadoop/init-hdfs.sh
37:HADOOP_DEPENDENCIES="commons-logging*.jar commons-io*.jar guava*.jar commons-configuration*.jar commons-collections*.jar slf4j-api*.jar protobuf-java*.jar commons-lang*.jar servlet-api-2.5.jar"src/common/hadoop/init-hcfs.groovy
74: :/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar /vagrant/init-hcfs.groovysrc/deb/hbase/control
39: * A REST-ful Web service gateway that supports XML, Protobuf, and binary data encoding optionssrc/charm/giraph/layer-giraph/reactive/giraph.py
41: # - hive-exec-0.11.0 protobuf classsrc/rpm/hbase/SPECS/hbase.spec
114: * A REST-ful Web service gateway that supports XML, Protobuf, and binary data encoding options
{code}
This is needed for patch _0001-Add-generic-GCC-support-for-atomic-operations.patch_ for supporting arm64 and ppc64el?
> Protocbuf build failed due to git.savannah.gnu.org is not accessible
> --------------------------------------------------------------------
>
> Key: BIGTOP-3274
> URL: https://issues.apache.org/jira/browse/BIGTOP-3274
> Project: Bigtop
> Issue Type: Bug
> Reporter: Jun He
> Priority: Major
>
> In BIGTOP-2698, protocbuf is set to build from source with patche and new config.guess from git.savannah.gnu.org.
> Recently when I was trying to build toolchain, I noticed that confgi.guess was failed to download due to git.savannah.gnu.org was not accessible.
> {noformat}
> Notice: /Stage[main]/Bigtop_toolchain::Protobuf/Exec[download protobuf]/returns: Dload Upload Total Spent Left Speed
> 0 0 0 0 0 0 0 0 --:--:-- 0:00:03 --:--:-- 0curl: (7) Failed to connect to git.savannah.gnu.org port 80: No route to host
> Error: /usr/bin/wget https://github.com/google/protobuf/releases/download/v2.5.0//protobuf-2.5.0.tar.gz && mkdir -p protobuf-2.5.0 && /bin/tar -xvzf protobuf-2.5.0.tar.gz -C protobuf-2.5.0 --strip-components=1 && cd protobuf-2.5.0 && /usr/bin/patch -p1 </usr/src/0001-Add-generic-GCC-support-for-atomic-operations.patch && curl -o config.guess 'http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD' && cp config.guess gtest/build-aux/ returned 7 instead of one of [0]
> Error: /Stage[main]/Bigtop_toolchain::Protobuf/Exec[download protobuf]/returns: change from notrun to 0 failed: /usr/bin/wget https://github.com/google/protobuf/releases/download/v2.5.0//protobuf-2.5.0.tar.gz && mkdir -p protobuf-2.5.0 && /bin/tar -xvzf protobuf-2.5.0.tar.gz -C protobuf-2.5.0 --strip-components=1 && cd protobuf-2.5.0 && /usr/bin/patch -p1 </usr/src/0001-Add-generic-GCC-support-for-atomic-operations.patch && curl -o config.guess 'http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD' && cp config.guess gtest/build-aux/ returned 7 instead of one of [0]
> Notice: /Stage[main]/Bigtop_toolchain::Protobuf/Exec[install protobuf]: Dependency Exec[download protobuf] has failures: true
> Warning: /Stage[main]/Bigtop_toolchain::Protobuf/Exec[install protobuf]: Skipping because of failed dependencies
> {noformat}
> To fix this I think we can use a included config.guess, just like the patch (_0001-Add-generic-GCC-support-for-atomic-operations.patch_), instead of downloading it every time. This will minimize failure due to network issue and keep a consistent file for the given v2.5 protocbuf
--
This message was sent by Atlassian Jira
(v8.3.4#803005)