[jira] [Commented] (TEZ-4300) Download protoc automatically compile/development time

["Martin Tzvetanov Grigorov (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TEZ-4300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489531#comment-17489531 ] 

Martin Tzvetanov Grigorov commented on TEZ-4300:
------------------------------------------------

Hey [~abstractdog] !

 

I've just tested it.

The build passes on Ubuntu 20.04.3 ARM64 with: mvn clean install -DskipTests -Dprotobuf.version=3.13.0 -pl "!tez-ui"

3.13.0 comes from org.apache.hadoop:hadoop-maven-plugins:3.3.1.

Trying without -Dprotobuf.version fails with:
{code:java}
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.3.1:protoc (compile-protoc) on project tez-api: org.apache.maven.plugin.MojoExecutionException: protoc version is 'libprotoc 3.13.0', expected version is '2.5.0' -> [Help 1]
 {code}
Trying with 3.19.2 fails with:
{code:java}
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.3.1:protoc (compile-protoc) on project tez-api: org.apache.maven.plugin.MojoExecutionException: protoc version is 'libprotoc 3.13.0', expected version is '3.19.2' -> [Help 1] {code}
So, it seems only 3.13.0 is OK for now.

 

I also needed to remove tez-ui from the reactor because it uses PhantomJS which is not available for linux aarch64.

> Download protoc automatically compile/development time
> ------------------------------------------------------
>
>                 Key: TEZ-4300
>                 URL: https://issues.apache.org/jira/browse/TEZ-4300
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: László Bodor
>            Assignee: László Bodor
>            Priority: Major
>         Attachments: tez_build_protobuf_3_7_1.log
>
>          Time Spent: 3h
>  Remaining Estimate: 0h
>
> Currently, Tez cannot be built out of the box because tez-api needs a protobuf compiler (protoc executable), which was also a problematic dependency in a sense that downloading/installing it is a burden on developers. Even if we have tons of [gists|https://gist.github.com/abstractdog/1eddeac9cc09815ded5253e3a83293a8] for that, it's still uncomfortable. Not to mention if we want to support more versions at the same time. 
> Shipping the typically used versions (2.5.0, 3.x for later) of the protoc executable could make sense, but I'm not sure at the moment if it has any license issues, [need to check|https://github.com/protocolbuffers/protobuf/blob/master/LICENSE], also it's not general, instead an intelligent script could do it for developers.
> For 2.5, it should be done from source: https://github.com/protocolbuffers/protobuf/releases/tag/v2.5.0
> But for newer versions, the protoc executable can be downloaded from the releases page: https://github.com/protocolbuffers/protobuf/releases/tag/v3.7.1
> https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protoc-3.7.1-osx-x86_64.zip
> https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protoc-3.7.1-linux-x86_64.zip



--
This message was sent by Atlassian Jira
(v8.20.1#820001)