[jira] [Created] (ARROW-9600) [Rust] When used as a crate dependency, arrow-flight is rebuilt on every invocation of cargo build

["Andrew Lamb (Jira)" <ji...@apache.org>]

Andrew Lamb created ARROW-9600:
----------------------------------

             Summary: [Rust] When used as a crate dependency, arrow-flight is rebuilt on every invocation of cargo build
                 Key: ARROW-9600
                 URL: https://issues.apache.org/jira/browse/ARROW-9600
             Project: Apache Arrow
          Issue Type: Bug
          Components: Rust
    Affects Versions: 1.0.0
            Reporter: Andrew Lamb


When used as a crate dependency, arrow-flight is rebuilt on every invocation of cargo build
h1. *Repro*:

Create a new repo, add `arrow=1.0.0` as a dependency, and then run `cargo build`

*Expected behavior:* After the first successful invocation of `cargo build`, arrow-flight will not recompile if no other changes are made.

*Actual behavior*: After every invocation of `cargo build`, arrow-flight is recompiled, even when nothing has changed
h1. Example

 
 Create a new crate
{code:java}
 alamb@ip-192-168-0-129 arrow_rebuilds % cargo new too_many_rebuilds --bin
 cargo new too_many_rebuilds --bin
 Created binary (application) `too_many_rebuilds` package
{code}
Add arrow as a dependency in Cargo.toml:
{code:java}
 diff --git a/Cargo.toml b/Cargo.toml
 index a239680..44ed358 100644
 — a/Cargo.toml
 +++ b/Cargo.toml
 @@ -5,3 +5,6 @@ authors = ["alamb <an...@nerdnetworks.org>"]
 edition = "2018"
 # See more keys and their definitions at [https://doc.rust-lang.org/cargo/reference/manifest.html]
 +
 +[dependencies]
 +arrow = "1.0.0"
{code}
Now, all invocations of `cargo build` will rebuild arrow, even though nothing in the code has changed:
{code:java}
 alamb@ip-192-168-0-129 too_many_rebuilds % cargo build
 cargo build
 Compiling arrow-flight v1.0.0
 Compiling arrow v1.0.0
 Compiling too_many_rebuilds v0.1.0 (/Users/alamb/Software/bugs/arrow_rebuilds/too_many_rebuilds)
 Finished dev [unoptimized + debuginfo] target(s) in 8.70s
 alamb@ip-192-168-0-129 too_many_rebuilds % cargo build
 cargo build
 Compiling arrow-flight v1.0.0
 Compiling arrow v1.0.0
 Compiling too_many_rebuilds v0.1.0 (/Users/alamb/Software/bugs/arrow_rebuilds/too_many_rebuilds)
 Finished dev [unoptimized + debuginfo] target(s) in 8.65s
{code}
You can see what is happening by checking out a fresh copy of arrow/master (no Cargo.log) and running `cargo build` – you'll see your local checkout has changes in rust/arrow-flight/src/arrow.flight.protocol.rs:
{code:java}
 alamb@ip-192-168-0-129 arrow % cd rust/arrow
 cd rust/arrow
 alamb@ip-192-168-0-129 arrow % git status
 git status
 On branch master
 Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean
 alamb@ip-192-168-0-129 arrow % cargo build
 cargo build
 Compiling futures-task v0.3.5
 ...
 Compiling arrow v2.0.0-SNAPSHOT (/Users/alamb/Software/arrow/rust/arrow)
 Finished dev [unoptimized + debuginfo] target(s) in 21.76s
 alamb@ip-192-168-0-129 arrow %

alamb@ip-192-168-0-129 arrow % git status
 git status
 On branch master
 Your branch is up to date with 'origin/master'.

Changes not staged for commit:
 (use "git add <file>..." to update what will be committed)
 (use "git restore <file>..." to discard changes in working directory)
 modified: ../arrow-flight/src/arrow.flight.protocol.rs

no changes added to commit (use "git add" and/or "git commit -a")
{code}
 # Root Cause Analysis

The issue is that the build.rs of arrow-flight calls `tonic_build` to auto generate `rust/arrow-flight/src/arrow.flight.protocol.rs`, which is also checked in (first done in [https://github.com/apache/arrow/commit/ec84b7b8102f227295f865c420496830c66a6281]).

This file and the version of tonic were updated on [https://github.com/apache/arrow/commit/7b49cbc23f22ed99eebf85cc0b9acb1f0d3f832f] on July 11, 2020

It turns out that the output of "tonic_build" depends on not only on the version of tonic, but also on the version of proc-macro2, and the version of proc-macro2 is not specifically pinned. 

`proc-macro2 = "1.0.19"` was released on July 19, 2020 ([https://crates.io/crates/proc-macro2/1.0.19]) and it appears to subtlety changes the resulting output of arrow.flight.protocol.rs; Thus the output no longer matches what is checked in. This means that anyone without a Cargo.lock file that pins proc-macro2 to 1.0.18 would get 1.0.19 and thus also a local modification during build.

h1. Workaround
If we pin Cargo.toml to use proc-macro2 1.0.18 the local modification stops.
 {code}
 proc-macro2 = "1.0.18"
 {code}

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)