You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/11/25 12:27:13 UTC

[GitHub] [arrow] alamb opened a new pull request #8769: ARROW-10710: [Rust] Revery tokio upgrade, go back to 0.2

alamb opened a new pull request #8769:
URL: https://github.com/apache/arrow/pull/8769


   ## Changes 
   
   This PR backs out the version upgrade to tokio done in https://github.com/apache/arrow/pull/8697 (see comment https://github.com/apache/arrow/pull/8697#issuecomment-732936572 by @rdettai for more details).
   
   ## Rationale:
   
   It seems as if the new version of tokio (0.3) has some fairly substantial changes, and some of the higher level libraries built on top of tokio that are used in the flight implementation do not yet support version 0.3 of tokio. This results in parts of the code (flight server) using the older version of tokio and parts of the code (datafusion) using newer versions of tokio which causes ... issues. 
   
   This issue can be seen in the flight example linked in https://issues.apache.org/jira/browse/ARROW-10710 and I saw it when I tried to upgrade to latest arrow in https://github.com/influxdata/influxdb_iox
   
   For example, https://crates.io/crates/tonic appears to require tokio < 0.3
   
   <img width="350" alt="Screen Shot 2020-11-25 at 7 10 21 AM" src="https://user-images.githubusercontent.com/490673/100227458-2eabbb00-2eef-11eb-82a8-eb11dad03bc2.png">
   
   
   
   cc @Dandandan  and @nevi-me 
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] Dandandan commented on a change in pull request #8769: ARROW-10710: [Rust] Revert tokio upgrade, go back to 0.2

Posted by GitBox <gi...@apache.org>.
Dandandan commented on a change in pull request #8769:
URL: https://github.com/apache/arrow/pull/8769#discussion_r530338275



##########
File path: rust/benchmarks/Cargo.toml
##########
@@ -30,5 +30,5 @@ arrow = { path = "../arrow" }
 parquet = { path = "../parquet" }
 datafusion = { path = "../datafusion" }
 structopt = { version = "0.3", default-features = false }
-tokio = { version = "0.3", features = ["macros", "rt", "rt-multi-thread"] }
+tokio = { version = "0.2", features = ["macros"] }

Review comment:
       I think you need to revert to "rt-core", "rt-threaded" ?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] jorgecarleitao removed a comment on pull request #8769: ARROW-10710: [Rust] Revert tokio upgrade, go back to 0.2

Posted by GitBox <gi...@apache.org>.
jorgecarleitao removed a comment on pull request #8769:
URL: https://github.com/apache/arrow/pull/8769#issuecomment-733687016


   :shipit: I also found a regression related to this update in `datafusion-python` on which the process hangs.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb commented on a change in pull request #8769: ARROW-10710: [Rust] Revert tokio upgrade, go back to 0.2

Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #8769:
URL: https://github.com/apache/arrow/pull/8769#discussion_r530342400



##########
File path: rust/benchmarks/Cargo.toml
##########
@@ -30,5 +30,5 @@ arrow = { path = "../arrow" }
 parquet = { path = "../parquet" }
 datafusion = { path = "../datafusion" }
 structopt = { version = "0.3", default-features = false }
-tokio = { version = "0.3", features = ["macros", "rt", "rt-multi-thread"] }
+tokio = { version = "0.2", features = ["macros"] }

Review comment:
       Thanks for the tip @Dandandan  -- it is strange. It seems to compile for me locally without doing this -- I am re-building from scratch and will also see what happens on CI tests. 




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb commented on pull request #8769: ARROW-10710: [Rust] Revert tokio upgrade, go back to 0.2

Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #8769:
URL: https://github.com/apache/arrow/pull/8769#issuecomment-733686362


   FYI @jorgecarleitao 


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb commented on pull request #8769: ARROW-10710: [Rust] Revert tokio upgrade, go back to 0.2

Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #8769:
URL: https://github.com/apache/arrow/pull/8769#issuecomment-733698117


   I am waiting for CI to finish running and then I will merge this in


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb commented on a change in pull request #8769: ARROW-10710: [Rust] Revert tokio upgrade, go back to 0.2

Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #8769:
URL: https://github.com/apache/arrow/pull/8769#discussion_r530353896



##########
File path: rust/benchmarks/Cargo.toml
##########
@@ -30,5 +30,5 @@ arrow = { path = "../arrow" }
 parquet = { path = "../parquet" }
 datafusion = { path = "../datafusion" }
 structopt = { version = "0.3", default-features = false }
-tokio = { version = "0.3", features = ["macros", "rt", "rt-multi-thread"] }
+tokio = { version = "0.2", features = ["macros"] }

Review comment:
       yes you are right :) -- fixing




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb closed pull request #8769: ARROW-10710: [Rust] Revert tokio upgrade, go back to 0.2

Posted by GitBox <gi...@apache.org>.
alamb closed pull request #8769:
URL: https://github.com/apache/arrow/pull/8769


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] jorgecarleitao commented on pull request #8769: ARROW-10710: [Rust] Revert tokio upgrade, go back to 0.2

Posted by GitBox <gi...@apache.org>.
jorgecarleitao commented on pull request #8769:
URL: https://github.com/apache/arrow/pull/8769#issuecomment-733687016


   :shipit: I also found a regression related to this update in `datafusion-python` on which the process hangs.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb commented on pull request #8769: ARROW-10710: [Rust] Revert tokio upgrade, go back to 0.2

Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #8769:
URL: https://github.com/apache/arrow/pull/8769#issuecomment-733708535


   CI is green -- merging this in!
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] Dandandan commented on a change in pull request #8769: ARROW-10710: [Rust] Revert tokio upgrade, go back to 0.2

Posted by GitBox <gi...@apache.org>.
Dandandan commented on a change in pull request #8769:
URL: https://github.com/apache/arrow/pull/8769#discussion_r530338275



##########
File path: rust/benchmarks/Cargo.toml
##########
@@ -30,5 +30,5 @@ arrow = { path = "../arrow" }
 parquet = { path = "../parquet" }
 datafusion = { path = "../datafusion" }
 structopt = { version = "0.3", default-features = false }
-tokio = { version = "0.3", features = ["macros", "rt", "rt-multi-thread"] }
+tokio = { version = "0.2", features = ["macros"] }

Review comment:
       I think you need to revert to use features "rt-core", "rt-threaded" ?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] Dandandan commented on a change in pull request #8769: ARROW-10710: [Rust] Revert tokio upgrade, go back to 0.2

Posted by GitBox <gi...@apache.org>.
Dandandan commented on a change in pull request #8769:
URL: https://github.com/apache/arrow/pull/8769#discussion_r530338379



##########
File path: rust/datafusion/Cargo.toml
##########
@@ -57,7 +57,7 @@ chrono = "0.4"
 async-trait = "0.1.41"
 futures = "0.3"
 pin-project-lite= "^0.2.0"
-tokio = { version = "0.3", features = ["macros", "rt", "rt-multi-thread"] }
+tokio = { version = "0.2", features = ["macros"] }

Review comment:
       Here as well "rt-core", "rt-threaded" ?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] github-actions[bot] commented on pull request #8769: ARROW-10710: [Rust] Revert tokio upgrade, go back to 0.2

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8769:
URL: https://github.com/apache/arrow/pull/8769#issuecomment-733679895


   https://issues.apache.org/jira/browse/ARROW-10710


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb commented on a change in pull request #8769: ARROW-10710: [Rust] Revert tokio upgrade, go back to 0.2

Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #8769:
URL: https://github.com/apache/arrow/pull/8769#discussion_r530339818



##########
File path: rust/benchmarks/Cargo.toml
##########
@@ -30,5 +30,5 @@ arrow = { path = "../arrow" }
 parquet = { path = "../parquet" }
 datafusion = { path = "../datafusion" }
 structopt = { version = "0.3", default-features = false }
-tokio = { version = "0.3", features = ["macros", "rt", "rt-multi-thread"] }
+tokio = { version = "0.2", features = ["macros"] }

Review comment:
       I don't think those are features in tokio 0.2 (annoyingly):
   
   If you add them, you get the following error
   
   ```
   diff --git a/rust/datafusion/Cargo.toml b/rust/datafusion/Cargo.toml
   index ab11a9520..bc7b67808 100644
   --- a/rust/datafusion/Cargo.toml
   +++ b/rust/datafusion/Cargo.toml
   @@ -57,7 +57,7 @@ chrono = "0.4"
    async-trait = "0.1.41"
    futures = "0.3"
    pin-project-lite= "^0.2.0"
   -tokio = { version = "0.2", features = ["macros"] }
   +tokio = { version = "0.2", features = ["macros", "rt", "rt-multi-thread"] }
    
    [dev-dependencies]
    rand = "0.7"
   ```
   
   
   ```
   error: failed to select a version for `tokio`.
       ... required by package `datafusion v3.0.0-SNAPSHOT (/Users/alamb/Software/arrow2/rust/datafusion)`
   versions that meet the requirements `=0.2.23` are: 0.2.23
   
   the package `datafusion` depends on `tokio`, with features: `rt` but `tokio` does not have these features.
   ```




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org