[GitHub] [arrow-datafusion] alamb commented on a change in pull request #683: Introduce (default) number of partitions option, use it in DataFusion/Ballista

[GitBox <gi...@apache.org>]

alamb commented on a change in pull request #683:
URL: https://github.com/apache/arrow-datafusion/pull/683#discussion_r664761587



##########
File path: datafusion/src/execution/context.rs
##########
@@ -639,6 +641,7 @@ impl Default for ExecutionConfig {
     fn default() -> Self {
         Self {
             concurrency: num_cpus::get(),
+            partitions: num_cpus::get(),

Review comment:
       For backwards compatibility what would you think about defaulting `partitions` to `None` and if it was not set using the value for `concurrency` instead? Otherwise users may have to tune two separate knobs when they may just want a big hammer "how many cores should data fusion try and keep busy"




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org