[GitHub] [arrow-ballista] andygrove commented on a diff in pull request #472: Add SchedulerConfig for the scheduler configurations, like event_loop_buffer_size, finished_job_data_clean_up_interval_seconds, finished_job_state_clean_up_interval_seconds

[GitBox <gi...@apache.org>]

andygrove commented on code in PR #472:
URL: https://github.com/apache/arrow-ballista/pull/472#discussion_r1010813058


##########
docs/source/user-guide/configs.md:
##########
@@ -58,3 +60,24 @@ In addition to Ballista-specific configuration settings, the following DataFusio
 | datafusion.explain.physical_plan_only           | Boolean | false   | When set to true, the explain statement will only print physical plans.                                                                                                                                                                                                                                                                                       |
 | datafusion.optimizer.filter_null_join_keys      | Boolean | false   | When set to true, the optimizer will insert filters before a join between a nullable and non-nullable column to filter out nulls on the nullable side. This filter can add additional overhead when the file format does not fully support predicate push down.                                                                                               |
 | datafusion.optimizer.skip_failed_rules          | Boolean | true    | When set to true, the logical plan optimizer will produce warning messages if any optimization rules produce errors and then proceed to the next rule. When set to false, any rules that produce errors will cause the query to fail.                                                                                                                         |
+
+## Ballista Scheduler Configuration Settings
+
+Besides the BallistaContext configuration settings, a few configuration settings for the Ballista scheduler to better
+manage the whole cluster are also needed to be taken care of.
+
+_Example: Specifying configuration options when starting the scheduler_
+
+```shell
+./ballista-scheduler --scheduler-policy push-staged --event-loop-buffer-size 1000000 --executor-slots-policy
+round-robin-local
+```
+
+| key                                            | type      | default      | description                                                                                                                                                                      |
+|------------------------------------------------|-----------|--------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| scheduler_policy                               | Utf8      | pull-staged  | Sets the scheduing policy for the scheduler, possible values: pull-staged, push-staged.                                                                                          |

Review Comment:
   The table is using underscore (`scheduler_policy`) but the example shell command is using hyphen (`scheduler-policy`).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org