You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/08/11 07:56:59 UTC
(datafusion) branch asf-site updated: Publish built docs triggered by a1f64e94b4d4d2eec6d4a8af448cedb86a075f75
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 8192778392 Publish built docs triggered by a1f64e94b4d4d2eec6d4a8af448cedb86a075f75
8192778392 is described below
commit 81927783928878003f6e08c69f839dd70aab44f8
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Tue Aug 11 07:56:53 2026 +0000
Publish built docs triggered by a1f64e94b4d4d2eec6d4a8af448cedb86a075f75
---
_sources/user-guide/configs.md.txt | 2 +-
searchindex.js | 2 +-
user-guide/configs.html | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/_sources/user-guide/configs.md.txt b/_sources/user-guide/configs.md.txt
index 860884e11f..a66ad3edf5 100644
--- a/_sources/user-guide/configs.md.txt
+++ b/_sources/user-guide/configs.md.txt
@@ -130,7 +130,7 @@ The following configuration settings are available:
| datafusion.execution.meta_fetch_concurrency | 32 | Number of files to read in parallel when inferring schema and statistics [...]
| datafusion.execution.minimum_parallel_output_files | 4 | Guarantees a minimum level of output files running in parallel. RecordBatches will be distributed in round robin fashion to each parallel writer. Each writer is closed and a new file opened once soft_max_rows_per_output_file is reached. [...]
| datafusion.execution.soft_max_rows_per_output_file | 50000000 | Target number of rows in output files when writing multiple. This is a soft max, so it can be exceeded slightly. There also will be one file smaller than the limit if the total number of rows written is not roughly divisible by the soft max [...]
-| datafusion.execution.max_buffered_batches_per_output_file | 2 | This is the maximum number of RecordBatches buffered for each output file being worked. Higher values can potentially give faster write performance at the cost of higher peak memory consumption [...]
+| datafusion.execution.max_buffered_batches_per_output_file | 2 | This is the maximum number of RecordBatches buffered for each output file being worked. Higher values can potentially give faster write performance at the cost of higher peak memory consumption. This budget is split evenly between two independent points in the write pipeline (see the demuxer diagram in #7791): how many files can be in flight from the demuxer to a writer task, and how m [...]
| datafusion.execution.listing_table_ignore_subdirectory | true | Should sub directories be ignored when scanning directories for data files. Defaults to true (ignores subdirectories), consistent with Hive. Note that this setting does not affect reading partitioned tables (e.g. `/table/year=2021/month=01/data.parquet`). [...]
| datafusion.execution.listing_table_factory_infer_partitions | true | Should a `ListingTable` created through the `ListingTableFactory` infer table partitions from Hive compliant directories. Defaults to true (partition columns are inferred and will be represented in the table schema). [...]
| datafusion.execution.enable_recursive_ctes | true | Should DataFusion support recursive CTEs [...]
diff --git a/searchindex.js b/searchindex.js
index f874401ee3..9391b5b1c8 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles":{"!=":[[74,"op-neq"]],"!~":[[74,"op-re-not-match"]],"!~*":[[74,"op-re-not-match-i"]],"!~~":[[74,"id19"]],"!~~*":[[74,"id20"]],"#":[[74,"op-bit-xor"]],"%":[[74,"op-modulo"]],"&":[[74,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[15,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[74,"op-multiply"]],"+":[[74,"op-plus"]],"-":[[74,"op-minus"]],"/":[[74,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
+Search.setIndex({"alltitles":{"!=":[[74,"op-neq"]],"!~":[[74,"op-re-not-match"]],"!~*":[[74,"op-re-not-match-i"]],"!~~":[[74,"id19"]],"!~~*":[[74,"id20"]],"#":[[74,"op-bit-xor"]],"%":[[74,"op-modulo"]],"&":[[74,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[15,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[74,"op-multiply"]],"+":[[74,"op-plus"]],"-":[[74,"op-minus"]],"/":[[74,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
diff --git a/user-guide/configs.html b/user-guide/configs.html
index 35eab36ee3..f7187283ff 100644
--- a/user-guide/configs.html
+++ b/user-guide/configs.html
@@ -747,7 +747,7 @@ example, to configure <code class="docutils literal notranslate"><span class="pr
</tr>
<tr class="row-odd"><td><p>datafusion.execution.max_buffered_batches_per_output_file</p></td>
<td><p>2</p></td>
-<td><p>This is the maximum number of RecordBatches buffered for each output file being worked. Higher values can potentially give faster write performance at the cost of higher peak memory consumption</p></td>
+<td><p>This is the maximum number of RecordBatches buffered for each output file being worked. Higher values can potentially give faster write performance at the cost of higher peak memory consumption. This budget is split evenly between two independent points in the write pipeline (see the demuxer diagram in #7791): how many files can be in flight from the demuxer to a writer task, and how many RecordBatches are buffered for a single file’s writer. Must be at least 2 so each half gets a [...]
</tr>
<tr class="row-even"><td><p>datafusion.execution.listing_table_ignore_subdirectory</p></td>
<td><p>true</p></td>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org