You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/10/14 23:16:49 UTC
(datafusion) branch asf-site updated: Publish built docs triggered by 53232ba2914d9cda09b887f7c2c016ea1cfa0a63
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 96d31c6ac9 Publish built docs triggered by 53232ba2914d9cda09b887f7c2c016ea1cfa0a63
96d31c6ac9 is described below
commit 96d31c6ac964f9a2560c155839e40bede51b217c
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Wed Oct 14 23:16:44 2026 +0000
Publish built docs triggered by 53232ba2914d9cda09b887f7c2c016ea1cfa0a63
---
_sources/user-guide/sql/format_options.md.txt | 19 +++++++++++++++
searchindex.js | 2 +-
user-guide/sql/format_options.html | 35 +++++++++++++++++++++++++++
user-guide/sql/index.html | 1 +
4 files changed, 56 insertions(+), 1 deletion(-)
diff --git a/_sources/user-guide/sql/format_options.md.txt b/_sources/user-guide/sql/format_options.md.txt
index 719fd5bd6b..738ed8b5de 100644
--- a/_sources/user-guide/sql/format_options.md.txt
+++ b/_sources/user-guide/sql/format_options.md.txt
@@ -78,6 +78,25 @@ In this example, we write the entire `source_table` out to a folder of Parquet f
# Available Options
+## Arrow Format Options
+
+The following options are available when writing [Arrow IPC](https://arrow.apache.org/docs/format/Columnar.html#ipc-file-format) files. Note: If any unsupported option is specified, an error will be raised and the query will fail.
+
+| Option | Description | Default Value |
+| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- |
+| COMPRESSION | Sets the compression codec used for the record batch bodies within the Arrow IPC file. Supported values are UNCOMPRESSED, LZ4_FRAME, and ZSTD. | LZ4_FRAME |
+| COMPRESSION_LEVEL | Sets the compression level to use with the `ZSTD` codec. Not supported by `LZ4_FRAME` or `UNCOMPRESSED`. If not specified, the default zstd compression level is used. | None |
+| ALIGNMENT | Sets the byte alignment used to pad buffers when writing the Arrow IPC file. Must be one of 8, 16, 32, or 64. | 64 |
+
+**Example:**
+
+```sql
+COPY source_table
+ TO '/tmp/foo.arrow'
+ STORED AS ARROW
+ OPTIONS('COMPRESSION' 'zstd', 'COMPRESSION_LEVEL' '15');
+```
+
## JSON Format Options
The following options are available when reading or writing JSON files. Note: If any unsupported option is specified, an error will be raised and the query will fail.
diff --git a/searchindex.js b/searchindex.js
index ccc8ae9978..f6b7f1c849 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles":{"!=":[[79,"op-neq"]],"!~":[[79,"op-re-not-match"]],"!~*":[[79,"op-re-not-match-i"]],"!~~":[[79,"id19"]],"!~~*":[[79,"id20"]],"#":[[79,"op-bit-xor"]],"%":[[79,"op-modulo"]],"&":[[79,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[17,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[79,"op-multiply"]],"+":[[79,"op-plus"]],"-":[[79,"op-minus"]],"/":[[79,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
+Search.setIndex({"alltitles":{"!=":[[79,"op-neq"]],"!~":[[79,"op-re-not-match"]],"!~*":[[79,"op-re-not-match-i"]],"!~~":[[79,"id19"]],"!~~*":[[79,"id20"]],"#":[[79,"op-bit-xor"]],"%":[[79,"op-modulo"]],"&":[[79,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[17,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[79,"op-multiply"]],"+":[[79,"op-plus"]],"-":[[79,"op-minus"]],"/":[[79,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
diff --git a/user-guide/sql/format_options.html b/user-guide/sql/format_options.html
index b3c1c037b4..11b8d3cd08 100644
--- a/user-guide/sql/format_options.html
+++ b/user-guide/sql/format_options.html
@@ -502,6 +502,40 @@ a<span class="p">;</span>b
</section>
<section id="available-options">
<h1>Available Options<a class="headerlink" href="#available-options" title="Link to this heading">#</a></h1>
+<section id="arrow-format-options">
+<h2>Arrow Format Options<a class="headerlink" href="#arrow-format-options" title="Link to this heading">#</a></h2>
+<p>The following options are available when writing <a class="reference external" href="https://arrow.apache.org/docs/format/Columnar.html#ipc-file-format">Arrow IPC</a> files. Note: If any unsupported option is specified, an error will be raised and the query will fail.</p>
+<div class="pst-scrollable-table-container"><table class="table">
+<thead>
+<tr class="row-odd"><th class="head"><p>Option</p></th>
+<th class="head"><p>Description</p></th>
+<th class="head"><p>Default Value</p></th>
+</tr>
+</thead>
+<tbody>
+<tr class="row-even"><td><p>COMPRESSION</p></td>
+<td><p>Sets the compression codec used for the record batch bodies within the Arrow IPC file. Supported values are UNCOMPRESSED, LZ4_FRAME, and ZSTD.</p></td>
+<td><p>LZ4_FRAME</p></td>
+</tr>
+<tr class="row-odd"><td><p>COMPRESSION_LEVEL</p></td>
+<td><p>Sets the compression level to use with the <code class="docutils literal notranslate"><span class="pre">ZSTD</span></code> codec. Not supported by <code class="docutils literal notranslate"><span class="pre">LZ4_FRAME</span></code> or <code class="docutils literal notranslate"><span class="pre">UNCOMPRESSED</span></code>. If not specified, the default zstd compression level is used.</p></td>
+<td><p>None</p></td>
+</tr>
+<tr class="row-even"><td><p>ALIGNMENT</p></td>
+<td><p>Sets the byte alignment used to pad buffers when writing the Arrow IPC file. Must be one of 8, 16, 32, or 64.</p></td>
+<td><p>64</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+<p><strong>Example:</strong></p>
+<div class="highlight-sql notranslate"><div class="highlight"><pre><span></span><span class="k">COPY</span><span class="w"> </span><span class="n">source_table</span>
+<span class="w"> </span><span class="k">TO</span><span class="w"> </span><span class="s1">'/tmp/foo.arrow'</span>
+<span class="w"> </span><span class="n">STORED</span><span class="w"> </span><span class="k">AS</span><span class="w"> </span><span class="n">ARROW</span>
+<span class="w"> </span><span class="k">OPTIONS</span><span class="p">(</span><span class="s1">'COMPRESSION'</span><span class="w"> </span><span class="s1">'zstd'</span><span class="p">,</span><span class="w"> </span><span class="s1">'COMPRESSION_LEVEL'</span><span class="w"> </span><span class="s1">'15'</span><span class="p">);</span>
+</pre></div>
+</div>
+</section>
<section id="json-format-options">
<h2>JSON Format Options<a class="headerlink" href="#json-format-options" title="Link to this heading">#</a></h2>
<p>The following options are available when reading or writing JSON files. Note: If any unsupported option is specified, an error will be raised and the query will fail.</p>
@@ -911,6 +945,7 @@ a<span class="p">;</span>b
</ul>
</li>
<li class="toc-h1 nav-item toc-entry"><a class="reference internal nav-link" href="#available-options">Available Options</a><ul class="pst-show_toc_level nav section-nav flex-column">
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#arrow-format-options">Arrow Format Options</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#json-format-options">JSON Format Options</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#csv-format-options">CSV Format Options</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#parquet-format-options">Parquet Format Options</a></li>
diff --git a/user-guide/sql/index.html b/user-guide/sql/index.html
index ff6397ab03..e8d3cd3150 100644
--- a/user-guide/sql/index.html
+++ b/user-guide/sql/index.html
@@ -563,6 +563,7 @@
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="format_options.html#available-options">Available Options</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="format_options.html#arrow-format-options">Arrow Format Options</a></li>
<li class="toctree-l2"><a class="reference internal" href="format_options.html#json-format-options">JSON Format Options</a></li>
<li class="toctree-l2"><a class="reference internal" href="format_options.html#csv-format-options">CSV Format Options</a></li>
<li class="toctree-l2"><a class="reference internal" href="format_options.html#parquet-format-options">Parquet Format Options</a></li>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org