You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/03/26 22:16:21 UTC
(datafusion) branch asf-site updated: Publish built docs triggered by 627fabaa7795ae654114882d6714f7f1065e926d
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 5775ef2079 Publish built docs triggered by 627fabaa7795ae654114882d6714f7f1065e926d
5775ef2079 is described below
commit 5775ef2079108181984a4ec0457fb4c51395d0c3
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Thu Mar 26 22:16:17 2026 +0000
Publish built docs triggered by 627fabaa7795ae654114882d6714f7f1065e926d
---
_sources/download.md.txt | 2 +-
.../library-user-guide/upgrading/54.0.0.md.txt | 38 ++++++++++++++-
_sources/user-guide/configs.md.txt | 2 +-
_sources/user-guide/crate-configuration.md.txt | 2 +-
_sources/user-guide/example-usage.md.txt | 2 +-
library-user-guide/upgrading/54.0.0.html | 56 ++++++++++++++++++----
searchindex.js | 2 +-
7 files changed, 89 insertions(+), 15 deletions(-)
diff --git a/_sources/download.md.txt b/_sources/download.md.txt
index 3be76a6acf..ed8fc06440 100644
--- a/_sources/download.md.txt
+++ b/_sources/download.md.txt
@@ -26,7 +26,7 @@ For example:
```toml
[dependencies]
-datafusion = "52.1.0"
+datafusion = "53.0.0"
```
While DataFusion is distributed via [crates.io] as a convenience, the
diff --git a/_sources/library-user-guide/upgrading/54.0.0.md.txt b/_sources/library-user-guide/upgrading/54.0.0.md.txt
index fcc7aed293..454a8de3d1 100644
--- a/_sources/library-user-guide/upgrading/54.0.0.md.txt
+++ b/_sources/library-user-guide/upgrading/54.0.0.md.txt
@@ -77,7 +77,7 @@ fn apply_expressions(
**Node whose only expressions are in `output_ordering()` (e.g. a synthetic test node with no owned expression fields):**
-````rust,ignore
+```rust,ignore
fn apply_expressions(
&self,
f: &mut dyn FnMut(&dyn PhysicalExpr) -> Result<TreeNodeRecursion>,
@@ -90,6 +90,7 @@ fn apply_expressions(
}
Ok(tnr)
}
+```
### `ExecutionPlan::partition_statistics` now returns `Arc<Statistics>`
@@ -101,7 +102,7 @@ fn apply_expressions(
fn partition_statistics(&self, partition: Option<usize>) -> Result<Statistics> {
Ok(Statistics::new_unknown(&self.schema()))
}
-````
+```
**After:**
@@ -166,3 +167,36 @@ let is_async = (func
.downcast_ref::<AsyncScalarUDF>()
.is_some();
```
+
+### Avro API and timestamp decoding changes
+
+DataFusion has switched to use `arrow-avro` (see [#17861]) when reading avro files
+which results in a few changes:
+
+- `DataFusionError::AvroError` has been removed.
+- `From<apache_avro::Error> for DataFusionError` has been removed.
+- Avro crate re-export changed:
+ - Before: `datafusion::apache_avro`
+ - After: `datafusion::arrow_avro`
+- **Avro timestamp logical type interpretation changed.** Notable effects:
+ - Avro `timestamp-*` logical types are read as UTC timezone-aware Arrow
+ timestamps (`Timestamp(..., Some("+00:00"))`)
+ - Avro `local-timestamp-*` logical types remain timezone-naive
+ (`Timestamp(..., None)`)
+
+**Who is affected:**
+
+- Users matching on `DataFusionError::AvroError`
+- Users importing `datafusion::apache_avro`
+- Users relying on previous Avro timestamp logical type behavior
+
+**Migration guide:**
+
+- Replace `datafusion::apache_avro` imports with `datafusion::arrow_avro`.
+- Update error handling code that matches on `DataFusionError::AvroError` to use
+ the current error surface.
+- Validate timestamp handling where timezone semantics matter:
+ `timestamp-*` is UTC timezone-aware, while `local-timestamp-*` is
+ timezone-naive.
+
+[#17861]: https://github.com/apache/datafusion/pull/17861
diff --git a/_sources/user-guide/configs.md.txt b/_sources/user-guide/configs.md.txt
index 56ab4d1539..e7ec8933a9 100644
--- a/_sources/user-guide/configs.md.txt
+++ b/_sources/user-guide/configs.md.txt
@@ -101,7 +101,7 @@ The following configuration settings are available:
| datafusion.execution.parquet.dictionary_page_size_limit | 1048576 | (writing) Sets best effort maximum dictionary page size, in bytes [...]
| datafusion.execution.parquet.statistics_enabled | page | (writing) Sets if statistics are enabled for any column Valid values are: "none", "chunk", and "page" These values are not case sensitive. If NULL, uses default parquet writer setting [...]
| datafusion.execution.parquet.max_row_group_size | 1048576 | (writing) Target maximum number of rows in each row group (defaults to 1M rows). Writing larger row groups requires more memory to write, but can get better compression and be faster to read. [...]
-| datafusion.execution.parquet.created_by | datafusion version 52.3.0 | (writing) Sets "created by" property [...]
+| datafusion.execution.parquet.created_by | datafusion version 53.0.0 | (writing) Sets "created by" property [...]
| datafusion.execution.parquet.column_index_truncate_length | 64 | (writing) Sets column index truncate length [...]
| datafusion.execution.parquet.statistics_truncate_length | 64 | (writing) Sets statistics truncate length. If NULL, uses default parquet writer setting [...]
| datafusion.execution.parquet.data_page_row_count_limit | 20000 | (writing) Sets best effort maximum number of rows in data page [...]
diff --git a/_sources/user-guide/crate-configuration.md.txt b/_sources/user-guide/crate-configuration.md.txt
index 44b4d39839..2acb2140ef 100644
--- a/_sources/user-guide/crate-configuration.md.txt
+++ b/_sources/user-guide/crate-configuration.md.txt
@@ -156,7 +156,7 @@ By default, Datafusion returns errors as a plain text message. You can enable mo
such as backtraces by enabling the `backtrace` feature to your `Cargo.toml` file like this:
```toml
-datafusion = { version = "52.1.0", features = ["backtrace"]}
+datafusion = { version = "53.0.0", features = ["backtrace"]}
```
Set environment [variables](https://doc.rust-lang.org/std/backtrace/index.html#environment-variables)
diff --git a/_sources/user-guide/example-usage.md.txt b/_sources/user-guide/example-usage.md.txt
index 83ba530d2b..fd755715ee 100644
--- a/_sources/user-guide/example-usage.md.txt
+++ b/_sources/user-guide/example-usage.md.txt
@@ -29,7 +29,7 @@ Find latest available Datafusion version on [DataFusion's
crates.io] page. Add the dependency to your `Cargo.toml` file:
```toml
-datafusion = "52.1.0"
+datafusion = "53.0.0"
tokio = { version = "1.0", features = ["rt-multi-thread"] }
```
diff --git a/library-user-guide/upgrading/54.0.0.html b/library-user-guide/upgrading/54.0.0.html
index e9a51801c3..ec14828ff6 100644
--- a/library-user-guide/upgrading/54.0.0.html
+++ b/library-user-guide/upgrading/54.0.0.html
@@ -481,14 +481,14 @@ to the main branch and are awaiting release in this version.</p>
<span class="w"> </span><span class="p">}</span>
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">tnr</span><span class="p">)</span>
<span class="p">}</span>
-
-<span class="p">###</span><span class="w"> </span><span class="err">`</span><span class="n">ExecutionPlan</span><span class="p">::</span><span class="n">partition_statistics</span><span class="err">`</span><span class="w"> </span><span class="n">now</span><span class="w"> </span><span class="n">returns</span><span class="w"> </span><span class="err">`</span><span class="n">Arc</span><span class="o"><</span><span class="n">Statistics</span><span class="o">></span><span class="err">`</span>
-
-<span class="err">`</span><span class="n">ExecutionPlan</span><span class="p">::</span><span class="n">partition_statistics</span><span class="err">`</span><span class="w"> </span><span class="n">now</span><span class="w"> </span><span class="n">returns</span><span class="w"> </span><span class="err">`</span><span class="nb">Result</span><span class="o"><</span><span class="n">Arc</span><span class="o"><</span><span class="n">Statistics</span><span class="o">>></span><span cl [...]
-
-<span class="o">**</span><span class="n">Before</span><span class="p">:</span><span class="o">**</span>
-<span class="err">```</span><span class="n">rust</span><span class="p">,</span><span class="n">ignore</span>
-<span class="k">fn</span><span class="w"> </span><span class="nf">partition_statistics</span><span class="p">(</span><span class="o">&</span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">partition</span><span class="p">:</span><span class="w"> </span><span class="nb">Option</span><span class="o"><</span><span class="kt">usize</span><span class="o">></span><span class="p">)</span><span class="w"> </span><span class="p">-></span><s [...]
+</pre></div>
+</div>
+</section>
+<section id="executionplan-partition-statistics-now-returns-arc-statistics">
+<h3><code class="docutils literal notranslate"><span class="pre">ExecutionPlan::partition_statistics</span></code> now returns <code class="docutils literal notranslate"><span class="pre">Arc<Statistics></span></code><a class="headerlink" href="#executionplan-partition-statistics-now-returns-arc-statistics" title="Link to this heading">#</a></h3>
+<p><code class="docutils literal notranslate"><span class="pre">ExecutionPlan::partition_statistics</span></code> now returns <code class="docutils literal notranslate"><span class="pre">Result<Arc<Statistics>></span></code> instead of <code class="docutils literal notranslate"><span class="pre">Result<Statistics></span></code>. This avoids cloning <code class="docutils literal notranslate"><span class="pre">Statistics</span></code> when it is shared across multiple con [...]
+<p><strong>Before:</strong></p>
+<div class="highlight-rust notranslate"><div class="highlight"><pre><span></span><span class="k">fn</span><span class="w"> </span><span class="nf">partition_statistics</span><span class="p">(</span><span class="o">&</span><span class="bp">self</span><span class="p">,</span><span class="w"> </span><span class="n">partition</span><span class="p">:</span><span class="w"> </span><span class="nb">Option</span><span class="o"><</span><span class="kt">usize</span><span class="o">></sp [...]
<span class="w"> </span><span class="nb">Ok</span><span class="p">(</span><span class="n">Statistics</span><span class="p">::</span><span class="n">new_unknown</span><span class="p">(</span><span class="o">&</span><span class="bp">self</span><span class="p">.</span><span class="n">schema</span><span class="p">()))</span>
<span class="p">}</span>
</pre></div>
@@ -548,6 +548,44 @@ the call to <code class="docutils literal notranslate"><span class="pre">.as_any
</pre></div>
</div>
</section>
+<section id="avro-api-and-timestamp-decoding-changes">
+<h3>Avro API and timestamp decoding changes<a class="headerlink" href="#avro-api-and-timestamp-decoding-changes" title="Link to this heading">#</a></h3>
+<p>DataFusion has switched to use <code class="docutils literal notranslate"><span class="pre">arrow-avro</span></code> (see <a class="reference external" href="https://github.com/apache/datafusion/pull/17861">#17861</a>) when reading avro files
+which results in a few changes:</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">DataFusionError::AvroError</span></code> has been removed.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">From<apache_avro::Error></span> <span class="pre">for</span> <span class="pre">DataFusionError</span></code> has been removed.</p></li>
+<li><p>Avro crate re-export changed:</p>
+<ul>
+<li><p>Before: <code class="docutils literal notranslate"><span class="pre">datafusion::apache_avro</span></code></p></li>
+<li><p>After: <code class="docutils literal notranslate"><span class="pre">datafusion::arrow_avro</span></code></p></li>
+</ul>
+</li>
+<li><p><strong>Avro timestamp logical type interpretation changed.</strong> Notable effects:</p>
+<ul>
+<li><p>Avro <code class="docutils literal notranslate"><span class="pre">timestamp-*</span></code> logical types are read as UTC timezone-aware Arrow
+timestamps (<code class="docutils literal notranslate"><span class="pre">Timestamp(...,</span> <span class="pre">Some("+00:00"))</span></code>)</p></li>
+<li><p>Avro <code class="docutils literal notranslate"><span class="pre">local-timestamp-*</span></code> logical types remain timezone-naive
+(<code class="docutils literal notranslate"><span class="pre">Timestamp(...,</span> <span class="pre">None)</span></code>)</p></li>
+</ul>
+</li>
+</ul>
+<p><strong>Who is affected:</strong></p>
+<ul class="simple">
+<li><p>Users matching on <code class="docutils literal notranslate"><span class="pre">DataFusionError::AvroError</span></code></p></li>
+<li><p>Users importing <code class="docutils literal notranslate"><span class="pre">datafusion::apache_avro</span></code></p></li>
+<li><p>Users relying on previous Avro timestamp logical type behavior</p></li>
+</ul>
+<p><strong>Migration guide:</strong></p>
+<ul class="simple">
+<li><p>Replace <code class="docutils literal notranslate"><span class="pre">datafusion::apache_avro</span></code> imports with <code class="docutils literal notranslate"><span class="pre">datafusion::arrow_avro</span></code>.</p></li>
+<li><p>Update error handling code that matches on <code class="docutils literal notranslate"><span class="pre">DataFusionError::AvroError</span></code> to use
+the current error surface.</p></li>
+<li><p>Validate timestamp handling where timezone semantics matter:
+<code class="docutils literal notranslate"><span class="pre">timestamp-*</span></code> is UTC timezone-aware, while <code class="docutils literal notranslate"><span class="pre">local-timestamp-*</span></code> is
+timezone-naive.</p></li>
+</ul>
+</section>
</section>
</section>
@@ -600,7 +638,9 @@ the call to <code class="docutils literal notranslate"><span class="pre">.as_any
<ul class="visible nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#datafusion-54-0-0">DataFusion 54.0.0</a><ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#executionplan-apply-expressions-is-now-a-required-method"><code class="docutils literal notranslate"><span class="pre">ExecutionPlan::apply_expressions</span></code> is now a required method</a></li>
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#executionplan-partition-statistics-now-returns-arc-statistics"><code class="docutils literal notranslate"><span class="pre">ExecutionPlan::partition_statistics</span></code> now returns <code class="docutils literal notranslate"><span class="pre">Arc<Statistics></span></code></a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#remove-as-any-from-scalarudfimpl">Remove <code class="docutils literal notranslate"><span class="pre">as_any</span></code> from <code class="docutils literal notranslate"><span class="pre">ScalarUDFImpl</span></code></a></li>
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#avro-api-and-timestamp-decoding-changes">Avro API and timestamp decoding changes</a></li>
</ul>
</li>
</ul>
diff --git a/searchindex.js b/searchindex.js
index b2166b2d03..6149e8a630 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles":{"!=":[[72,"op-neq"]],"!~":[[72,"op-re-not-match"]],"!~*":[[72,"op-re-not-match-i"]],"!~~":[[72,"id19"]],"!~~*":[[72,"id20"]],"#":[[72,"op-bit-xor"]],"%":[[72,"op-modulo"]],"&":[[72,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[14,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[72,"op-multiply"]],"+":[[72,"op-plus"]],"-":[[72,"op-minus"]],"/":[[72,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
+Search.setIndex({"alltitles":{"!=":[[72,"op-neq"]],"!~":[[72,"op-re-not-match"]],"!~*":[[72,"op-re-not-match-i"]],"!~~":[[72,"id19"]],"!~~*":[[72,"id20"]],"#":[[72,"op-bit-xor"]],"%":[[72,"op-modulo"]],"&":[[72,"op-bit-and"]],"(relation, name) tuples in logical fields and logical columns are unique":[[14,"relation-name-tuples-in-logical-fields-and-logical-columns-are-unique"]],"*":[[72,"op-multiply"]],"+":[[72,"op-plus"]],"-":[[72,"op-minus"]],"/":[[72,"op-divide"]],"1. Array Literal Con [...]
\ No newline at end of file
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org