You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by al...@apache.org on 2025/03/04 11:38:20 UTC
(datafusion) branch main updated: Fix documentation warnings and error if anymore occur (#14952)
This is an automated email from the ASF dual-hosted git repository.
alamb pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion.git
The following commit(s) were added to refs/heads/main by this push:
new ec222096b9 Fix documentation warnings and error if anymore occur (#14952)
ec222096b9 is described below
commit ec222096b9d750614a2e5a27950129b172ea11f4
Author: Amos Aidoo <ai...@gmail.com>
AuthorDate: Tue Mar 4 12:38:16 2025 +0100
Fix documentation warnings and error if anymore occur (#14952)
* feat: treat sphinx-build warnings as errors
this should fail during build until warnings are fixed
* fix: ./gen is not valid link in current context
replaces the link with a code backticks
* fix: replace with valid docs.rs link
* fix: respect hierarchy from H2 -> H3
* apply prettier fixes
* Add build.sh to the CI check
* tweak
* fix
* Update spans elsewhere
---------
Co-authored-by: Andrew Lamb <an...@nerdnetworks.org>
---
.github/workflows/docs.yaml | 17 ++++++++++++
.github/workflows/docs_pr.yaml | 30 +++++++++++++++++++++-
datafusion/common/src/config.rs | 2 +-
.../sqllogictest/test_files/information_schema.slt | 2 +-
docs/build.sh | 2 +-
docs/source/contributor-guide/howtos.md | 4 +--
docs/source/library-user-guide/query-optimizer.md | 4 +--
docs/source/user-guide/configs.md | 2 +-
8 files changed, 54 insertions(+), 9 deletions(-)
diff --git a/.github/workflows/docs.yaml b/.github/workflows/docs.yaml
index 0b43339f57..5f1b2c1395 100644
--- a/.github/workflows/docs.yaml
+++ b/.github/workflows/docs.yaml
@@ -1,3 +1,20 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
on:
push:
branches:
diff --git a/.github/workflows/docs_pr.yaml b/.github/workflows/docs_pr.yaml
index 3fad08643a..d3c901c5b7 100644
--- a/.github/workflows/docs_pr.yaml
+++ b/.github/workflows/docs_pr.yaml
@@ -15,6 +15,7 @@
# specific language governing permissions and limitations
# under the License.
+# Tests for Docs that runs on PRs
name: Docs
concurrency:
@@ -48,7 +49,34 @@ jobs:
uses: ./.github/actions/setup-builder
with:
rust-version: stable
- - name: Run doctests
+ - name: Run doctests (embedded rust examples)
run: cargo test --doc --features avro,json
- name: Verify Working Directory Clean
run: git diff --exit-code
+
+ # Test doc build
+ linux-test-doc-build:
+ name: Test doc build
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+ with:
+ submodules: true
+ fetch-depth: 1
+ - name: Setup Python
+ uses: actions/setup-python@v5
+ with:
+ python-version: "3.12"
+ - name: Install doc dependencies
+ run: |
+ set -x
+ python3 -m venv venv
+ source venv/bin/activate
+ pip install -r docs/requirements.txt
+ - name: Build docs html and check for warnings
+ run: |
+ set -x
+ source venv/bin/activate
+ cd docs
+ ./build.sh # fails on errors
+
diff --git a/datafusion/common/src/config.rs b/datafusion/common/src/config.rs
index 65ecd40327..e2f61cfc08 100644
--- a/datafusion/common/src/config.rs
+++ b/datafusion/common/src/config.rs
@@ -253,7 +253,7 @@ config_namespace! {
pub support_varchar_with_length: bool, default = true
/// When set to true, the source locations relative to the original SQL
- /// query (i.e. [`Span`](sqlparser::tokenizer::Span)) will be collected
+ /// query (i.e. [`Span`](https://docs.rs/sqlparser/latest/sqlparser/tokenizer/struct.Span.html)) will be collected
/// and recorded in the logical plan nodes.
pub collect_spans: bool, default = false
diff --git a/datafusion/sqllogictest/test_files/information_schema.slt b/datafusion/sqllogictest/test_files/information_schema.slt
index b0538b5e65..5c2f730a35 100644
--- a/datafusion/sqllogictest/test_files/information_schema.slt
+++ b/datafusion/sqllogictest/test_files/information_schema.slt
@@ -355,7 +355,7 @@ datafusion.optimizer.repartition_sorts true Should DataFusion execute sorts in a
datafusion.optimizer.repartition_windows true Should DataFusion repartition data using the partitions keys to execute window functions in parallel using the provided `target_partitions` level
datafusion.optimizer.skip_failed_rules false When set to true, the logical plan optimizer will produce warning messages if any optimization rules produce errors and then proceed to the next rule. When set to false, any rules that produce errors will cause the query to fail
datafusion.optimizer.top_down_join_key_reordering true When set to true, the physical plan optimizer will run a top down process to reorder the join keys
-datafusion.sql_parser.collect_spans false When set to true, the source locations relative to the original SQL query (i.e. [`Span`](sqlparser::tokenizer::Span)) will be collected and recorded in the logical plan nodes.
+datafusion.sql_parser.collect_spans false When set to true, the source locations relative to the original SQL query (i.e. [`Span`](https://docs.rs/sqlparser/latest/sqlparser/tokenizer/struct.Span.html)) will be collected and recorded in the logical plan nodes.
datafusion.sql_parser.dialect generic Configure the SQL dialect used by DataFusion's parser; supported values include: Generic, MySQL, PostgreSQL, Hive, SQLite, Snowflake, Redshift, MsSQL, ClickHouse, BigQuery, Ansi, DuckDB and Databricks.
datafusion.sql_parser.enable_ident_normalization true When set to true, SQL parser will normalize ident (convert ident to lowercase when not quoted)
datafusion.sql_parser.enable_options_value_normalization false When set to true, SQL parser will normalize options value (convert value to lowercase). Note that this option is ignored and will be removed in the future. All case-insensitive values are normalized automatically.
diff --git a/docs/build.sh b/docs/build.sh
index 14464fab40..73516e8e9c 100755
--- a/docs/build.sh
+++ b/docs/build.sh
@@ -28,4 +28,4 @@ sed -i -e 's/\.\.\/\.\.\/\.\.\//https:\/\/github.com\/apache\/arrow-datafusion\/
python rustdoc_trim.py
-make SOURCEDIR=`pwd`/temp html
+make SOURCEDIR=`pwd`/temp SPHINXOPTS=-W html
diff --git a/docs/source/contributor-guide/howtos.md b/docs/source/contributor-guide/howtos.md
index 556242751f..89a1bc7360 100644
--- a/docs/source/contributor-guide/howtos.md
+++ b/docs/source/contributor-guide/howtos.md
@@ -141,9 +141,9 @@ taplo fmt
## How to update protobuf/gen dependencies
-The prost/tonic code can be generated by running `./regen.sh`, which in turn invokes the Rust binary located in [gen](./gen)
+The prost/tonic code can be generated by running `./regen.sh`, which in turn invokes the Rust binary located in `./gen`
-This is necessary after modifying the protobuf definitions or altering the dependencies of [gen](./gen), and requires a
+This is necessary after modifying the protobuf definitions or altering the dependencies of `./gen`, and requires a
valid installation of [protoc] (see [installation instructions] for details).
```bash
diff --git a/docs/source/library-user-guide/query-optimizer.md b/docs/source/library-user-guide/query-optimizer.md
index af27bb7505..03cd7b5bbb 100644
--- a/docs/source/library-user-guide/query-optimizer.md
+++ b/docs/source/library-user-guide/query-optimizer.md
@@ -401,7 +401,7 @@ interval arithmetic to take an expression such as `a > 2500 AND a <= 5000` and
build an accurate selectivity estimate that can then be used to find more efficient
plans.
-#### `AnalysisContext` API
+### `AnalysisContext` API
The `AnalysisContext` serves as a shared knowledge base during expression evaluation
and boundary analysis. Think of it as a dynamic repository that maintains information about:
@@ -414,7 +414,7 @@ What makes `AnalysisContext` particularly powerful is its ability to propagate i
through the expression tree. As each node in the expression tree is analyzed, it can both
read from and write to this shared context, allowing for sophisticated boundary analysis and inference.
-#### `ColumnStatistics` for Cardinality Estimation
+### `ColumnStatistics` for Cardinality Estimation
Column statistics form the foundation of optimization decisions. Rather than just tracking
simple metrics, DataFusion's `ColumnStatistics` provides a rich set of information including:
diff --git a/docs/source/user-guide/configs.md b/docs/source/user-guide/configs.md
index 8c4aad5107..f29fbb6745 100644
--- a/docs/source/user-guide/configs.md
+++ b/docs/source/user-guide/configs.md
@@ -127,5 +127,5 @@ Environment variables are read during `SessionConfig` initialisation so they mus
| datafusion.sql_parser.enable_options_value_normalization | false | When set to true, SQL parser will normalize options value (convert value to lowercase). Note that this option is ignored and will be removed in the future. All case-insensitive values are normalized automatically. [...]
| datafusion.sql_parser.dialect | generic | Configure the SQL dialect used by DataFusion's parser; supported values include: Generic, MySQL, PostgreSQL, Hive, SQLite, Snowflake, Redshift, MsSQL, ClickHouse, BigQuery, Ansi, DuckDB and Databricks. [...]
| datafusion.sql_parser.support_varchar_with_length | true | If true, permit lengths for `VARCHAR` such as `VARCHAR(20)`, but ignore the length. If false, error if a `VARCHAR` with a length is specified. The Arrow type system does not have a notion of maximum string length and thus DataFusion can not enforce such limits. [...]
-| datafusion.sql_parser.collect_spans | false | When set to true, the source locations relative to the original SQL query (i.e. [`Span`](sqlparser::tokenizer::Span)) will be collected and recorded in the logical plan nodes. [...]
+| datafusion.sql_parser.collect_spans | false | When set to true, the source locations relative to the original SQL query (i.e. [`Span`](https://docs.rs/sqlparser/latest/sqlparser/tokenizer/struct.Span.html)) will be collected and recorded in the logical plan nodes. [...]
| datafusion.sql_parser.recursion_limit | 50 | Specifies the recursion depth limit when parsing complex SQL Queries [...]
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org