You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cloudberry.apache.org by "MisterRaindrop (via GitHub)" <gi...@apache.org> on 2026/10/16 07:23:00 UTC
[PR] lance_fdw: add a foreign-data wrapper that scans Lance datasets [cloudberry]
MisterRaindrop opened a new pull request, #2140:
URL: https://github.com/apache/cloudberry/pull/2140
Part of #2139 (1 of 4 for contrib/lance_fdw).
### What does this PR do?
Adds `contrib/lance_fdw`, a read-only foreign-data wrapper that reads Lance
datasets, local or on S3, in place, with the scan spread over every segment
by fragment and pinned to one dataset version per statement. Every A-tier
Arrow type is converted; anything else is refused by name. Arrow batches,
which lance allocates outside palloc, are charged to the vmem tracker.
It is built only with `configure --with-lance` (or with PGXS and
`LANCE_C_PREFIX`) and links an installed `liblance_c`. No Rust code, crates or
cargo invocation enter this tree; `contrib/lance_fdw/ci/build-lance-c.sh`
builds the tested lance-c revision for CI. The only bundled third-party code
is Apache Arrow nanoarrow 0.7.0 (Apache-2.0).
CI: a new `ic-lance-fdw` entry builds lance-c (cached), builds the extension
with PGXS, generates the test datasets with pylance, serves them from a local
SeaweedFS S3 gateway, and runs the 15 regression suites.
### Type of Change
- [x] New feature (non-breaking change)
### Test Plan
- [x] Integration tests added/updated (15 pg_regress suites)
- [x] Passed `make installcheck` (contrib/lance_fdw, Rocky 8, 9 and 10 in CI)
### Impact
**Dependencies:** an installed liblance_c when configured with `--with-lance`;
nothing otherwise.
### Checklist
- [ ] Followed [contribution guide](https://cloudberry.apache.org/contribute/code)
- [x] Added/updated documentation (contrib/lance_fdw/README.md)
- [ ] Reviewed code for security implications
- [x] This PR contains AI-assisted code generation
- [ ] Requested review from [cloudberry committers](https://github.com/orgs/apache/teams/cloudberry-committers)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] lance_fdw: add a foreign-data wrapper that scans Lance datasets [cloudberry]
Posted by "tuhaihe (via GitHub) via commits" <co...@cloudberry.apache.org>.
tuhaihe commented on code in PR #2140:
URL: https://github.com/apache/cloudberry/pull/2140#discussion_r4463434106
##########
contrib/lance_fdw/ci/build-lance-c.sh:
##########
@@ -0,0 +1,77 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# contrib/lance_fdw/ci/build-lance-c.sh - build liblance_c for CI.
+#
+# lance_fdw links liblance_c, which is written in Rust and is not part of the
+# Apache Cloudberry source tree. This builds the tested revision from source
+# and installs its header and library under PREFIX:
+#
+# PREFIX/include/lance/lance.h
+# PREFIX/lib/liblance_c.so
+#
+# Usage: build-lance-c.sh PREFIX (as root: it installs protoc and Rust)
+# A PREFIX that already holds the library, e.g. restored from a cache, is kept.
+
+set -euo pipefail
+
+# The lance-c revision lance_fdw is tested against. Its C ABI is not stable
+# yet, so this moves only together with the code that uses it.
+LANCE_C_REPO=${LANCE_C_REPO:-https://github.com/lance-format/lance-c}
+LANCE_C_COMMIT=${LANCE_C_COMMIT:-c43774ce81a2351e61c72a9065a76bd72a727673}
+RUST_VERSION=${RUST_VERSION:-1.91.0}
+PROTOC_VERSION=${PROTOC_VERSION:-24.3}
+
+prefix=${1:?usage: build-lance-c.sh PREFIX}
+
+if [[ -f "$prefix/lib/liblance_c.so" && -f "$prefix/include/lance/lance.h" ]]; then
+ echo "build-lance-c: $prefix already holds liblance_c, nothing to build"
+ exit 0
+fi
+
+work=$(mktemp -d)
+trap 'rm -rf "$work"' EXIT
+
+case $(uname -m) in
+ x86_64) protoc_arch=x86_64 ;;
+ aarch64) protoc_arch=aarch_64 ;;
+ *) echo "build-lance-c: no protoc build for $(uname -m)" >&2; exit 1 ;;
+esac
+curl -fsSL --retry 5 --retry-connrefused -o "$work/protoc.zip" \
+ "https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOC_VERSION}/protoc-${PROTOC_VERSION}-linux-${protoc_arch}.zip"
+unzip -q -o "$work/protoc.zip" -d "$work/protoc" bin/protoc 'include/*'
+export PROTOC=$work/protoc/bin/protoc
+"$PROTOC" --version
+
+export RUSTUP_HOME=$work/rustup CARGO_HOME=$work/cargo
+curl -fsSL --retry 5 --retry-connrefused https://sh.rustup.rs |
+ sh -s -- -y --no-modify-path --profile minimal --default-toolchain "$RUST_VERSION"
+export PATH=$CARGO_HOME/bin:$PATH
+rustc --version
+
+git init -q "$work/lance-c"
+git -C "$work/lance-c" fetch -q --depth 1 "$LANCE_C_REPO" "$LANCE_C_COMMIT"
+git -C "$work/lance-c" checkout -q FETCH_HEAD
+echo "build-lance-c: building lance-c $LANCE_C_COMMIT"
+(cd "$work/lance-c" && cargo build --release --locked)
Review Comment:
We can also ship the `liblance` in the future. FYI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] lance_fdw: add a foreign-data wrapper that scans Lance datasets [cloudberry]
Posted by "MisterRaindrop (via GitHub) via commits" <co...@cloudberry.apache.org>.
MisterRaindrop commented on code in PR #2140:
URL: https://github.com/apache/cloudberry/pull/2140#discussion_r4477283504
##########
contrib/lance_fdw/src/lance_fdw.h:
##########
@@ -0,0 +1,50 @@
+/*-------------------------------------------------------------------------
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ * lance_fdw.h
+ * Definitions shared by every lance_fdw module.
+ *
+ * Include this instead of <lance/lance.h> or <nanoarrow/nanoarrow.h>: both
+ * declare the Arrow C data interface structs, but only nanoarrow.h defines the
+ * ARROW_FLAG_* macros and it guards that whole block on
+ * ARROW_FLAG_DICTIONARY_ORDERED. Pulling in lance.h first would therefore
+ * leave the flags undefined and make nanoarrow.h declare ArrowArrayStream a
+ * second time, so the order below is not cosmetic.
+ *
+ * Copyright (c) 2026, Apache Cloudberry (incubating) contributors
Review Comment:
Removed in e09e2b0d6f2. It was the only line like it in lance_fdw.
##########
configure.ac:
##########
@@ -2134,6 +2152,10 @@ if test "$with_libxml" = yes ; then
AC_CHECK_HEADER(libxml/parser.h, [], [AC_MSG_ERROR([header file <libxml/parser.h> is required for XML support])])
fi
+if test "$with_lance" = yes ; then
+ AC_CHECK_HEADER(lance/lance.h, [], [AC_MSG_ERROR([header file <lance/lance.h> is required for Lance support])])
+fi
+
Review Comment:
I don't think there are any. liblance_c is lance_fdw's only external dependency; the link check also passes `-lpthread -ldl -lm`, which it needs. protoc is only used to build lance-c itself: neither lance.h nor lance_fdw uses protobuf, and the module doesn't link it. So once a liblance_c is installed, configure has nothing else to look for. The protoc requirement is checked where it applies, in `ci/build-lance-c.sh`, which uses the image's protoc if it is 3.15 or newer and otherwise downloads a pinned, checksummed one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] lance_fdw: add a foreign-data wrapper that scans Lance datasets [cloudberry]
Posted by "tuhaihe (via GitHub) via commits" <co...@cloudberry.apache.org>.
tuhaihe commented on PR #2140:
URL: https://github.com/apache/cloudberry/pull/2140#issuecomment-6622102707
Review by Claude Code, FYI.
---
[pr-2140.md](https://github.com/user-attachments/files/34350780/pr-2140.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] lance_fdw: add a foreign-data wrapper that scans Lance datasets [cloudberry]
Posted by "tuhaihe (via GitHub) via commits" <co...@cloudberry.apache.org>.
tuhaihe commented on code in PR #2140:
URL: https://github.com/apache/cloudberry/pull/2140#discussion_r4463428329
##########
contrib/lance_fdw/ci/build-lance-c.sh:
##########
@@ -0,0 +1,77 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# contrib/lance_fdw/ci/build-lance-c.sh - build liblance_c for CI.
+#
+# lance_fdw links liblance_c, which is written in Rust and is not part of the
+# Apache Cloudberry source tree. This builds the tested revision from source
+# and installs its header and library under PREFIX:
+#
+# PREFIX/include/lance/lance.h
+# PREFIX/lib/liblance_c.so
+#
+# Usage: build-lance-c.sh PREFIX (as root: it installs protoc and Rust)
+# A PREFIX that already holds the library, e.g. restored from a cache, is kept.
+
+set -euo pipefail
+
+# The lance-c revision lance_fdw is tested against. Its C ABI is not stable
+# yet, so this moves only together with the code that uses it.
+LANCE_C_REPO=${LANCE_C_REPO:-https://github.com/lance-format/lance-c}
+LANCE_C_COMMIT=${LANCE_C_COMMIT:-c43774ce81a2351e61c72a9065a76bd72a727673}
+RUST_VERSION=${RUST_VERSION:-1.91.0}
+PROTOC_VERSION=${PROTOC_VERSION:-24.3}
+
+prefix=${1:?usage: build-lance-c.sh PREFIX}
+
+if [[ -f "$prefix/lib/liblance_c.so" && -f "$prefix/include/lance/lance.h" ]]; then
+ echo "build-lance-c: $prefix already holds liblance_c, nothing to build"
+ exit 0
+fi
+
+work=$(mktemp -d)
+trap 'rm -rf "$work"' EXIT
+
+case $(uname -m) in
+ x86_64) protoc_arch=x86_64 ;;
+ aarch64) protoc_arch=aarch_64 ;;
+ *) echo "build-lance-c: no protoc build for $(uname -m)" >&2; exit 1 ;;
+esac
+curl -fsSL --retry 5 --retry-connrefused -o "$work/protoc.zip" \
+ "https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOC_VERSION}/protoc-${PROTOC_VERSION}-linux-${protoc_arch}.zip"
+unzip -q -o "$work/protoc.zip" -d "$work/protoc" bin/protoc 'include/*'
+export PROTOC=$work/protoc/bin/protoc
+"$PROTOC" --version
Review Comment:
We should already ship the `protoc` in our development images.
```bash
# Rocky 8
[gpadmin@cdw ~]$ protoc --version
libprotoc 3.5.0
# Rocky 9
[gpadmin@cdw ~]$ protoc --version
libprotoc 3.14.0
# Rocky 10
[gpadmin@cdw ~]$ protoc --version
libprotoc 3.19.6
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] lance_fdw: add a foreign-data wrapper that scans Lance datasets [cloudberry]
Posted by "MisterRaindrop (via GitHub) via commits" <co...@cloudberry.apache.org>.
MisterRaindrop commented on PR #2140:
URL: https://github.com/apache/cloudberry/pull/2140#issuecomment-6649083370
Thanks for the thorough review. Everything below is folded into the two commits (now `a34953b8d08` and `62834278016`, rebased onto main). I reproduced each finding on the old code first, on a 3-primary demo cluster with the PR's fixtures, then checked the fix against the same reproduction. The suites now pass 16/16 under the planner, and 15/15 under ORCA with the new `installcheck-orca` target.
### Must fix
**1. `base_uri` did not confine non-superusers.** Reproduced, and it was wider than the two cases you listed. A plain role with only `USAGE` on a `file:///tmp/confined_empty` server read the 15 rows of `frag_3.lance` in seven ways:
- an absolute path or a `file://` uri;
- `../../…`;
- `%2e%2e/%2E%2e/…`, because the Rust uri parser decodes `%2e` before it resolves the path;
- `IMPORT FOREIGN SCHEMA … LIMIT TO ("../../…")`;
- its own server with a local `base_uri`, once it had `USAGE` on the wrapper;
- the host-credential fallback: with keys in gpadmin's `~/.aws/credentials`, a user mapping without keys read an S3 table.
Creating a server with `aws_endpoint 'http://169.254.169.254'` also succeeded. Now, following file_fdw and postgres_fdw:
- For a non-superuser, the `uri` must stay inside `base_uri`. That rules out an absolute uri, a `..` component (also spelled `%2e`), a backslash, `%2f` and `%5c`. A member of `pg_read_server_files` may still give an absolute local path. `IMPORT` checks each `LIMIT TO` name before it opens the dataset, because the generated table's validator would only run after the open.
- A local `base_uri` needs `pg_read_server_files`. `aws_endpoint` and `allow_http` need a superuser.
- A non-superuser reading an object store needs `aws_access_key_id` and `aws_secret_access_key` in its user mapping, and only `s3://` takes them, so any other scheme is refused as well. A superuser may set the new user mapping option `credentials_required 'false'` (postgres_fdw's `password_required`) for instance roles or public buckets. This is checked whenever a dataset is opened, against the user mapping in use (`checkAsUser`, as before).
- The uri and server checks run when the option is set, as in file_fdw, so a superuser can still create a table on any path and grant `SELECT` on it.
A `.` component and dots inside a name (`a..b`) are still accepted. `%252e%252e` is decoded only once and comes back as not found inside the base. The new `privileges` suite covers all of this, and it fails on the old code.
**2. `mpp_execute 'any'` broke one-version-per-statement.** Reproduced with an 'any' table hash-joined to a 1M-row distributed table while pylance appended 1000 ids per version. Every correct answer is a multiple of 1000.
| Code | Torn counts | QEs logging different versions in one statement |
|---|---|---|
| Old | 1 of 12 queries (78340) | 3 of 10 queries (e.g. 4 / 7 / 8) |
| New | 0 of 33 queries | 0 of 9 queries |
The QD now pins the version under 'any' and publishes it with a width of 0, which the QEs read as "every fragment". This is the same plan slot the all-segments path uses, and cached plans republish. A QE under 'any' with no published units is an error, as it is under all segments. The README no longer says "one process reads everything". `snapshot` gains 'any'-joined-to-distributed cases at the latest and at a pinned version, and a prepared statement run twice. The race itself can't be staged in pg_regress, so the numbers above come from the repro script.
### Suggestions
**3. Threads.** Worse than it looked: `cpu_threads` and `io_threads` did not bound the threads you counted. Thread names showed 11 `tokio-rt-worker` threads per backend on 10 cores whatever the two GUCs said. Those belong to lance-c's own runtime (`Builder::new_multi_thread()` with no worker count), which only `TOKIO_WORKER_THREADS` sizes. `LANCE_CPU_THREADS` sizes lance's separate `lance-cpu` pool, and `LANCE_IO_THREADS` is not a thread count at all; it is the number of I/O requests in flight.
So there is a new `lance_fdw.runtime_threads` (sets `TOKIO_WORKER_THREADS`). It and `cpu_threads` default to 2, and `io_threads` stays 0 (lance's per-store default; 4 would throttle S3). With 2/2 on 10 cores and 3 segments, a segment backend went from 13 threads to 5. A 5M-row scan went from 4.2 to 4.5 s locally and from 2.5 to 2.8 s over S3. The README's deploy section is rewritten around these numbers.
**4. Signals.** Synchronous signals (SEGV, BUS, FPE, ILL, TRAP, SYS) are no longer blocked. On latency, I don't think the mask is what defers a cancel. A backend's handlers only set a flag for the next `CHECK_FOR_INTERRUPTS()`, and none runs inside a lance-c call anyway, so a cancel waits for the call to return either way. That wait is the real problem, and measuring it was sobering: against an endpoint that never answers, `SELECT … ` with `statement_timeout = '1s'` returned after 125 s, spent in object_store retries.
`connect_timeout`, `timeout`, `client_max_retries` and `client_retry_timeout` are now server options passed to object_store and lance. With `1s / 2s / 0 / 5` the same statement failed after 8 s. The two retry options are validated at DDL time, because lance silently keeps its defaults for an unparsable value.
I didn't take the other two options:
- Warming the pool once wouldn't hold, because tokio keeps adding blocking threads later.
- `on_thread_start` would need a lance-c change.
A real fix for latency would be to read plain scans through `lance_scanner_poll_next` with the latch, as #2142 already does for vector search. I'd rather do that as a follow-up than grow this PR further.
**5. ORCA.** ORCA does plan these scans itself (`Optimizer: GPORCA`). The new `make installcheck-orca` runs every suite except `scan_core` under `optimizer=on`, into `test/feature/orca/`, and `ic-lance-fdw` now runs both targets. All 15 pass. `scan_core` is left out only because one check forces a nested loop through `enable_hashjoin`, which ORCA does not read.
**6. In-tree build.** All three reproduced and fixed:
- `readelf` showed `RUNPATH [/usr/local/cloudberry-db/lib]` only. The module now gets a run path to the `--with-libraries` directories, and is `[…/cloudberry-db/lib:/usr/local/lance-c/lib]`.
- Without `test/regress.mk`, an in-tree `installcheck` (and `installcheck-orca`) prints why it is skipping and succeeds. PGXS, which is what CI uses, still fails, so a broken fixture step can't pass silently.
- configure now probes `lance_scanner_set_batch_size_bytes`, the newest function the code calls (lance-c 6db08fd).
**7. Downloads.**
- protoc and `rustup-init` (now downloaded directly instead of `curl | sh`; its sha256 matches the one rustup publishes) in `build-lance-c.sh`, and SeaweedFS in `prepare-tests.sh`, are pinned and checked with sha256 for both arches.
- The pylance and friends in `requirements.txt` were already pinned exactly.
On the protoc inline comment, I tried it. lance's build scripts pass `--experimental_allow_proto3_optional`, but the `substrait` crate's doesn't, and with Rocky 9's 3.14 the build fails with `substrait/type.proto: This file contains proto3 optional fields, but --experimental_allow_proto3_optional was not set`. The script now uses the image's protoc when it is 3.15 or newer, which today means Rocky 10's 3.19, and downloads the pinned 24.3 otherwise.
On shipping liblance in the images: agreed. `build-lance-c.sh` already keeps a prefix that holds the library, so an image that installs it under `/usr/local/lance-c` would just skip the build.
**8. LICENSE.** nanoarrow is now listed in the top-level LICENSE, next to the other vendored Apache-2.0 code.
**9. Stale references.** Fixed `test/README.md` → `test/fixtures/README.md` and the two in the signal comment. While doing that I found more leftovers from the standalone repository: `creds.sql`, `import.sql`, `test/fixtures/Makefile` and `conftest.py` referred to `test/run/run.sh` and `env.sh`, which aren't in this tree. The `creds` comment now says the log grep is a manual check.
**10. Nits.**
- The converters now also check each stream column's name against the one asked for.
- `lance_fdw_cache_stats()` no longer starts lance-c in a backend that hasn't opened a dataset. It returns a row of NULLs, and `install` checks that no tokio or lance thread appears afterwards.
#2141–#2143 will be rebased onto this, together with their own review fixes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] lance_fdw: add a foreign-data wrapper that scans Lance datasets [cloudberry]
Posted by "tuhaihe (via GitHub) via commits" <co...@cloudberry.apache.org>.
tuhaihe commented on code in PR #2140:
URL: https://github.com/apache/cloudberry/pull/2140#discussion_r4476237848
##########
contrib/lance_fdw/src/lance_fdw.h:
##########
@@ -0,0 +1,50 @@
+/*-------------------------------------------------------------------------
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ * lance_fdw.h
+ * Definitions shared by every lance_fdw module.
+ *
+ * Include this instead of <lance/lance.h> or <nanoarrow/nanoarrow.h>: both
+ * declare the Arrow C data interface structs, but only nanoarrow.h defines the
+ * ARROW_FLAG_* macros and it guards that whole block on
+ * ARROW_FLAG_DICTIONARY_ORDERED. Pulling in lance.h first would therefore
+ * leave the flags undefined and make nanoarrow.h declare ArrowArrayStream a
+ * second time, so the order below is not cosmetic.
+ *
+ * Copyright (c) 2026, Apache Cloudberry (incubating) contributors
Review Comment:
```suggestion
```
There is no need. Let us remove this line.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org
Re: [PR] lance_fdw: add a foreign-data wrapper that scans Lance datasets [cloudberry]
Posted by "tuhaihe (via GitHub) via commits" <co...@cloudberry.apache.org>.
tuhaihe commented on code in PR #2140:
URL: https://github.com/apache/cloudberry/pull/2140#discussion_r4476255697
##########
configure.ac:
##########
@@ -2134,6 +2152,10 @@ if test "$with_libxml" = yes ; then
AC_CHECK_HEADER(libxml/parser.h, [], [AC_MSG_ERROR([header file <libxml/parser.h> is required for XML support])])
fi
+if test "$with_lance" = yes ; then
+ AC_CHECK_HEADER(lance/lance.h, [], [AC_MSG_ERROR([header file <lance/lance.h> is required for Lance support])])
+fi
+
Review Comment:
If other dependencies, like `libprotoc`, are also required dependencies, we can add their checks too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org