You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "atiaomar1978-hub (via GitHub)" <gi...@apache.org> on 2026/07/24 18:34:21 UTC
[PR] CAMEL-24255: Add camel-duckdb component and test-infra [camel]
atiaomar1978-hub opened a new pull request, #25102:
URL: https://github.com/apache/camel/pull/25102
## Summary
- Adds **camel-duckdb** (Preview): JDBC-based producer for embedded DuckDB with operations `execute`, `query`, `insert`, `copy`, and `ping`.
- Adds **camel-test-infra-duckdb** with in-process embedded database (temp file or `:memory:`) and optional remote JDBC URL via system properties — no Docker required for default tests.
- Documents the component and adds a **4.22** upgrade-guide entry; uses `org.duckdb:duckdb_jdbc` **1.3.1.0**.
Implements [CAMEL-24255](https://issues.apache.org/jira/browse/CAMEL-24255).
_AI-generated on behalf of atiaomar1978-hub_
## Test plan
- [ ] `./mvnw -pl test-infra/camel-test-infra-duckdb,components/camel-duckdb -am test`
- [ ] `./mvnw -pl components/camel-duckdb -Psourcecheck` (regenerate catalog/docs metadata if the build updates generated files)
- AssertJ coverage: `DuckDbComponentTest`, `DuckDbJdbcSupportTest`, `DuckDbProducerTest`, `DuckDbProducerIT`
Made with [Cursor](https://cursor.com)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24255: Add camel-duckdb component and test-infra [camel]
Posted by "atiaomar1978-hub (via GitHub)" <gi...@apache.org>.
atiaomar1978-hub commented on PR #25102:
URL: https://github.com/apache/camel/pull/25102#issuecomment-5077625760
symlinks !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24255: Add camel-duckdb component and test-infra [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25102:
URL: https://github.com/apache/camel/pull/25102#issuecomment-5077592042
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:robot: CI automation will test this PR automatically.
:camel: Apache Camel Committers, please review the following items:
* First-time contributors **require MANUAL approval** for the GitHub Actions to run
* You can use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot although they are normally detected and executed by CI.
* You can label PRs using `skip-tests` and `test-dependents` to fine-tune the checks executed by this PR.
* Build and test logs are available in the summary page. **Only** [Apache Camel committers](https://camel.apache.org/community/team/#committers) have access to the summary.
:warning: Be careful when sharing logs. Review their contents before sharing them publicly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24255: Add camel-duckdb component and test-infra [camel]
Posted by "atiaomar1978-hub (via GitHub)" <gi...@apache.org>.
atiaomar1978-hub commented on PR #25102:
URL: https://github.com/apache/camel/pull/25102#issuecomment-5078964249
## CI follow-up
Addressed the failing checks on this PR:
### PR doc validation ([run 30125289181](https://github.com/apache/camel/actions/runs/30125289121))
Antora could not resolve `example$json/duckdb.json` for `duckdb-component.adoc`.
**Fix:** `5433e67351e` — commit `prepare-doc-symlinks` output:
- `docs/components/modules/ROOT/examples/json/duckdb.json`
- `docs/components/modules/ROOT/pages/duckdb-component.adoc` → symlink to component docs (same pattern as other components)
### Regen / uncommitted changes — Java 25 ([run 30150483119](https://github.com/apache/camel/actions/runs/30150483119))
After `./etc/scripts/regen.sh`, CI failed on a dirty tree: missing entry in `dsl/camel-kamelet-main/src/generated/resources/camel-component-known-dependencies.properties`.
**Fix:** `d5e212eb56d` — add:
`org.apache.camel.component.duckdb.DuckDbComponent=camel:duckdb`
### Local tests (OpenJDK 21)
```bash
./mvnw -pl components/camel-duckdb,test-infra/camel-test-infra-duckdb test failsafe:integration-test
```
**21 tests** (18 unit + 3 IT) — all green.
PR description updated accordingly. CI should re-run on `d5e212eb56d`.
_AI-generated comment on behalf of [atiaomar1978-hub](https://github.com/atiaomar1978-hub)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24255: Add camel-duckdb component and test-infra [camel]
Posted by "atiaomar1978-hub (via GitHub)" <gi...@apache.org>.
atiaomar1978-hub commented on PR #25102:
URL: https://github.com/apache/camel/pull/25102#issuecomment-5073703403
## Bugbot follow-up — addressed in c3a05971b30
Re-ran Bugbot on the branch and fixed all three reported findings:
| Severity | Issue | Fix |
|----------|-------|-----|
| **High** | Published docs / upgrade guide still showed `duckdb:databasePath[/table]` (e.g. `duckdb:analytics.db/events`), which no longer matches the component and would build wrong JDBC URLs | Synced `docs/components/.../duckdb-component.adoc`, catalog docs copy, and the 4.22 upgrade-guide example to `duckdb:databasePath?table=...` |
| **Medium** | `CamelDuckDbDatabasePath` was documented in catalog/constants but never read in `DuckDbProducer` | Producer now reads the header and opens a short-lived connection to the override path for that exchange |
| **Medium** | Endpoint reused one JDBC `Connection` across concurrent route threads without synchronization | Added `DuckDbConnectionScope` + lock around the endpoint-owned shared connection for each producer invocation; DataSource and header-override connections remain per-call and are closed after use. Docs note preferring a shared `DataSource` for concurrent producers |
**Tests:** `./mvnw -pl components/camel-duckdb test failsafe:integration-test` — 18 unit + 3 IT, all green (including `databasePathHeaderOverridesEndpointDatabase`).
_Cursor on behalf of atiaomar1978-hub_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24255: Add camel-duckdb component and test-infra [camel]
Posted by "gnodet (via GitHub)" <gi...@apache.org>.
gnodet commented on code in PR #25102:
URL: https://github.com/apache/camel/pull/25102#discussion_r3647499505
##########
components/camel-duckdb/src/main/java/org/apache/camel/component/duckdb/DuckDbException.java:
##########
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.duckdb;
+
+public class DuckDbException extends Exception {
Review Comment:
**Compilation error:** `DuckDbException` extends `Exception` (checked), but `buildCopySql()` and `resolveFormat()` in `DuckDbProducer` throw it without declaring `throws DuckDbException` in their method signatures. This will cause `javac` to reject the code.
Suggested fix — change to `RuntimeException` (more consistent with typical Camel exception patterns):
```suggestion
public class DuckDbException extends RuntimeException {
```
##########
components/camel-duckdb/src/main/java/org/apache/camel/component/duckdb/DuckDbProducer.java:
##########
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.duckdb;
+
+import java.io.File;
+import java.sql.Connection;
+import java.sql.ResultSet;
+import java.sql.Statement;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+
+import org.apache.camel.Exchange;
+import org.apache.camel.Message;
+import org.apache.camel.WrappedFile;
+import org.apache.camel.support.DefaultProducer;
+import org.apache.camel.util.ObjectHelper;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class DuckDbProducer extends DefaultProducer {
+
+ private static final Logger LOG = LoggerFactory.getLogger(DuckDbProducer.class);
+
+ private final DuckDbEndpoint endpoint;
+
+ public DuckDbProducer(DuckDbEndpoint endpoint) {
+ super(endpoint);
+ this.endpoint = endpoint;
+ }
+
+ @Override
+ public void process(Exchange exchange) throws Exception {
+ DuckDbOperation operation = resolveOperation(exchange);
+ try (ConnectionScope scope = ConnectionScope.open(endpoint)) {
+ Connection connection = scope.connection();
+ switch (operation) {
+ case EXECUTE -> doExecute(exchange, connection);
+ case QUERY -> doQuery(exchange, connection);
+ case INSERT -> doInsert(exchange, connection);
+ case COPY -> doCopy(exchange, connection);
+ case PING -> doPing(exchange, connection);
+ default -> throw new DuckDbException("Unsupported operation: " + operation);
+ }
+ }
+ }
+
+ private void doExecute(Exchange exchange, Connection connection) throws Exception {
+ String sql = resolveSql(exchange);
+ if (ObjectHelper.isEmpty(sql)) {
+ throw new DuckDbException("SQL is required for execute (message body or query option)");
+ }
+ long updated;
+ try (Statement statement = connection.createStatement()) {
+ updated = statement.executeUpdate(sql);
+ }
+ setRowsWritten(exchange, updated);
+ exchange.getMessage().setBody(updated);
+ }
+
+ private void doQuery(Exchange exchange, Connection connection) throws Exception {
+ String sql = resolveSql(exchange);
+ if (ObjectHelper.isEmpty(sql)) {
+ throw new DuckDbException("SQL is required for query (message body, query option or CamelDuckDbQuery header)");
+ }
+ try (Statement statement = connection.createStatement();
+ ResultSet resultSet = statement.executeQuery(sql)) {
+ List<Map<String, Object>> rows = DuckDbResultSetMapper.toListMap(resultSet);
+ Message message = exchange.getMessage();
+ message.setHeader(DuckDbConstants.ROWS_READ, (long) rows.size());
+ if (endpoint.getResultFormat() == DuckDbResultFormat.JSON) {
+ message.setBody(DuckDbResultSetMapper.toJson(rows));
+ } else {
+ message.setBody(rows);
+ }
+ }
+ }
+
+ private void doInsert(Exchange exchange, Connection connection) throws Exception {
+ String table = resolveTable(exchange);
+ if (ObjectHelper.isEmpty(table)) {
+ throw new DuckDbException(
+ "A table is required for insert. Provide databasePath/table, the table option, or the "
+ + DuckDbConstants.TABLE + " header.");
+ }
+ List<?> body = exchange.getIn().getBody(List.class);
+ if (body == null) {
+ throw new DuckDbException("Insert operation expects the message body to be a List of rows");
+ }
+ long written = DuckDbInsertSupport.insertMaps(connection, table, body, endpoint.getBatchSize());
+ LOG.debug("Inserted {} rows into {}", written, table);
+ setRowsWritten(exchange, written);
+ }
+
+ private void doCopy(Exchange exchange, Connection connection) throws Exception {
+ String table = resolveTable(exchange);
+ if (ObjectHelper.isEmpty(table)) {
+ throw new DuckDbException("A table is required for copy operations");
+ }
+ String path = resolveFilePath(exchange);
+ String format = resolveFormat(path);
+ String sql = buildCopySql(table, path, format);
+ long updated;
+ try (Statement statement = connection.createStatement()) {
+ updated = statement.executeUpdate(sql);
+ }
+ setRowsWritten(exchange, updated);
+ }
+
+ private void doPing(Exchange exchange, Connection connection) throws Exception {
+ boolean ok;
+ try (Statement statement = connection.createStatement();
+ ResultSet rs = statement.executeQuery("SELECT 1")) {
+ ok = rs.next() && rs.getInt(1) == 1;
+ }
+ Message message = exchange.getMessage();
+ message.setHeader(DuckDbConstants.PING_OK, ok);
+ message.setBody(ok);
+ }
+
+ private String buildCopySql(String table, String path, String format) {
Review Comment:
This method throws checked `DuckDbException` in the `default` branch but the signature does not declare `throws`. This is part of the compilation error described on `DuckDbException.java` — if `DuckDbException` is changed to extend `RuntimeException`, this is resolved automatically.
##########
components/camel-duckdb/src/main/java/org/apache/camel/component/duckdb/DuckDbInsertSupport.java:
##########
@@ -0,0 +1,84 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.duckdb;
+
+import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+
+final class DuckDbInsertSupport {
+
+ private DuckDbInsertSupport() {
+ }
+
+ static long insertMaps(Connection connection, String table, List<?> rows, int batchSize) throws Exception {
+ if (rows.isEmpty()) {
+ return 0;
+ }
+ if (!(rows.get(0) instanceof Map)) {
+ throw new DuckDbException("Insert body must be a List of Map entries when using operation=insert");
+ }
+ @SuppressWarnings("unchecked")
+ List<Map<String, Object>> maps = (List<Map<String, Object>>) rows;
+ List<String> columns = new ArrayList<>(maps.get(0).keySet());
+ if (columns.isEmpty()) {
+ throw new DuckDbException("Insert row maps must contain at least one column");
+ }
+
+ String sql = buildInsertSql(table, columns);
+ long written = 0;
+ int effectiveBatch = batchSize > 0 ? batchSize : maps.size();
+ try (PreparedStatement ps = connection.prepareStatement(sql)) {
+ for (int i = 0; i < maps.size(); i++) {
+ Map<String, Object> row = maps.get(i);
+ for (int c = 0; c < columns.size(); c++) {
+ ps.setObject(c + 1, row.get(columns.get(c)));
+ }
+ ps.addBatch();
+ if ((i + 1) % effectiveBatch == 0 || i + 1 == maps.size()) {
+ int[] counts = ps.executeBatch();
+ for (int count : counts) {
+ if (count >= 0) {
+ written += count;
+ } else {
+ written += 1;
+ }
+ }
+ }
+ }
+ }
+ return written;
+ }
+
+ private static String buildInsertSql(String table, List<String> columns) {
+ StringBuilder sql = new StringBuilder("INSERT INTO ");
+ sql.append(table).append(" (");
Review Comment:
**Defense-in-depth:** Table and column names are interpolated directly into SQL without quoting. While route authors are trusted per Camel's security model, wrapping identifiers in double quotes provides defense against reserved words and special characters in identifiers:
```java
sql.append('"').append(table).append('"').append(" (");
```
Same applies to column names in this method and in `DuckDbProducer.buildCopySql()`.
##########
components/camel-duckdb/src/main/java/org/apache/camel/component/duckdb/DuckDbEndpoint.java:
##########
@@ -0,0 +1,214 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.duckdb;
+
+import java.sql.Connection;
+import java.sql.DriverManager;
+
+import javax.sql.DataSource;
+
+import org.apache.camel.Category;
+import org.apache.camel.Consumer;
+import org.apache.camel.Processor;
+import org.apache.camel.Producer;
+import org.apache.camel.spi.Metadata;
+import org.apache.camel.spi.UriEndpoint;
+import org.apache.camel.spi.UriParam;
+import org.apache.camel.spi.UriPath;
+import org.apache.camel.support.DefaultEndpoint;
+import org.apache.camel.util.ObjectHelper;
+
+/**
+ * Interact with <a href="https://duckdb.org/">DuckDB</a>, the in-process analytical SQL database, for embedded
+ * analytics workloads.
+ */
+@UriEndpoint(firstVersion = "4.22.0", scheme = "duckdb", title = "DuckDB",
+ syntax = "duckdb:databasePath", category = { Category.DATABASE, Category.BIGDATA },
+ producerOnly = true, headersClass = DuckDbConstants.class)
+public class DuckDbEndpoint extends DefaultEndpoint {
+
+ private DataSource dataSource;
+ private Connection connection;
+ private Connection injectedConnection;
+ private boolean connectionOwned;
+
+ @UriPath(description = "Database path (:memory: or a file path). An optional /table segment sets the default table.")
+ private String databasePath = ":memory:";
+ @UriParam(description = "Target table for insert and copy operations. Can also be set in the path as databasePath/table"
+ + " or overridden with the CamelDuckDbTable header.")
+ private String table;
+ @UriParam(description = "Full JDBC URL override. When set, databasePath is not used to build the URL.")
+ private String jdbcUrl;
+ @UriParam(defaultValue = "EXECUTE", description = "The operation to perform: execute, query, insert, copy or ping.")
+ private DuckDbOperation operation = DuckDbOperation.EXECUTE;
+ @UriParam(description = "Static SQL for query when the message body is empty.")
+ private String query;
+ @UriParam(defaultValue = "0",
+ description = "Batch size for insert when the body is a List. 0 inserts all rows in one JDBC batch.")
+ private int batchSize;
+ @UriParam(defaultValue = "auto",
+ description = "File format for copy: csv, parquet, json or auto (inferred from the file extension).")
+ private String format = "auto";
+ @UriParam(defaultValue = "false", description = "Open the embedded database file in read-only mode when supported.")
+ private boolean readOnly;
Review Comment:
**Dead parameter:** The `readOnly` `@UriParam` is declared and documented, but is never referenced in `doStart()`, `resolveJdbcUrl()`, or anywhere in connection setup. Users setting `readOnly=true` would expect read-only mode but it has no effect.
Either implement it (e.g., append `?access_mode=read_only` to the JDBC URL or pass a `Properties` object to `DriverManager.getConnection()`) or remove it entirely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24255: Add camel-duckdb component and test-infra [camel]
Posted by "atiaomar1978-hub (via GitHub)" <gi...@apache.org>.
atiaomar1978-hub commented on PR #25102:
URL: https://github.com/apache/camel/pull/25102#issuecomment-5074232490
@gnodet — addressed the two follow-up items from your latest review in c8edc75cb4e:
1. **BOM ordering** — `camel-duckdb` is now after `camel-dsl-support` and before `camel-dynamic-router` in `bom/camel-bom/pom.xml` (`dsl` < `duc`).
2. **Unrelated script** — removed `scripts/dev-java-env.sh` from this changeset.
Tests still green locally (18 unit + 3 IT). Thanks again for the review.
_Cursor on behalf of atiaomar1978-hub_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24255: Add camel-duckdb component and test-infra [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #25102:
URL: https://github.com/apache/camel/pull/25102
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24255: Add camel-duckdb component and test-infra [camel]
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on PR #25102:
URL: https://github.com/apache/camel/pull/25102#issuecomment-5077848060
There are uncommitted changes
HEAD detached at pull/25102/merge
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: dsl/camel-kamelet-main/src/generated/resources/camel-component-known-dependencies.properties
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24255: Add camel-duckdb component and test-infra [camel]
Posted by "atiaomar1978-hub (via GitHub)" <gi...@apache.org>.
atiaomar1978-hub commented on PR #25102:
URL: https://github.com/apache/camel/pull/25102#issuecomment-5079052501
## CI fixes (latest)
Follow-up commits on `CAMEL-24255-camel-duckdb` to unblock checks:
### 1. PR doc validation — `5433e67351e`
Antora could not resolve `example$json/duckdb.json` for `duckdb-component.adoc`.
- Symlink `docs/components/modules/ROOT/examples/json/duckdb.json` → generated catalog JSON (same as other components).
- Symlink `docs/components/modules/ROOT/pages/duckdb-component.adoc` → `components/camel-duckdb/src/main/docs/…` (drop duplicated copy).
### 2. Regen / uncommitted changes (Java 25) — `d5e212eb56d`
After `./etc/scripts/regen.sh`, CI failed on a dirty tree: missing kamelet entry.
- Add `org.apache.camel.component.duckdb.DuckDbComponent=camel:duckdb` to `dsl/camel-kamelet-main/src/generated/resources/camel-component-known-dependencies.properties`.
### 3. Incremental tests (Java 17) — [run 30162647542](https://github.com/apache/camel/actions/runs/30162647542/job/89690636388) — `157c5756780`
Incremental build died during parent POM grep analysis (`echo | awk` → SIGPIPE under `pipefail` when this PR touches `parent/pom.xml`).
- Fix `extractPomDiff` in `.github/actions/incremental-build/incremental-build.sh` to use a here-string instead of a pipe.
PR description updated. CI should re-run on `157c5756780`.
_AI-generated comment on behalf of [atiaomar1978-hub](https://github.com/atiaomar1978-hub)._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24255: Add camel-duckdb component and test-infra [camel]
Posted by "atiaomar1978-hub (via GitHub)" <gi...@apache.org>.
atiaomar1978-hub commented on PR #25102:
URL: https://github.com/apache/camel/pull/25102#issuecomment-5073620531
Thanks for the thorough review @gnodet — all four points are addressed in 7d9c51a34a8.
- **`DuckDbException` checked exception** — changed to extend `RuntimeException` as you suggested, which also clears the `buildCopySql()` / `resolveFormat()` compilation errors.
- **Alphabetical ordering** — `camel-duckdb` now sits after `camel-dropbox` in the BOM, `catalog/camel-allcomponents/pom.xml`, `components/pom.xml` and the docs nav, and `duckdb-jdbc-version` moved to the D section of `parent/pom.xml`.
- **Dead `readOnly` parameter** — implemented rather than removed. `doStart()` now passes `duckdb.read_only=true` via a `Properties` object to `DriverManager.getConnection()`, so DuckDB itself rejects writes (`Cannot execute statement of type "INSERT" on database ... attached in read-only mode!`). `DuckDbProducerTest#readOnlyPreventsWrites` asserts both that reads still work and that the insert fails.
- **Unquoted identifiers** — added `quoteIdentifier` / `quoteQualifiedIdentifier` in `DuckDbJdbcSupport`, applied to the table and column names in `buildInsertSql()` and to the table in `buildCopySql()`. Qualified names such as `main.events` are quoted per segment so the qualification survives, and identifiers the route author already quoted are passed through untouched. There is an end-to-end test inserting into a table named `order` with columns `select` and `group`.
One extra fix while I was in there, called out separately in the PR description: the `databasePath[/table]` path convention split the remaining URI at the **first** `/`, so `duckdb:/data/warehouse.db` silently resolved to an in-memory database with `tmp/warehouse.db` as the table. Since the declared endpoint syntax is already just `duckdb:databasePath`, I removed the path convention — the table now comes from the `table` option or the `CamelDuckDbTable` header. Happy to split that into its own commit or revert it if you would rather keep the path form.
Local run is green: 17 unit + 3 integration tests, formatter/impsort clean, `duckdb.json` regenerated.
_Cursor on behalf of atiaomar1978-hub_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] CAMEL-24255: Add camel-duckdb component and test-infra [camel]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25102:
URL: https://github.com/apache/camel/pull/25102#issuecomment-5079231612
<!-- ci-tested-modules -->
:test_tube: **CI tested the following changed modules:**
- `bom/camel-bom`
- `catalog/camel-allcomponents`
- `catalog/camel-catalog`
- `components`
- `components/camel-duckdb`
- `core/camel-main`
- `docs`
- `dsl/camel-componentdsl`
- `dsl/camel-endpointdsl`
- `dsl/camel-kamelet-main`
- `parent`
- `test-infra`
- `test-infra/camel-test-infra-duckdb`
> :information_source: Dependent modules were not tested because the total number of affected modules exceeded the threshold (50). Use the `test-dependents` label to force testing all dependents.
:white_check_mark: **POM dependency changes: targeted tests included**
Changed properties: duckdb-jdbc-version
<details><summary>Modules affected by dependency changes (1)</summary>
- `:camel-duckdb`
</details>
---
<details><summary>:microscope: Scalpel shadow comparison — Scalpel: 70 tested, 23 compile-only — current: 63 all tested</summary>
[Maveniverse Scalpel](https://github.com/maveniverse/scalpel) detected **93 affected modules** (current approach: 63).
<details><summary>:warning: Modules only in Scalpel (30)</summary>
- `apache-camel`
- `camel-allcomponents`
- `camel-bom`
- `camel-catalog`
- `camel-catalog-console`
- `camel-catalog-lucene`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-componentdsl`
- `camel-csimple-maven-plugin`
- `camel-endpointdsl`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-kubernetes`
- `camel-jbang-plugin-test`
- `camel-kamelet-main`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `docs`
- `dummy-component`
</details>
Changed properties: duckdb-jdbc-version
Changed managed dependencies: org.apache.camel:camel-duckdb
**Skip-tests mode would test 70 modules** (10 direct + 60 downstream), **skip tests for 23** (generated code, meta-modules)
<details><summary>Modules Scalpel would test (70)</summary>
- `camel-a2a`
- `camel-allcomponents`
- `camel-as2`
- `camel-bom`
- `camel-catalog`
- `camel-clickup`
- `camel-componentdsl`
- `camel-consul`
- `camel-core-all`
- `camel-csv`
- `camel-duckdb`
- `camel-endpointdsl`
- `camel-fhir`
- `camel-groovy`
- `camel-hazelcast`
- `camel-hl7`
- `camel-ibm-watsonx-ai`
- `camel-iso8583`
- `camel-java-joor-dsl`
- `camel-jbang-mcp`
- `camel-jbang-plugin-mcp`
- `camel-jbang-plugin-route-parser`
- `camel-jbang-plugin-tui`
- `camel-jbang-plugin-validate`
- `camel-jetty`
- `camel-jsonpath`
- `camel-kamelet`
- `camel-kamelet-main`
- `camel-kamelet-main-support`
- `camel-knative-http`
- `camel-launcher-container`
- `camel-mail`
- `camel-mail-microsoft-oauth`
- `camel-main`
- `camel-maven-plugin`
- `camel-micrometer-prometheus`
- `camel-microprofile-health`
- `camel-netty`
- `camel-netty-http`
- `camel-oauth`
- `camel-observability-services`
- `camel-once`
- `camel-openapi-validator`
- `camel-platform-http-main`
- `camel-platform-http-vertx`
- `camel-plc4x`
- `camel-pubnub`
- `camel-quickfix`
- `camel-rest-openapi`
- `camel-restdsl-openapi-plugin`
- `camel-salesforce`
- `camel-salesforce-codegen`
- `camel-salesforce-maven-plugin`
- `camel-spring-main`
- `camel-syslog`
- `camel-telegram`
- `camel-test-infra-duckdb`
- `camel-test-main-junit5`
- `camel-test-main-junit6`
- `camel-vertx-websocket`
- `camel-webhook`
- `camel-whatsapp`
- `camel-xml-io-dsl`
- `camel-xml-jaxb-dsl`
- `camel-xml-jaxb-dsl-test-definition`
- `camel-xml-jaxb-dsl-test-spring`
- `camel-yaml-dsl-validator`
- `camel-yaml-dsl-validator-maven-plugin`
- `camel-zookeeper`
- `docs`
</details>
<details><summary>Modules with tests skipped (23)</summary>
- `apache-camel`
- `camel-catalog-console`
- `camel-catalog-lucene`
- `camel-catalog-maven`
- `camel-catalog-suggest`
- `camel-csimple-maven-plugin`
- `camel-endpointdsl-support`
- `camel-itest`
- `camel-jbang-core`
- `camel-jbang-it`
- `camel-jbang-main`
- `camel-jbang-plugin-edit`
- `camel-jbang-plugin-generate`
- `camel-jbang-plugin-kubernetes`
- `camel-jbang-plugin-test`
- `camel-launcher`
- `camel-report-maven-plugin`
- `camel-route-parser`
- `camel-yaml-dsl`
- `camel-yaml-dsl-deserializers`
- `camel-yaml-dsl-maven-plugin`
- `coverage`
- `dummy-component`
</details>
> :information_source: Shadow mode — Scalpel observes but does not affect test execution. [Learn more](https://github.com/maveniverse/scalpel)
</details>
:warning: **Some tests are disabled on GitHub Actions** (`@DisabledIfSystemProperty(named = "ci.env.name")`) and require manual verification:
- `components`: 101 test(s) disabled on GitHub Actions
<details><summary>Build reactor — dependencies compiled but only changed modules were tested (96 modules)</summary>
- Camel :: AI :: A2A
- Camel :: AS2 :: Component
- Camel :: All Components Sync point
- Camel :: All Core Sync point
- Camel :: Assembly
- Camel :: BOM
- Camel :: CSV
- Camel :: Catalog :: CSimple Maven Plugin (deprecated)
- Camel :: Catalog :: Camel Catalog
- Camel :: Catalog :: Camel Report Maven Plugin
- Camel :: Catalog :: Camel Route Parser
- Camel :: Catalog :: Console
- Camel :: Catalog :: Dummy Component
- Camel :: Catalog :: Lucene (deprecated)
- Camel :: Catalog :: Maven
- Camel :: Catalog :: Suggest
- Camel :: ClickUp
- Camel :: Component DSL
- Camel :: Components
- Camel :: Consul
- Camel :: Coverage
- Camel :: Docs
- Camel :: DuckDB
- Camel :: Endpoint DSL
- Camel :: Endpoint DSL :: Support
- Camel :: FHIR
- Camel :: Groovy
- Camel :: HL7
- Camel :: HazelCast
- Camel :: IBM :: watsonx.ai
- Camel :: ISO-8583
- Camel :: Integration Tests
- Camel :: JBang :: Core
- Camel :: JBang :: Integration tests
- Camel :: JBang :: MCP
- Camel :: JBang :: Main
- Camel :: JBang :: Plugin :: Edit
- Camel :: JBang :: Plugin :: Generate
- Camel :: JBang :: Plugin :: Kubernetes
- Camel :: JBang :: Plugin :: MCP
- Camel :: JBang :: Plugin :: Route Parser
- Camel :: JBang :: Plugin :: TUI
- Camel :: JBang :: Plugin :: Testing
- Camel :: JBang :: Plugin :: Validate
- Camel :: JSon Path
- Camel :: Java DSL with jOOR
- Camel :: Jetty
- Camel :: Kamelet
- Camel :: Kamelet Main
- Camel :: Kamelet Main :: Support
- Camel :: Knative HTTP
- Camel :: Launcher
- Camel :: Launcher :: Container
- Camel :: Mail
- Camel :: Mail :: Microsoft OAuth
- Camel :: Main
- Camel :: Maven Plugins :: Camel Maven Plugin
- Camel :: Maven Plugins :: OpenApi REST DSL Generator
- Camel :: MicroProfile :: Health
- Camel :: Micrometer :: Prometheus
- Camel :: Netty
- Camel :: Netty HTTP
- Camel :: OAuth
- Camel :: Observability Services
- Camel :: Once
- Camel :: OpenAPI :: Validator
- Camel :: PLC4X
- Camel :: Parent
- Camel :: Platform HTTP :: Main
- Camel :: Platform HTTP :: Vert.x
- Camel :: PubNub
- Camel :: QuickFIX/J
- Camel :: REST OpenApi
- Camel :: Salesforce
- Camel :: Salesforce :: CodeGen
- Camel :: Salesforce :: Maven Plugin
- Camel :: Spring Main
- Camel :: Syslog
- Camel :: Telegram
- Camel :: Test :: Main :: JUnit5
- Camel :: Test :: Main :: JUnit6
- Camel :: Test Infra
- Camel :: Test Infra :: DuckDB
- Camel :: Vert.x :: WebSocket
- Camel :: Webhook
- Camel :: Whatsapp
- Camel :: XML DSL Jaxb :: Test :: Definition
- Camel :: XML DSL Jaxb :: Test :: Spring
- Camel :: XML DSL with camel-xml-io
- Camel :: XML DSL with camel-xml-jaxb
- Camel :: YAML DSL
- Camel :: YAML DSL :: Deserializers
- Camel :: YAML DSL :: Maven Plugins
- Camel :: YAML DSL :: Validator
- Camel :: YAML DSL :: Validator Maven Plugin
- Camel :: Zookeeper
</details>
---
:gear: [View full build and test results](https://github.com/apache/camel/actions/runs/30163517931)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org