You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@resilientdb.apache.org by "Burnfireblaze (via GitHub)" <gi...@apache.org> on 2026/01/12 04:09:02 UTC
[PR] Team 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Burnfireblaze opened a new pull request, #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224
# Description
This PR introduces ReSQL, a SQL-based service layer for ResilientDB that enables relational querying capabilities on top of the existing distributed ledger execution model. The ReSQL service provides an alternative or replacement for traditional key-value storage with a structured relational storage model backed by DuckDB, allowing decentralized applications to interact with ResilientDB using standard SQL semantics rather than low-level KV operations.
## Under the hood
### Storage & Execution
- Integrates DuckDB as an embedded SQL backend within ResilientDB.
- Extends the existing KV executor to support SQL-based read and write execution paths.
- Adds runtime flags for enabling and configuring ReSQL mode.
- Introduces relational metadata and schema management via DuckDB catalogs.
- Enables relational table creation, insertion, selection, and update semantics over decentralized application data.
### Service Layer
Implements a dedicated ReSQL service responsible for:
- Initializing and managing the embedded DuckDB instance.
- Handling SQL connection contexts and query execution lifecycles.
- Parsing and executing SQL statements.
- Returning structured tabular results to clients.
- Providing a higher-level interface suitable for decentralized application workflows.
### Tooling & Scripts
- Adds standalone startup scripts for launching the ReSQL service.
- Adds test tooling (test_sql) for validating execution of SQL workloads.
- Provides developer-facing utilities to bootstrap relational schemas and queries on ResilientDB.
## Motivation
ResilientDB currently exposes a key-value storage abstraction, which limits expressiveness for applications requiring relational schemas, joins, range queries, or analytical operators. ReSQL introduces an embedded relational interface backed by SQL, enabling application developers to model state with structured schemas without relying on external DBMS systems.
## Compatibility & Deployment
- ReSQL is an optional module and does not modify or interfere with ResilientDB’s core consensus or ledger protocols.
- KV-based applications remain fully compatible without changes.
- SQL execution can be selectively enabled through runtime flags.
## Testing & Validation
Validation included:
- Data insertion and retrieval queries.
- Selection and projection operators.
- Basic relational schema management.
- End-to-end execution tests via test_sql.
## Benefits
- Enables RDBMS-style semantics for decentralized applications.
- Allows structured data modeling and querying without external services.
- Leverages DuckDB’s in-process analytical engine for efficient SQL execution.
- Improves developer ergonomics through familiar SQL abstractions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "Burnfireblaze (via GitHub)" <gi...@apache.org>.
Burnfireblaze commented on PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#issuecomment-4230028547
@Bismanpal-Singh Could you please approve trigger the workflows? We have pushed code to fix the workflow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "cjcchen (via GitHub)" <gi...@apache.org>.
cjcchen commented on code in PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#discussion_r3090639684
##########
service/kv/kv_service.cpp:
##########
@@ -82,6 +124,8 @@ int main(int argc, char** argv) {
auto server = GenerateResDBServer(
config_file, private_key_file, cert_file,
- std::make_unique<KVExecutor>(NewStorage(db_path, config_data)), nullptr);
+ std::make_unique<KVExecutor>(
+ NewStorage(db_path, config_data, enable_duckdb, duckdb_path)),
Review Comment:
added one comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "Burnfireblaze (via GitHub)" <gi...@apache.org>.
Burnfireblaze commented on PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#issuecomment-4230304457
@cjcchen Could you please review and approve?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "cjcchen (via GitHub)" <gi...@apache.org>.
cjcchen commented on code in PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#discussion_r2908502062
##########
service/tools/config/server.config:
##########
Review Comment:
why remove the header?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "Burnfireblaze (via GitHub)" <gi...@apache.org>.
Burnfireblaze commented on code in PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#discussion_r3054869264
##########
service/tools/config/interface/service.config:
##########
@@ -25,4 +25,4 @@
]
}
-
+5 127.0.0.1 20005
Review Comment:
Resolved
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "Burnfireblaze (via GitHub)" <gi...@apache.org>.
Burnfireblaze commented on code in PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#discussion_r3054869264
##########
service/tools/config/interface/service.config:
##########
@@ -25,4 +25,4 @@
]
}
-
+5 127.0.0.1 20005
Review Comment:
@cjcchen Resolved this comment
##########
service/tools/config/server.config:
##########
Review Comment:
@cjcchen Resolved this comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "Burnfireblaze (via GitHub)" <gi...@apache.org>.
Burnfireblaze commented on PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#issuecomment-4027538150
@cjcchen and @harish876 Could you please review this pull request and let us know of the changes we need to do to get this PR merged?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "cjcchen (via GitHub)" <gi...@apache.org>.
cjcchen commented on code in PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#discussion_r2908503436
##########
service/tools/config/interface/service.config:
##########
@@ -25,4 +25,4 @@
]
}
-
+5 127.0.0.1 20005
Review Comment:
do you need this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "Burnfireblaze (via GitHub)" <gi...@apache.org>.
Burnfireblaze commented on code in PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#discussion_r3054871819
##########
chain/storage/duckdb.cpp:
##########
@@ -0,0 +1,99 @@
+#include "chain/storage/duckdb.h"
+
+#include <glog/logging.h>
+
+#include <exception>
+#include <memory>
+#include <string>
+
+#include "duckdb.hpp"
+
+namespace resdb {
+namespace storage {
+
+std::unique_ptr<Storage> NewResQL(const std::string& path,
+ const DuckDBInfo& config) {
+ DuckDBInfo cfg = config;
+ cfg.set_path(path);
+
+ return std::make_unique<ResQL>(cfg);
+}
+
+ResQL::ResQL(const DuckDBInfo& config) : config_(config) {
Review Comment:
@cjcchen Resolved this comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "Burnfireblaze (via GitHub)" <gi...@apache.org>.
Burnfireblaze commented on PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#issuecomment-4210608551
@cjcchen We have resolved your comments? Could you please approve if no other changes are required?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "Burnfireblaze (via GitHub)" <gi...@apache.org>.
Burnfireblaze commented on code in PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#discussion_r3054871204
##########
test_sql.sh:
##########
@@ -0,0 +1,145 @@
+#!/bin/bash
Review Comment:
@cjcchen Resolved this comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "Bismanpal-Singh (via GitHub)" <gi...@apache.org>.
Bismanpal-Singh commented on PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#issuecomment-4274362449
The workflows have passed, please approve if it's ready to merge @cjcchen , we'll move it to development branch first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "cjcchen (via GitHub)" <gi...@apache.org>.
cjcchen commented on code in PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#discussion_r2908501093
##########
test_sql.sh:
##########
@@ -0,0 +1,145 @@
+#!/bin/bash
Review Comment:
source header
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "cjcchen (via GitHub)" <gi...@apache.org>.
cjcchen commented on code in PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#discussion_r2908475975
##########
chain/storage/duckdb.cpp:
##########
@@ -0,0 +1,99 @@
+#include "chain/storage/duckdb.h"
+
+#include <glog/logging.h>
+
+#include <exception>
+#include <memory>
+#include <string>
+
+#include "duckdb.hpp"
+
+namespace resdb {
+namespace storage {
+
+std::unique_ptr<Storage> NewResQL(const std::string& path,
+ const DuckDBInfo& config) {
+ DuckDBInfo cfg = config;
+ cfg.set_path(path);
+
+ return std::make_unique<ResQL>(cfg);
+}
+
+ResQL::ResQL(const DuckDBInfo& config) : config_(config) {
Review Comment:
The filename is dockdb. Why is this called ResQL?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "cjcchen (via GitHub)" <gi...@apache.org>.
cjcchen commented on code in PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#discussion_r3090638763
##########
service/kv/kv_service.cpp:
##########
@@ -82,6 +124,8 @@ int main(int argc, char** argv) {
auto server = GenerateResDBServer(
config_file, private_key_file, cert_file,
- std::make_unique<KVExecutor>(NewStorage(db_path, config_data)), nullptr);
+ std::make_unique<KVExecutor>(
+ NewStorage(db_path, config_data, enable_duckdb, duckdb_path)),
Review Comment:
move enable_duckdb into config_data and reuse the db_path
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "Burnfireblaze (via GitHub)" <gi...@apache.org>.
Burnfireblaze commented on PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224#issuecomment-4272987324
@cjcchen Could you please review and approve?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 9 - Added ReSQL - DuckDB support [incubator-resilientdb]
Posted by "cjcchen (via GitHub)" <gi...@apache.org>.
cjcchen merged PR #224:
URL: https://github.com/apache/incubator-resilientdb/pull/224
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@resilientdb.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org