[DISCUSS] [ASTERIXDB-3695] A Model Context Protocol (MCP) server

[Vivek Gangavarapu <ga...@gmail.com>]

Hello all,

A short update on the MCP server work under ASTERIXDB-3695, so the dev list
knows what's coming over the next few weeks.

The goal is an MCP server for AsterixDB. MCP (Model Context Protocol) is
the emerging standard that lets LLMs and AI agents — Claude Desktop, IDE
assistants, and similar clients — call external tools in a structured way.
The server lets one of these agents explore an AsterixDB instance and run
queries against it, without anyone writing custom glue for each client. It
runs as a separate sidecar process and talks to the cluster over the
existing HTTP API, so nothing inside AsterixDB itself has to change.

A few principles are shaping the design. The server is strictly read-only —
every query is sent with readonly=true, so the CC itself rejects anything
that isn't a query; there are no writes and no DDL. The cluster stays the
single source of truth: the gateway keeps no query state of its own, so it
can't drift out of sync with the CC. Output is bounded by default, with
time and size limits on every query, so an agent can't accidentally pull an
unbounded result into its context window. And the tooling is columnar-aware
— when it reports a dataset's schema it also reports whether the dataset is
row- or column-formatted, so the agent can reason about access patterns
instead of guessing.

Beyond running queries, a big part of the value is giving the agent
accurate context so it stops guessing. Two directions here. First,
functions: the plan is to expose the built-in function catalog along with
full signatures — argument counts, types, return shape — plus any
user-defined functions read from the metadata, so the agent uses real
function names and arities instead of inventing them. Second, query plans:
the query service can already return the optimized logical plan and the
Hyracks job (the same ones the web UI visualizes), and feeding those back
to the agent as context is a genuinely interesting lever — it lets the
model see how its query was actually compiled and executed, reason about
index usage, and refine the query rather than guessing blindly. That
plan/job introspection is one of the things I'm most keen to build out.

On where it stands: there's an early prototype working end-to-end against a
live cluster. For development and testing I'm driving it with a local
Ollama model (a small quantized 7B), so the full loop — natural-language
request to tool call to executed SQL++ and back — can be exercised offline
without depending on a hosted model. An LLM client can already list the
datasets in an instance, inspect a chosen dataset's schema, and run a query
through the server and get results back; that path is proven, not just
mocked. That first slice covers query execution plus schema and dataset
discovery. The next slices build on it: asynchronous handling for
long-running queries, the function catalog and plan/index introspection
above, and the remaining safety guardrails. It's written in Python on the
official MCP SDK; I'll share the repo and a README once the layout settles.

Early feedback is very welcome — especially from people who know the query
service and metadata internals well, since that's where I most want to
avoid wrong assumptions. Happy to take questions.

Thanks,
 Vivek

Re: [DISCUSS] [ASTERIXDB-3695] A Model Context Protocol (MCP) server

[Vivek Gangavarapu <ga...@gmail.com>]

 Hey Ian,

 Thanks, this is really helpful — all three points land.

On the async API: good to know the recent changes were a wholesale
improvement rather than a tweak. The async lifecycle (submit / poll / fetch
/ cancel) is exactly the next slice I'm building, so I'll work against the
current master version of that API and not lean on any older behavior or
stale docs. If you happen to remember the change ID or rough timeframe of
that rework, I'd love to read the patch so I build on top of the right
semantics from the start.

On result size: that matches what I found digging through the query service
params — there's no server-side byte or row cap beyond LIMIT n. So my plan
is exactly that, in two layers: the server nudges a LIMIT into the
statement, and then it also counts rows and bytes as the result streams
back and truncates with a clear "result was truncated, here's how to narrow
it" hint. The goal is just to keep an agent from accidentally pulling an
unbounded result into its context window — the cluster stays the source of
truth, the gateway only bounds what it forwards.

And thanks for the pointer to Calvin's schema-inference patch (19044) —
that's a great fit. Right now my 'get_schema' reads the declared type out
of the Metadata datatype, which for open datasets only really tells you the
key fields. An array_schema-style inference is exactly what's needed to
recover the actual shape, and running it over a bounded sample of a dataset
is a clean way to estimate the schema without a full scan. Since it isn't
merged yet I won't depend on it, but I'll design the schema path so it can
slot in as an enrichment once it lands.

 Thanks,

  Vivek


On Fri, 29 May 2026 at 00:18, Ian Maxon <im...@apache.org> wrote:

> Hey Toby,
> Good to hear from you, it's been a minute.
> I would say it's used a fair amount. There was a discussion thread
> earlier in the year about LLM generated code, and the consensus as I
> recall it was that basically it is fine as long as it's marked as such
> and not passed off as your own writing.
> In my personal experience within AsterixDB, it's a help for sure, but
> you have to clearly define real constraints and tests that Claude Code
> or some other agentic harness has to pass before it claims it's done
> with what you asked for. An AGENTS.md file or similar is essential.
> Otherwise it's very easy to end up wasting more time fixing and
> refactoring broken or junky code that it spat out than it would have
> been to just write it in the first place. It can be very good for
> doing tasks that are complex, but rote. For example adding new
> functions in AsterixDB is a bit of a chore, but it's a well worn path.
> I find Claude and other agents pretty decent at these tasks of
> basically doing a minor variant of what has already been done as long
> as you give a clear example to follow.
> Probably the biggest task I gave it yet was to upgrade the web UI from
> a very old version of Angular (11 -> 21) and it did okay with it.
> However the lack of any real tests in this area was a major hindrance
> to getting good output, along with the jankiness of a headless
> chromium setup inside a sandbox. Thankfully it is a relatively simple
> component however, so I was able to do some minor tweaks to what it
> had generated to get a visually acceptable result.
>
> - Ian
>
> On Thu, May 28, 2026 at 8:56 AM Toby Lehman <sk...@gmail.com> wrote:
> >
> > Hey … how much AI are you guys employing in either design or code
> writing?
> > I’m wondering if AI can actually help yet with complex applications.
> >
> > TJL
> >
> > On Wed, May 27, 2026 at 11:26 PM Mike Carey <dt...@gmail.com> wrote:
> >
> > > Wow - sounds like great progress so far - very nice!
> > >
> > > Cheers,
> > >
> > > Mike
> > >
> > > On 5/27/26 11:01 PM, Vivek Gangavarapu wrote:
> > > > Hello all,
> > > >
> > > > A short update on the MCP server work under ASTERIXDB-3695, so the
> dev
> > > list
> > > > knows what's coming over the next few weeks.
> > > >
> > > > The goal is an MCP server for AsterixDB. MCP (Model Context
> Protocol) is
> > > > the emerging standard that lets LLMs and AI agents — Claude Desktop,
> IDE
> > > > assistants, and similar clients — call external tools in a structured
> > > way.
> > > > The server lets one of these agents explore an AsterixDB instance
> and run
> > > > queries against it, without anyone writing custom glue for each
> client.
> > > It
> > > > runs as a separate sidecar process and talks to the cluster over the
> > > > existing HTTP API, so nothing inside AsterixDB itself has to change.
> > > >
> > > > A few principles are shaping the design. The server is strictly
> > > read-only —
> > > > every query is sent with readonly=true, so the CC itself rejects
> anything
> > > > that isn't a query; there are no writes and no DDL. The cluster
> stays the
> > > > single source of truth: the gateway keeps no query state of its own,
> so
> > > it
> > > > can't drift out of sync with the CC. Output is bounded by default,
> with
> > > > time and size limits on every query, so an agent can't accidentally
> pull
> > > an
> > > > unbounded result into its context window. And the tooling is
> > > columnar-aware
> > > > — when it reports a dataset's schema it also reports whether the
> dataset
> > > is
> > > > row- or column-formatted, so the agent can reason about access
> patterns
> > > > instead of guessing.
> > > >
> > > > Beyond running queries, a big part of the value is giving the agent
> > > > accurate context so it stops guessing. Two directions here. First,
> > > > functions: the plan is to expose the built-in function catalog along
> with
> > > > full signatures — argument counts, types, return shape — plus any
> > > > user-defined functions read from the metadata, so the agent uses real
> > > > function names and arities instead of inventing them. Second, query
> > > plans:
> > > > the query service can already return the optimized logical plan and
> the
> > > > Hyracks job (the same ones the web UI visualizes), and feeding those
> back
> > > > to the agent as context is a genuinely interesting lever — it lets
> the
> > > > model see how its query was actually compiled and executed, reason
> about
> > > > index usage, and refine the query rather than guessing blindly. That
> > > > plan/job introspection is one of the things I'm most keen to build
> out.
> > > >
> > > > On where it stands: there's an early prototype working end-to-end
> > > against a
> > > > live cluster. For development and testing I'm driving it with a local
> > > > Ollama model (a small quantized 7B), so the full loop —
> natural-language
> > > > request to tool call to executed SQL++ and back — can be exercised
> > > offline
> > > > without depending on a hosted model. An LLM client can already list
> the
> > > > datasets in an instance, inspect a chosen dataset's schema, and run a
> > > query
> > > > through the server and get results back; that path is proven, not
> just
> > > > mocked. That first slice covers query execution plus schema and
> dataset
> > > > discovery. The next slices build on it: asynchronous handling for
> > > > long-running queries, the function catalog and plan/index
> introspection
> > > > above, and the remaining safety guardrails. It's written in Python
> on the
> > > > official MCP SDK; I'll share the repo and a README once the layout
> > > settles.
> > > >
> > > > Early feedback is very welcome — especially from people who know the
> > > query
> > > > service and metadata internals well, since that's where I most want
> to
> > > > avoid wrong assumptions. Happy to take questions.
> > > >
> > > > Thanks,
> > > >   Vivek
> > > >
>

Re: [DISCUSS] [ASTERIXDB-3695] A Model Context Protocol (MCP) server

[Ian Maxon <im...@apache.org>]

Hey Toby,
Good to hear from you, it's been a minute.
I would say it's used a fair amount. There was a discussion thread
earlier in the year about LLM generated code, and the consensus as I
recall it was that basically it is fine as long as it's marked as such
and not passed off as your own writing.
In my personal experience within AsterixDB, it's a help for sure, but
you have to clearly define real constraints and tests that Claude Code
or some other agentic harness has to pass before it claims it's done
with what you asked for. An AGENTS.md file or similar is essential.
Otherwise it's very easy to end up wasting more time fixing and
refactoring broken or junky code that it spat out than it would have
been to just write it in the first place. It can be very good for
doing tasks that are complex, but rote. For example adding new
functions in AsterixDB is a bit of a chore, but it's a well worn path.
I find Claude and other agents pretty decent at these tasks of
basically doing a minor variant of what has already been done as long
as you give a clear example to follow.
Probably the biggest task I gave it yet was to upgrade the web UI from
a very old version of Angular (11 -> 21) and it did okay with it.
However the lack of any real tests in this area was a major hindrance
to getting good output, along with the jankiness of a headless
chromium setup inside a sandbox. Thankfully it is a relatively simple
component however, so I was able to do some minor tweaks to what it
had generated to get a visually acceptable result.

- Ian

On Thu, May 28, 2026 at 8:56 AM Toby Lehman <sk...@gmail.com> wrote:
>
> Hey … how much AI are you guys employing in either design or code writing?
> I’m wondering if AI can actually help yet with complex applications.
>
> TJL
>
> On Wed, May 27, 2026 at 11:26 PM Mike Carey <dt...@gmail.com> wrote:
>
> > Wow - sounds like great progress so far - very nice!
> >
> > Cheers,
> >
> > Mike
> >
> > On 5/27/26 11:01 PM, Vivek Gangavarapu wrote:
> > > Hello all,
> > >
> > > A short update on the MCP server work under ASTERIXDB-3695, so the dev
> > list
> > > knows what's coming over the next few weeks.
> > >
> > > The goal is an MCP server for AsterixDB. MCP (Model Context Protocol) is
> > > the emerging standard that lets LLMs and AI agents — Claude Desktop, IDE
> > > assistants, and similar clients — call external tools in a structured
> > way.
> > > The server lets one of these agents explore an AsterixDB instance and run
> > > queries against it, without anyone writing custom glue for each client.
> > It
> > > runs as a separate sidecar process and talks to the cluster over the
> > > existing HTTP API, so nothing inside AsterixDB itself has to change.
> > >
> > > A few principles are shaping the design. The server is strictly
> > read-only —
> > > every query is sent with readonly=true, so the CC itself rejects anything
> > > that isn't a query; there are no writes and no DDL. The cluster stays the
> > > single source of truth: the gateway keeps no query state of its own, so
> > it
> > > can't drift out of sync with the CC. Output is bounded by default, with
> > > time and size limits on every query, so an agent can't accidentally pull
> > an
> > > unbounded result into its context window. And the tooling is
> > columnar-aware
> > > — when it reports a dataset's schema it also reports whether the dataset
> > is
> > > row- or column-formatted, so the agent can reason about access patterns
> > > instead of guessing.
> > >
> > > Beyond running queries, a big part of the value is giving the agent
> > > accurate context so it stops guessing. Two directions here. First,
> > > functions: the plan is to expose the built-in function catalog along with
> > > full signatures — argument counts, types, return shape — plus any
> > > user-defined functions read from the metadata, so the agent uses real
> > > function names and arities instead of inventing them. Second, query
> > plans:
> > > the query service can already return the optimized logical plan and the
> > > Hyracks job (the same ones the web UI visualizes), and feeding those back
> > > to the agent as context is a genuinely interesting lever — it lets the
> > > model see how its query was actually compiled and executed, reason about
> > > index usage, and refine the query rather than guessing blindly. That
> > > plan/job introspection is one of the things I'm most keen to build out.
> > >
> > > On where it stands: there's an early prototype working end-to-end
> > against a
> > > live cluster. For development and testing I'm driving it with a local
> > > Ollama model (a small quantized 7B), so the full loop — natural-language
> > > request to tool call to executed SQL++ and back — can be exercised
> > offline
> > > without depending on a hosted model. An LLM client can already list the
> > > datasets in an instance, inspect a chosen dataset's schema, and run a
> > query
> > > through the server and get results back; that path is proven, not just
> > > mocked. That first slice covers query execution plus schema and dataset
> > > discovery. The next slices build on it: asynchronous handling for
> > > long-running queries, the function catalog and plan/index introspection
> > > above, and the remaining safety guardrails. It's written in Python on the
> > > official MCP SDK; I'll share the repo and a README once the layout
> > settles.
> > >
> > > Early feedback is very welcome — especially from people who know the
> > query
> > > service and metadata internals well, since that's where I most want to
> > > avoid wrong assumptions. Happy to take questions.
> > >
> > > Thanks,
> > >   Vivek
> > >

Re: [DISCUSS] [ASTERIXDB-3695] A Model Context Protocol (MCP) server

[Toby Lehman <sk...@gmail.com>]

Hey … how much AI are you guys employing in either design or code writing?
I’m wondering if AI can actually help yet with complex applications.

TJL

On Wed, May 27, 2026 at 11:26 PM Mike Carey <dt...@gmail.com> wrote:

> Wow - sounds like great progress so far - very nice!
>
> Cheers,
>
> Mike
>
> On 5/27/26 11:01 PM, Vivek Gangavarapu wrote:
> > Hello all,
> >
> > A short update on the MCP server work under ASTERIXDB-3695, so the dev
> list
> > knows what's coming over the next few weeks.
> >
> > The goal is an MCP server for AsterixDB. MCP (Model Context Protocol) is
> > the emerging standard that lets LLMs and AI agents — Claude Desktop, IDE
> > assistants, and similar clients — call external tools in a structured
> way.
> > The server lets one of these agents explore an AsterixDB instance and run
> > queries against it, without anyone writing custom glue for each client.
> It
> > runs as a separate sidecar process and talks to the cluster over the
> > existing HTTP API, so nothing inside AsterixDB itself has to change.
> >
> > A few principles are shaping the design. The server is strictly
> read-only —
> > every query is sent with readonly=true, so the CC itself rejects anything
> > that isn't a query; there are no writes and no DDL. The cluster stays the
> > single source of truth: the gateway keeps no query state of its own, so
> it
> > can't drift out of sync with the CC. Output is bounded by default, with
> > time and size limits on every query, so an agent can't accidentally pull
> an
> > unbounded result into its context window. And the tooling is
> columnar-aware
> > — when it reports a dataset's schema it also reports whether the dataset
> is
> > row- or column-formatted, so the agent can reason about access patterns
> > instead of guessing.
> >
> > Beyond running queries, a big part of the value is giving the agent
> > accurate context so it stops guessing. Two directions here. First,
> > functions: the plan is to expose the built-in function catalog along with
> > full signatures — argument counts, types, return shape — plus any
> > user-defined functions read from the metadata, so the agent uses real
> > function names and arities instead of inventing them. Second, query
> plans:
> > the query service can already return the optimized logical plan and the
> > Hyracks job (the same ones the web UI visualizes), and feeding those back
> > to the agent as context is a genuinely interesting lever — it lets the
> > model see how its query was actually compiled and executed, reason about
> > index usage, and refine the query rather than guessing blindly. That
> > plan/job introspection is one of the things I'm most keen to build out.
> >
> > On where it stands: there's an early prototype working end-to-end
> against a
> > live cluster. For development and testing I'm driving it with a local
> > Ollama model (a small quantized 7B), so the full loop — natural-language
> > request to tool call to executed SQL++ and back — can be exercised
> offline
> > without depending on a hosted model. An LLM client can already list the
> > datasets in an instance, inspect a chosen dataset's schema, and run a
> query
> > through the server and get results back; that path is proven, not just
> > mocked. That first slice covers query execution plus schema and dataset
> > discovery. The next slices build on it: asynchronous handling for
> > long-running queries, the function catalog and plan/index introspection
> > above, and the remaining safety guardrails. It's written in Python on the
> > official MCP SDK; I'll share the repo and a README once the layout
> settles.
> >
> > Early feedback is very welcome — especially from people who know the
> query
> > service and metadata internals well, since that's where I most want to
> > avoid wrong assumptions. Happy to take questions.
> >
> > Thanks,
> >   Vivek
> >

Re: [DISCUSS] [ASTERIXDB-3695] A Model Context Protocol (MCP) server

[Mike Carey <dt...@gmail.com>]

Wow - sounds like great progress so far - very nice!

Cheers,

Mike

On 5/27/26 11:01 PM, Vivek Gangavarapu wrote:
> Hello all,
>
> A short update on the MCP server work under ASTERIXDB-3695, so the dev list
> knows what's coming over the next few weeks.
>
> The goal is an MCP server for AsterixDB. MCP (Model Context Protocol) is
> the emerging standard that lets LLMs and AI agents — Claude Desktop, IDE
> assistants, and similar clients — call external tools in a structured way.
> The server lets one of these agents explore an AsterixDB instance and run
> queries against it, without anyone writing custom glue for each client. It
> runs as a separate sidecar process and talks to the cluster over the
> existing HTTP API, so nothing inside AsterixDB itself has to change.
>
> A few principles are shaping the design. The server is strictly read-only —
> every query is sent with readonly=true, so the CC itself rejects anything
> that isn't a query; there are no writes and no DDL. The cluster stays the
> single source of truth: the gateway keeps no query state of its own, so it
> can't drift out of sync with the CC. Output is bounded by default, with
> time and size limits on every query, so an agent can't accidentally pull an
> unbounded result into its context window. And the tooling is columnar-aware
> — when it reports a dataset's schema it also reports whether the dataset is
> row- or column-formatted, so the agent can reason about access patterns
> instead of guessing.
>
> Beyond running queries, a big part of the value is giving the agent
> accurate context so it stops guessing. Two directions here. First,
> functions: the plan is to expose the built-in function catalog along with
> full signatures — argument counts, types, return shape — plus any
> user-defined functions read from the metadata, so the agent uses real
> function names and arities instead of inventing them. Second, query plans:
> the query service can already return the optimized logical plan and the
> Hyracks job (the same ones the web UI visualizes), and feeding those back
> to the agent as context is a genuinely interesting lever — it lets the
> model see how its query was actually compiled and executed, reason about
> index usage, and refine the query rather than guessing blindly. That
> plan/job introspection is one of the things I'm most keen to build out.
>
> On where it stands: there's an early prototype working end-to-end against a
> live cluster. For development and testing I'm driving it with a local
> Ollama model (a small quantized 7B), so the full loop — natural-language
> request to tool call to executed SQL++ and back — can be exercised offline
> without depending on a hosted model. An LLM client can already list the
> datasets in an instance, inspect a chosen dataset's schema, and run a query
> through the server and get results back; that path is proven, not just
> mocked. That first slice covers query execution plus schema and dataset
> discovery. The next slices build on it: asynchronous handling for
> long-running queries, the function catalog and plan/index introspection
> above, and the remaining safety guardrails. It's written in Python on the
> official MCP SDK; I'll share the repo and a README once the layout settles.
>
> Early feedback is very welcome — especially from people who know the query
> service and metadata internals well, since that's where I most want to
> avoid wrong assumptions. Happy to take questions.
>
> Thanks,
>   Vivek
>

Re: [DISCUSS] [ASTERIXDB-3695] A Model Context Protocol (MCP) server

[Ian Maxon <im...@apache.org>]

Hey Vivek,
Sounds very neat overall, thanks for the update. One thing that comes
to mind as something to watch out for is some changes in the async api
that were merged in relatively recently. They're basically a wholesale
improvement, the original version had a lot of strange behavior. The
other thing I would be curious about is how to handle the result size.
I don't know of a way to do that off the top of my head outside of
LIMIT n, at least from the server side.
Another patch to check out might be Calvin's work on schema inference,
one of the patches is here:
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/19044 . It's not
merged so I guess it can't really be used just yet, but certainly it
could be useful for knowing the schema of a dataset, or estimating it
from the Sample dataset.

- Ian

On Wed, May 27, 2026 at 11:02 PM Vivek Gangavarapu
<ga...@gmail.com> wrote:
>
> Hello all,
>
> A short update on the MCP server work under ASTERIXDB-3695, so the dev list
> knows what's coming over the next few weeks.
>
> The goal is an MCP server for AsterixDB. MCP (Model Context Protocol) is
> the emerging standard that lets LLMs and AI agents — Claude Desktop, IDE
> assistants, and similar clients — call external tools in a structured way.
> The server lets one of these agents explore an AsterixDB instance and run
> queries against it, without anyone writing custom glue for each client. It
> runs as a separate sidecar process and talks to the cluster over the
> existing HTTP API, so nothing inside AsterixDB itself has to change.
>
> A few principles are shaping the design. The server is strictly read-only —
> every query is sent with readonly=true, so the CC itself rejects anything
> that isn't a query; there are no writes and no DDL. The cluster stays the
> single source of truth: the gateway keeps no query state of its own, so it
> can't drift out of sync with the CC. Output is bounded by default, with
> time and size limits on every query, so an agent can't accidentally pull an
> unbounded result into its context window. And the tooling is columnar-aware
> — when it reports a dataset's schema it also reports whether the dataset is
> row- or column-formatted, so the agent can reason about access patterns
> instead of guessing.
>
> Beyond running queries, a big part of the value is giving the agent
> accurate context so it stops guessing. Two directions here. First,
> functions: the plan is to expose the built-in function catalog along with
> full signatures — argument counts, types, return shape — plus any
> user-defined functions read from the metadata, so the agent uses real
> function names and arities instead of inventing them. Second, query plans:
> the query service can already return the optimized logical plan and the
> Hyracks job (the same ones the web UI visualizes), and feeding those back
> to the agent as context is a genuinely interesting lever — it lets the
> model see how its query was actually compiled and executed, reason about
> index usage, and refine the query rather than guessing blindly. That
> plan/job introspection is one of the things I'm most keen to build out.
>
> On where it stands: there's an early prototype working end-to-end against a
> live cluster. For development and testing I'm driving it with a local
> Ollama model (a small quantized 7B), so the full loop — natural-language
> request to tool call to executed SQL++ and back — can be exercised offline
> without depending on a hosted model. An LLM client can already list the
> datasets in an instance, inspect a chosen dataset's schema, and run a query
> through the server and get results back; that path is proven, not just
> mocked. That first slice covers query execution plus schema and dataset
> discovery. The next slices build on it: asynchronous handling for
> long-running queries, the function catalog and plan/index introspection
> above, and the remaining safety guardrails. It's written in Python on the
> official MCP SDK; I'll share the repo and a README once the layout settles.
>
> Early feedback is very welcome — especially from people who know the query
> service and metadata internals well, since that's where I most want to
> avoid wrong assumptions. Happy to take questions.
>
> Thanks,
>  Vivek