You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ossie.apache.org by mahmoud hassanen <ma...@gmail.com> on 2026/10/09 12:51:20 UTC
[DISCUSS] Native ClickHouse SQL dialect converter (#546)
Hi everyone,
I opened GitHub issue #546 to propose adding a native ClickHouse dialect
converter under converters/clickhouse:
https://github.com/apache/ossie/issues/546
Because ClickHouse relies on specialized aggregate functions (e.g.,
uniqExact, quantileExact) and native time-bucketing functions (e.g.,
toStartOfMonth), generic ANSI SQL converters produce invalid or inefficient
queries when targeting ClickHouse.
I am preparing a Python implementation with unit tests covering core metric
mappings and plan to open a Pull Request shortly. Any feedback or
suggestions on the dialect design are welcome!
Best regards,
Mahmoud Hassanen
Re: [DISCUSS] Native ClickHouse SQL dialect converter (#546)
Posted by mahmoud hassanen <ma...@gmail.com>.
Hi JB,
Thank you for the guidance! Enabling native ClickHouse SQL expressions
(such as uniqExact, quantileExactInclusive, and toStartOfMonth) under
expression.dialects as well as compiling Ossie metric models into native
ClickHouse SQL queries is exactly our end-to-end goal.
To fulfill the specification requirements you outlined:
1. Core Spec PR Opened: I submitted PR #558 against core-spec/ registering
CLICKHOUSE in the Dialect enum across spec.md, ossie-schema.json, and
spec.yaml:
https://github.com/apache/ossie/pull/558
2. Mailing List Discussion: This email thread serves as our 7-day [DISCUSS]
thread for PR #558 to gather community feedback before proceeding to a
formal vote.
3. Converter Repository Alignment: For PR #548 (the ClickHouse converter
mapper implementation), I am fully aligned with transitioning or placing it
into apache/ossie-converters as that repository migration completes.
Thank you again for guiding this through the proper ASF governance workflow!
Best regards,
Mahmoud Hassanen
Re: [DISCUSS] Native ClickHouse SQL dialect converter (#546)
Posted by mahmoud hassanen <ma...@gmail.com>.
Hi JB,
Thank you for the detailed clarification! That helps align the scope with
Ossie's architecture and repository boundaries.
To clarify our target end-to-end workflow:
The primary objective is to allow Ossie semantic models to compile and
translate metrics into native ClickHouse SQL expressions. Standard ANSI SQL
transpilation generates inefficient or invalid queries when targeting
ClickHouse, because ClickHouse relies on specialized native functions and
combinators for optimal performance and exact semantics.
I opened PR #548 (https://github.com/apache/ossie/pull/548) to introduce
this initial dialect mapper, where we implemented:
- Native Aggregates: Mapping `count_distinct` to `uniqExact()`,
`approx_distinct` to `uniqCombined64()`, and `median`/`percentile` to
`quantileExactInclusive()` (to ensure exact continuous interpolation
matching `PERCENTILE_CONT` in reference engines like DuckDB).
- Native Time Bucketing: Mapping temporal grains to ClickHouse's
specialized `toStartOf*` functions (`toStartOfDay`, `toStartOfWeek`,
`toStartOfMonth`, `toStartOfQuarter`, `toStartOfYear`).
- Safe Division: Wrapping denominators with `nullIf(denominator, 0)` to
prevent division-by-zero runtime exceptions.
Having worked extensively with ClickHouse in production data platform
architectures, I plan to contribute further dialect capabilities in
follow-up increments, such as sub-daily time grains (`toStartOfHour`,
`toStartOfMinute`), aggregate combinators (`-If`, `-Distinct`), and
query-level features (`WITH TOTALS`, `SAMPLE`).
Based on your governance guidance:
1. Core Spec Proposal (Dialect Enum): I will open a PR against `core-spec/`
to officially add `CLICKHOUSE` to the `Dialect` enum in `core-spec/spec.md`
and `core-spec/ossie-schema.json`. I am happy to initiate the 7-day dev@
discussion thread and formal community vote for this spec addition.
2. Repository Alignment: Once the dialect is approved in the core spec, I
will ensure the converter components align with the migration to
`apache/ossie-converters` as that repo transition takes place.
Thank you again for guiding this contribution through the proper ASF
governance process
Best regards,
Mahmoud Hassanen
On Sat, Oct 10, 2026, 8:58 AM Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> Hi,
>
> It would be great to have the new converters added directly to
> https://github.com/apache/ossie-converters. I am currently preparing the
> move, so I can include the new converters as part of that process.
>
> Ossie converters are designed to translate between Ossie's vendor-neutral
> semantic model format and other vendor semantic layer definitions (e.g.,
> Snowflake, dbt semantic models, Cube, ThoughtSpot TML, etc.). Ossie itself
> is a semantic model standard, not a query engine or SQL transpiler that
> generates execution queries at runtime (see our guidance on Model
> Declaration vs. Querying in the expression language spec).
> Could you clarify the expected input and output artifacts for this
> converter? Does ClickHouse have a specific semantic modeling format you are
> targeting, or are you looking to generate ClickHouse SQL queries from an
> Ossie model?
>
> If the goal is to enable authoring metric and field expressions directly in
> ClickHouse SQL (such as uniqExact or toStartOfMonth) under
> expression.dialects:
>
> expression:
> dialects:
> - dialect: ANSI_SQL
> expression: COUNT(DISTINCT user_id)
> - dialect: CLICKHOUSE
> expression: uniqExact(user_id)
>
> That is actually a core specification change, not a converter. Supported
> dialects are governed by the Dialect enum in core-spec/spec.md and
> core-spec/ossie-schema.json. Adding a new dialect requires following the
> specification change workflow: opening a PR against core-spec/, holding a
> minimum 7-day discussion on dev@, followed by a formal community vote.
>
> Could you share a bit more detail on your target end-to-end workflow? That
> will help us guide you to the right place (whether that's a spec proposal
> for a new dialect, a converter in ossie-converters, or both).
>
> Regards,
> JB
>
>
> On Fri, Oct 9, 2026 at 1:53 PM mahmoud hassanen <
> mahmoudhassanen99@gmail.com>
> wrote:
>
> > Hi everyone,
> >
> > I opened GitHub issue #546 to propose adding a native ClickHouse dialect
> > converter under converters/clickhouse:
> > https://github.com/apache/ossie/issues/546
> >
> > Because ClickHouse relies on specialized aggregate functions (e.g.,
> > uniqExact, quantileExact) and native time-bucketing functions (e.g.,
> > toStartOfMonth), generic ANSI SQL converters produce invalid or
> inefficient
> > queries when targeting ClickHouse.
> >
> > I am preparing a Python implementation with unit tests covering core
> metric
> > mappings and plan to open a Pull Request shortly. Any feedback or
> > suggestions on the dialect design are welcome!
> >
> > Best regards,
> > Mahmoud Hassanen
> >
>
Re: [DISCUSS] Native ClickHouse SQL dialect converter (#546)
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi,
It would be great to have the new converters added directly to
https://github.com/apache/ossie-converters. I am currently preparing the
move, so I can include the new converters as part of that process.
Ossie converters are designed to translate between Ossie's vendor-neutral
semantic model format and other vendor semantic layer definitions (e.g.,
Snowflake, dbt semantic models, Cube, ThoughtSpot TML, etc.). Ossie itself
is a semantic model standard, not a query engine or SQL transpiler that
generates execution queries at runtime (see our guidance on Model
Declaration vs. Querying in the expression language spec).
Could you clarify the expected input and output artifacts for this
converter? Does ClickHouse have a specific semantic modeling format you are
targeting, or are you looking to generate ClickHouse SQL queries from an
Ossie model?
If the goal is to enable authoring metric and field expressions directly in
ClickHouse SQL (such as uniqExact or toStartOfMonth) under
expression.dialects:
expression:
dialects:
- dialect: ANSI_SQL
expression: COUNT(DISTINCT user_id)
- dialect: CLICKHOUSE
expression: uniqExact(user_id)
That is actually a core specification change, not a converter. Supported
dialects are governed by the Dialect enum in core-spec/spec.md and
core-spec/ossie-schema.json. Adding a new dialect requires following the
specification change workflow: opening a PR against core-spec/, holding a
minimum 7-day discussion on dev@, followed by a formal community vote.
Could you share a bit more detail on your target end-to-end workflow? That
will help us guide you to the right place (whether that's a spec proposal
for a new dialect, a converter in ossie-converters, or both).
Regards,
JB
On Fri, Oct 9, 2026 at 1:53 PM mahmoud hassanen <ma...@gmail.com>
wrote:
> Hi everyone,
>
> I opened GitHub issue #546 to propose adding a native ClickHouse dialect
> converter under converters/clickhouse:
> https://github.com/apache/ossie/issues/546
>
> Because ClickHouse relies on specialized aggregate functions (e.g.,
> uniqExact, quantileExact) and native time-bucketing functions (e.g.,
> toStartOfMonth), generic ANSI SQL converters produce invalid or inefficient
> queries when targeting ClickHouse.
>
> I am preparing a Python implementation with unit tests covering core metric
> mappings and plan to open a Pull Request shortly. Any feedback or
> suggestions on the dialect design are welcome!
>
> Best regards,
> Mahmoud Hassanen
>