You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ossie.apache.org by Justin Talbot via dev <de...@ossie.apache.org> on 2026/08/05 00:24:41 UTC
[DISCUSS] PRs #246 (Foundational Semantics) and #237 (Compliance Suite) — Request for Extended Review Before Vote
Hi all,
We've raised concerns about the foundational semantics document (
https://github.com/apache/ossie/pull/246, and the compliance suite that
builds on it in https://github.com/apache/ossie/pull/237) in our working
group discussions, and we want to make sure they're on the record before
any vote is called.
We appreciate the work Will has put into these, and we're engaging with the
semantics document in good faith because we want it to be as clear as
possible if this direction proceeds. But our engagement with the details
shouldn't be read as endorsement of the direction or the document's
readiness.
We have a few key concerns with the current proposal.
BI vendor buy-in
Ossie's success depends on vendor adoption. So our first question about any
direction, including one we'd prefer ourselves, is whether the vendors who
would implement it have said they can and will. That hasn't been
established here. We're not aware of any BI vendor that has tried
implementing the proposed semantics or querying through the proposed query
model. We think that experience is necessary before standardization.
We also expect it to be harder than it looks. Some of the specified
semantics, particularly join direction, fan-out prevention, and M:N
resolution, conflict with choices existing tools have already made and
their users depend on. Vendors will also want to layer their own logic on
top of an imported model, and it isn't clear whether these semantics allow
that. Tableau (composed data sources) and Power BI (model composition)
flatten composed graphs into a single graph, so a composed model should
behave like one built from scratch. That's the kind of thing we'd expect an
implementation attempt to surface.
At 1,308 lines specifying the core of how a semantic layer behaves, this is
a lot to adopt as normative on the limited review it has had so far. We'd
like broader review from the vendors who would implement it, with evidence
they find the semantics feasible and useful. We're happy to help collect
realistic query patterns to test against.
The query model only demonstrates trivial queries, and punts the hard parts
Section 5 of the foundational semantics document defines a basic query
model and assumes it can be extended later. We don't think that gap can be
deferred.
Section 5 shows only that the semantics can answer trivial queries: no
joins beyond the model's own relationships, no subqueries, no window
functions. Real BI tools don't generate such simple queries. Tableau, Power
BI, and their peers emit multi-way joins, subqueries, and window functions
to deliver rich analytics, like LOD calculations, Top-N filters, and
period-over-period comparisons. Those are the hard cases for a semantics
layer, because they're where fan-out, re-aggregation, grain, and filter
propagation actually bite.
The spec also mixes two kinds of content. Some guarantees are close to
universally true, like not silently duplicating a measure. Others are
opinionated: join direction, path selection, and filter flow are places
where tools have made different, deliberate choices, and matching all of
them probably needs tunable knobs rather than one normative answer. Most of
that opinionated territory is punted. We need to work through those
opinionated parts before adopting the specification as normative. If we
ratify now and the semantics or query model later can't express common BI
functionality, we'll be changing a normative spec that vendors were asked
to build against.
One clarification, since it comes up: the in-flight expression language
proposal covers the expressions used to define dimensions and measures
inside a model. It doesn't cover how consumers query a model. The latter is
our concern here.
This work implicitly makes an architectural decision that hasn't been
settled
The interchange side of Ossie, importing a model definition, has broad
agreement in principle. But #246 also commits the project, implicitly, to a
particular query architecture: the model is queried as a black box, like a
single opaque table, with the constituent tables and their relationships
hidden behind the semantic layer.
That is a significant choice, and we don't think the group has explicitly
discussed or agreed on it. It's also a hard one to make work. A black box
leaves three bad options: live with the limited query capability Section 5
currently specifies, invent SQL extensions that are hard for tools to
integrate against, or accept non-SQL semantics where the model looks like a
table but doesn't behave like one when queried. None of these is cheap for
a BI tool.
We'd rather have narrow, well-defined semantics that work end to end for a
broad set of vendors than more powerful semantics that aren't well defined
or widely adopted. There's a real alternative in that direction: a more
transparent interface where a BI tool can see the constituent tables and
control how they compose. A tool could then express much of its current
behavior, filter flow for instance, in ordinary SQL, instead of Ossie
having to standardize a new query model for it.
There's a related open question on interchange itself: how much of the
semantics must be strictly portable versus left to the consuming tool. #246
takes a strong position, very strict semantic agreement, and we don't think
that position has been settled either.
We'd like the group to make this architectural decision explicitly rather
than have it fall out of merging these PRs. This is also why the sequencing
matters now rather than later: #237 builds a compliance suite on top of
these semantics, which would begin defining Ossie conformance in terms of
them. We're planning to write up the transparent-interface direction as an
alternative for the group to weigh, and we don't think one branch of this
fork should be ratified before the options have been compared.
Readiness of the PR
The #246 description summary is blank and the checklist is entirely
unchecked. For a normative spec of this scope, we'd expect a clear written
summary of what's being adopted and what tradeoffs were considered before
it goes to a vote.
What we're asking for
Before any [VOTE] is called:
1.
Broader review from the vendors who would implement this, with evidence
they see the semantics as feasible and useful, and some indication of who
plans to implement, by what mechanism, and on what timeline.
2.
A demonstration that these semantics extend to realistic BI queries
(e.g. joins, subqueries, window functions, LOD calcs, Top-N filters, filter
flow), not just the trivial cases in Section 5. We're glad to help assemble
these patterns.
3.
An explicit discussion of the query architecture (black-box vs
transparent) and of how much semantics must be portable on interchange,
decided on its own merits rather than as a side effect of merging these PRs.
4.
A substantive PR description summarizing what's proposed and what
tradeoffs were considered.
We'll follow up with a written alternative on the query architecture
question so there's something concrete to compare. Thanks!
Justin Talbot
Chris Eubank
Databricks
Re: [DISCUSS] PRs #246 (Foundational Semantics) and #237 (Compliance Suite) — Request for Extended Review Before Vote
Posted by Chris Eubank via dev <de...@ossie.apache.org>.
Hi all,
Recapping the metric language working group meeting on 2026-08-26 for those
who weren't there. This meeting covered the discussion in this thread.
Please correct me if I've mischaracterized anything!
The discussion centered on the layered model in this document
<https://docs.google.com/document/d/1EgG5vOlJSubm01S1AGMJ4MKcMcjxBhJHenGKWMr849E/edit?tab=t.0#heading=h.k8nomy2jicfx>.
The two middle layers are the ones this thread is about: Layer 2 (metrics)
and Layer 3 (the dimensional model). We agreed that Will’s PRs (#246
<https://github.com/apache/ossie/pull/246>, #237
<https://github.com/apache/ossie/pull/237>) belong to Layer 3 and our
proposal
<https://docs.google.com/document/d/1uKc3x9NlvV9O9UPdvlnCTDxsc-XWsQrxb0ylG45H-IQ/edit?tab=t.0#heading=h.5vtm2vjqu4mp>
belongs to Layer 2. Concretely, this addresses the sequencing concern that
opened this thread: #246/#237 proceed as Layer 3 rather than as the
"foundational" semantics, and Layer 2 proceeds in parallel.
The proposed next steps were:
1.
A PR capturing the framing spec for the four layers of the Ossie model
(POC: Will)
2.
Reframe the in-flight #246/#237 from "foundational" to Layer 3 within
that framing (POC: Will)
3.
Spec & compliance PRs for Layer 2 (POCs: Justin, Chris)
4.
A community BI query corpus [initial PR #344
<https://github.com/apache/ossie/pull/344>] (POC: Chris)
Khushboo - agreed. The expression language covers how a model is defined;
the open gap is how it's queried. We should build reference implementations
& integration guides so the community can better understand / test out the
layers.
Kind regards,
Chris Eubank
On Fri, Aug 28, 2026 at 9:04 PM Khushboo Bhatia <kh...@gmail.com>
wrote:
> Hey Justin,
> > We'd rather have narrow, well-defined semantics that work end to end for
> a
> > broad set of vendors than more powerful semantics that aren't well
> defined
> > or widely adopted. There's a real alternative in that direction: a more
> > transparent interface where a BI tool can see the constituent tables and
> > control how they compose. A tool could then express much of its current
> > behavior, filter flow for instance, in ordinary SQL, instead of Ossie
> > having to standardize a new query model for it.
>
> I do agree with this statement of yours. Now that we have an expression
> language for Ossie, I believe what would be most valuable now is a query
> language spec and its reference implementation.
>
> Khushboo
>
> On Tue, Aug 11, 2026 at 3:37 PM Will Pugh <wi...@apache.org> wrote:
>
> > Yes. I would love any additional review, although, I expect that having
> > an implementation makes reviewing the behaviours easier.
> >
> > In the sub-committee we did specifically say we want to shrink the scope
> > of the foundational semantics so that we did not boil the ocean. I still
> > think that is a better approach then trying to get in everything at once.
> >
> > It would be great to either have more specific feedback on the PR about
> > what semantics you think don't work or to get code out so we can actually
> > work with the model and change what does not work. Having examples would
> > be great, but pausing this work makes it harder, rather than easier.
> >
> > If there are places that the semantics for fan-out, etc are not
> > sufficient, then please bring them up in a more specific way. I've been
> > following the PR and don't see issues raised there.
> >
> > We've had a discussion going for a while
> > https://github.com/apache/ossie/discussions/46 where the approach we
> want
> > to go is something standard. The thing with a standard is that there
> needs
> > to be a specified correct answer. Things like the join directions need
> to
> > be specified so that we all get the same answer. You can look at the
> > current OSI spec, and it allows joins, and it allows calcs that cross
> > tables. So, in order to address the model, we need to define what it
> will
> > do. We can always relax things later or make things more configurable if
> > we run into cases that are not fully supported.
> >
> > Given how big this space is, I'm not sure that a big-bang approach is the
> > best one.
> >
> > --Will
> >
> > On 2026/08/05 00:24:41 Justin Talbot via dev wrote:
> > > Hi all,
> > >
> > > We've raised concerns about the foundational semantics document (
> > > https://github.com/apache/ossie/pull/246, and the compliance suite
> that
> > > builds on it in https://github.com/apache/ossie/pull/237) in our
> working
> > > group discussions, and we want to make sure they're on the record
> before
> > > any vote is called.
> > >
> > > We appreciate the work Will has put into these, and we're engaging with
> > the
> > > semantics document in good faith because we want it to be as clear as
> > > possible if this direction proceeds. But our engagement with the
> details
> > > shouldn't be read as endorsement of the direction or the document's
> > > readiness.
> > >
> > > We have a few key concerns with the current proposal.
> > >
> > > BI vendor buy-in
> > >
> > > Ossie's success depends on vendor adoption. So our first question about
> > any
> > > direction, including one we'd prefer ourselves, is whether the vendors
> > who
> > > would implement it have said they can and will. That hasn't been
> > > established here. We're not aware of any BI vendor that has tried
> > > implementing the proposed semantics or querying through the proposed
> > query
> > > model. We think that experience is necessary before standardization.
> > >
> > > We also expect it to be harder than it looks. Some of the specified
> > > semantics, particularly join direction, fan-out prevention, and M:N
> > > resolution, conflict with choices existing tools have already made and
> > > their users depend on. Vendors will also want to layer their own logic
> on
> > > top of an imported model, and it isn't clear whether these semantics
> > allow
> > > that. Tableau (composed data sources) and Power BI (model composition)
> > > flatten composed graphs into a single graph, so a composed model should
> > > behave like one built from scratch. That's the kind of thing we'd
> expect
> > an
> > > implementation attempt to surface.
> > >
> > > At 1,308 lines specifying the core of how a semantic layer behaves,
> this
> > is
> > > a lot to adopt as normative on the limited review it has had so far.
> We'd
> > > like broader review from the vendors who would implement it, with
> > evidence
> > > they find the semantics feasible and useful. We're happy to help
> collect
> > > realistic query patterns to test against.
> > >
> > > The query model only demonstrates trivial queries, and punts the hard
> > parts
> > >
> > > Section 5 of the foundational semantics document defines a basic query
> > > model and assumes it can be extended later. We don't think that gap can
> > be
> > > deferred.
> > >
> > > Section 5 shows only that the semantics can answer trivial queries: no
> > > joins beyond the model's own relationships, no subqueries, no window
> > > functions. Real BI tools don't generate such simple queries. Tableau,
> > Power
> > > BI, and their peers emit multi-way joins, subqueries, and window
> > functions
> > > to deliver rich analytics, like LOD calculations, Top-N filters, and
> > > period-over-period comparisons. Those are the hard cases for a
> semantics
> > > layer, because they're where fan-out, re-aggregation, grain, and filter
> > > propagation actually bite.
> > >
> > > The spec also mixes two kinds of content. Some guarantees are close to
> > > universally true, like not silently duplicating a measure. Others are
> > > opinionated: join direction, path selection, and filter flow are places
> > > where tools have made different, deliberate choices, and matching all
> of
> > > them probably needs tunable knobs rather than one normative answer.
> Most
> > of
> > > that opinionated territory is punted. We need to work through those
> > > opinionated parts before adopting the specification as normative. If we
> > > ratify now and the semantics or query model later can't express common
> BI
> > > functionality, we'll be changing a normative spec that vendors were
> asked
> > > to build against.
> > >
> > > One clarification, since it comes up: the in-flight expression language
> > > proposal covers the expressions used to define dimensions and measures
> > > inside a model. It doesn't cover how consumers query a model. The
> latter
> > is
> > > our concern here.
> > >
> > > This work implicitly makes an architectural decision that hasn't been
> > > settled
> > >
> > > The interchange side of Ossie, importing a model definition, has broad
> > > agreement in principle. But #246 also commits the project, implicitly,
> > to a
> > > particular query architecture: the model is queried as a black box,
> like
> > a
> > > single opaque table, with the constituent tables and their
> relationships
> > > hidden behind the semantic layer.
> > >
> > > That is a significant choice, and we don't think the group has
> explicitly
> > > discussed or agreed on it. It's also a hard one to make work. A black
> box
> > > leaves three bad options: live with the limited query capability
> Section
> > 5
> > > currently specifies, invent SQL extensions that are hard for tools to
> > > integrate against, or accept non-SQL semantics where the model looks
> > like a
> > > table but doesn't behave like one when queried. None of these is cheap
> > for
> > > a BI tool.
> > >
> > > We'd rather have narrow, well-defined semantics that work end to end
> for
> > a
> > > broad set of vendors than more powerful semantics that aren't well
> > defined
> > > or widely adopted. There's a real alternative in that direction: a more
> > > transparent interface where a BI tool can see the constituent tables
> and
> > > control how they compose. A tool could then express much of its current
> > > behavior, filter flow for instance, in ordinary SQL, instead of Ossie
> > > having to standardize a new query model for it.
> > >
> > > There's a related open question on interchange itself: how much of the
> > > semantics must be strictly portable versus left to the consuming tool.
> > #246
> > > takes a strong position, very strict semantic agreement, and we don't
> > think
> > > that position has been settled either.
> > >
> > > We'd like the group to make this architectural decision explicitly
> rather
> > > than have it fall out of merging these PRs. This is also why the
> > sequencing
> > > matters now rather than later: #237 builds a compliance suite on top of
> > > these semantics, which would begin defining Ossie conformance in terms
> of
> > > them. We're planning to write up the transparent-interface direction as
> > an
> > > alternative for the group to weigh, and we don't think one branch of
> this
> > > fork should be ratified before the options have been compared.
> > >
> > > Readiness of the PR
> > >
> > > The #246 description summary is blank and the checklist is entirely
> > > unchecked. For a normative spec of this scope, we'd expect a clear
> > written
> > > summary of what's being adopted and what tradeoffs were considered
> before
> > > it goes to a vote.
> > >
> > > What we're asking for
> > >
> > > Before any [VOTE] is called:
> > >
> > >
> > > 1.
> > >
> > > Broader review from the vendors who would implement this, with
> > evidence
> > > they see the semantics as feasible and useful, and some indication
> of
> > who
> > > plans to implement, by what mechanism, and on what timeline.
> > > 2.
> > >
> > > A demonstration that these semantics extend to realistic BI queries
> > > (e.g. joins, subqueries, window functions, LOD calcs, Top-N filters,
> > filter
> > > flow), not just the trivial cases in Section 5. We're glad to help
> > assemble
> > > these patterns.
> > > 3.
> > >
> > > An explicit discussion of the query architecture (black-box vs
> > > transparent) and of how much semantics must be portable on
> > interchange,
> > > decided on its own merits rather than as a side effect of merging
> > these PRs.
> > > 4.
> > >
> > > A substantive PR description summarizing what's proposed and what
> > > tradeoffs were considered.
> > >
> > >
> > > We'll follow up with a written alternative on the query architecture
> > > question so there's something concrete to compare. Thanks!
> > >
> > > Justin Talbot
> > >
> > > Chris Eubank
> > >
> > > Databricks
> > >
> >
>
Re: [DISCUSS] PRs #246 (Foundational Semantics) and #237 (Compliance Suite) — Request for Extended Review Before Vote
Posted by Khushboo Bhatia <kh...@gmail.com>.
Hey Justin,
> We'd rather have narrow, well-defined semantics that work end to end for a
> broad set of vendors than more powerful semantics that aren't well defined
> or widely adopted. There's a real alternative in that direction: a more
> transparent interface where a BI tool can see the constituent tables and
> control how they compose. A tool could then express much of its current
> behavior, filter flow for instance, in ordinary SQL, instead of Ossie
> having to standardize a new query model for it.
I do agree with this statement of yours. Now that we have an expression
language for Ossie, I believe what would be most valuable now is a query
language spec and its reference implementation.
Khushboo
On Tue, Aug 11, 2026 at 3:37 PM Will Pugh <wi...@apache.org> wrote:
> Yes. I would love any additional review, although, I expect that having
> an implementation makes reviewing the behaviours easier.
>
> In the sub-committee we did specifically say we want to shrink the scope
> of the foundational semantics so that we did not boil the ocean. I still
> think that is a better approach then trying to get in everything at once.
>
> It would be great to either have more specific feedback on the PR about
> what semantics you think don't work or to get code out so we can actually
> work with the model and change what does not work. Having examples would
> be great, but pausing this work makes it harder, rather than easier.
>
> If there are places that the semantics for fan-out, etc are not
> sufficient, then please bring them up in a more specific way. I've been
> following the PR and don't see issues raised there.
>
> We've had a discussion going for a while
> https://github.com/apache/ossie/discussions/46 where the approach we want
> to go is something standard. The thing with a standard is that there needs
> to be a specified correct answer. Things like the join directions need to
> be specified so that we all get the same answer. You can look at the
> current OSI spec, and it allows joins, and it allows calcs that cross
> tables. So, in order to address the model, we need to define what it will
> do. We can always relax things later or make things more configurable if
> we run into cases that are not fully supported.
>
> Given how big this space is, I'm not sure that a big-bang approach is the
> best one.
>
> --Will
>
> On 2026/08/05 00:24:41 Justin Talbot via dev wrote:
> > Hi all,
> >
> > We've raised concerns about the foundational semantics document (
> > https://github.com/apache/ossie/pull/246, and the compliance suite that
> > builds on it in https://github.com/apache/ossie/pull/237) in our working
> > group discussions, and we want to make sure they're on the record before
> > any vote is called.
> >
> > We appreciate the work Will has put into these, and we're engaging with
> the
> > semantics document in good faith because we want it to be as clear as
> > possible if this direction proceeds. But our engagement with the details
> > shouldn't be read as endorsement of the direction or the document's
> > readiness.
> >
> > We have a few key concerns with the current proposal.
> >
> > BI vendor buy-in
> >
> > Ossie's success depends on vendor adoption. So our first question about
> any
> > direction, including one we'd prefer ourselves, is whether the vendors
> who
> > would implement it have said they can and will. That hasn't been
> > established here. We're not aware of any BI vendor that has tried
> > implementing the proposed semantics or querying through the proposed
> query
> > model. We think that experience is necessary before standardization.
> >
> > We also expect it to be harder than it looks. Some of the specified
> > semantics, particularly join direction, fan-out prevention, and M:N
> > resolution, conflict with choices existing tools have already made and
> > their users depend on. Vendors will also want to layer their own logic on
> > top of an imported model, and it isn't clear whether these semantics
> allow
> > that. Tableau (composed data sources) and Power BI (model composition)
> > flatten composed graphs into a single graph, so a composed model should
> > behave like one built from scratch. That's the kind of thing we'd expect
> an
> > implementation attempt to surface.
> >
> > At 1,308 lines specifying the core of how a semantic layer behaves, this
> is
> > a lot to adopt as normative on the limited review it has had so far. We'd
> > like broader review from the vendors who would implement it, with
> evidence
> > they find the semantics feasible and useful. We're happy to help collect
> > realistic query patterns to test against.
> >
> > The query model only demonstrates trivial queries, and punts the hard
> parts
> >
> > Section 5 of the foundational semantics document defines a basic query
> > model and assumes it can be extended later. We don't think that gap can
> be
> > deferred.
> >
> > Section 5 shows only that the semantics can answer trivial queries: no
> > joins beyond the model's own relationships, no subqueries, no window
> > functions. Real BI tools don't generate such simple queries. Tableau,
> Power
> > BI, and their peers emit multi-way joins, subqueries, and window
> functions
> > to deliver rich analytics, like LOD calculations, Top-N filters, and
> > period-over-period comparisons. Those are the hard cases for a semantics
> > layer, because they're where fan-out, re-aggregation, grain, and filter
> > propagation actually bite.
> >
> > The spec also mixes two kinds of content. Some guarantees are close to
> > universally true, like not silently duplicating a measure. Others are
> > opinionated: join direction, path selection, and filter flow are places
> > where tools have made different, deliberate choices, and matching all of
> > them probably needs tunable knobs rather than one normative answer. Most
> of
> > that opinionated territory is punted. We need to work through those
> > opinionated parts before adopting the specification as normative. If we
> > ratify now and the semantics or query model later can't express common BI
> > functionality, we'll be changing a normative spec that vendors were asked
> > to build against.
> >
> > One clarification, since it comes up: the in-flight expression language
> > proposal covers the expressions used to define dimensions and measures
> > inside a model. It doesn't cover how consumers query a model. The latter
> is
> > our concern here.
> >
> > This work implicitly makes an architectural decision that hasn't been
> > settled
> >
> > The interchange side of Ossie, importing a model definition, has broad
> > agreement in principle. But #246 also commits the project, implicitly,
> to a
> > particular query architecture: the model is queried as a black box, like
> a
> > single opaque table, with the constituent tables and their relationships
> > hidden behind the semantic layer.
> >
> > That is a significant choice, and we don't think the group has explicitly
> > discussed or agreed on it. It's also a hard one to make work. A black box
> > leaves three bad options: live with the limited query capability Section
> 5
> > currently specifies, invent SQL extensions that are hard for tools to
> > integrate against, or accept non-SQL semantics where the model looks
> like a
> > table but doesn't behave like one when queried. None of these is cheap
> for
> > a BI tool.
> >
> > We'd rather have narrow, well-defined semantics that work end to end for
> a
> > broad set of vendors than more powerful semantics that aren't well
> defined
> > or widely adopted. There's a real alternative in that direction: a more
> > transparent interface where a BI tool can see the constituent tables and
> > control how they compose. A tool could then express much of its current
> > behavior, filter flow for instance, in ordinary SQL, instead of Ossie
> > having to standardize a new query model for it.
> >
> > There's a related open question on interchange itself: how much of the
> > semantics must be strictly portable versus left to the consuming tool.
> #246
> > takes a strong position, very strict semantic agreement, and we don't
> think
> > that position has been settled either.
> >
> > We'd like the group to make this architectural decision explicitly rather
> > than have it fall out of merging these PRs. This is also why the
> sequencing
> > matters now rather than later: #237 builds a compliance suite on top of
> > these semantics, which would begin defining Ossie conformance in terms of
> > them. We're planning to write up the transparent-interface direction as
> an
> > alternative for the group to weigh, and we don't think one branch of this
> > fork should be ratified before the options have been compared.
> >
> > Readiness of the PR
> >
> > The #246 description summary is blank and the checklist is entirely
> > unchecked. For a normative spec of this scope, we'd expect a clear
> written
> > summary of what's being adopted and what tradeoffs were considered before
> > it goes to a vote.
> >
> > What we're asking for
> >
> > Before any [VOTE] is called:
> >
> >
> > 1.
> >
> > Broader review from the vendors who would implement this, with
> evidence
> > they see the semantics as feasible and useful, and some indication of
> who
> > plans to implement, by what mechanism, and on what timeline.
> > 2.
> >
> > A demonstration that these semantics extend to realistic BI queries
> > (e.g. joins, subqueries, window functions, LOD calcs, Top-N filters,
> filter
> > flow), not just the trivial cases in Section 5. We're glad to help
> assemble
> > these patterns.
> > 3.
> >
> > An explicit discussion of the query architecture (black-box vs
> > transparent) and of how much semantics must be portable on
> interchange,
> > decided on its own merits rather than as a side effect of merging
> these PRs.
> > 4.
> >
> > A substantive PR description summarizing what's proposed and what
> > tradeoffs were considered.
> >
> >
> > We'll follow up with a written alternative on the query architecture
> > question so there's something concrete to compare. Thanks!
> >
> > Justin Talbot
> >
> > Chris Eubank
> >
> > Databricks
> >
>
Re: [DISCUSS] PRs #246 (Foundational Semantics) and #237 (Compliance Suite) — Request for Extended Review Before Vote
Posted by Will Pugh <wi...@apache.org>.
Yes. I would love any additional review, although, I expect that having an implementation makes reviewing the behaviours easier.
In the sub-committee we did specifically say we want to shrink the scope of the foundational semantics so that we did not boil the ocean. I still think that is a better approach then trying to get in everything at once.
It would be great to either have more specific feedback on the PR about what semantics you think don't work or to get code out so we can actually work with the model and change what does not work. Having examples would be great, but pausing this work makes it harder, rather than easier.
If there are places that the semantics for fan-out, etc are not sufficient, then please bring them up in a more specific way. I've been following the PR and don't see issues raised there.
We've had a discussion going for a while https://github.com/apache/ossie/discussions/46 where the approach we want to go is something standard. The thing with a standard is that there needs to be a specified correct answer. Things like the join directions need to be specified so that we all get the same answer. You can look at the current OSI spec, and it allows joins, and it allows calcs that cross tables. So, in order to address the model, we need to define what it will do. We can always relax things later or make things more configurable if we run into cases that are not fully supported.
Given how big this space is, I'm not sure that a big-bang approach is the best one.
--Will
On 2026/08/05 00:24:41 Justin Talbot via dev wrote:
> Hi all,
>
> We've raised concerns about the foundational semantics document (
> https://github.com/apache/ossie/pull/246, and the compliance suite that
> builds on it in https://github.com/apache/ossie/pull/237) in our working
> group discussions, and we want to make sure they're on the record before
> any vote is called.
>
> We appreciate the work Will has put into these, and we're engaging with the
> semantics document in good faith because we want it to be as clear as
> possible if this direction proceeds. But our engagement with the details
> shouldn't be read as endorsement of the direction or the document's
> readiness.
>
> We have a few key concerns with the current proposal.
>
> BI vendor buy-in
>
> Ossie's success depends on vendor adoption. So our first question about any
> direction, including one we'd prefer ourselves, is whether the vendors who
> would implement it have said they can and will. That hasn't been
> established here. We're not aware of any BI vendor that has tried
> implementing the proposed semantics or querying through the proposed query
> model. We think that experience is necessary before standardization.
>
> We also expect it to be harder than it looks. Some of the specified
> semantics, particularly join direction, fan-out prevention, and M:N
> resolution, conflict with choices existing tools have already made and
> their users depend on. Vendors will also want to layer their own logic on
> top of an imported model, and it isn't clear whether these semantics allow
> that. Tableau (composed data sources) and Power BI (model composition)
> flatten composed graphs into a single graph, so a composed model should
> behave like one built from scratch. That's the kind of thing we'd expect an
> implementation attempt to surface.
>
> At 1,308 lines specifying the core of how a semantic layer behaves, this is
> a lot to adopt as normative on the limited review it has had so far. We'd
> like broader review from the vendors who would implement it, with evidence
> they find the semantics feasible and useful. We're happy to help collect
> realistic query patterns to test against.
>
> The query model only demonstrates trivial queries, and punts the hard parts
>
> Section 5 of the foundational semantics document defines a basic query
> model and assumes it can be extended later. We don't think that gap can be
> deferred.
>
> Section 5 shows only that the semantics can answer trivial queries: no
> joins beyond the model's own relationships, no subqueries, no window
> functions. Real BI tools don't generate such simple queries. Tableau, Power
> BI, and their peers emit multi-way joins, subqueries, and window functions
> to deliver rich analytics, like LOD calculations, Top-N filters, and
> period-over-period comparisons. Those are the hard cases for a semantics
> layer, because they're where fan-out, re-aggregation, grain, and filter
> propagation actually bite.
>
> The spec also mixes two kinds of content. Some guarantees are close to
> universally true, like not silently duplicating a measure. Others are
> opinionated: join direction, path selection, and filter flow are places
> where tools have made different, deliberate choices, and matching all of
> them probably needs tunable knobs rather than one normative answer. Most of
> that opinionated territory is punted. We need to work through those
> opinionated parts before adopting the specification as normative. If we
> ratify now and the semantics or query model later can't express common BI
> functionality, we'll be changing a normative spec that vendors were asked
> to build against.
>
> One clarification, since it comes up: the in-flight expression language
> proposal covers the expressions used to define dimensions and measures
> inside a model. It doesn't cover how consumers query a model. The latter is
> our concern here.
>
> This work implicitly makes an architectural decision that hasn't been
> settled
>
> The interchange side of Ossie, importing a model definition, has broad
> agreement in principle. But #246 also commits the project, implicitly, to a
> particular query architecture: the model is queried as a black box, like a
> single opaque table, with the constituent tables and their relationships
> hidden behind the semantic layer.
>
> That is a significant choice, and we don't think the group has explicitly
> discussed or agreed on it. It's also a hard one to make work. A black box
> leaves three bad options: live with the limited query capability Section 5
> currently specifies, invent SQL extensions that are hard for tools to
> integrate against, or accept non-SQL semantics where the model looks like a
> table but doesn't behave like one when queried. None of these is cheap for
> a BI tool.
>
> We'd rather have narrow, well-defined semantics that work end to end for a
> broad set of vendors than more powerful semantics that aren't well defined
> or widely adopted. There's a real alternative in that direction: a more
> transparent interface where a BI tool can see the constituent tables and
> control how they compose. A tool could then express much of its current
> behavior, filter flow for instance, in ordinary SQL, instead of Ossie
> having to standardize a new query model for it.
>
> There's a related open question on interchange itself: how much of the
> semantics must be strictly portable versus left to the consuming tool. #246
> takes a strong position, very strict semantic agreement, and we don't think
> that position has been settled either.
>
> We'd like the group to make this architectural decision explicitly rather
> than have it fall out of merging these PRs. This is also why the sequencing
> matters now rather than later: #237 builds a compliance suite on top of
> these semantics, which would begin defining Ossie conformance in terms of
> them. We're planning to write up the transparent-interface direction as an
> alternative for the group to weigh, and we don't think one branch of this
> fork should be ratified before the options have been compared.
>
> Readiness of the PR
>
> The #246 description summary is blank and the checklist is entirely
> unchecked. For a normative spec of this scope, we'd expect a clear written
> summary of what's being adopted and what tradeoffs were considered before
> it goes to a vote.
>
> What we're asking for
>
> Before any [VOTE] is called:
>
>
> 1.
>
> Broader review from the vendors who would implement this, with evidence
> they see the semantics as feasible and useful, and some indication of who
> plans to implement, by what mechanism, and on what timeline.
> 2.
>
> A demonstration that these semantics extend to realistic BI queries
> (e.g. joins, subqueries, window functions, LOD calcs, Top-N filters, filter
> flow), not just the trivial cases in Section 5. We're glad to help assemble
> these patterns.
> 3.
>
> An explicit discussion of the query architecture (black-box vs
> transparent) and of how much semantics must be portable on interchange,
> decided on its own merits rather than as a side effect of merging these PRs.
> 4.
>
> A substantive PR description summarizing what's proposed and what
> tradeoffs were considered.
>
>
> We'll follow up with a written alternative on the query architecture
> question so there's something concrete to compare. Thanks!
>
> Justin Talbot
>
> Chris Eubank
>
> Databricks
>