Re: [DISCUSS] Dataset-scoped metrics

[Klahr Joshua via dev <de...@ossie.apache.org>]

 I think this makes sense, and agree it would be good to get this out soon.  The one question I have is how/if we can allow data set scoped metrics to refer to either fields (defined in the dataset) OR columns (from the data set's source columns).  Is this something that could be supported? I think this would be desirable but can create ambiguity if field_names and column_names collide...
Thoughts?
    On Wednesday, September 30, 2026 at 05:04:28 AM GMT+1, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:  
 
 Hi all,

I completely agree with Chris's proposal to tighten the scope here.
Decoupling the placement (datasets[].metrics) from the larger,
in-flight semantic questions (expression composition, traversal paths,
grain locking) is the pragmatic way to get a clean spec vote (I love
pragmatic approaches :)).

A few specific thoughts on the three restrictions and the rollout:
1. Unqualified references and unified namespace are spot on.
Single-part references (SUM(amount) instead of orders.amount) neatly
eliminate the circularity issue in DDL/parser contexts that Justin
pointed out. Enforcing a single namespace across fields and metrics
within the dataset also elliminites the naming collision ambiguity.
2. In the wording, we should be explicit about whether "source fields"
refers to the underlying physical table columns or the dataset's
declared fields (datasets[].fields). In Ossie, metrics should
aggregate the dataset's declared/modeled fields (or we need clear
precedence if both are permitted). Tightening this phrasing will avoid
confusion during the vote.
3. While I support deferring complex AST evaluation or cross-dialect
query semantics to follow-up, I would prefer we do not strip the
validation entirely from this PR. The JSON schema update
(ossie-schema.json) and basic structural validation in validate.py
(uniqueness across fields + metrics, checking for unqualified
references, and null guards) should land together with the spec. In
Ossie, the schema and validator act as our executable spec: keeping
baseline structural checks ensures the vote lands on a solid,
verifiable foundation.

Overall, this narrowed scope gives us a safe, backward-compatible
baseline that we can relax later one expression composition lands.

Happy to review the updated PR and spec wording once ready.

Regards
JB

On Tue, Sep 29, 2026 at 11:50 PM Chris Eubank
<ch...@databricks.com> wrote:
>
> Hi Josh,
>
> Thank you for putting together this proposal for dataset-scoped metrics. It's a fundamental building block for several in-flight extensions: expression resolution / composability (thread), relational query semantics (thread) and dimensional query semantics (thread).
>
> Several open comments remain, mostly concerning those follow-up features (e.g. expression composition). I'm wondering if we can tighten this PR's scope to land the minimal spec extension needed for a spec vote, and defer the rest. The validation logic could even move to a follow-up PR.
>
> Concretely, I'd propose these three restrictions:
>
> 1. Datasets may declare metrics. Within a dataset, names must be unique across fields and metrics.
> 2. A dataset's metric expression may reference only that dataset's source fields. We can lift this restriction once the expression-composition rules land.
> 3. Those references must be single-part (unqualified) within the expression and must resolve to the dataset's source fields.
>
> Would this narrower scope work for you as a path to a spec vote? If so, I'd be glad to help however is most useful - happy to pair on the scoped wording or take a first pass for us to iterate on.
>
> Kind regards,
> Chris
>
> On Thu, Aug 27, 2026 at 2:20 PM Khushboo Bhatia <kb...@apache.org> wrote:
>>
>> Hey Josh,
>> Thanks for starting this discussion. This issue has been raised in multiple discussions on the repo.
>>
>> I am generally supportive of adding these. Leaving some comments on the PR for the subtle differences between semantic model scoped metrics vs dataset scoped.
>>
>> Khushboo
>>
>> On 2026/08/27 16:57:13 Klahr Joshua via dev wrote:
>> >
>> >
>> >
>> > Hi all,
>> >
>> > I'd like to propose allowing metrics to be declared on an individual dataset (datasets[].metrics) in addition to the semantic model (semantic_model.metrics), using the same metric structure in both placements.
>> >
>> > PR: https://github.com/apache/ossie/pull/343
>> >
>> > Motivation
>> >
>> > Today every metric must be declared at the semantic model level, even when it aggregates a single dataset's own fields. Three costs follow from that.
>> >
>> > First, a dataset is not independently interpretable — you cannot tell from a dataset what metrics exist over it without scanning the whole model, and simple aggregations end up separated from the fields they depend on.
>> >
>> > Second, and more concretely: round-trip fidelity through Ossie is reduced. Several formats we convert from attach metrics to a specific entity — Omni has per-view measures, Honeydew has per-entity metric files, WisdomAI has per-table measures, and dbt MetricFlow binds metrics to a semantic model. Because Ossie has no dataset-scoped placement, the converters in this repository flatten all of those to the model level on import, and the entity association is not represented in the resulting document.
>> >
>> > On export, four converters independently reconstruct that association by scanning expressions for dataset.column qualifiers and falling back to the first dataset when none is found. The Honeydew converter emits a runtime warning when it does so. Two converters stash the owning entity in vendor-specific custom_extensions, which round-trips within a single vendor's toolchain but is not something a generic Ossie consumer can produce or interpret.
>> >
>> > Four converters reimplementing the same heuristic suggests the spec is missing a place to put information the source formats already carry.
>> >
>> > Third, export placement can be coarser than the target supports. The Snowflake converter maps every Ossie model-level metric to Snowflake's top-level metrics block, including metrics referencing only one table that would fit the narrower tables[].metrics position. The converter cannot distinguish the two cases because the Ossie document does not record the distinction.
>> >
>> > What's proposed
>> >
>> > datasets[].metrics, using the identical Metric structure. Scoping rules:
>> >
>> >    -
>> > A dataset-scoped metric's expression MUST only reference fields of its declaring dataset, and MUST NOT traverse relationships. Cross-dataset metrics remain model-scoped.
>> >
>> >    -
>> > Names MUST be unique within their dataset — two datasets MAY each declare item_count.
>> >
>> >    -
>> > A dataset-scoped name MUST NOT collide with any model-scoped metric name.
>> >
>> >    -
>> > Referenced from outside the dataset as dataset_name.metric_name.
>> >
>> >
>> > Worth stating explicitly because it's easy to misread: rule 1 constrains the expression, not the query. A dataset-scoped metric can still be grouped by or filtered on dimensions from other datasets via relationships, since grouping dimensions come from the consumer at query time.
>> >
>> > Prior art
>> >
>> > Separating single-entity aggregations from cross-entity calculations is established practice. Four of the five systems I looked at do it — AtScale SML (metric bound to one dataset + column, with metric_calc as a separate type), Cube (measures within cubes), dbt MetricFlow (in-model vs top-level), and Snowflake semantic views (tables[].metrics vs top-level derived). Databricks UC metric views take a different approach, with one flat scope per view and joins declared inside it.
>> >
>> > On naming, the proposal extends Ossie's own existing convention rather than importing one: field names are already unique within a dataset rather than globally, and metric expressions already reference them as dataset.field. Applying the same scoped-uniqueness and qualified-reference pattern to metrics keeps the model internally consistent.
>> >
>> > On strictness it sits within the range the ecosystem spans — SML is more restrictive (one column, one aggregation method), Snowflake semantic views more permissive (table-level metrics may traverse relationships with a declared join path).
>> >
>> > Compatibility
>> >
>> > Additive and backward compatible. datasets[].metrics is optional and existing models remain valid unchanged. Because a dataset-scoped expression resolves within its own dataset, it is already valid as a model-scoped metric, so consumers wanting a single flat namespace can hoist without rewriting expressions. No converter changes are included; existing converters read semantic_model.metrics and that behaviour is untouched.
>> >
>> > Two open questions I'd particularly like input on
>> >
>> >    -
>> > Should a dataset-scoped metric eventually be allowed to traverse relationships when the join path is declared explicitly? The PR takes the conservative position, since relaxing later is backward compatible while tightening is not.
>> >
>> >    -
>> > Separately, I noticed the spec does not say whether one metric expression may reference another metric — there's no mechanism and no prohibition, while every system above has one. It's ambiguous rather than merely absent: SUM(orders.amount) / total_orders passes validation today because total_orders parses as an unqualified column. My inclination is to handle that separately since it needs aggregation-composition semantics settled, but the two features interact. The PR includes a sketch of what the reference-scoping rules would be, in case it's worth fixing those now for forward compatibility.
>> >
>> >
>> > Validation
>> >
>> > Three of the rules are semantic constraints JSON Schema cannot express, so validate.py gains checks for duplicate names within a dataset, collisions with model-scoped names, and cross-dataset references (table qualifiers extracted via sqlglot). Scope checking skips rather than guesses for dialects sqlglot cannot parse. The TPC-DS example is updated to demonstrate both placements.
>> >
>> > AI disclosure
>> >
>> > This contribution was developed with AI assistance (Snowflake Cortex Code). I have reviewed all changes and am personally responsible for the content per the ASF Generative Tooling Guidance.
>> >
>> > Feedback and pushback welcome. Minimum discussion period is 7 days per the contribution guidelines.
>> >
>> > Best,
>> > Josh Klahr
>> >
>> >
>> >
>> >
>> >
  

Re: [DISCUSS] Dataset-scoped metrics

[Chris Eubank via dev <de...@ossie.apache.org>]

Hi Josh,

I think we'd ultimately want to support both. To your point, though, that
introduces some naming ambiguity. Damian raised a related issue in this
thread <https://lists.apache.org/thread/bc9v9o0d2702gwm978km13mg6v16hzv1>
around
naming for model-level metrics as well.

Naming rules - especially as we define how expressions can be composed -
feel like a broader topic that deserves its own PR. My understanding from
that thread is that Damian and Justin plan to create a joint PR covering
this area.

For this PR, I proposed restricting references to source columns for
consistency with fields, and getting the core concept of dataset-level
metrics into the spec without resolving the broader naming questions. The
naming work can then build on and reference the concepts established here.

Kind regards,
Chris

On Thu, Oct 1, 2026 at 10:44 AM Klahr Joshua <jk...@yahoo.com> wrote:

> I think this makes sense, and agree it would be good to get this out
> soon.  The one question I have is how/if we can allow data set scoped
> metrics to refer to either fields (defined in the dataset) OR columns (from
> the data set's source columns).  Is this something that could be supported?
> I think this would be desirable but can create ambiguity if field_names and
> column_names collide...
>
> Thoughts?
>
> On Wednesday, September 30, 2026 at 05:04:28 AM GMT+1, Jean-Baptiste
> Onofré <jb...@nanthrax.net> wrote:
>
>
> Hi all,
>
> I completely agree with Chris's proposal to tighten the scope here.
> Decoupling the placement (datasets[].metrics) from the larger,
> in-flight semantic questions (expression composition, traversal paths,
> grain locking) is the pragmatic way to get a clean spec vote (I love
> pragmatic approaches :)).
>
> A few specific thoughts on the three restrictions and the rollout:
> 1. Unqualified references and unified namespace are spot on.
> Single-part references (SUM(amount) instead of orders.amount) neatly
> eliminate the circularity issue in DDL/parser contexts that Justin
> pointed out. Enforcing a single namespace across fields and metrics
> within the dataset also elliminites the naming collision ambiguity.
> 2. In the wording, we should be explicit about whether "source fields"
> refers to the underlying physical table columns or the dataset's
> declared fields (datasets[].fields). In Ossie, metrics should
> aggregate the dataset's declared/modeled fields (or we need clear
> precedence if both are permitted). Tightening this phrasing will avoid
> confusion during the vote.
> 3. While I support deferring complex AST evaluation or cross-dialect
> query semantics to follow-up, I would prefer we do not strip the
> validation entirely from this PR. The JSON schema update
> (ossie-schema.json) and basic structural validation in validate.py
> (uniqueness across fields + metrics, checking for unqualified
> references, and null guards) should land together with the spec. In
> Ossie, the schema and validator act as our executable spec: keeping
> baseline structural checks ensures the vote lands on a solid,
> verifiable foundation.
>
> Overall, this narrowed scope gives us a safe, backward-compatible
> baseline that we can relax later one expression composition lands.
>
> Happy to review the updated PR and spec wording once ready.
>
> Regards
> JB
>
> On Tue, Sep 29, 2026 at 11:50 PM Chris Eubank
> <ch...@databricks.com> wrote:
> >
> > Hi Josh,
> >
> > Thank you for putting together this proposal for dataset-scoped metrics.
> It's a fundamental building block for several in-flight extensions:
> expression resolution / composability (thread), relational query semantics
> (thread) and dimensional query semantics (thread).
> >
> > Several open comments remain, mostly concerning those follow-up features
> (e.g. expression composition). I'm wondering if we can tighten this PR's
> scope to land the minimal spec extension needed for a spec vote, and defer
> the rest. The validation logic could even move to a follow-up PR.
> >
> > Concretely, I'd propose these three restrictions:
> >
> > 1. Datasets may declare metrics. Within a dataset, names must be unique
> across fields and metrics.
> > 2. A dataset's metric expression may reference only that dataset's
> source fields. We can lift this restriction once the expression-composition
> rules land.
> > 3. Those references must be single-part (unqualified) within the
> expression and must resolve to the dataset's source fields.
> >
> > Would this narrower scope work for you as a path to a spec vote? If so,
> I'd be glad to help however is most useful - happy to pair on the scoped
> wording or take a first pass for us to iterate on.
> >
> > Kind regards,
> > Chris
> >
> > On Thu, Aug 27, 2026 at 2:20 PM Khushboo Bhatia <kb...@apache.org>
> wrote:
> >>
> >> Hey Josh,
> >> Thanks for starting this discussion. This issue has been raised in
> multiple discussions on the repo.
> >>
> >> I am generally supportive of adding these. Leaving some comments on the
> PR for the subtle differences between semantic model scoped metrics vs
> dataset scoped.
> >>
> >> Khushboo
> >>
> >> On 2026/08/27 16:57:13 Klahr Joshua via dev wrote:
> >> >
> >> >
> >> >
> >> > Hi all,
> >> >
> >> > I'd like to propose allowing metrics to be declared on an individual
> dataset (datasets[].metrics) in addition to the semantic model
> (semantic_model.metrics), using the same metric structure in both
> placements.
> >> >
> >> > PR: https://github.com/apache/ossie/pull/343
> >> >
> >> > Motivation
> >> >
> >> > Today every metric must be declared at the semantic model level, even
> when it aggregates a single dataset's own fields. Three costs follow from
> that.
> >> >
> >> > First, a dataset is not independently interpretable — you cannot tell
> from a dataset what metrics exist over it without scanning the whole model,
> and simple aggregations end up separated from the fields they depend on.
> >> >
> >> > Second, and more concretely: round-trip fidelity through Ossie is
> reduced. Several formats we convert from attach metrics to a specific
> entity — Omni has per-view measures, Honeydew has per-entity metric files,
> WisdomAI has per-table measures, and dbt MetricFlow binds metrics to a
> semantic model. Because Ossie has no dataset-scoped placement, the
> converters in this repository flatten all of those to the model level on
> import, and the entity association is not represented in the resulting
> document.
> >> >
> >> > On export, four converters independently reconstruct that association
> by scanning expressions for dataset.column qualifiers and falling back to
> the first dataset when none is found. The Honeydew converter emits a
> runtime warning when it does so. Two converters stash the owning entity in
> vendor-specific custom_extensions, which round-trips within a single
> vendor's toolchain but is not something a generic Ossie consumer can
> produce or interpret.
> >> >
> >> > Four converters reimplementing the same heuristic suggests the spec
> is missing a place to put information the source formats already carry.
> >> >
> >> > Third, export placement can be coarser than the target supports. The
> Snowflake converter maps every Ossie model-level metric to Snowflake's
> top-level metrics block, including metrics referencing only one table that
> would fit the narrower tables[].metrics position. The converter cannot
> distinguish the two cases because the Ossie document does not record the
> distinction.
> >> >
> >> > What's proposed
> >> >
> >> > datasets[].metrics, using the identical Metric structure. Scoping
> rules:
> >> >
> >> >    -
> >> > A dataset-scoped metric's expression MUST only reference fields of
> its declaring dataset, and MUST NOT traverse relationships. Cross-dataset
> metrics remain model-scoped.
> >> >
> >> >    -
> >> > Names MUST be unique within their dataset — two datasets MAY each
> declare item_count.
> >> >
> >> >    -
> >> > A dataset-scoped name MUST NOT collide with any model-scoped metric
> name.
> >> >
> >> >    -
> >> > Referenced from outside the dataset as dataset_name.metric_name.
> >> >
> >> >
> >> > Worth stating explicitly because it's easy to misread: rule 1
> constrains the expression, not the query. A dataset-scoped metric can still
> be grouped by or filtered on dimensions from other datasets via
> relationships, since grouping dimensions come from the consumer at query
> time.
> >> >
> >> > Prior art
> >> >
> >> > Separating single-entity aggregations from cross-entity calculations
> is established practice. Four of the five systems I looked at do it —
> AtScale SML (metric bound to one dataset + column, with metric_calc as a
> separate type), Cube (measures within cubes), dbt MetricFlow (in-model vs
> top-level), and Snowflake semantic views (tables[].metrics vs top-level
> derived). Databricks UC metric views take a different approach, with one
> flat scope per view and joins declared inside it.
> >> >
> >> > On naming, the proposal extends Ossie's own existing convention
> rather than importing one: field names are already unique within a dataset
> rather than globally, and metric expressions already reference them as
> dataset.field. Applying the same scoped-uniqueness and qualified-reference
> pattern to metrics keeps the model internally consistent.
> >> >
> >> > On strictness it sits within the range the ecosystem spans — SML is
> more restrictive (one column, one aggregation method), Snowflake semantic
> views more permissive (table-level metrics may traverse relationships with
> a declared join path).
> >> >
> >> > Compatibility
> >> >
> >> > Additive and backward compatible. datasets[].metrics is optional and
> existing models remain valid unchanged. Because a dataset-scoped expression
> resolves within its own dataset, it is already valid as a model-scoped
> metric, so consumers wanting a single flat namespace can hoist without
> rewriting expressions. No converter changes are included; existing
> converters read semantic_model.metrics and that behaviour is untouched.
> >> >
> >> > Two open questions I'd particularly like input on
> >> >
> >> >    -
> >> > Should a dataset-scoped metric eventually be allowed to traverse
> relationships when the join path is declared explicitly? The PR takes the
> conservative position, since relaxing later is backward compatible while
> tightening is not.
> >> >
> >> >    -
> >> > Separately, I noticed the spec does not say whether one metric
> expression may reference another metric — there's no mechanism and no
> prohibition, while every system above has one. It's ambiguous rather than
> merely absent: SUM(orders.amount) / total_orders passes validation today
> because total_orders parses as an unqualified column. My inclination is to
> handle that separately since it needs aggregation-composition semantics
> settled, but the two features interact. The PR includes a sketch of what
> the reference-scoping rules would be, in case it's worth fixing those now
> for forward compatibility.
> >> >
> >> >
> >> > Validation
> >> >
> >> > Three of the rules are semantic constraints JSON Schema cannot
> express, so validate.py gains checks for duplicate names within a dataset,
> collisions with model-scoped names, and cross-dataset references (table
> qualifiers extracted via sqlglot). Scope checking skips rather than guesses
> for dialects sqlglot cannot parse. The TPC-DS example is updated to
> demonstrate both placements.
> >> >
> >> > AI disclosure
> >> >
> >> > This contribution was developed with AI assistance (Snowflake Cortex
> Code). I have reviewed all changes and am personally responsible for the
> content per the ASF Generative Tooling Guidance.
> >> >
> >> > Feedback and pushback welcome. Minimum discussion period is 7 days
> per the contribution guidelines.
> >> >
> >> > Best,
> >> > Josh Klahr
> >> >
> >> >
> >> >
> >> >
> >> >
>