You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ossie.apache.org by "eisber (via GitHub)" <gi...@apache.org> on 2026/08/14 17:33:40 UTC
[PR] feat: register MICROSOFT as a well-known vendor token and reconcile the vendor lists [ossie]
eisber opened a new pull request, #328:
URL: https://github.com/apache/ossie/pull/328
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
## Summary
Registers the Microsoft Tabular model (TMSL / TMDL / TOM) as a well-known `custom_extensions` vendor, so Power BI and Fabric semantic models can be a first-class spoke in the hub-and-spoke converter ecosystem.
This is the **registration slice only** — no converter implementation. It is deliberately small so the naming question can be settled before code depends on the answer.
### The token is `MICROSOFT`, not `POWER_BI`
There are currently two tokens in flight for the same vendor: #250 adds `MICROSOFT` to `Vendor.examples`, while work in progress elsewhere uses `POWER_BI` in the converters table. If both land, the spec ships contradictory guidance and `custom_extensions` written by one tool are silently invisible to the other.
This PR settles it on `MICROSOFT`, agreeing with #250:
1. **One object model, four products.** The Tabular model — scripted as TMSL, authored as TMDL, manipulated through TOM — is the storage format for Power BI semantic models, Fabric semantic models, Azure Analysis Services *and* SQL Server Analysis Services. A `model.bim` does not record which produced it, and a converter reading one cannot tell. A product-named token is therefore wrong for three of the four inputs.
2. **It matches existing precedent.** Ossie tokens name the organization — `SNOWFLAKE`, `DATABRICKS`, `GOODDATA`, `HONEYDEW` — and `SALESFORCE` already covers a differently-branded product, the Tableau semantic layer.
3. **It costs no renames.** #299 is currently renaming the GSF converter directory and package to `nvidia` to match its token. Under `MICROSOFT` a `converters/microsoft/` directory and an `ossie_microsoft` package already agree; under `POWER_BI` they would need exactly the churn #299 is doing.
The honest objection to a company token is that it over-covers: Microsoft ships semantic surfaces that are not Tabular models. That is answered **inside the payload**, with a `format` discriminator in the envelope, rather than by minting a second vendor token. This picks the recoverable error over the unrecoverable one — a payload labelled too broadly can be narrowed within the envelope, whereas a token that is too narrow cannot be widened once it is written into files in the wild.
**"Register both" is explicitly rejected.** `MICROSOFT` is canonical for reading *and* writing, and readers must **not** accept `POWER_BI` as an alias. A converter that stashes under one token and reads the other loses data on round-trip, and a reader that quietly accepts two spellings makes a disagreeing writer undetectable — the first symptom would be a silently lost round trip.
### Reconciling the vendor lists
Five places enumerated well-known vendors, and **all five already disagreed with each other on `main`**. This PR reconciles them rather than appending to one:
| Location | Was missing | Now |
|---|---|---|
| `core-spec/spec.md` "Vendor Names" | `OMNI`, `NVIDIA_GSF` | canonical list, + `MICROSOFT` |
| `converters/README.md` "Supported Vendors" | `GOODDATA`, `HONEYDEW` | + `MICROSOFT`, and now points at the canonical list |
| `core-spec/osi-schema.json` `Vendor.examples` | `HONEYDEW`, `OMNI`, `NVIDIA_GSF` | + `MICROSOFT` |
| `core-spec/spec.yaml` comment | 4 tokens | resynced to the canonical list |
| `python/src/ossie/models.py` `OSIVendor` | `HONEYDEW`, `OMNI`, `NVIDIA_GSF` | + `MICROSOFT`, + docstring |
`core-spec/spec.md` is now named as the single canonical list, and the other four point at it, so they cannot silently drift apart again.
`vendor_name` remains a **free-form string**. Nothing here turns it into an enum — the schema keeps `{"type": "string", "examples": [...]}`, the description now says outright that the examples are not a closed set, and `OSICustomExtension.vendor_name` stays typed `str`. `OSIVendor` is documented as advisory.
### Drive-by fix
The `Custom Extensions` example block used `vendor_name: Databricks` — the only mixed-case token in a document that is SCREAMING_SNAKE throughout, and inconsistent with the `DATABRICKS` row in the table directly above it. Changed to `DATABRICKS`. Flagging it explicitly rather than burying it: it is a one-word docs change with no schema effect (`vendor_name` is free-form, so both were always valid), but it is a change to a published example.
### What gives the registration substance
`docs/vendor_extensions/microsoft.md` documents what actually belongs in `custom_extensions[MICROSOFT]` — otherwise a registration is just a table row. It covers the constructs Ossie core cannot express, per level (model / dataset / field / metric / relationship):
- DAX expressions (and where they *don't* go), format strings, display folders, `isHidden` visibility
- hierarchies, calculation groups, perspectives
- RLS and OLS roles
- partitions and storage mode — Import / DirectQuery / Dual / **Direct Lake**
- lineage tags, `sortByColumn`, `summarizeBy`, relationship cardinality and cross-filter behaviour
Design points worth reviewing:
- **Keys are TMSL property names spelled exactly as TMSL spells them.** No synonyms, no re-casing — so any key in a payload can be looked up directly in the public reference.
- **Preservation is a deny-list, not an allow-list.** A converter should carry across TMSL properties it has never heard of. New properties appear over time; an allow-list turns each one into silent data loss.
- **A `_v` version and a `format` discriminator** in the envelope, with "fail loudly, never guess" rules for both.
- **The extension carries only what has no core home.** A measure's DAX belongs in `metrics[].expression` under the `DAX` dialect, *not* in the extension. There is an explicit table of constructs that must stay in core fields, so the two cannot become competing sources of truth.
- **Security is called out as absent.** An Ossie document converted from a model with RLS/OLS describes the *unsecured* shape and must not be treated as a security boundary.
Everything in it is reconstructible from public documentation — learn.microsoft.com's TMSL/TMDL/TOM/DAX/Direct Lake references and the public `microsoft/semantic-link-labs` repository, all cited at the bottom of the file.
`examples/microsoft_extension.yaml` is a worked example covering model, dataset, field, metric and relationship placement.
## Relationship to #250
This PR **agrees with and depends on nothing from** #250, and the two can merge in either order.
- #250 adds `MICROSOFT` to `Vendor.examples`; this PR's `Vendor.examples` line is a strict superset of #250's, so the conflict is a trivial one-line rebase whichever lands first.
- The `DAX` dialect — `$defs.Dialect`, the spec dialect table, `OSIDialect`, and the `validate.py` skip list — comes entirely from #250 and is **not duplicated here**. `docs/vendor_extensions/microsoft.md` defers to #250 by reference.
- Because of that, `examples/microsoft_extension.yaml` deliberately carries **no `DAX` dialect entry** yet, so it validates on today's `main`. Once #250 lands, the DAX-dialect form documented in the vendor doc becomes usable.
Worth knowing while #250 is open: `validate.py` currently hands a `DAX` expression to sqlglot as if it were SQL. Simple DAX happens to parse — `CALCULATE(SUM('Sales'[Amount]), 'Date'[Year] = 2024)` is accepted — while realistic DAX using `VAR` / `RETURN` fails. #250's skip-list entry fixes this; noting it here as supporting evidence for that part of #250.
## Other overlapping PRs
- **#299** (GSF → NVIDIA rename) — no file overlap. Its directory/package rename is cited above as precedent for token choice.
- **#306** (converter/schema alignment) — also edits `python/src/ossie/models.py`, but removes `vendors: Optional[list[OSIVendor]]` from the root model, which is different lines from this change. `OSIVendor` remains an exported public enum, so both changes are compatible.
- **#313** (Honeydew rebrand) and **#288** (OSI → Ossie rename) — no logical overlap, but #288 touches 135 files, so rebase order matters mechanically.
- **#323** (root-level `ai_context` / `custom_extensions`) — no conflict; this PR deliberately places nothing at the document root.
## Open questions for reviewers
1. **`SEMANTIDO`** appears in `OSIVendor` and in no other list — not in the spec, the schema, or any converter. I left it untouched rather than guess. Someone who knows what it is should either document or remove it.
2. **RLS role membership can name identities.** Should a converter preserve `members`, or drop it as unnecessary personal data? The doc flags the question rather than deciding it.
3. **`ORIONBELT` and `POLARIS`** have converters in-repo but no registered vendor token.
4. **Is `docs/vendor_extensions/` the right home** for per-vendor payload conventions, or should this live under `core-spec/`? It is a new convention introduced by this PR.
5. **Should `converters/README.md` keep its own vendor table at all**, or just link the canonical one? Two tables that must agree is what caused the current drift.
## Related Issues
Relates to #250.
## Checklist
### Specification
- [x] Spec changes are included in `core-spec/` and follow the existing structure
- [ ] Spec changes have been discussed on the mailing list or in a linked issue — **not yet done.** Per `CONTRIBUTING.md`, changes touching `core-spec/` need a `dev@ossie.apache.org` thread and a minimum 7-day discussion period. This PR is opened to give that discussion something concrete to point at; the dev@ announcement still needs to be sent before it can merge.
- [x] Breaking changes to the spec are clearly called out in the summary — there are none. Adding examples to a free-form string field and adding rows to a documentation table cannot invalidate an existing document.
### Ontology
- [x] Ontology changes in `ontology/` are consistent with spec changes — n/a, no ontology change.
- [x] New or modified terms are defined and documented
### Converters
- [x] Converter logic in `converters/` is updated to reflect spec or ontology changes — n/a by design; this is the registration slice only, no converter is added or changed.
- [x] New converters include tests under the converter's test directory — n/a, no new converter.
### Validation
- [x] Validation rules in `validation/` are updated if the spec changed — no change needed; `vendor_name` is free-form, so no validation rule keys off the token. (The `DAX` skip-list entry belongs to #250.)
- [x] New validation cases are covered by tests
### Documentation
- [x] `docs/` is updated to reflect any user-facing changes
- [x] New features or behaviors are documented with examples where appropriate
- [x] `CONTRIBUTING.md` is updated if the contribution process changed — n/a, unchanged.
### Examples
- [x] `examples/` are added or updated for any new spec constructs or converter support
### Tests
- [x] All existing tests pass — `python validation/validate.py` passes on `examples/microsoft_extension.yaml`, `examples/tpcds_semantic_model.yaml` and `examples/flights.yaml`; `pytest python/tests` is 9 passed.
- [x] New functionality is covered by tests — the new example is exercised by the validator.
### Compliance
- [x] ASF license headers are present on all new source files — both new files carry the standard header.
- [x] No third-party dependencies are added without PMC/IPMC approval — none added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: register MICROSOFT as a well-known vendor token and reconcile the vendor lists [ossie]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #328:
URL: https://github.com/apache/ossie/pull/328#discussion_r3785953089
##########
examples/microsoft_extension.yaml:
##########
@@ -0,0 +1,223 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Worked example of the MICROSOFT vendor extension.
+#
+# The Microsoft Tabular model (TMSL / TMDL / TOM) backs Power BI semantic models,
+# Fabric semantic models, Azure Analysis Services and SQL Server Analysis Services.
+# Constructs that Ossie core can express are expressed in core fields; everything else
+# is preserved in a `MICROSOFT` custom_extensions entry, keyed by TMSL property name.
+#
+# See docs/vendor_extensions/microsoft.md for the full key catalogue.
+#
+# NOTE: metric expressions below use the ANSI_SQL dialect. A real Tabular measure is
+# DAX, and belongs in a `DAX` dialect entry — that enum member is added by
+# apache/ossie#250, so this example stays portable until #250 merges.
+
+version: 0.2.0.dev0
+semantic_model:
+ - name: sales_analytics
+ description: Sales analytics model imported from a Microsoft Tabular model
+ datasets:
+ - name: sales
+ source: Sales
+ description: Sales fact table
+ primary_key:
+ - sales_order_line_id
+ fields:
+ - name: sales_order_line_id
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: SalesOrderLineID
+ datatype: Integer
+ custom_extensions:
+ # A surrogate key: present and queryable, but curated out of the field
+ # list. Ossie has no visibility concept, so without this an AI consumer
+ # cannot tell it apart from a business column.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "isHidden": true,
+ "summarizeBy": "none",
+ "lineageTag": "2f4a1c7e-8b3d-4e15-9a62-0d7c5b1e9f38"
+ }'
+ - name: extended_amount
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: ExtendedAmount
+ datatype: Decimal
+ description: Line item amount
+ custom_extensions:
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "formatString": "\\$#,0.00;(\\$#,0.00);\\$#,0.00",
+ "displayFolder": "Amounts",
+ "summarizeBy": "sum"
+ }'
+ - name: order_date
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: OrderDate
+ datatype: Date
+ dimension:
+ is_time: true
+ custom_extensions:
+ # A Direct Lake table. `dataset.source` is a single name, so the storage mode
+ # and the partition shape have nowhere else to live — and the mode decides
+ # whether the model holds data or delegates to OneLake at query time.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "partitions": [
+ {
+ "name": "Sales",
+ "mode": "directLake",
+ "source": {"type": "entity", "entityName": "Sales", "schemaName": "dbo"}
+ }
+ ],
+ "lineageTag": "6c0b9d24-3f77-4a51-b8ee-2a9d4c6f1b03"
+ }'
+
+ - name: date
+ source: Date
+ description: Date dimension
+ primary_key:
+ - date_key
+ fields:
+ - name: date_key
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: DateKey
+ datatype: Integer
+ - name: month_name
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: MonthName
+ datatype: String
+ custom_extensions:
+ # Without sortByColumn, "April" sorts before "August" and every chart
+ # built on this column is silently in the wrong order.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "sortByColumn": "MonthNumber",
+ "displayFolder": "Calendar"
+ }'
+ - name: calendar_date
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: CalendarDate
+ datatype: Date
+ dimension:
+ is_time: true
+ custom_extensions:
+ # dataCategory: Time marks the date table; the hierarchy is an ordered drill
+ # path, and the ordering is the whole content of the construct.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "dataCategory": "Time",
+ "hierarchies": [
+ {
+ "name": "Calendar",
+ "levels": [
+ {"name": "Year", "ordinal": 0, "column": "CalendarYear"},
+ {"name": "Month", "ordinal": 1, "column": "MonthName"},
+ {"name": "Date", "ordinal": 2, "column": "CalendarDate"}
+ ]
+ }
+ ]
+ }'
+
+ relationships:
+ - name: sales_to_date
+ from: sales
+ to: date
+ from_columns:
+ - order_date
+ to_columns:
+ - calendar_date
+ custom_extensions:
+ # Ossie has no inactive-join concept. Recording an inactive relationship as an
+ # ordinary one would misrepresent the join graph, so the flag is preserved and
+ # the relationship reported.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "isActive": true,
Review Comment:
The comment describes preserving an **inactive** relationship, but the example payload sets `"isActive": true`. Either flip the example to `false` to match the narrative, or adjust the comment so readers aren’t misled about what’s being demonstrated.
##########
examples/microsoft_extension.yaml:
##########
@@ -0,0 +1,223 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Worked example of the MICROSOFT vendor extension.
+#
+# The Microsoft Tabular model (TMSL / TMDL / TOM) backs Power BI semantic models,
+# Fabric semantic models, Azure Analysis Services and SQL Server Analysis Services.
+# Constructs that Ossie core can express are expressed in core fields; everything else
+# is preserved in a `MICROSOFT` custom_extensions entry, keyed by TMSL property name.
+#
+# See docs/vendor_extensions/microsoft.md for the full key catalogue.
+#
+# NOTE: metric expressions below use the ANSI_SQL dialect. A real Tabular measure is
+# DAX, and belongs in a `DAX` dialect entry — that enum member is added by
+# apache/ossie#250, so this example stays portable until #250 merges.
+
+version: 0.2.0.dev0
+semantic_model:
+ - name: sales_analytics
+ description: Sales analytics model imported from a Microsoft Tabular model
+ datasets:
+ - name: sales
+ source: Sales
+ description: Sales fact table
+ primary_key:
+ - sales_order_line_id
+ fields:
+ - name: sales_order_line_id
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: SalesOrderLineID
+ datatype: Integer
+ custom_extensions:
+ # A surrogate key: present and queryable, but curated out of the field
+ # list. Ossie has no visibility concept, so without this an AI consumer
+ # cannot tell it apart from a business column.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "isHidden": true,
+ "summarizeBy": "none",
+ "lineageTag": "2f4a1c7e-8b3d-4e15-9a62-0d7c5b1e9f38"
+ }'
+ - name: extended_amount
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: ExtendedAmount
+ datatype: Decimal
+ description: Line item amount
+ custom_extensions:
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "formatString": "\\$#,0.00;(\\$#,0.00);\\$#,0.00",
+ "displayFolder": "Amounts",
+ "summarizeBy": "sum"
+ }'
+ - name: order_date
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: OrderDate
+ datatype: Date
+ dimension:
+ is_time: true
+ custom_extensions:
+ # A Direct Lake table. `dataset.source` is a single name, so the storage mode
+ # and the partition shape have nowhere else to live — and the mode decides
+ # whether the model holds data or delegates to OneLake at query time.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "partitions": [
+ {
+ "name": "Sales",
+ "mode": "directLake",
+ "source": {"type": "entity", "entityName": "Sales", "schemaName": "dbo"}
+ }
+ ],
+ "lineageTag": "6c0b9d24-3f77-4a51-b8ee-2a9d4c6f1b03"
+ }'
+
+ - name: date
+ source: Date
+ description: Date dimension
+ primary_key:
+ - date_key
+ fields:
+ - name: date_key
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: DateKey
+ datatype: Integer
+ - name: month_name
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: MonthName
+ datatype: String
+ custom_extensions:
+ # Without sortByColumn, "April" sorts before "August" and every chart
+ # built on this column is silently in the wrong order.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "sortByColumn": "MonthNumber",
+ "displayFolder": "Calendar"
+ }'
+ - name: calendar_date
Review Comment:
The MICROSOFT extension for `month_name` sets `sortByColumn` to `MonthNumber`, but this example model does not define a `MonthNumber`/`month_number` field in the `date` dataset. That makes the example internally inconsistent and would prevent a faithful round-trip back to a Tabular model (the sort column has no representation in core fields). Consider adding the sort column as a (possibly hidden) field in the dataset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: register MICROSOFT as a well-known vendor token and reconcile the vendor lists [ossie]
Posted by "eisber (via GitHub)" <gi...@apache.org>.
eisber closed pull request #328: feat: register MICROSOFT as a well-known vendor token and reconcile the vendor lists
URL: https://github.com/apache/ossie/pull/328
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: register MICROSOFT as a well-known vendor token and reconcile the vendor lists [ossie]
Posted by "eisber (via GitHub)" <gi...@apache.org>.
eisber commented on code in PR #328:
URL: https://github.com/apache/ossie/pull/328#discussion_r3786034544
##########
examples/microsoft_extension.yaml:
##########
@@ -0,0 +1,223 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Worked example of the MICROSOFT vendor extension.
+#
+# The Microsoft Tabular model (TMSL / TMDL / TOM) backs Power BI semantic models,
+# Fabric semantic models, Azure Analysis Services and SQL Server Analysis Services.
+# Constructs that Ossie core can express are expressed in core fields; everything else
+# is preserved in a `MICROSOFT` custom_extensions entry, keyed by TMSL property name.
+#
+# See docs/vendor_extensions/microsoft.md for the full key catalogue.
+#
+# NOTE: metric expressions below use the ANSI_SQL dialect. A real Tabular measure is
+# DAX, and belongs in a `DAX` dialect entry — that enum member is added by
+# apache/ossie#250, so this example stays portable until #250 merges.
+
+version: 0.2.0.dev0
+semantic_model:
+ - name: sales_analytics
+ description: Sales analytics model imported from a Microsoft Tabular model
+ datasets:
+ - name: sales
+ source: Sales
+ description: Sales fact table
+ primary_key:
+ - sales_order_line_id
+ fields:
+ - name: sales_order_line_id
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: SalesOrderLineID
+ datatype: Integer
+ custom_extensions:
+ # A surrogate key: present and queryable, but curated out of the field
+ # list. Ossie has no visibility concept, so without this an AI consumer
+ # cannot tell it apart from a business column.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "isHidden": true,
+ "summarizeBy": "none",
+ "lineageTag": "2f4a1c7e-8b3d-4e15-9a62-0d7c5b1e9f38"
+ }'
+ - name: extended_amount
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: ExtendedAmount
+ datatype: Decimal
+ description: Line item amount
+ custom_extensions:
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "formatString": "\\$#,0.00;(\\$#,0.00);\\$#,0.00",
+ "displayFolder": "Amounts",
+ "summarizeBy": "sum"
+ }'
+ - name: order_date
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: OrderDate
+ datatype: Date
+ dimension:
+ is_time: true
+ custom_extensions:
+ # A Direct Lake table. `dataset.source` is a single name, so the storage mode
+ # and the partition shape have nowhere else to live — and the mode decides
+ # whether the model holds data or delegates to OneLake at query time.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "partitions": [
+ {
+ "name": "Sales",
+ "mode": "directLake",
+ "source": {"type": "entity", "entityName": "Sales", "schemaName": "dbo"}
+ }
+ ],
+ "lineageTag": "6c0b9d24-3f77-4a51-b8ee-2a9d4c6f1b03"
+ }'
+
+ - name: date
+ source: Date
+ description: Date dimension
+ primary_key:
+ - date_key
+ fields:
+ - name: date_key
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: DateKey
+ datatype: Integer
+ - name: month_name
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: MonthName
+ datatype: String
+ custom_extensions:
+ # Without sortByColumn, "April" sorts before "August" and every chart
+ # built on this column is silently in the wrong order.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "sortByColumn": "MonthNumber",
+ "displayFolder": "Calendar"
+ }'
+ - name: calendar_date
Review Comment:
Good catch, and it was worse than one dangling reference — fixed in e6efc8e.
`MonthNumber` now exists as a real field (`month_number`, TMSL column `MonthNumber`), hidden with `summarizeBy: none` as a sort key normally is. While fixing it I found the same defect a second time in the same dataset: the `Calendar` hierarchy's `Year` level names `CalendarYear`, which was also undefined. That is now a field too, so every reference in the example resolves — verified mechanically, 0 dangling.
The real gap was in the doc, not the example. It never stated the rule, so the example had nothing to be wrong against. `docs/vendor_extensions/microsoft.md` now has a **"Which name space a value refers to"** section covering both halves of the trap:
1. These references are **TMSL** names, not Ossie field names. A converter emitting Ossie field `month_name` for TMSL column `MonthName` must keep `sortByColumn` pointing at `MonthName` — translating it into the Ossie name space breaks replay.
2. A payload must not reference a Tabular object that has no Ossie object. Your phrasing is the point exactly: preserving a sort key *only* inside a vendor payload is not preservation. It dangles for every consumer that reads core fields, and the sort order silently loses its target.
Which is also the general answer to "possibly hidden": a column existing purely to support another construct gets emitted as a normal field with `isHidden: true`, rather than dropped as an implementation detail. The doc says so now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat: register MICROSOFT as a well-known vendor token and reconcile the vendor lists [ossie]
Posted by "eisber (via GitHub)" <gi...@apache.org>.
eisber commented on code in PR #328:
URL: https://github.com/apache/ossie/pull/328#discussion_r3786036305
##########
examples/microsoft_extension.yaml:
##########
@@ -0,0 +1,223 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Worked example of the MICROSOFT vendor extension.
+#
+# The Microsoft Tabular model (TMSL / TMDL / TOM) backs Power BI semantic models,
+# Fabric semantic models, Azure Analysis Services and SQL Server Analysis Services.
+# Constructs that Ossie core can express are expressed in core fields; everything else
+# is preserved in a `MICROSOFT` custom_extensions entry, keyed by TMSL property name.
+#
+# See docs/vendor_extensions/microsoft.md for the full key catalogue.
+#
+# NOTE: metric expressions below use the ANSI_SQL dialect. A real Tabular measure is
+# DAX, and belongs in a `DAX` dialect entry — that enum member is added by
+# apache/ossie#250, so this example stays portable until #250 merges.
+
+version: 0.2.0.dev0
+semantic_model:
+ - name: sales_analytics
+ description: Sales analytics model imported from a Microsoft Tabular model
+ datasets:
+ - name: sales
+ source: Sales
+ description: Sales fact table
+ primary_key:
+ - sales_order_line_id
+ fields:
+ - name: sales_order_line_id
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: SalesOrderLineID
+ datatype: Integer
+ custom_extensions:
+ # A surrogate key: present and queryable, but curated out of the field
+ # list. Ossie has no visibility concept, so without this an AI consumer
+ # cannot tell it apart from a business column.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "isHidden": true,
+ "summarizeBy": "none",
+ "lineageTag": "2f4a1c7e-8b3d-4e15-9a62-0d7c5b1e9f38"
+ }'
+ - name: extended_amount
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: ExtendedAmount
+ datatype: Decimal
+ description: Line item amount
+ custom_extensions:
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "formatString": "\\$#,0.00;(\\$#,0.00);\\$#,0.00",
+ "displayFolder": "Amounts",
+ "summarizeBy": "sum"
+ }'
+ - name: order_date
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: OrderDate
+ datatype: Date
+ dimension:
+ is_time: true
+ custom_extensions:
+ # A Direct Lake table. `dataset.source` is a single name, so the storage mode
+ # and the partition shape have nowhere else to live — and the mode decides
+ # whether the model holds data or delegates to OneLake at query time.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "partitions": [
+ {
+ "name": "Sales",
+ "mode": "directLake",
+ "source": {"type": "entity", "entityName": "Sales", "schemaName": "dbo"}
+ }
+ ],
+ "lineageTag": "6c0b9d24-3f77-4a51-b8ee-2a9d4c6f1b03"
+ }'
+
+ - name: date
+ source: Date
+ description: Date dimension
+ primary_key:
+ - date_key
+ fields:
+ - name: date_key
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: DateKey
+ datatype: Integer
+ - name: month_name
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: MonthName
+ datatype: String
+ custom_extensions:
+ # Without sortByColumn, "April" sorts before "August" and every chart
+ # built on this column is silently in the wrong order.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "sortByColumn": "MonthNumber",
+ "displayFolder": "Calendar"
+ }'
+ - name: calendar_date
+ expression:
+ dialects:
+ - dialect: ANSI_SQL
+ expression: CalendarDate
+ datatype: Date
+ dimension:
+ is_time: true
+ custom_extensions:
+ # dataCategory: Time marks the date table; the hierarchy is an ordered drill
+ # path, and the ordering is the whole content of the construct.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "dataCategory": "Time",
+ "hierarchies": [
+ {
+ "name": "Calendar",
+ "levels": [
+ {"name": "Year", "ordinal": 0, "column": "CalendarYear"},
+ {"name": "Month", "ordinal": 1, "column": "MonthName"},
+ {"name": "Date", "ordinal": 2, "column": "CalendarDate"}
+ ]
+ }
+ ]
+ }'
+
+ relationships:
+ - name: sales_to_date
+ from: sales
+ to: date
+ from_columns:
+ - order_date
+ to_columns:
+ - calendar_date
+ custom_extensions:
+ # Ossie has no inactive-join concept. Recording an inactive relationship as an
+ # ordinary one would misrepresent the join graph, so the flag is preserved and
+ # the relationship reported.
+ - vendor_name: MICROSOFT
+ data: '{
+ "_v": 1,
+ "format": "TMSL",
+ "isActive": true,
Review Comment:
Correct, the two contradicted each other — fixed in e6efc8e by taking the second option, adjusting the comment rather than the payload.
The payload was the right half. Flipping it to `false` would have made the example demonstrate the one thing the doc tells converters not to do: an inactive relationship emitted as an *ordinary* Ossie relationship asserts a join the engine does not apply by default, so any consumer reading core fields would treat a deliberately-inactive join as live. Since this is the example's only relationship, that would have been the sole thing it showed about relationships.
What the payload actually demonstrates — and what the comment failed to say — is that `isActive` is written on **every** relationship, active ones included. Ossie has no inactive-join concept, so absence of the flag is ambiguous: a reader cannot distinguish "active" from "converted by a tool that dropped it". Recording it unconditionally is what makes the flag meaningful.
Both the comment and the `isActive` row in `docs/vendor_extensions/microsoft.md` now state this, including the prohibition on emitting the inactive case as an ordinary relationship, which the doc previously only implied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org