[I] Add a converter for Alibaba Cloud Hologres Semantic View [ossie]

["TimothyDing (via GitHub)" <gi...@apache.org>]

TimothyDing opened a new issue, #319:
URL: https://github.com/apache/ossie/issues/319

   ## Summary
   
   I would like to contribute a bidirectional converter between Apache Ossie and **Alibaba Cloud Hologres Semantic View**, a feature available since Hologres V5.0.0.
   
   Per CONTRIBUTING.md I am opening this issue first to discuss the approach before submitting a PR.
   
   ## Background
   
   A Hologres Semantic View is an in-database object declaring physical tables, their relationships, business dimensions and metrics. Queries reference dimensions and `AGG(metric)` instead of repeating joins and aggregations, and the engine aggregates each metric within its own minimal join subtree so a one-to-many join cannot inflate a total. Hologres also publishes a structured YAML model for every view, intended for BI tools and NL2SQL agents.
   
   ## Why it maps cleanly onto Ossie
   
   | Ossie | Hologres Semantic View |
   |---|---|
   | `dataset` + `source` + `primary_key` | `TABLES (alias AS schema.table PRIMARY KEY (...))` |
   | `relationship` (`from` many -> `to` one) | `RELATIONSHIPS (name AS from(cols) REFERENCES to(cols))` |
   | `dataset.fields[]` | `DIMENSIONS (alias.name AS expr)` |
   | model-level `metrics[]` | `METRICS (owner.name AS agg_expr)` |
   
   The relationship model is a particularly close fit: Hologres requires `from` to be the many side and the `REFERENCES` target to be the referenced table's primary key, which matches Ossie's existing FK-pair semantics exactly.
   
   ## Proposed shape
   
   The two directions are asymmetric, because of how Hologres exposes definitions:
   
   - **Export (Ossie -> Hologres)** emits `CREATE SEMANTIC VIEW` **SQL DDL text**. Hologres has no YAML import function, so DDL is the only way to create a view. This would be the first converter in the repo whose output is not YAML — I would like feedback on whether that is acceptable, or whether it should be shaped differently.
   - **Import (Hologres -> Ossie)** consumes the `model_yaml` that Hologres publishes in the `hologres.hg_semantic_view_properties` system table.
   
   ## Spec implication
   
   Hologres expressions are PostgreSQL-compatible. Labelling something like `col::text` as `ANSI_SQL` would misrepresent its portability, so I propose adding a `HOLOGRES` token to the `Dialect` enum in `core-spec/osi-schema.json` (mapping to sqlglot's `postgres` in `validation/validate.py`).
   
   I recognise this is a **specification change** under CONTRIBUTING.md and carries the higher bar (7-day discussion, 3 binding +1). I would therefore propose splitting it into a separate, small PR, and can keep the converter itself on `ANSI_SQL` only if the community prefers no spec change — as the NVIDIA GSF converter does.
   
   ## Known limitations to document
   
   Hologres constrains what a Semantic View can express, so some valid Ossie models cannot be converted. Notably, using the repository's own `examples/tpcds_semantic_model.yaml`, 2 of its 5 metrics (`customer_lifetime_value`, `store_productivity`) are cross-dataset ratios and have no Semantic View form:
   
   - definition expressions must be row-level over a **single** table
   - aggregates limited to `count` / `sum` / `avg` / `min` / `max`
   - no derived, ratio, or filtered metrics
   - a `REFERENCES` target must be the target table's primary key
   - no `CREATE OR REPLACE` / `ALTER SEMANTIC VIEW`
   
   My preference is to fail closed and name the offending field, with an opt-in flag to skip unconvertible metrics.
   
   ## Status
   
   I have a working implementation, verified against a real Hologres 5.0.0 instance: the generated DDL executes, the resulting views return correct results (including the fan-out case where a joined detail table must not inflate a total), and importing Hologres' own readback of that DDL reproduces the original Ossie model. Happy to open the PR once there is agreement on the approach — particularly on the DDL-output question and the `HOLOGRES` dialect.
   
   Disclosure: this implementation was produced with AI assistance. In line with the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html), I have reviewed the code and take full responsibility for it; the behaviour described above was verified by me against a live instance.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Add a converter for Alibaba Cloud Hologres Semantic View [ossie]

["TimothyDing (via GitHub)" <gi...@apache.org>]

TimothyDing commented on issue #319:
URL: https://github.com/apache/ossie/issues/319#issuecomment-5261242380

   Opened the PR for this: #320. It contains no specification changes, per the discussion above — the converter reads and writes `ANSI_SQL` only. The one design question that would benefit from a maintainer's view is whether an export direction may emit SQL DDL rather than a data document (background in the comment above).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Add a converter for Alibaba Cloud Hologres Semantic View [ossie]

["TimothyDing (via GitHub)" <gi...@apache.org>]

TimothyDing commented on issue #319:
URL: https://github.com/apache/ossie/issues/319#issuecomment-5260706264

   On the one open question above — whether it is acceptable for a converter's export direction to emit SQL DDL rather than YAML — I went looking for whether the CLI plugin contract already constrains this, and it appears not to.
   
   In `cli/internal/plugin/plugin.go`, the declared input format is asymmetric by design:
   
   ```go
   Accepts []string // file extensions e.g. [".yaml"]; populated on ToOssie only
   ```
   
   - `convert.to_ossie` (vendor → Ossie) carries both `invoke` and `accepts`, and `validate()` requires both (`plugin.go:60-63`).
   - `convert.from_ossie` (Ossie → vendor) carries **only** `invoke` (`plugin.go:44-46`), and `validate()` requires only that (`plugin.go:64-65`). There is no field in which a plugin declares what its export direction produces.
   
   So a plugin whose `from_ossie` writes `.sql` does not appear to violate the manifest as it stands, while the import direction here consumes Hologres' published `model_yaml` and fits `accepts: [".yaml"]` naturally.
   
   Two caveats I do not want to overstate this with:
   
   1. `runConvert` is currently a stub (`cli/cmd/convert.go:46` prints `not yet implemented`), so this contract is not yet exercised end to end and could still change. If the intent is for the CLI to chain converters — vendor A → Ossie → vendor B — then an export that emits DDL is a terminal step and cannot be an intermediate one. That is a real asymmetry, just not one the manifest currently forbids.
   2. The constraint is Hologres', not a preference: `hg_create_semantic_view_from_yaml()` does not exist, and the documentation directs cross-environment rebuilds to the original `ddl_text`. Emitting the `model_yaml` shape instead would be symmetrical with the other spokes but produce a file Hologres cannot consume, leaving users to hand-write the DDL anyway.
   
   Since Snowflake Semantic Views are also created through DDL, whatever is decided here will likely set the precedent for that shape of platform, so I am happy to follow the community's preference rather than assume mine.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org