You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ossie.apache.org by "OGsiji (via GitHub)" <gi...@apache.org> on 2026/09/17 17:47:34 UTC

[PR] fix(validation): report empty names/refs and survive nested input [ossie]

OGsiji opened a new pull request, #414:
URL: https://github.com/apache/ossie/pull/414

   ## Summary
   
   Fixes #408. `validation/validate.py` silently skipped validation or crashed with a raw traceback on several inputs that the JSON Schema itself accepts — `name`, `from`, and `to` are strings with **no `minLength`**, so `""` is schema-valid and reaches the semantic checks.
   
   ## Bugs fixed
   
   1. **Empty-string names bypassed uniqueness checks.** `validate_unique_names` collected names with a truthiness filter (`if d.get("name")`), so duplicate `""` dataset/field/metric/relationship names were dropped before duplicate detection. Now filtered with `is not None`.
   2. **Empty-string relationship endpoints were never flagged.** `validate_references` used `if from_ds` / `if to_ds`, so a relationship with `from: ""` or `to: ""` silently passed instead of being reported as an unknown dataset. Now `is not None`.
   3. **Deeply nested SQL crashed the validator.** `validate_sql_expression` only caught `ParseError`/`TokenError`; pathologically nested expressions raised an uncaught `RecursionError`. Now caught and reported as a `[SQL]` diagnostic.
   4. **Deeply nested YAML crashed with a traceback.** `main()` only caught `yaml.YAMLError`; deeply nested flow collections raise `RecursionError` during composition. Now caught, exits cleanly.
   
   ## Tests
   
   Added regression tests in both suites:
   - `validation/tests/test_validate.py` (pytest): empty-name duplicates, missing-name skip, empty `from`/`to` reported, missing-endpoint skip, deeply nested SQL diagnostic.
   - `validation/test_validate.py` (unittest): deeply nested YAML exits cleanly without a traceback, empty endpoint reported end-to-end.
   
   All Validation CI steps pass locally on Python 3.11 and 3.12:
   - `uv run validation/test_validate.py` — 41 passed
   - pytest `validation/tests/` — 55 passed
   - `uv run validation/validate.py examples/tpcds_semantic_model.yaml` — PASSED
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(validation): report empty names/refs and survive nested input [ossie]

Posted by "jbonofre (via GitHub)" <gi...@apache.org>.
jbonofre merged PR #414:
URL: https://github.com/apache/ossie/pull/414


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(validation): report empty names/refs and survive nested input [ossie]

Posted by "OGsiji (via GitHub)" <gi...@apache.org>.
OGsiji commented on PR #414:
URL: https://github.com/apache/ossie/pull/414#issuecomment-5796355531

   Thanks both. Reworked the fix and force-pushed a single commit (174491f) rebased onto current `main`.
   
   **@jbonofre point 1 (schema is the real fix):** added `minLength: 1` to every required identifier string in `core-spec/ossie-schema.json`: `name` on model/dataset/field/metric/relationship, plus dataset `source` and relationship `from`/`to`. Empty values now fail with a clear `[Schema] ... should be non-empty` before semantic checks run, so a **single** empty identifier is caught at the contract level, not just duplicates. (@kayemkim: confirmed your single-field-and-dataset `""` repro now fails instead of printing `Validation PASSED`.)
   
   **@jbonofre point 2 (consistency):** reverted the earlier `is not None` changes in `validate_unique_names` and `validate_references`. With the schema fix they were redundant, and reverting removes the inconsistency you flagged, where the dataset lookup used a truthiness filter while the endpoint checks used `is not None`. Both are plain again.
   
   **@kayemkim's exact suggestion on the SQL guard:** the first parse attempt is now `except (ParseError, TokenError, RecursionError): pass`, and the SELECT-wrapped retry reports the "too deeply nested" line. `validate_sql_expression(123, "ANSI_SQL", "ctx")` raises `TypeError` again (verified), while 5000 nested parentheses still produce the diagnostic. No bare `except Exception` anywhere.
   
   **Rebase:** carried over cleanly; `OSSIE_SQL_2026` (#439) is preserved in the schema and accepted by the validator.
   
   Tests: schema-level empty-identifier cases (unittest and pytest) replace the semantic ones, plus a non-string-expression case. validation-ci is green on 3.11 and 3.12 (unittest 41, pytest 55, canonical example passes).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(validation): report empty names/refs and survive nested input [ossie]

Posted by "jbonofre (via GitHub)" <gi...@apache.org>.
jbonofre commented on PR #414:
URL: https://github.com/apache/ossie/pull/414#issuecomment-5797536560

   @OGsiji awesome! Many thanks! I'm doing a new pass.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(validation): report empty names/refs and survive nested input [ossie]

Posted by "kayemkim (via GitHub)" <gi...@apache.org>.
kayemkim commented on PR #414:
URL: https://github.com/apache/ossie/pull/414#issuecomment-5753655072

   Following up because one of the two nits in my review pointed the wrong way. I only said the `except (ParseError, TokenError)` clause was covered by the broader one. What I had in mind was keeping the narrow clause and adding `RecursionError` to it, not dropping it, and I should have spelled that out.
   
   Measured on this branch merged into current main (df81044): `validate_sql_expression(123, "ANSI_SQL", "ctx")` returns `None` because the retry parses `SELECT 123` without complaint, so a non-string expression is reported as valid SQL, which is the case raised above. Main raises `TypeError` there. Changing the first attempt to `except (ParseError, TokenError, RecursionError): pass` brings the `TypeError` back, still turns 5000 nested parentheses into the "too deeply nested" line, and both validation suites pass with it (55 pytest cases, unittest OK).
   
   On the empty-name point I can confirm the gap. A document with a single dataset and a single field both named `""` prints `Validation PASSED` on main and on this branch. My earlier reproduction only used the duplicate and dangling-reference shapes from #408, which is why it didn't show up. `core-spec/ossie-schema.json` has no `minLength` anywhere today, so `minLength: 1` on `name`, `from` and `to` would be the first one, and with it the `is not None` changes become consistency fixes rather than the thing that closes the bug.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(validation): report empty names/refs and survive nested input [ossie]

Posted by "jbonofre (via GitHub)" <gi...@apache.org>.
jbonofre commented on code in PR #414:
URL: https://github.com/apache/ossie/pull/414#discussion_r4053960146


##########
validation/validate.py:
##########
@@ -211,16 +214,20 @@ def validate_references(data: dict) -> list[str]:
     errors = []
 
     model_name = model.get("name", "<unnamed>")
+    # Exclude falsy dataset names so a relationship pointing at "" is reported
+    # as unknown rather than matched.
     datasets = {d.get("name"): d for d in model.get("datasets", []) if d.get("name")}

Review Comment:
   The fix (about `minLength: 1`) is helping there too.
   
   This data lookup (`{d.get("name"): d for d in ... if d.get("name")}`) still uses a truth filter, while the `from_ds`/`to_ds` checks two lines below were fixed to `is not None`. That inconsistency means a dataset actually named `""` gets excluded from this dict, so any relationship referencing it is wrongly reporter as "unknown dataset ''" instead of being caught as an invalid name. 



##########
validation/validate.py:
##########
@@ -292,7 +299,9 @@ def validate_sql_expression(expr: str, dialect: str, context: str) -> str | None
         # Try parsing as expression first (for field expressions like "column_name")
         sqlglot.parse_one(expr, dialect=sqlglot_dialect)
         return None
-    except (ParseError, TokenError):
+    except Exception:

Review Comment:
   Broadening this from `except (ParseError, TokenError)` to bare `exception Exception: pass` silently swallows any exception type on the first parse attempt, not just the `RecursionError` this seems aimed at.
   It means `validate_sql_expression(123, 'ANSI_SQL', 'ctx')` (non string `expr`) used to raise a loud `TypeError`. Now it's swallowed and the SELECT wrapped retry (`parse_one('SELECT 123', ...)`) spuriously succeeds, so the function reports the input as valid SQL.
   
   I recommend catching `RecursionError` specifically here (in addition to `ParseError`/`TokenError`) rather than a bare `Exception`, so genuine bugs still surface instead of being masked as "valid".



##########
validation/validate.py:
##########
@@ -176,25 +176,28 @@ def validate_unique_names(data: dict) -> list[str]:
 
     model_name = model.get("name", "<unnamed>")
 
+    # Guard on "is not None", not truthiness: "" is a schema-valid name (no
+    # minLength) and must not slip past duplicate detection.
+
     # Check unique dataset names
-    dataset_names = [d.get("name") for d in model.get("datasets", []) if d.get("name")]
+    dataset_names = [d.get("name") for d in model.get("datasets", []) if d.get("name") is not None]

Review Comment:
   The `is not None` fix here only helps duplicate-name detection see an empty string. It doesn't fix the actual bug: a single (non duplicate) dataset/field/metric/relationship named `""` still passes validation entirely, `validate_unique_names` only fires on 2+ occurrences, and nothing else checks for empty names.
   
   I suggest fixing this at the schema level instead: add `minLength: 1` to the `name`/`from`/`to` string properties in `core-spec/ossie-schema.json` (Dataset, Field, Metric, Relationship, SemanticModel). That rejects `""` with a clear `[Schema]` error before semantic checks even run, and fixes this consistently across all four call sites instead of patching them individually.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(validation): report empty names/refs and survive nested input [ossie]

Posted by "OGsiji (via GitHub)" <gi...@apache.org>.
OGsiji commented on PR #414:
URL: https://github.com/apache/ossie/pull/414#issuecomment-5736887640

   Thanks for the thorough review! Addressed both nits in 2c8c4a5:
   
   - **Redundant `except`** — collapsed the first parse attempt's `except (ParseError, TokenError): pass` into the broader `except Exception: pass` that already subsumed it. `ParseError`/`TokenError` stay imported for the second attempt's actual `[SQL]` diagnostic.
   - **Inert `noqa: BLE001`** — dropped both, since `validation/` has no ruff config.
   - **Comments** — trimmed to the file's one-line style, keeping only the non-obvious "why" (schema has no `minLength`; deep input surfaces as `RecursionError`, not the parser/YAML error types).
   
   No behaviour change; validation-ci steps (unittest 41, pytest 55, canonical example) stay green on 3.11–3.14.
   
   On the #271 overlap: happy to take the second-mover rebase whenever it lands — the conflict is limited to the `main()` `except` ladder and the added pytest cases.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org