You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ossie.apache.org by "christianeu-db (via GitHub)" <gi...@apache.org> on 2026/09/17 06:14:33 UTC
[I] OrionBelt converter drops a field's logical datatype in both directions [ossie]
christianeu-db opened a new issue, #409:
URL: https://github.com/apache/ossie/issues/409
## Summary
Neither conversion direction carries a field's logical type through the
canonical spec `datatype` field:
- **Ossie->OBML** ignores the declared `datatype` and derives OBML
`abstractType` from a column-name heuristic, so a `String` field can be
emitted as numeric and vice-versa (**wrong** type).
- **OBML->Ossie** never writes the spec `datatype` field at all - the type is
stashed only inside an OrionBelt `custom_extensions` blob (**missing** type).
A round-trip appears lossless only because the converter is self-consistent
with its own non-spec key; any other spec-compliant tool sees wrong or absent
types.
## Root cause
**Ossie->OBML** (ossie_to_obml.py):
```python
ossie_type = field.get("data_type", "") # (1) wrong key
if ossie_type and ossie_type in OSSIE_TO_OBML_TYPE: # (2) wrong-cased map
abstract_type = OSSIE_TO_OBML_TYPE[ossie_type]
else:
abstract_type = self._infer_obml_type(field)
```
1. **Wrong key.** The spec field is `datatype`, no underscore, so the branch is dead
2. **Wrong-cased map.** The spec enum is capitalized, but
`OSSIE_TO_OBML_TYPE` is keyed lowercase with different terms.
Even after fixing the key, case-normalizing is not enough:
`DateTime`/`DateTimeTz`/`Decimal`/`Float` have no key - so case-normalizing is not enough
**OBML->Ossie: canonical field never written** (`obml_to_ossie.py`).
The type goes only into a vendor extension, under the non-spec key
`data_type` with a lowercase value; `field["datatype"]` is not set.
## Illustrative Repro
```python
# Ossie -> OBML: declared datatype ignored
for f, want in [({"name": "total_amount", "datatype": "String"}, "string"),
({"name": "customer_id", "datatype": "Integer"}, "int")]:
_, col = OssietoOBML(ossie={})._convert_field(dict(f))
got = col["abstractType"]
print(f"fwd {f['datatype']:8} -> {got:8} (want {want}) [{'OK' if got==want else 'BUG'}]")
# OBML -> Ossie: canonical datatype never emitted
field = OBMLtoOssie(obml={})._convert_column("customer_id", {"code": "customer_id",
"abstractType": "int"}, "Orders", {})
print("rev canonical 'datatype' present?", "datatype" in field, "[BUG]" if "datatype" not in field else "")
```
## Why this survived
The existing fixtures (test_ossie_converter_vendors, test_ossie_metric_no_silent_loss)
use the same wrong key `data_type`, so the suite stays green with the bug present.
Drop-in test (currently fails). Suggested home:
`tests/test_ossie_metric_no_silent_loss.py`, which already guards this "no
silent loss on field conversion" path.
```python
def test_explicit_datatype_roundtrips():
_, col = OssietoOBML(ossie={})._convert_field({"name": "total_amount", "datatype": "String"})
assert col["abstractType"] == "string" # fwd: currently 'float'
field = OBMLtoOssie(obml={})._convert_column("total_amount",
{"code": "total_amount", "abstractType": "string"}, "Orders", {})
assert field.get("datatype") == "String" # rev: currently missing
```
## Suggested fix
- **Ossie->OBML:** read `field.get("datatype")` (fix the comment); rebuild
`OSSIE_TO_OBML_TYPE` on the spec enum
- **OBML->Ossie:** set the canonical `field["datatype"]` (capitalized enum value)
in addition to any extension bookkeeping.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] OrionBelt converter drops a field's logical datatype in both directions [ossie]
Posted by "christianeu-db (via GitHub)" <gi...@apache.org>.
christianeu-db commented on issue #409:
URL: https://github.com/apache/ossie/issues/409#issuecomment-5709832060
cc @ralfbecher if you'd like to a take a first look. otherwise, I'm happy to take a pass
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] OrionBelt converter drops a field's logical datatype in both directions [ossie]
Posted by "ralfbecher (via GitHub)" <gi...@apache.org>.
ralfbecher commented on issue #409:
URL: https://github.com/apache/ossie/issues/409#issuecomment-5709920839
Thanks for the detailed report, confirmed on `main`. This is already fixed in the downstream `osi-orionbelt` package (reads/emits the canonical `datatype` on fields and metrics, maps the full `DataType` enum, keeps the extension only for the lossless OBML round trip); it just was never ported back here. I'll open a PR porting it with your drop-in test. Since #396 touches the same files, happy to either land first or rebase onto it, whichever the reviewers prefer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] OrionBelt converter drops a field's logical datatype in both directions [ossie]
Posted by "christianeu-db (via GitHub)" <gi...@apache.org>.
christianeu-db commented on issue #409:
URL: https://github.com/apache/ossie/issues/409#issuecomment-5710108061
> Thanks for the detailed report, confirmed on `main`. This is already fixed in the downstream `osi-orionbelt` package (reads/emits the canonical `datatype` on fields and metrics, maps the full `DataType` enum, keeps the extension only for the lossless OBML round trip); it just was never ported back here. I'll open a PR porting it with your drop-in test. Since [#396](https://github.com/apache/ossie/pull/396) touches the same files, happy to either land first or rebase onto it, whichever the reviewers prefer.
Wow - thanks for the very speedy turnaround! I'd bias towards getting in this change early since it is a data loss issue but defer to @jbonofre, esp since JB is reviewing the large refactor pr & is working on other converters
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] OrionBelt converter drops a field's logical datatype in both directions [ossie]
Posted by "jbonofre (via GitHub)" <gi...@apache.org>.
jbonofre closed issue #409: OrionBelt converter drops a field's logical datatype in both directions
URL: https://github.com/apache/ossie/issues/409
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org