You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@tsfile.apache.org by "Young-Leo (via GitHub)" <gi...@apache.org> on 2026/10/13 12:26:06 UTC
[PR] [Python] Fix tree-model reads and indexing in TsFileDataFrame [tsfile]
Young-Leo opened a new pull request, #1011:
URL: https://github.com/apache/tsfile/pull/1011
TsFileDataFrame could reject valid tree-model datasets when different devices used the same measurement name with different physical types. For example, `root.a.b.temperature` (DOUBLE) and `root.c.d.temperature` (INT32) failed index construction with `incompatible physical type for root.temperature`. Without the index, scanning both devices through a shared tree-table column could also decode values using the wrong type.
This change makes numeric tree sequences usable through the existing sequence, statistics, and training-window APIs.
### Changes
- Validate tree-model physical types per full logical series path. Allow independent devices to use different types for the same measurement name, while retaining same-series shard validation and table-model schema checks.
- Represent heterogeneous synthetic fields with a float64 logical type and a mixed physical-type marker. Preserve the Dataset Index v1 record layout; actual decoding uses each locator's TimeseriesMetadata.
- Read unindexed tree sequences through exact metadata locators instead of scanning all devices through a shared column type. Preserve timestamps, aligned null positions, and float64 output.
- Filter column lookup by FIELD role so measurements such as `_col_1` do not resolve to synthetic path tags.
- Initialize aligned time decoders from the chunk-header encoding, with INT64 timestamp values, in both serial and parallel paths. This fixes reads of Java-generated aligned tree files using PLAIN time encoding.
- Add 29 tree-model regression cases, Java-generated V4 fixtures for PLAIN and TS_2DIFF time encoding, fixture generation source, and English/Chinese usage documentation.
### Validation
- 325 Python tests passed across `test_tree_dataset_training.py`, `test_tsfile_dataset.py`, `test_dataset_index.py`, and `test_dataset_read_options.py`.
- 149 C++ reader, aligned-data, prepared-series, and Dataset Index tests passed.
- The configured Python/C++ Dataset Index interoperability test passed separately with an actual heterogeneous tree index, bringing the C++ total to 150 passing tests.
- Black checks, C++ Spotless formatting, and `git diff --check` passed.
Coverage includes single-file and multi-file datasets, independent measurement types, full-path reads, statistics, positional and time-range windows, index reload without metadata rescanning, name collisions, and aligned nulls.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@tsfile.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org