You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "XiaoHongbo-Hope (via GitHub)" <gi...@apache.org> on 2026/07/05 03:18:22 UTC
[PR] [python][ray] Add distributed read_by_row_id for data-evolution tables [paimon]
XiaoHongbo-Hope opened a new pull request, #8465:
URL: https://github.com/apache/paimon/pull/8465
### Purpose
### Tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [python][ray] Add distributed read_by_row_id for data-evolution tables [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #8465:
URL: https://github.com/apache/paimon/pull/8465#issuecomment-4885336586
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [python][ray] Add distributed read_by_row_id for data-evolution tables [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #8465:
URL: https://github.com/apache/paimon/pull/8465
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [python][ray] Add distributed read_by_row_id for data-evolution tables [paimon]
Posted by "QuakeWang (via GitHub)" <gi...@apache.org>.
QuakeWang commented on code in PR #8465:
URL: https://github.com/apache/paimon/pull/8465#discussion_r3524465731
##########
paimon-python/pypaimon/ray/data_evolution_merge_join.py:
##########
@@ -584,6 +584,139 @@ def _apply_group(group: pa.Table) -> pa.Table:
return all_msgs, num_updated, action_row_ids
+def _read_output_schema(table, read_cols: Sequence[str]) -> "pa.Schema":
+ """Result schema: each projected column's type plus int64 ``_ROW_ID``, in
+ ``read_cols`` order. Shared by the empty-result paths so they can't drift."""
+ from pypaimon.schema.data_types import PyarrowFieldParser
+ from pypaimon.table.special_fields import SpecialFields
+
+ rid = SpecialFields.ROW_ID.name
+ full = PyarrowFieldParser.from_paimon_schema(table.table_schema.fields)
+ return pa.schema([
+ (col, pa.int64() if col == rid else full.field(col).type)
Review Comment:
This schema construction drops field nullability because `(name, type)` defaults to nullable fields. That makes empty `read_by_row_id` results expose a different schema from non-empty reads, especially for `_ROW_ID`, which should be non-nullable per `SpecialFields.ROW_ID`.
Could we preserve `full.field(col)` for table columns and use `pa.field(rid, pa.int64(), nullable=False)` for `_ROW_ID`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org