[PR] [python] Fix union global index reader semantics [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi opened a new pull request, #8742:
URL: https://github.com/apache/paimon/pull/8742

   ## What changed
   
   - propagate unsupported scalar results across every child of `UnionGlobalIndexReader`
   - make the default batch-vector fan-out non-blocking and preserve asynchronous failures
   - close every union child before propagating the first close failure
   - add focused regression tests for scalar, batch-vector, and close semantics
   
   ## Why
   
   The Python union reader skipped a child result of `None` for scalar predicates. Because `None` means unsupported rather than no matches, a union spanning multiple row ranges could return only the supported ranges and silently omit matching rows. Java already propagates the unsupported result for this case.
   
   The Python batch-vector default also called `Future.result()` before returning, which blocked asynchronous readers and could surface child failures synchronously.
   
   ## Impact
   
   Scalar global-index scans now fall back safely instead of returning incomplete results. Batch-vector callers receive an immediately returned Future whose results retain input order and whose failures remain asynchronous.
   
   ## Tests
   
   - `python3 -m pytest pypaimon/tests/union_global_index_reader_test.py pypaimon/tests/global_index_evaluator_test.py pypaimon/tests/vector_search_filter_test.py pypaimon/tests/lumina_vector_index_test.py pypaimon/tests/vindex_vector_index_test.py -q`
     - `80 passed, 1 skipped, 15 subtests passed`
   - `pycodestyle` and `pyflakes` on changed files
   - Python license-header check: 642 files valid
   - `git diff --check`
   
   The bundled `flake8` launcher could not start because its plugin manager is incompatible with the installed `importlib-metadata`; the underlying style and unused-symbol checks were run separately.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [python] Fix union global index reader semantics [paimon]

["JingsongLi (via GitHub)" <gi...@apache.org>]

JingsongLi merged PR #8742:
URL: https://github.com/apache/paimon/pull/8742


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [python] Fix union global index reader semantics [paimon]

["leaves12138 (via GitHub)" <gi...@apache.org>]

leaves12138 commented on code in PR #8742:
URL: https://github.com/apache/paimon/pull/8742#discussion_r3612111495


##########
paimon-python/pypaimon/globalindex/union_global_index_reader.py:
##########
@@ -54,6 +59,9 @@ def on_done(_):
                         for f in futures:
                             current = f.result()
                             if current is None:
+                                if propagate_unsupported:
+                                    all_done.set_result(None)

Review Comment:
   `propagate_unsupported` returns as soon as the first `None` is encountered, so a failed future later in `futures` is never observed. For example, `[completed(None), failed(RuntimeError(...))]` completes this union successfully with `None`; reversing the readers raises. Java’s `CompletableFuture.allOf` propagates any child failure, and the previous Python loop also reached the failed future. Please materialize all child results first (for example, `results = [f.result() for f in futures]`) and only then apply the unsupported check/union, and add a regression test for both reader orders.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org