You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org> on 2026/10/26 17:33:48 UTC

[Impala-ASF-CR] PROTOTYPE IMPALA-15656: Release CTE exchangers when consumers never opened

Hello Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/25095

to look at the new patch set (#2).

Change subject: PROTOTYPE IMPALA-15656: Release CTE exchangers when consumers never opened
......................................................................

PROTOTYPE IMPALA-15656: Release CTE exchangers when consumers never opened

A query that fails while CTE consumers are still opening hangs: the
producer's LocalExchanger::Release() waits for every consumer to call
CloseConsumer(), but a consumer only finds its exchanger at the end of
CTEConsumerNode::Open(). Consumers that fail or are cancelled before
that point never close it, so the producer's fragment instance blocks
forever and the query keeps its memory until the impalad restarts.

Once the query is cancelled, Release() now waits only for the consumers
that opened the exchanger, then frees all cells. A consumer that opens
afterwards gets no exchanger and returns eos, which is safe because the
query is already cancelled.

Testing:
- TPC-DS q14_1 at SF3000, 8 executors, PLANNER=CALCITE,
  CTE_THRESHOLD=1, MT_DOP=8 (runs out of memory): its memory was
  released 5-11 s after the failure instead of being held until a
  restart. No new tests.

Change-Id: Ic2e2f4090bb2270a33be396d759e435cd8defefa
Assisted-by: Claude Opus 5.5 (Claude Code)
---
M be/src/exec/cte-consumer-node.cc
M be/src/exec/cte-producer-node.cc
M be/src/runtime/local-exchanger.cc
M be/src/runtime/local-exchanger.h
4 files changed, 38 insertions(+), 15 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/25095/2
-- 
To view, visit http://gerrit.cloudera.org:8080/25095
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic2e2f4090bb2270a33be396d759e435cd8defefa
Gerrit-Change-Number: 25095
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>