You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Henry Robinson (Code Review)" <ge...@cloudera.org> on 2016/10/21 22:41:58 UTC
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to sink
Henry Robinson has uploaded a new change for review.
http://gerrit.cloudera.org:8080/4787
Change subject: IMPALA-4335: Don't send 0-row batches to sink
......................................................................
IMPALA-4335: Don't send 0-row batches to sink
This patchr restores some behaviour from pre-IMPALA-2905 where the
PlanFragmentExecutor would not send 0-row batches to the sink. Although
0-row batches are legal, they're not very useful for clients to
receive (and clients may not correctly process them).
No query was found which reliably produced 0-row batches, so no test is
added.
Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
---
M be/src/runtime/coordinator.h
M be/src/runtime/plan-fragment-executor.cc
2 files changed, 3 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/4787/1
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to sink
Posted by "Henry Robinson (Code Review)" <ge...@cloudera.org>.
Henry Robinson has restored this change.
Change subject: IMPALA-4335: Don't send 0-row batches to sink
......................................................................
Restored
Let's try this again.
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: restore
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to sink
Posted by "Henry Robinson (Code Review)" <ge...@cloudera.org>.
Hello Matthew Jacobs, Alex Behm,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4787
to look at the new patch set (#3).
Change subject: IMPALA-4335: Don't send 0-row batches to sink
......................................................................
IMPALA-4335: Don't send 0-row batches to sink
This patch restores some behaviour from pre-IMPALA-2905 where we would
not send 0-row batches to the client. Although 0-row batches are legal,
they're not very useful for clients to receive (and clients may not
correctly process them).
No query was found which reliably produced 0-row batches, so no test is
added.
Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
---
M be/src/exec/plan-root-sink.cc
M be/src/runtime/coordinator.h
2 files changed, 6 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/4787/3
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to sink
Posted by "Matthew Jacobs (Code Review)" <ge...@cloudera.org>.
Matthew Jacobs has posted comments on this change.
Change subject: IMPALA-4335: Don't send 0-row batches to sink
......................................................................
Patch Set 2: Code-Review+2
Thanks. I verified it fixed case w/ the KuduScanNode.
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to sink
Posted by "Henry Robinson (Code Review)" <ge...@cloudera.org>.
Hello Matthew Jacobs,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4787
to look at the new patch set (#2).
Change subject: IMPALA-4335: Don't send 0-row batches to sink
......................................................................
IMPALA-4335: Don't send 0-row batches to sink
This patch restores some behaviour from pre-IMPALA-2905 where we would
not send 0-row batches to the client. Although 0-row batches are legal,
they're not very useful for clients to receive (and clients may not
correctly process them).
No query was found which reliably produced 0-row batches, so no test is
added.
Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
---
M be/src/exec/plan-root-sink.cc
M be/src/runtime/coordinator.h
2 files changed, 5 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/4787/2
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to clients
Posted by "Internal Jenkins (Code Review)" <ge...@cloudera.org>.
Internal Jenkins has posted comments on this change.
Change subject: IMPALA-4335: Don't send 0-row batches to clients
......................................................................
Patch Set 4: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to sink
Posted by "Dan Hecht (Code Review)" <ge...@cloudera.org>.
Dan Hecht has posted comments on this change.
Change subject: IMPALA-4335: Don't send 0-row batches to sink
......................................................................
Patch Set 3: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4787/3//COMMIT_MSG
Commit Message:
Line 7: IMPALA-4335: Don't send 0-row batches to sink
update for the new fix
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to sink
Posted by "Henry Robinson (Code Review)" <ge...@cloudera.org>.
Henry Robinson has abandoned this change.
Change subject: IMPALA-4335: Don't send 0-row batches to sink
......................................................................
Abandoned
Turns out that some sinks need 0-row batches, to manage their lifetime so we can't just drop them on the floor at the PFE. I'll redo this.
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: abandon
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to clients
Posted by "Alex Behm (Code Review)" <ge...@cloudera.org>.
Alex Behm has posted comments on this change.
Change subject: IMPALA-4335: Don't send 0-row batches to clients
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4787/2/be/src/exec/plan-root-sink.cc
File be/src/exec/plan-root-sink.cc:
Line 88: // Don't enter the loop if batch->num_rows() == 0; no point triggering the consumer with
> It's benign, but it's true: we don't want to trigger the consumer (why pay
Yes, but we're not fixing this because we're saving some perf. We're making this change to not break clients.
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to clients
Posted by "Internal Jenkins (Code Review)" <ge...@cloudera.org>.
Internal Jenkins has submitted this change and it was merged.
Change subject: IMPALA-4335: Don't send 0-row batches to clients
......................................................................
IMPALA-4335: Don't send 0-row batches to clients
This patch restores some behaviour from pre-IMPALA-2905 where we would
not send 0-row batches to the client. Although 0-row batches are legal,
they're not very useful for clients to receive (and clients may not
correctly process them).
No query was found which reliably produced 0-row batches, so no test is
added.
Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Reviewed-on: http://gerrit.cloudera.org:8080/4787
Reviewed-by: Henry Robinson <he...@cloudera.com>
Tested-by: Internal Jenkins
---
M be/src/exec/plan-root-sink.cc
M be/src/runtime/coordinator.h
2 files changed, 6 insertions(+), 3 deletions(-)
Approvals:
Henry Robinson: Looks good to me, approved
Internal Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 5
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to sink
Posted by "Henry Robinson (Code Review)" <ge...@cloudera.org>.
Henry Robinson has posted comments on this change.
Change subject: IMPALA-4335: Don't send 0-row batches to sink
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4787/2/be/src/exec/plan-root-sink.cc
File be/src/exec/plan-root-sink.cc:
Line 88: // Don't enter the loop if batch->num_rows() == 0; no point triggering the consumer with
> This sounds rather benign. Mention that some clients may not handle empty b
It's benign, but it's true: we don't want to trigger the consumer (why pay a context switch to do no work and force another RPC?).
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to clients
Posted by "Henry Robinson (Code Review)" <ge...@cloudera.org>.
Henry Robinson has posted comments on this change.
Change subject: IMPALA-4335: Don't send 0-row batches to clients
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/4787/2/be/src/exec/plan-root-sink.cc
File be/src/exec/plan-root-sink.cc:
Line 88: // Don't enter the loop if batch->num_rows() == 0; no point triggering the consumer with
> Yes, but we're not fixing this because we're saving some perf. We're making
I'm doing it for both reasons? It's a sensible optimization to make, now that I've noticed it. Anyhow, comment now refers to both.
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to clients
Posted by "Henry Robinson (Code Review)" <ge...@cloudera.org>.
Henry Robinson has posted comments on this change.
Change subject: IMPALA-4335: Don't send 0-row batches to clients
......................................................................
Patch Set 4: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to clients
Posted by "Henry Robinson (Code Review)" <ge...@cloudera.org>.
Hello Matthew Jacobs, Alex Behm, Dan Hecht,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4787
to look at the new patch set (#4).
Change subject: IMPALA-4335: Don't send 0-row batches to clients
......................................................................
IMPALA-4335: Don't send 0-row batches to clients
This patch restores some behaviour from pre-IMPALA-2905 where we would
not send 0-row batches to the client. Although 0-row batches are legal,
they're not very useful for clients to receive (and clients may not
correctly process them).
No query was found which reliably produced 0-row batches, so no test is
added.
Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
---
M be/src/exec/plan-root-sink.cc
M be/src/runtime/coordinator.h
2 files changed, 6 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/4787/4
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dh...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to sink
Posted by "Matthew Jacobs (Code Review)" <ge...@cloudera.org>.
Matthew Jacobs has posted comments on this change.
Change subject: IMPALA-4335: Don't send 0-row batches to sink
......................................................................
Patch Set 1: Code-Review+1
(1 comment)
Thanks. Not sure why it doesn't repro for you, Michael Brown repro'd this on his machine and I was able to repro it. I'll test this on my machine.
http://gerrit.cloudera.org:8080/#/c/4787/1//COMMIT_MSG
Commit Message:
PS1, Line 9: patchr
patch
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-4335: Don't send 0-row batches to sink
Posted by "Alex Behm (Code Review)" <ge...@cloudera.org>.
Alex Behm has posted comments on this change.
Change subject: IMPALA-4335: Don't send 0-row batches to sink
......................................................................
Patch Set 2: Code-Review+1
(1 comment)
I suspect that whatever test we add may become stale pretty easily. The empty batch behavior can change with various parameters that we're likely to tune.
http://gerrit.cloudera.org:8080/#/c/4787/2/be/src/exec/plan-root-sink.cc
File be/src/exec/plan-root-sink.cc:
Line 88: // Don't enter the loop if batch->num_rows() == 0; no point triggering the consumer with
This sounds rather benign. Mention that some clients may not handle empty batches correctly, so we cannot return such batches.
--
To view, visit http://gerrit.cloudera.org:8080/4787
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I7d339c1f9a55d9d75fb0e97d16b3176cc34f2171
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Alex Behm <al...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj...@cloudera.com>
Gerrit-HasComments: Yes