You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@impala.apache.org by "Lars Volker (Code Review)" <ge...@cloudera.org> on 2016/05/06 22:22:21 UTC

[Impala-CR](cdh5-trunk) PREVIEW

Lars Volker has uploaded a new patch set (#20).

Change subject: PREVIEW
......................................................................

PREVIEW

IMPALA-2979: Fix scheduling on remote hosts

Also fixes: IMPALA-2400, IMPALA-3019, IMPALA-3043

This change fixes scheduling scan-ranges on remote hosts by adding
remote backend selection capability to SimpleScheduler. Prior to this
change the scheduler would try to select a local backend even when
remote scheduling was requested.

This change also changes remote backend selection from round-robin to
random to prevent convoying, which could happen when different
independent schedulers had the same internal state, e.g. after a
cluster restart. To use the old behavior set the query option
SCHEDULE_RANDOM_BACKEND to false.

This change also fixes IMPALA-2400: Unpredictable locality behavior
for reading Parquet files

This change also fixes IMPALA-3019: next_nonlocal_backend_entry_ is
reset on empty Statestore messages

This change also fixes IMPALA-3043: SimpleScheduler does not handle
hosts with multiple IP addresses correctly

This change also does some minor clean-up in scheduler.h and
simple-scheduler.{h,cc}.

Change-Id: I044f83806fcde820fcb38047cf6b8e780d803858
---
M be/CMakeLists.txt
M be/src/scheduling/scheduler.h
M be/src/scheduling/simple-scheduler-test.cc
M be/src/scheduling/simple-scheduler.cc
M be/src/scheduling/simple-scheduler.h
M be/src/service/query-options.cc
M be/src/service/query-options.h
M common/thrift/ImpalaInternalService.thrift
M common/thrift/ImpalaService.thrift
M fe/src/main/java/com/cloudera/impala/analysis/TableRef.java
M fe/src/test/java/com/cloudera/impala/analysis/AnalyzeStmtsTest.java
M fe/src/test/java/com/cloudera/impala/analysis/ParserTest.java
M fe/src/test/java/com/cloudera/impala/analysis/ToSqlTest.java
13 files changed, 548 insertions(+), 287 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/00/2200/20
-- 
To view, visit http://gerrit.cloudera.org:8080/2200
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I044f83806fcde820fcb38047cf6b8e780d803858
Gerrit-PatchSet: 20
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv...@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <ma...@cloudera.com>
Gerrit-Reviewer: Mostafa Mokhtar <mm...@cloudera.com>
Gerrit-Reviewer: anujphadke <ap...@cloudera.com>