You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org> on 2019/08/05 19:38:52 UTC

[Impala-ASF-CR] IMPALA-8832: Fix HTTP client protocol to work with Apache Knox

Thomas Tauber-Marshall has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14008


Change subject: IMPALA-8832: Fix HTTP client protocol to work with Apache Knox
......................................................................

IMPALA-8832: Fix HTTP client protocol to work with Apache Knox

This patch fixes two bugs with Impala's HTTP client protocol:
- THttpServer transports are no longer wrapped in TBufferedTransports.
  THttpServer already has its own support for buffering to process one
  HTTP request at a time, and wrapping it in a TBufferedTransport
  interferes with this, in some cases causing client requests to
  either not be processed or to recieve multiple responses.
- Fixes a bug in THttpTransport where when a chunked HTTP request is
  finished being processed, the 'readHeaders_' variable is never reset
  and further requests over the connection are not processed.

Testing:
- Tested by proxying beeline connections to Impala through Apache Knox

Change-Id: I5c9d934a654a9e6aaf9207fa5856f956baaacf55
---
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication.cc
M be/src/rpc/thrift-server.cc
M be/src/transport/THttpTransport.cpp
4 files changed, 47 insertions(+), 58 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/14008/1
-- 
To view, visit http://gerrit.cloudera.org:8080/14008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I5c9d934a654a9e6aaf9207fa5856f956baaacf55
Gerrit-Change-Number: 14008
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>

[Impala-ASF-CR] IMPALA-8832: Fix HTTP client protocol to work with Apache Knox

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14008 )

Change subject: IMPALA-8832: Fix HTTP client protocol to work with Apache Knox
......................................................................


Patch Set 3: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/14008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c9d934a654a9e6aaf9207fa5856f956baaacf55
Gerrit-Change-Number: 14008
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 06 Aug 2019 09:10:51 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8832: Fix HTTP client protocol to work with Apache Knox

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14008 )

Change subject: IMPALA-8832: Fix HTTP client protocol to work with Apache Knox
......................................................................


Patch Set 1: Code-Review+2

(4 comments)

http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/rpc/auth-provider.h
File be/src/rpc/auth-provider.h:

http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/rpc/auth-provider.h@63
PS1, Line 63: geenrated
typo: generated


http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:

http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/rpc/authentication.cc@1029
PS1, Line 1029: NULL
nit:nullptr


http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/rpc/authentication.cc@1098
PS1, Line 1098: NULL
nit: nullptr


http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/transport/THttpTransport.cpp
File be/src/transport/THttpTransport.cpp:

http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/transport/THttpTransport.cpp@81
PS1, Line 81:   readHeaders_ = true;
It looks like upstream thrift still has these issues - worth filing a bug against them? https://github.com/apache/thrift/blob/master/lib/cpp/src/thrift/transport/THttpTransport.cpp



-- 
To view, visit http://gerrit.cloudera.org:8080/14008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c9d934a654a9e6aaf9207fa5856f956baaacf55
Gerrit-Change-Number: 14008
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 05 Aug 2019 21:32:49 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8832: Fix HTTP client protocol to work with Apache Knox

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14008 )

Change subject: IMPALA-8832: Fix HTTP client protocol to work with Apache Knox
......................................................................

IMPALA-8832: Fix HTTP client protocol to work with Apache Knox

This patch fixes two bugs with Impala's HTTP client protocol:
- THttpServer transports are no longer wrapped in TBufferedTransports.
  THttpServer already has its own support for buffering to process one
  HTTP request at a time, and wrapping it in a TBufferedTransport
  interferes with this, in some cases causing client requests to
  either not be processed or to recieve multiple responses.
- Fixes a bug in THttpTransport where when a chunked HTTP request is
  finished being processed, the 'readHeaders_' variable is never reset
  and further requests over the connection are not processed.

Testing:
- Tested by proxying beeline connections to Impala through Apache Knox

Change-Id: I5c9d934a654a9e6aaf9207fa5856f956baaacf55
Reviewed-on: http://gerrit.cloudera.org:8080/14008
Reviewed-by: Thomas Tauber-Marshall <tm...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication.cc
M be/src/rpc/thrift-server.cc
M be/src/transport/THttpTransport.cpp
4 files changed, 47 insertions(+), 58 deletions(-)

Approvals:
  Thomas Tauber-Marshall: Looks good to me, approved
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/14008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I5c9d934a654a9e6aaf9207fa5856f956baaacf55
Gerrit-Change-Number: 14008
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-8832: Fix HTTP client protocol to work with Apache Knox

Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/14008 )

Change subject: IMPALA-8832: Fix HTTP client protocol to work with Apache Knox
......................................................................


Patch Set 2: Code-Review+2

(4 comments)

carrying forward

http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/rpc/auth-provider.h
File be/src/rpc/auth-provider.h:

http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/rpc/auth-provider.h@63
PS1, Line 63: generated
> typo: generated
Done


http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/rpc/authentication.cc
File be/src/rpc/authentication.cc:

http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/rpc/authentication.cc@1029
PS1, Line 1029: ansp
> nit:nullptr
Done


http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/rpc/authentication.cc@1098
PS1, Line 1098: ut_t
> nit: nullptr
Done


http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/transport/THttpTransport.cpp
File be/src/transport/THttpTransport.cpp:

http://gerrit.cloudera.org:8080/#/c/14008/1/be/src/transport/THttpTransport.cpp@81
PS1, Line 81:   readHeaders_ = true;
> It looks like upstream thrift still has these issues - worth filing a bug a
THRIFT-4916



-- 
To view, visit http://gerrit.cloudera.org:8080/14008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c9d934a654a9e6aaf9207fa5856f956baaacf55
Gerrit-Change-Number: 14008
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 05 Aug 2019 22:10:49 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-8832: Fix HTTP client protocol to work with Apache Knox

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14008 )

Change subject: IMPALA-8832: Fix HTTP client protocol to work with Apache Knox
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/4139/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/14008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c9d934a654a9e6aaf9207fa5856f956baaacf55
Gerrit-Change-Number: 14008
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 05 Aug 2019 20:23:03 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8832: Fix HTTP client protocol to work with Apache Knox

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14008 )

Change subject: IMPALA-8832: Fix HTTP client protocol to work with Apache Knox
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4736/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/14008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c9d934a654a9e6aaf9207fa5856f956baaacf55
Gerrit-Change-Number: 14008
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 06 Aug 2019 04:18:15 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8832: Fix HTTP client protocol to work with Apache Knox

Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Hello Tim Armstrong, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14008

to look at the new patch set (#2).

Change subject: IMPALA-8832: Fix HTTP client protocol to work with Apache Knox
......................................................................

IMPALA-8832: Fix HTTP client protocol to work with Apache Knox

This patch fixes two bugs with Impala's HTTP client protocol:
- THttpServer transports are no longer wrapped in TBufferedTransports.
  THttpServer already has its own support for buffering to process one
  HTTP request at a time, and wrapping it in a TBufferedTransport
  interferes with this, in some cases causing client requests to
  either not be processed or to recieve multiple responses.
- Fixes a bug in THttpTransport where when a chunked HTTP request is
  finished being processed, the 'readHeaders_' variable is never reset
  and further requests over the connection are not processed.

Testing:
- Tested by proxying beeline connections to Impala through Apache Knox

Change-Id: I5c9d934a654a9e6aaf9207fa5856f956baaacf55
---
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication.cc
M be/src/rpc/thrift-server.cc
M be/src/transport/THttpTransport.cpp
4 files changed, 47 insertions(+), 58 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/14008/2
-- 
To view, visit http://gerrit.cloudera.org:8080/14008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5c9d934a654a9e6aaf9207fa5856f956baaacf55
Gerrit-Change-Number: 14008
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-8832: Fix HTTP client protocol to work with Apache Knox

Posted by "Thomas Tauber-Marshall (Code Review)" <ge...@cloudera.org>.
Thomas Tauber-Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/14008 )

Change subject: IMPALA-8832: Fix HTTP client protocol to work with Apache Knox
......................................................................


Patch Set 3: Code-Review+2

carrying forward


-- 
To view, visit http://gerrit.cloudera.org:8080/14008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c9d934a654a9e6aaf9207fa5856f956baaacf55
Gerrit-Change-Number: 14008
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 06 Aug 2019 04:17:57 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-8832: Fix HTTP client protocol to work with Apache Knox

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/14008 )

Change subject: IMPALA-8832: Fix HTTP client protocol to work with Apache Knox
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/4143/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/14008
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c9d934a654a9e6aaf9207fa5856f956baaacf55
Gerrit-Change-Number: 14008
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tm...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 05 Aug 2019 22:53:40 +0000
Gerrit-HasComments: No