You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Kurt Deschler (Jira)" <ji...@apache.org> on 2021/08/04 14:15:00 UTC
[jira] [Comment Edited] (IMPALA-10354) impala-shell hs2-http 3x
slower than hs2 with high-latency network
[ https://issues.apache.org/jira/browse/IMPALA-10354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17393199#comment-17393199 ]
Kurt Deschler edited comment on IMPALA-10354 at 8/4/21, 2:14 PM:
-----------------------------------------------------------------
Without changing the overall protocol, the potential low-hanging fruit would be:
1) Try to keep the HTTP connection open across requests. Each RPC is currently reconnecting. Connect represents around 1/3 of the wall time.
2) Use larger receive buffers. getresponse() times vary by up to 4x which suggests that there are multiple round trips being made. Receive is around 2/3 of the wall time.
was (Author: kdeschle):
Without changing the overall protocol, the potential low-hanging fruit would be:
1) Try to keep the HTTP connection open across requests. Connect represents around 1/3 of the wall time.
2) Use larger receive buffers. getresponse() times vary by up to 4x which suggests that there are multiple round trips being made. Receive is around 2/3 of the wall time.
> impala-shell hs2-http 3x slower than hs2 with high-latency network
> ------------------------------------------------------------------
>
> Key: IMPALA-10354
> URL: https://issues.apache.org/jira/browse/IMPALA-10354
> Project: IMPALA
> Issue Type: Bug
> Components: Clients
> Reporter: Kurt Deschler
> Priority: Major
> Attachments: createdrop.sql, hs2.svg, http.svg
>
>
> Performance testing of simple create/drop function with simulated high network latency shows a 3x performance difference between the hs2 and hs2-http protocols.
> Simulated latency:
> sudo tc qdisc add dev lo root handle 1: prio priomap 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
> sudo tc qdisc add dev lo parent 1:2 handle 20: netem delay 1000ms
> sudo tc filter add dev lo parent 1:0 protocol ip u32 match ip dport 28000 0xffff flowid 1:2
> sudo tc filter add dev lo parent 1:0 protocol ip u32 match ip dport 21050 0xffff flowid 1:2
> See attached flamegraphs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org