You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues-all@impala.apache.org by "Sameera Wijerathne (Jira)" <ji...@apache.org> on 2022/06/28 08:33:00 UTC

[jira] [Created] (IMPALA-11400) Kudu scan bottleneck due to sharing a single Kudu client for multiple tablet scans

|  ![](cid:jira-generated-image-avatar-e34c54fa-c758-4e7d-af1a-15e75193e254) |
[Sameera
Wijerathne](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=sameerawi)
**created** an issue  
---|---  
|  
---  
|  [IMPALA](https://issues.apache.org/jira/browse/IMPALA) / [![Bug](cid:jira-
generated-image-
avatar-096a3c00-58d7-4cbf-8f83-776baf5606e9)](https://issues.apache.org/jira/browse/IMPALA-11400)
[IMPALA-11400](https://issues.apache.org/jira/browse/IMPALA-11400)  
---  
[Kudu scan bottleneck due to sharing a single Kudu client for multiple tablet
scans](https://issues.apache.org/jira/browse/IMPALA-11400)  
| Issue Type: |  ![Bug](cid:jira-generated-image-
avatar-096a3c00-58d7-4cbf-8f83-776baf5606e9) Bug  
---|---  
Affects Versions: |  Impala 4.1.0  
Assignee: |  Unassigned  
Attachments: |  0.JPG, 1.JPG, 2.JPG, 2.jpeg, 3.JPG, 4.JPG, 5.JPG,
Impala_1.png, Impala_2.png, Kudu_1.png, Kudu_2.png, WhatsApp Image 2022-06-07
at 10.39.27 PM.jpeg  
Components: |  Backend  
Created: |  28/Jun/22 08:32  
Fix Versions: |  Impala 4.2.0  
Labels: |  performance  
Priority: |  ![Major](cid:jira-generated-image-static-
major-d8a89c57-cd9d-4894-bd00-8ec93433c775) Major  
Reporter: |  [Sameera
Wijerathne](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=sameerawi)  
|

This issue was observed when impala queries large datasets resides in Kudu.
Even single ImpalaD is scanning multiple kudu tablets, it shows a slowness to
retrive data eventhough ImpalaD makes parrellel scans. Reason for this is
ImpalaD only uses a single Kudu client for multiple scans but
KuduScanner::NextBatch runs on a single thread. So it's rpc reactor thread
utilizes upto a single core and bottlenecks all parrelel scans.

This behaviour makes Impala clusters that scans kudu cannot be vertically
scales to the maximum performance/cores of a node.

Please refer the screenshots from Kudu slack channel for more information.  
  
---  
|  |  [ ![Add Comment](cid:jira-generated-image-static-comment-
icon-511b0294-df23-42dd-803e-65d4a465e594)
](https://issues.apache.org/jira/browse/IMPALA-11400#add-comment "Add
Comment") |  [Add
Comment](https://issues.apache.org/jira/browse/IMPALA-11400#add-comment "Add
Comment")  
---|---  
  
|  This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9) |  |
![Atlassian logo](https://issues.apache.org/jira/images/mail/atlassian-email-
logo.png)  
---