[jira] [Created] (PHOENIX-5281) 95%+ of request time spent in Phoenix driver

["Laran Evans (JIRA)" <ji...@apache.org>]

Laran Evans created PHOENIX-5281:
------------------------------------

             Summary: 95%+ of request time spent in Phoenix driver
                 Key: PHOENIX-5281
                 URL: https://issues.apache.org/jira/browse/PHOENIX-5281
             Project: Phoenix
          Issue Type: Bug
         Environment: We're running on Azure using HDInsight for HBase.
            Reporter: Laran Evans
         Attachments: Screen Shot 2019-05-14 at 6.14.29 PM.png, Screen Shot 2019-05-14 at 6.14.29 PM.png, Screen Shot 2019-05-14 at 6.38.39 PM.png

Our application consistently spends 95% of it's time in what shows up in New Relic as "Reading database result". From what I've read, this usually means that the time is being spent in the database driver, which in our case is 



org.apache.phoenix:phoenix-core:4.7.0-HBase-1.1



I have profiled the application and all I see is this:

!Screen Shot 2019-05-14 at 6.14.29 PM.png!

All I see is lots of time in poll().

Here's the line of code from Phoenix:

[https://github.com/apache/phoenix/blob/4.7.0-HBase-1.1/phoenix-core/src/main/java/org/apache/phoenix/job/AbstractRoundRobinQueue.java#L136]

It isn't clear to me why this takes so long?

Here you see that 96% of the response time is stuck in that one section. It seems that the two select queries themselves are very fast. All the time is spent in reading the result set.

!Screen Shot 2019-05-14 at 6.38.39 PM.png!

What are some likely reasons why this might be happening? And how can we fix it?

Thanks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)