You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Billy Pearson (JIRA)" <ji...@apache.org> on 2009/02/05 18:29:59 UTC
[jira] Created: (HBASE-1185) wrong request/sec in the gui reporting
wrong
wrong request/sec in the gui reporting wrong
--------------------------------------------
Key: HBASE-1185
URL: https://issues.apache.org/jira/browse/HBASE-1185
Project: Hadoop HBase
Issue Type: Bug
Components: client, regionserver
Affects Versions: 0.19.0
Environment: 0.19.0
Reporter: Billy Pearson
Priority: Minor
Fix For: 0.20.0
I am seeing lower number of request in the masters gui then I have seen in 0.18.0 while scanning.
I thank part of it is we moved to report per sec request not per 3 secs so the request should be 1/3 of the old numbers I was getting.
hbase.client.scanner.caching is not the reason the request are under reported.
I set hbase.client.scanner.caching = 1 and still get about 2K request a sec in the gui
but when the job is done I take records / job time and get 36,324/ records /sec. So
there must be some caching out side of the hbase.client.scanner.caching making the
request per sec lower then it should be. I know it running faster then reported just thought
it might give some new users the wrong impression that request/sec = read/write /sec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-1185) wrong request/sec in the gui reporting
wrong
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-1185:
--------------------------------------
Attachment: hbase-1185.patch
Billy, try this patch (against 0.19 branch) to see if you have better numbers.
> wrong request/sec in the gui reporting wrong
> --------------------------------------------
>
> Key: HBASE-1185
> URL: https://issues.apache.org/jira/browse/HBASE-1185
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.19.0
> Environment: 0.19.0
> Reporter: Billy Pearson
> Priority: Minor
> Fix For: 0.20.0
>
> Attachments: hbase-1185.patch
>
>
> I am seeing lower number of request in the masters gui then I have seen in 0.18.0 while scanning.
> I thank part of it is we moved to report per sec request not per 3 secs so the request should be 1/3 of the old numbers I was getting.
> hbase.client.scanner.caching is not the reason the request are under reported.
> I set hbase.client.scanner.caching = 1 and still get about 2K request a sec in the gui
> but when the job is done I take records / job time and get 36,324/ records /sec. So
> there must be some caching out side of the hbase.client.scanner.caching making the
> request per sec lower then it should be. I know it running faster then reported just thought
> it might give some new users the wrong impression that request/sec = read/write /sec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1185) wrong request/sec in the gui
reporting wrong
Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678944#action_12678944 ]
Billy Pearson commented on HBASE-1185:
--------------------------------------
I tested on 0.19.1 branch from a month or so ago.
> wrong request/sec in the gui reporting wrong
> --------------------------------------------
>
> Key: HBASE-1185
> URL: https://issues.apache.org/jira/browse/HBASE-1185
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.19.0
> Environment: 0.19.0
> Reporter: Billy Pearson
> Priority: Minor
> Fix For: 0.20.0
>
> Attachments: hbase-1185.patch
>
>
> I am seeing lower number of request in the masters gui then I have seen in 0.18.0 while scanning.
> I thank part of it is we moved to report per sec request not per 3 secs so the request should be 1/3 of the old numbers I was getting.
> hbase.client.scanner.caching is not the reason the request are under reported.
> I set hbase.client.scanner.caching = 1 and still get about 2K request a sec in the gui
> but when the job is done I take records / job time and get 36,324/ records /sec. So
> there must be some caching out side of the hbase.client.scanner.caching making the
> request per sec lower then it should be. I know it running faster then reported just thought
> it might give some new users the wrong impression that request/sec = read/write /sec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1185) wrong request/sec in the gui
reporting wrong
Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678493#action_12678493 ]
Billy Pearson commented on HBASE-1185:
--------------------------------------
I get my second number from a MR Job so yes number of records by the total job runtime
what I was saying about the 2 secs before in the past we reported
request in the gui on a 3 sec interval now we take the count and report it per sec. pre 0.19.0 I thank
0.18.0 maybe before that its been a while.
I thank the problem I am seeing is coming from the hbase.client.scanner.caching if it is set to 20 is that counted as 1 request or as 20 when we are scanning?
> wrong request/sec in the gui reporting wrong
> --------------------------------------------
>
> Key: HBASE-1185
> URL: https://issues.apache.org/jira/browse/HBASE-1185
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.19.0
> Environment: 0.19.0
> Reporter: Billy Pearson
> Priority: Minor
> Fix For: 0.20.0
>
>
> I am seeing lower number of request in the masters gui then I have seen in 0.18.0 while scanning.
> I thank part of it is we moved to report per sec request not per 3 secs so the request should be 1/3 of the old numbers I was getting.
> hbase.client.scanner.caching is not the reason the request are under reported.
> I set hbase.client.scanner.caching = 1 and still get about 2K request a sec in the gui
> but when the job is done I take records / job time and get 36,324/ records /sec. So
> there must be some caching out side of the hbase.client.scanner.caching making the
> request per sec lower then it should be. I know it running faster then reported just thought
> it might give some new users the wrong impression that request/sec = read/write /sec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1185) wrong request/sec in the gui
reporting wrong
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678731#action_12678731 ]
Jean-Daniel Cryans commented on HBASE-1185:
-------------------------------------------
Billy, I can confirm that it's scanner pre-fetching. In HRS :
{code}
public RowResult[] next(final long scannerId, int nbRows) throws IOException {
checkOpen();
requestCount.incrementAndGet();
ArrayList<RowResult> resultSets = new ArrayList<RowResult>();
try {
String scannerName = String.valueOf(scannerId);
InternalScanner s = scanners.get(scannerName);
if (s == null) {
throw new UnknownScannerException("Name: " + scannerName);
}
this.leases.renewLease(scannerName);
for(int i = 0; i < nbRows; i++) {
...
{code}
We increment for the whole batch of rows. I take the blame since it's my modification. The fix will be easy.
> wrong request/sec in the gui reporting wrong
> --------------------------------------------
>
> Key: HBASE-1185
> URL: https://issues.apache.org/jira/browse/HBASE-1185
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.19.0
> Environment: 0.19.0
> Reporter: Billy Pearson
> Priority: Minor
> Fix For: 0.20.0
>
>
> I am seeing lower number of request in the masters gui then I have seen in 0.18.0 while scanning.
> I thank part of it is we moved to report per sec request not per 3 secs so the request should be 1/3 of the old numbers I was getting.
> hbase.client.scanner.caching is not the reason the request are under reported.
> I set hbase.client.scanner.caching = 1 and still get about 2K request a sec in the gui
> but when the job is done I take records / job time and get 36,324/ records /sec. So
> there must be some caching out side of the hbase.client.scanner.caching making the
> request per sec lower then it should be. I know it running faster then reported just thought
> it might give some new users the wrong impression that request/sec = read/write /sec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1185) wrong request/sec in the gui
reporting wrong
Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678750#action_12678750 ]
Billy Pearson commented on HBASE-1185:
--------------------------------------
no big deal just thought it should report correctly sense we where under reporting the speeds.
> wrong request/sec in the gui reporting wrong
> --------------------------------------------
>
> Key: HBASE-1185
> URL: https://issues.apache.org/jira/browse/HBASE-1185
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.19.0
> Environment: 0.19.0
> Reporter: Billy Pearson
> Priority: Minor
> Fix For: 0.20.0
>
>
> I am seeing lower number of request in the masters gui then I have seen in 0.18.0 while scanning.
> I thank part of it is we moved to report per sec request not per 3 secs so the request should be 1/3 of the old numbers I was getting.
> hbase.client.scanner.caching is not the reason the request are under reported.
> I set hbase.client.scanner.caching = 1 and still get about 2K request a sec in the gui
> but when the job is done I take records / job time and get 36,324/ records /sec. So
> there must be some caching out side of the hbase.client.scanner.caching making the
> request per sec lower then it should be. I know it running faster then reported just thought
> it might give some new users the wrong impression that request/sec = read/write /sec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1185) wrong request/sec in the gui
reporting wrong
Posted by "Billy Pearson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678936#action_12678936 ]
Billy Pearson commented on HBASE-1185:
--------------------------------------
+1 the numbers look good now
thanks for the patch
> wrong request/sec in the gui reporting wrong
> --------------------------------------------
>
> Key: HBASE-1185
> URL: https://issues.apache.org/jira/browse/HBASE-1185
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.19.0
> Environment: 0.19.0
> Reporter: Billy Pearson
> Priority: Minor
> Fix For: 0.20.0
>
> Attachments: hbase-1185.patch
>
>
> I am seeing lower number of request in the masters gui then I have seen in 0.18.0 while scanning.
> I thank part of it is we moved to report per sec request not per 3 secs so the request should be 1/3 of the old numbers I was getting.
> hbase.client.scanner.caching is not the reason the request are under reported.
> I set hbase.client.scanner.caching = 1 and still get about 2K request a sec in the gui
> but when the job is done I take records / job time and get 36,324/ records /sec. So
> there must be some caching out side of the hbase.client.scanner.caching making the
> request per sec lower then it should be. I know it running faster then reported just thought
> it might give some new users the wrong impression that request/sec = read/write /sec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1185) wrong request/sec in the gui
reporting wrong
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678123#action_12678123 ]
Lars George commented on HBASE-1185:
------------------------------------
HI Stack,
When I look at the code in the HRegionServer.run() I see this:
this.serverInfo.setLoad(new HServerLoad(requestCount.get(),
onlineRegions.size()));
this.requestCount.set(0);
Which effectively means that this number is aggregated during the "message-interval" timeframe of the default 3 seconds. Then it handed over to the master and reset. The master UI takes this number and display it divided by the message-interval to get the requests per second value.
But since this in not actually aggregating the total number of request *per* HRegionServer it is always just a snapshot of what happened in the last three seconds. Once the job ends and no further requests are done it drops down to zero within 3 seconds.
Which I think is OK for a gauge. But what Billy is asking for sounds like something different, i.e. the load as an average over time - similar maybe to the unix "w" output.
Am I wrong here?
Thanks,
Lars
> wrong request/sec in the gui reporting wrong
> --------------------------------------------
>
> Key: HBASE-1185
> URL: https://issues.apache.org/jira/browse/HBASE-1185
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.19.0
> Environment: 0.19.0
> Reporter: Billy Pearson
> Priority: Minor
> Fix For: 0.20.0
>
>
> I am seeing lower number of request in the masters gui then I have seen in 0.18.0 while scanning.
> I thank part of it is we moved to report per sec request not per 3 secs so the request should be 1/3 of the old numbers I was getting.
> hbase.client.scanner.caching is not the reason the request are under reported.
> I set hbase.client.scanner.caching = 1 and still get about 2K request a sec in the gui
> but when the job is done I take records / job time and get 36,324/ records /sec. So
> there must be some caching out side of the hbase.client.scanner.caching making the
> request per sec lower then it should be. I know it running faster then reported just thought
> it might give some new users the wrong impression that request/sec = read/write /sec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-1185) wrong request/sec in the gui
reporting wrong
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans resolved HBASE-1185.
---------------------------------------
Resolution: Fixed
Fix Version/s: 0.19.1
Assignee: Jean-Daniel Cryans
Committed to branch and trunk. Thanks trying it out Billy!
> wrong request/sec in the gui reporting wrong
> --------------------------------------------
>
> Key: HBASE-1185
> URL: https://issues.apache.org/jira/browse/HBASE-1185
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.19.0
> Environment: 0.19.0
> Reporter: Billy Pearson
> Assignee: Jean-Daniel Cryans
> Priority: Minor
> Fix For: 0.19.1, 0.20.0
>
> Attachments: hbase-1185.patch
>
>
> I am seeing lower number of request in the masters gui then I have seen in 0.18.0 while scanning.
> I thank part of it is we moved to report per sec request not per 3 secs so the request should be 1/3 of the old numbers I was getting.
> hbase.client.scanner.caching is not the reason the request are under reported.
> I set hbase.client.scanner.caching = 1 and still get about 2K request a sec in the gui
> but when the job is done I take records / job time and get 36,324/ records /sec. So
> there must be some caching out side of the hbase.client.scanner.caching making the
> request per sec lower then it should be. I know it running faster then reported just thought
> it might give some new users the wrong impression that request/sec = read/write /sec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-1185) wrong request/sec in the gui
reporting wrong
Posted by "Lars George (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-1185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678122#action_12678122 ]
Lars George commented on HBASE-1185:
------------------------------------
Hi Billy,
I am trying to understand what the issue is exactly. I can see what the UI does to report request per interval. You say you have changed yours from 3 to 1 sec and therefore would expect the number to be a third from your actual numbers?
Where do you get the second number from? Or in other words, how do you compute the "records / job time"? Based on the number of the table you are scanning? For example, if you had 10,000 rows and scan it using a M/R job, you divide the number of records by the total job runtime?
And what you want is that the UI's "total requests" number to be close to the above?
Thanks,
Lars
> wrong request/sec in the gui reporting wrong
> --------------------------------------------
>
> Key: HBASE-1185
> URL: https://issues.apache.org/jira/browse/HBASE-1185
> Project: Hadoop HBase
> Issue Type: Bug
> Components: client, regionserver
> Affects Versions: 0.19.0
> Environment: 0.19.0
> Reporter: Billy Pearson
> Priority: Minor
> Fix For: 0.20.0
>
>
> I am seeing lower number of request in the masters gui then I have seen in 0.18.0 while scanning.
> I thank part of it is we moved to report per sec request not per 3 secs so the request should be 1/3 of the old numbers I was getting.
> hbase.client.scanner.caching is not the reason the request are under reported.
> I set hbase.client.scanner.caching = 1 and still get about 2K request a sec in the gui
> but when the job is done I take records / job time and get 36,324/ records /sec. So
> there must be some caching out side of the hbase.client.scanner.caching making the
> request per sec lower then it should be. I know it running faster then reported just thought
> it might give some new users the wrong impression that request/sec = read/write /sec.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.