You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2008/07/30 00:13:31 UTC
[jira] Updated: (HBASE-511) Do exponential backoff in clients on
NSRE, WRE, ISE, etc.
[ https://issues.apache.org/jira/browse/HBASE-511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Purtell updated HBASE-511:
---------------------------------
Attachment: hbase-511-r680847.patch
A client of ours attempting rapid inserts was seeing occasional RetriesExhastedExceptions containing NSREs coinciding with region splits on a loaded cluster. Thought implementing this might be a good idea. Patch attached.
* Implements a BSD TCP inspired exponential backoff table which is a bit more aggressive for the first few steps and is capped at 64x beyond 10 steps.
* Increases the default hbase.client.retries.number to 10.
* Makes HBaseAdmin.disableTable() and HBaseAdmin.enableTable() conform to retry/backoff expectations.
If it's not too late to sneak this into 0.2.0 that would be great.
Passes all tests:
[mkdir] Created dir: /usr/src/Hadoop/hbase/build/test/logs
[junit] Running org.apache.hadoop.hbase.TestClassMigration
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.089 sec
[junit] Running org.apache.hadoop.hbase.TestCompare
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.023 sec
[junit] Running org.apache.hadoop.hbase.TestEmptyMetaInfo
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 33.4 sec
[junit] Running org.apache.hadoop.hbase.TestGlobalMemcacheLimit
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 48.896 sec
[junit] Running org.apache.hadoop.hbase.TestHBaseCluster
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 42.316 sec
[junit] Running org.apache.hadoop.hbase.TestInfoServers
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 14.305 sec
[junit] Running org.apache.hadoop.hbase.TestMasterAdmin
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 25.271 sec
[junit] Running org.apache.hadoop.hbase.TestMergeMeta
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 42.335 sec
[junit] Running org.apache.hadoop.hbase.TestMergeTable
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 34.483 sec
[junit] Running org.apache.hadoop.hbase.TestRegionRebalancing
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 160.758 sec
[junit] Running org.apache.hadoop.hbase.TestScanMultipleVersions
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 19.321 sec
[junit] Running org.apache.hadoop.hbase.TestScannerAPI
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 19.307 sec
[junit] Running org.apache.hadoop.hbase.TestSerialization
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 0.311 sec
[junit] Running org.apache.hadoop.hbase.TestTable
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 406.79 sec
[junit] Running org.apache.hadoop.hbase.TestToString
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.012 sec
[junit] Running org.apache.hadoop.hbase.client.TestBatchUpdate
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 384.138 sec
[junit] Running org.apache.hadoop.hbase.client.TestHTable
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 73.964 sec
[junit] Running org.apache.hadoop.hbase.client.TestListTables
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 28.287 sec
[junit] Running org.apache.hadoop.hbase.client.TestScannerTimes
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 21.332 sec
[junit] Running org.apache.hadoop.hbase.client.TestTimestamp
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 19.374 sec
[junit] Running org.apache.hadoop.hbase.filter.TestInclusiveStopRowFilter
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.009 sec
[junit] Running org.apache.hadoop.hbase.filter.TestPageRowFilter
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.006 sec
[junit] Running org.apache.hadoop.hbase.filter.TestRegExpRowFilter
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 0.027 sec
[junit] Running org.apache.hadoop.hbase.filter.TestRowFilterAfterWrite
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 23.276 sec
[junit] Running org.apache.hadoop.hbase.filter.TestRowFilterOnMultipleFamilies
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 19.349 sec
[junit] Running org.apache.hadoop.hbase.filter.TestRowFilterSet
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.116 sec
[junit] Running org.apache.hadoop.hbase.filter.TestStopRowFilter
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.009 sec
[junit] Running org.apache.hadoop.hbase.filter.TestWhileMatchRowFilter
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.013 sec
[junit] Running org.apache.hadoop.hbase.io.TestHbaseObjectWritable
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.107 sec
[junit] Running org.apache.hadoop.hbase.mapred.TestTableIndex
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 37.485 sec
[junit] Running org.apache.hadoop.hbase.mapred.TestTableMapReduce
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 41.644 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestBloomFilters
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 29.372 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestCompaction
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 12.289 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestDeleteAll
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 5.376 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestDeleteFamily
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 4.53 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestGet
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 10.359 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestGet2
[junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 20.046 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestHLog
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 14.831 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestHMemcache
[junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 0.318 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestHRegion
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 11.349 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestHRegionInfo
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.125 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestHStoreFile
[junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 17.777 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestLogRolling
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 60.413 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestScanner
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 6.35 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestSplit
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 30.259 sec
[junit] Running org.apache.hadoop.hbase.regionserver.TestTimestamp
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 25.868 sec
[junit] Running org.apache.hadoop.hbase.util.TestBase64
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.074 sec
[junit] Running org.apache.hadoop.hbase.util.TestKeying
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.007 sec
[junit] Running org.apache.hadoop.hbase.util.TestMergeTool
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 19.345 sec
[junit] Running org.apache.hadoop.hbase.util.TestMigrate
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 33.471 sec
[junit] Running org.apache.hadoop.hbase.util.TestRootPath
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.017 sec
[junit] Running org.onelab.test.TestFilter
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.089 sec
BUILD SUCCESSFUL
Total time: 30 minutes 20 seconds
> Do exponential backoff in clients on NSRE, WRE, ISE, etc.
> ---------------------------------------------------------
>
> Key: HBASE-511
> URL: https://issues.apache.org/jira/browse/HBASE-511
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.16.0, 0.1.0, 0.2.0
> Reporter: stack
> Priority: Minor
> Attachments: hbase-511-r680847.patch
>
>
> Jim Firby suggestion is that we do expotential backoff retrying locating regions, committing edits, etc.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.