[jira] [Commented] (HBASE-12338) Client side scanning prefetching.

["Yi Deng (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-12338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14218376#comment-14218376 ] 

Yi Deng commented on HBASE-12338:
---------------------------------

The copy performance evaulation results:

$ for i in `seq 1 10`; do ./bin/hbase org.apache.hadoop.hbase.CopyPerformanceEvaluation -ch 10000 -st perf -dt perfdst -t prefetchcopy 2>/dev/null; done | grep throughput
throughput  : 65.9 MB/s
throughput  : 64.6 MB/s
throughput  : 58.4 MB/s
throughput  : 64.0 MB/s
throughput  : 61.8 MB/s
throughput  : 64.7 MB/s
throughput  : 67.2 MB/s
throughput  : 61.2 MB/s
throughput  : 61.1 MB/s
throughput  : 74.2 MB/s


$ for i in `seq 1 10`; do ./bin/hbase org.apache.hadoop.hbase.CopyPerformanceEvaluation -ch 10000 -st perf -dt perfdst -t copy 2>/dev/null; done | grep throughput
throughput  : 40.7 MB/s
throughput  : 38.4 MB/s
throughput  : 37.4 MB/s
throughput  : 40.6 MB/s
throughput  : 39.6 MB/s
throughput  : 40.7 MB/s
throughput  : 39.0 MB/s
throughput  : 38.4 MB/s
throughput  : 39.8 MB/s

> Client side scanning prefetching.
> ---------------------------------
>
>                 Key: HBASE-12338
>                 URL: https://issues.apache.org/jira/browse/HBASE-12338
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Client
>    Affects Versions: 1.0.0, 2.0.0, 0.98.6.1
>            Reporter: Yi Deng
>            Assignee: Yi Deng
>              Labels: prefetch, results, scanner
>         Attachments: 0001-Add-ScanPrefetcher-for-client-side-scanning-prefetch.patch, 0001-ScanPrefetcher.patch, 2.0-0001-Add-ScanPrefetcher-for-client-side-scanning-prefetch.patch
>
>
> Since server side prefetching was not proved to be a good way to prefetch, we need to do it on client side.
> This is a wrapper class that takes any instance of `ResultScanner` as the underneath scanning component. The class will schedule the scanning in a background thread. There is a buffering queue storing prefetched results, whose's length is configurable. The prefetcher will release the thread if the queue is full and wait for results to be consumed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)