You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "JinHyuk Kim (Jira)" <ji...@apache.org> on 2026/04/25 09:08:00 UTC

[jira] [Work started] (HBASE-30115) Introduce approximate progress estimation for TableRecordReader based on row key position

     [ https://issues.apache.org/jira/browse/HBASE-30115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on HBASE-30115 started by JinHyuk Kim.
-------------------------------------------
> Introduce approximate progress estimation for TableRecordReader based on row key position
> -----------------------------------------------------------------------------------------
>
>                 Key: HBASE-30115
>                 URL: https://issues.apache.org/jira/browse/HBASE-30115
>             Project: HBase
>          Issue Type: Task
>          Components: mapreduce
>            Reporter: JinHyuk Kim
>            Assignee: JinHyuk Kim
>            Priority: Minor
>              Labels: pull-request-available
>         Attachments: mapreduce-progress-0.png, mapreduce-progress-after.png
>
>
> h1. Background
> Currently, {{TableRecordReaderImpl.getProgress()}} always returns {*}0{*}, providing no progress feedback to the MapReduce framework. This makes it impossible for users to monitor scan progress during long-running jobs.
> !mapreduce-progress-0.png|width=1095,height=236!
>  
> h1. Suggestion
> This patch estimates progress by converting row keys to numeric values and computing the fraction of the key space covered so far: {{{}(current - start) / (stop - start){}}}.
> Since the {{TableInputFormat}} splitter sets start/stop row keys from region boundaries, they are only empty for the table's very first region (empty start) or last region (empty stop). In those cases, we *probe* the table with a forward or reverse scan (limit 1) to discover the actual boundary row key.
>                                                                                                                                                       The implementation is pluggable via {{hbase.mapreduce.rowkey.progress.class}} configuration:
>  * {{ByteBasedRowKeyProgress}} (default) : treats row keys as raw bytes. Works well for most key designs.
>  * {{HexPrefixRowKeyProgress}} : interprets leading bytes as hex characters ([0-9a-f]). Gives accurate linear progress for tables using hex-encoded hash prefixes (e.g. MD5). The raw byte approach is inaccurate for hex keys because there are large byte gaps between '9'→'a' (0x39→0x61) and between "0f"→"10" (0x3066→0x3130) that don't correspond to actual key distance. The prefix length is configurable via {{hbase.mapreduce.rowkey.progress.hex.prefix.length}} (default 4). Bytes beyond the prefix are ignored, so non-hex suffixes do not affect progress.
>  * Users can implement the {{RowKeyProgress}} interface for custom key encoding strategies.
> After this change, you can monitor the progress in this way.
>  
> !mapreduce-progress-after.png|width=1792,height=119!
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)