You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "JinHyuk Kim (Jira)" <ji...@apache.org> on 2026/04/25 09:08:00 UTC
[jira] [Work started] (HBASE-30115) Introduce approximate progress estimation for TableRecordReader based on row key position
[ https://issues.apache.org/jira/browse/HBASE-30115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HBASE-30115 started by JinHyuk Kim.
-------------------------------------------
> Introduce approximate progress estimation for TableRecordReader based on row key position
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-30115
> URL: https://issues.apache.org/jira/browse/HBASE-30115
> Project: HBase
> Issue Type: Task
> Components: mapreduce
> Reporter: JinHyuk Kim
> Assignee: JinHyuk Kim
> Priority: Minor
> Labels: pull-request-available
> Attachments: mapreduce-progress-0.png, mapreduce-progress-after.png
>
>
> h1. Background
> Currently, {{TableRecordReaderImpl.getProgress()}} always returns {*}0{*}, providing no progress feedback to the MapReduce framework. This makes it impossible for users to monitor scan progress during long-running jobs.
> !mapreduce-progress-0.png|width=1095,height=236!
>
> h1. Suggestion
> This patch estimates progress by converting row keys to numeric values and computing the fraction of the key space covered so far: {{{}(current - start) / (stop - start){}}}.
> Since the {{TableInputFormat}} splitter sets start/stop row keys from region boundaries, they are only empty for the table's very first region (empty start) or last region (empty stop). In those cases, we *probe* the table with a forward or reverse scan (limit 1) to discover the actual boundary row key.
> The implementation is pluggable via {{hbase.mapreduce.rowkey.progress.class}} configuration:
> * {{ByteBasedRowKeyProgress}} (default) : treats row keys as raw bytes. Works well for most key designs.
> * {{HexPrefixRowKeyProgress}} : interprets leading bytes as hex characters ([0-9a-f]). Gives accurate linear progress for tables using hex-encoded hash prefixes (e.g. MD5). The raw byte approach is inaccurate for hex keys because there are large byte gaps between '9'→'a' (0x39→0x61) and between "0f"→"10" (0x3066→0x3130) that don't correspond to actual key distance. The prefix length is configurable via {{hbase.mapreduce.rowkey.progress.hex.prefix.length}} (default 4). Bytes beyond the prefix are ignored, so non-hex suffixes do not affect progress.
> * Users can implement the {{RowKeyProgress}} interface for custom key encoding strategies.
> After this change, you can monitor the progress in this way.
>
> !mapreduce-progress-after.png|width=1792,height=119!
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)