[jira] [Created] (HBASE-24380) Improve WAL splitting log lines to enable sessionizatino

["Andrew Kyle Purtell (Jira)" <ji...@apache.org>]

Andrew Kyle Purtell created HBASE-24380:
-------------------------------------------

             Summary: Improve WAL splitting log lines to enable sessionizatino
                 Key: HBASE-24380
                 URL: https://issues.apache.org/jira/browse/HBASE-24380
             Project: HBase
          Issue Type: Improvement
          Components: logging, Operability, wal
            Reporter: Andrew Kyle Purtell


Looking to reconstruct a timeline from write of recovered.edits file back to start of WAL file split, with a bunch of unrelated activity in the meantime, there isn't a consistent token that links split file write messages (which include store path including region hash) to beginning of WAL splitting activity. Sessonizing by host doesn't work because work can bounce around through retries. Thread context names in the logs vary and can be like [nds1-225-fra:60020-7] or [fb472085572ba72e96f1] (trailing digits of region hash) or [splits-1589016325868] . 

We could have WALSplitter get the current time when starting the split of a WAL file and have it log this timestamp in every line as a splitting session identifier.

Related, we should track the time of split task execution end to end and export a metric that captures it.

It might also be worthwhile to wire up more of WAL splitting to TaskMonitor status logging. If we do this we can also enable status journal logging, so when splitting is down, a line will appear in the log that has the list of all status messages recorded during splitting and the time delta in milliseconds between them. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)