[jira] [Created] (HBASE-22899) logging improvements for snapshot operations w/large manifests

["Sean Busbey (Jira)" <ji...@apache.org>]

Sean Busbey created HBASE-22899:
-----------------------------------

             Summary: logging improvements for snapshot operations w/large manifests
                 Key: HBASE-22899
                 URL: https://issues.apache.org/jira/browse/HBASE-22899
             Project: HBase
          Issue Type: Improvement
          Components: snapshots
            Reporter: Sean Busbey


working through some issues with snapshotting in the presence of a substantial (millions-of) hfiles.

log messages about the state of creating or restoring span multiple hundreds of MB of logs, so more context in individual lines would make it easier to orient myself.

RestoreSnapshotHelper has a few different stanzas that essentially all say "I'm creating an HFileLink for this thing". e.g.

{code}
    for (SnapshotRegionManifest.FamilyFiles familyFiles: manifest.getFamilyFilesList()) {
      Path familyDir = new Path(regionDir, familyFiles.getFamilyName().toStringUtf8());
      for (SnapshotRegionManifest.StoreFile storeFile: familyFiles.getStoreFilesList()) {
        LOG.info("Adding HFileLink " + storeFile.getName() + " to table=" + tableName);
        restoreStoreFile(familyDir, snapshotRegionInfo, storeFile, createBackRefs);
      }
    }
{code}

If would be helpful if

1) these stanzas offered something in the logged text to tell them apart (I think the coincidentally differ in log level, but that's esoteric and easy to lose over time)
2) it would be nice if we included the snapshot name in the log message so I can more easily pull out all log lines related to the snapshot
3) these stanzas indicated "progress" of sorts by giving how many storefiles are present and where we are in that list.

sort of like this bit we log when creating the snapshot manifest:

{code}
      // 2.2. iterate through all the store's files and create "references".
      for (int i = 0, sz = storeFiles.size(); i < sz; i++) {
        HStoreFile storeFile = storeFiles.get(i);
        monitor.rethrowException();

        // create "reference" to this store file.
        LOG.debug("Adding reference for file (" + (i+1) + "/" + sz + "): " + storeFile.getPath());
        visitor.storeFile(regionData, familyData, storeFile.getFileInfo());
      }
{code}

And speaking of the above bit in SnapshotManifest, it would be nice if it included the name of the snapshot in the log message so it's easier to pull out all log lines related to the snapshot.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)