[jira] [Comment Edited] (HBASE-5238) Add a log4j category for all edits to META/ROOT

["Andrew Purtell (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380240#comment-14380240 ] 

Andrew Purtell edited comment on HBASE-5238 at 3/25/15 5:05 PM:
----------------------------------------------------------------

META has a fixed system defined schema so I think in this case we don't care if column families are printed.

Keys can definitely be sensitive, we should avoid logging those. One can argue that META is already world readable but a persisted log can leak and the information inside can be read independently of access routes to the cluster. For example, the cluster might be firewalled to prevent random access to META, because keys are sensitive, but here we've slipped them into the log, those log lines are now up in Splunk or Elasticsearch, and network ACLs grant wider access to those.

Edit: We're already printing META keys in the log for various activities like splits, compactions. I think we can proceed with the patch here and file a followup for general META key log sanitizing, if that's a thing we want to pursue.


was (Author: apurtell):
META has a fixed system defined schema so I think in this case we don't care if column families are printed.

Keys can definitely be sensitive, we should avoid logging those. One can argue that META is already world readable but a persisted log can leak and the information inside can be read independently of access routes to the cluster. For example, the cluster might be firewalled to prevent random access to META, because keys are sensitive, but here we've slipped them into the log, those log lines are now up in Splunk or Elasticsearch, and network ACLs grant wider access to those.

> Add a log4j category for all edits to META/ROOT
> -----------------------------------------------
>
>                 Key: HBASE-5238
>                 URL: https://issues.apache.org/jira/browse/HBASE-5238
>             Project: HBase
>          Issue Type: New Feature
>          Components: regionserver
>    Affects Versions: 2.0.0
>            Reporter: Todd Lipcon
>            Assignee: Andrey Stepachev
>            Priority: Minor
>              Labels: beginner
>         Attachments: HBASE-5238.patch, HBASE-5238.patch, HBASE-5238.v2.patch, meta2.log
>
>
> Occasionally we run into bugs that have corrected META and written some bad data to META/ROOT but it's difficult to understand the order in which things happened. One option is to dump the HLog contents from the servers that hosted META at that time, but then it's interspersed with all other data. It would be nice to add a Log4j Logger to which we log all edits being applied to META and ROOT in textual form at DEBUG level. Then it would be easier to do a cluster-wide log grep to see what happened when.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)