You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org> on 2012/09/18 02:45:07 UTC
[jira] [Created] (HBASE-6804) [replication] lower the amount of
logging to a more human-readable level
Jean-Daniel Cryans created HBASE-6804:
-----------------------------------------
Summary: [replication] lower the amount of logging to a more human-readable level
Key: HBASE-6804
URL: https://issues.apache.org/jira/browse/HBASE-6804
Project: HBase
Issue Type: Improvement
Reporter: Jean-Daniel Cryans
Fix For: 0.96.0
We need stop logging every time replication decides to do something. It used to be extremely useful when the code base was younger but now it should be possible to bring it down while keeping it relevant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6804) [replication] lower the amount of
logging to a more human-readable level
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-6804:
--------------------------------------
Attachment: HBASE-6804-0.94-v2.patch
Refresh of the patch, it's currently running on all clusters at StumbleUpon. It addresses Stack's comment too.
> [replication] lower the amount of logging to a more human-readable level
> ------------------------------------------------------------------------
>
> Key: HBASE-6804
> URL: https://issues.apache.org/jira/browse/HBASE-6804
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.96.0
>
> Attachments: HBASE-6804-0.94.patch, HBASE-6804-0.94-v2.patch
>
>
> We need stop logging every time replication decides to do something. It used to be extremely useful when the code base was younger but now it should be possible to bring it down while keeping it relevant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6804) [replication] lower the amount of
logging to a more human-readable level
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-6804:
--------------------------------------
Attachment: HBASE-6804-0.94-v3.patch
Oh and the refreshed 0.94 patch.
> [replication] lower the amount of logging to a more human-readable level
> ------------------------------------------------------------------------
>
> Key: HBASE-6804
> URL: https://issues.apache.org/jira/browse/HBASE-6804
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.96.0
>
> Attachments: HBASE-6804-0.94.patch, HBASE-6804-0.94-v2.patch, HBASE-6804-0.94-v3.patch, HBASE-6804.patch
>
>
> We need stop logging every time replication decides to do something. It used to be extremely useful when the code base was younger but now it should be possible to bring it down while keeping it relevant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6804) [replication] lower the amount of
logging to a more human-readable level
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459056#comment-13459056 ]
Jean-Daniel Cryans commented on HBASE-6804:
-------------------------------------------
bq. A pool of one thread only seems odd... is it just because you want to do this scheduleAtFixedRate?
Yeah.
bq. Threads usually have their host as prefix; check out other places in server where we name threads.
Will do.
> [replication] lower the amount of logging to a more human-readable level
> ------------------------------------------------------------------------
>
> Key: HBASE-6804
> URL: https://issues.apache.org/jira/browse/HBASE-6804
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Fix For: 0.96.0
>
> Attachments: HBASE-6804-0.94.patch
>
>
> We need stop logging every time replication decides to do something. It used to be extremely useful when the code base was younger but now it should be possible to bring it down while keeping it relevant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6804) [replication] lower the amount of
logging to a more human-readable level
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans reassigned HBASE-6804:
-----------------------------------------
Assignee: Jean-Daniel Cryans
> [replication] lower the amount of logging to a more human-readable level
> ------------------------------------------------------------------------
>
> Key: HBASE-6804
> URL: https://issues.apache.org/jira/browse/HBASE-6804
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.96.0
>
> Attachments: HBASE-6804-0.94.patch, HBASE-6804-0.94-v2.patch
>
>
> We need stop logging every time replication decides to do something. It used to be extremely useful when the code base was younger but now it should be possible to bring it down while keeping it relevant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6804) [replication] lower the amount of
logging to a more human-readable level
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-6804:
--------------------------------------
Attachment: HBASE-6804.patch
Patch for trunk based on the v2 0.94 patch.
> [replication] lower the amount of logging to a more human-readable level
> ------------------------------------------------------------------------
>
> Key: HBASE-6804
> URL: https://issues.apache.org/jira/browse/HBASE-6804
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.96.0
>
> Attachments: HBASE-6804-0.94.patch, HBASE-6804-0.94-v2.patch, HBASE-6804.patch
>
>
> We need stop logging every time replication decides to do something. It used to be extremely useful when the code base was younger but now it should be possible to bring it down while keeping it relevant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-6804) [replication] lower the amount of
logging to a more human-readable level
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans resolved HBASE-6804.
---------------------------------------
Resolution: Fixed
Release Note: Replication will not log a lot less messages than before. There's no way to "re-active" those old log lines. Configure replication.stats.thread.period.seconds to set how often replication stats are printed in the log, by default it's 5 minutes.
Hadoop Flags: Reviewed
Committed to trunk. Since it's borderline a new feature, I'd rather not put this in 0.94.
> [replication] lower the amount of logging to a more human-readable level
> ------------------------------------------------------------------------
>
> Key: HBASE-6804
> URL: https://issues.apache.org/jira/browse/HBASE-6804
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.96.0
>
> Attachments: HBASE-6804-0.94.patch, HBASE-6804-0.94-v2.patch, HBASE-6804-0.94-v3.patch, HBASE-6804.patch
>
>
> We need stop logging every time replication decides to do something. It used to be extremely useful when the code base was younger but now it should be possible to bring it down while keeping it relevant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6804) [replication] lower the amount of
logging to a more human-readable level
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-6804:
-------------------------
Priority: Critical (was: Major)
> [replication] lower the amount of logging to a more human-readable level
> ------------------------------------------------------------------------
>
> Key: HBASE-6804
> URL: https://issues.apache.org/jira/browse/HBASE-6804
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.96.0
>
> Attachments: HBASE-6804-0.94.patch
>
>
> We need stop logging every time replication decides to do something. It used to be extremely useful when the code base was younger but now it should be possible to bring it down while keeping it relevant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6804) [replication] lower the amount of
logging to a more human-readable level
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509419#comment-13509419 ]
Hudson commented on HBASE-6804:
-------------------------------
Integrated in HBase-TRUNK #3591 (See [https://builds.apache.org/job/HBase-TRUNK/3591/])
HBASE-6804 [replication] lower the amount of logging to a more human-readable level (Revision 1416735)
Result = FAILURE
jdcryans :
Files :
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/MetricsSink.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java
* /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java
* /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplication.java
> [replication] lower the amount of logging to a more human-readable level
> ------------------------------------------------------------------------
>
> Key: HBASE-6804
> URL: https://issues.apache.org/jira/browse/HBASE-6804
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.96.0
>
> Attachments: HBASE-6804-0.94.patch, HBASE-6804-0.94-v2.patch, HBASE-6804-0.94-v3.patch, HBASE-6804.patch
>
>
> We need stop logging every time replication decides to do something. It used to be extremely useful when the code base was younger but now it should be possible to bring it down while keeping it relevant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6804) [replication] lower the amount of
logging to a more human-readable level
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472774#comment-13472774 ]
stack commented on HBASE-6804:
------------------------------
+1 on commit. Can this go back into 0.94 too?
> [replication] lower the amount of logging to a more human-readable level
> ------------------------------------------------------------------------
>
> Key: HBASE-6804
> URL: https://issues.apache.org/jira/browse/HBASE-6804
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Assignee: Jean-Daniel Cryans
> Priority: Critical
> Fix For: 0.96.0
>
> Attachments: HBASE-6804-0.94.patch, HBASE-6804-0.94-v2.patch
>
>
> We need stop logging every time replication decides to do something. It used to be extremely useful when the code base was younger but now it should be possible to bring it down while keeping it relevant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6804) [replication] lower the amount of
logging to a more human-readable level
Posted by "Jean-Daniel Cryans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jean-Daniel Cryans updated HBASE-6804:
--------------------------------------
Attachment: HBASE-6804-0.94.patch
Attaching a first pass for 0.94 but I'm planning to just commit to 0.96.
I added a scheduled executor in {{Replication}} that prints out all the statistics for both the sources and the sink every 5 minutes by default. I had to add glue methods to get the stats all the way from {{ReplicationSource}} to {{Replication}} which is not super clean.
This is what the output looks like for a the sink:
bq. 2012-09-17 17:43:08,418 INFO [Replication Statistics #0] regionserver.Replication$ReplicationStatisticsThread(281): Sink: age in ms of last applied edit: 15644, total replicated edits: 7350
And for a source:
bq. 2012-09-17 17:42:19,887 INFO [Replication Statistics #0] regionserver.Replication$ReplicationStatisticsThread(281): Normal source for cluster 2: Total replicated edits: 1473, currently replicating from: hdfs://localhost:53235/user/jdcryans/hbase/.logs/h-25-185.sfo.stumble.net,53250,1347928807372/h-25-185.sfo.stumble.net%2C53250%2C1347928807372.1347928936043 at position: 3347
I think I need to do some more work on the "normal" EOFs that occur when tail'ing a log file.
> [replication] lower the amount of logging to a more human-readable level
> ------------------------------------------------------------------------
>
> Key: HBASE-6804
> URL: https://issues.apache.org/jira/browse/HBASE-6804
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Fix For: 0.96.0
>
> Attachments: HBASE-6804-0.94.patch
>
>
> We need stop logging every time replication decides to do something. It used to be extremely useful when the code base was younger but now it should be possible to bring it down while keeping it relevant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6804) [replication] lower the amount of
logging to a more human-readable level
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459052#comment-13459052 ]
stack commented on HBASE-6804:
------------------------------
Thats a lot of removed DEBUG logging.
A pool of one thread only seems odd... is it just because you want to do this scheduleAtFixedRate?
+ private final ScheduledExecutorService scheduleThreadPool =
+ Executors.newScheduledThreadPool(1,
+ new ThreadFactoryBuilder()
+ .setNameFormat("Replication Statistics #%d")
+ .setDaemon(true)
+ .build());
Threads usually have their host as prefix; check out other places in server where we name threads.
Patch seems good. +1 on commit.
> [replication] lower the amount of logging to a more human-readable level
> ------------------------------------------------------------------------
>
> Key: HBASE-6804
> URL: https://issues.apache.org/jira/browse/HBASE-6804
> Project: HBase
> Issue Type: Improvement
> Reporter: Jean-Daniel Cryans
> Fix For: 0.96.0
>
> Attachments: HBASE-6804-0.94.patch
>
>
> We need stop logging every time replication decides to do something. It used to be extremely useful when the code base was younger but now it should be possible to bring it down while keeping it relevant.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira