You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Hudson (Jira)" <ji...@apache.org> on 2021/07/23 06:50:00 UTC
[jira] [Commented] (HBASE-26108) add option to disable scanMetrics
in TableSnapshotInputFormat
[ https://issues.apache.org/jira/browse/HBASE-26108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17386000#comment-17386000 ]
Hudson commented on HBASE-26108:
--------------------------------
Results for branch branch-2.3
[build #259 on builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/259/]: (/) *{color:green}+1 overall{color}*
----
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/259/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2) report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/259/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3) report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/259/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11 report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/259/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> add option to disable scanMetrics in TableSnapshotInputFormat
> -------------------------------------------------------------
>
> Key: HBASE-26108
> URL: https://issues.apache.org/jira/browse/HBASE-26108
> Project: HBase
> Issue Type: Improvement
> Affects Versions: 2.3.5
> Reporter: Huaxiang Sun
> Assignee: Huaxiang Sun
> Priority: Major
> Fix For: 2.3.6, 3.0.0-alpha-2, 2.4.5
>
>
> When running spark job with TableSnapshotInputFormat, we found that scan is very slower. We found that scanMetrics is hardcoded as enabled, spark's
> newAPIHadoopRDD uses DummyReporter in hadoop, which causes the following exception and 80% cpu time is spent on this exception handling.
> Need to provide an option to disable scanMetrics.
> java.base@11.0.5/java.lang.Throwable.fillInStackTrace(Native Method)
> java.base@11.0.5/java.lang.Throwable.fillInStackTrace(Throwable.java:787) => holding Monitor(java.util.MissingResourceException@258206255})
> java.base@11.0.5/java.lang.Throwable.<init>(Throwable.java:292)
> java.base@11.0.5/java.lang.Exception.<init>(Exception.java:84)
> java.base@11.0.5/java.lang.RuntimeException.<init>(RuntimeException.java:80)
> java.base@11.0.5/java.util.MissingResourceException.<init>(MissingResourceException.java:85)
> java.base@11.0.5/java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:2055)
> java.base@11.0.5/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1689)
> java.base@11.0.5/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1593)
> java.base@11.0.5/java.util.ResourceBundle.getBundle(ResourceBundle.java:1284)
> app//org.apache.hadoop.mapreduce.util.ResourceBundles.getBundle(ResourceBundles.java:37)
> app//org.apache.hadoop.mapreduce.util.ResourceBundles.getValue(ResourceBundles.java:56) => holding Monitor(java.lang.Class@545605549})
> app//org.apache.hadoop.mapreduce.util.ResourceBundles.getCounterGroupName(ResourceBundles.java:77)
> app//org.apache.hadoop.mapreduce.counters.CounterGroupFactory.newGroup(CounterGroupFactory.java:94)
> app//org.apache.hadoop.mapreduce.counters.AbstractCounters.getGroup(AbstractCounters.java:227)
> app//org.apache.hadoop.mapreduce.counters.AbstractCounters.findCounter(AbstractCounters.java:154)
> app//org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl$DummyReporter.getCounter(TaskAttemptContextImpl.java:110)
> app//org.apache.hadoop.mapreduce.task.TaskAttemptContextImpl.getCounter(TaskAttemptContextImpl.java:76)
> org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.updateCounters(TableRecordReaderImpl.java:311)
> org.apache.hadoop.hbase.mapreduce.TableSnapshotInputFormat$TableSnapshotRegionRecordReader.nextKeyValue(TableSnapshotInputFormat.java:167)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)