You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by zheng wang <18...@qq.com> on 2020/07/21 13:00:21 UTC
回复: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
This log info just as a warning that cant make it disappear for now, but will not impact anything, so you can just ignore it in local mode.
------------------ 原始邮件 ------------------
发件人: "user" <subharaj.manna@gmail.com>;
发送时间: 2020年7月21日(星期二) 晚上9:19
收件人: "Hbase-User"<user@hbase.apache.org>;
主题: Re: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
Thanks for replying.
Yes it is a single node hbase cluster. I am not specifying any storage
policy. Looking at the HStore
<https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274>
code it appears even if no storage policy is specified it will take HOT.
Can you explain this a bit more how can I get around this error or in a
single node hbase cluster I should be ignoring this?
On Tue, Jul 21, 2020 at 3:03 PM zheng wang <18031031@qq.com> wrote:
> LocalFileSystem? The&nbsp;setStoragePolicy could only be used in
> distributed hdfs.
> &nbsp;
>
>
> ------------------&nbsp;原始邮件&nbsp;------------------
> 发件人:
> "user"
> <
> subharaj.manna@gmail.com&gt;;
> 发送时间:&nbsp;2020年7月21日(星期二) 下午5:58
> 收件人:&nbsp;"Hbase-User"<user@hbase.apache.org&gt;;
>
> 主题:&nbsp;HBase 2.1.0 - NoSuchMethodException
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
>
>
>
> Hi
>
> I am using HBase 2.1.0 with Hadoop 3.0.0. In hbase master logs I am
> seeing a warning like below
>
> 2020-07-20 06:02:24,859 WARN&nbsp; [StoreOpener-1588230740-1]
> util.CommonFSUtils: FileSystem doesn't support setStoragePolicy;
> HDFS-6584, HDFS-9345 not available. This is normal and expected on
> earlier Hadoop versions.
> java.lang.NoSuchMethodException:
>
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path,
> java.lang.String)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> java.lang.Class.getDeclaredMethod(Class.java:2130)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:577)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:558)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:526)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:194)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> org.apache.hadoop.hbase.regionserver.HStore.<init&gt;(HStore.java:264)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> java.lang.Thread.run(Thread.java:748)
>
> As per my understanding, this error should not be coming with Hadoop
> 3.0.0. Can someone let me know if my understanding is correct or what
> could be going wrong here?
回复: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
Posted by zheng wang <18...@qq.com>.
Seems getDeclaredMethod does not include the methods that extends from parent.
------------------ 原始邮件 ------------------
发件人: "user" <subharaj.manna@gmail.com>;
发送时间: 2020年7月22日(星期三) 晚上6:44
收件人: "Hbase-User"<user@hbase.apache.org>;
主题: Re: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
Yes I am using hadoop 3.0.0.
On Wed, Jul 22, 2020 at 12:31 PM zheng wang <18031031@qq.com> wrote:
> Are you sure you are using hadoop3.0.0?
>
>
>
>
> ------------------&nbsp;原始邮件&nbsp;------------------
> 发件人:
> "user"
> <
> subharaj.manna@gmail.com&gt;;
> 发送时间:&nbsp;2020年7月22日(星期三) 凌晨3:17
> 收件人:&nbsp;"Hbase-User"<user@hbase.apache.org&gt;;
>
> 主题:&nbsp;Re: HBase 2.1.0 - NoSuchMethodException
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
>
>
>
> Thanks for the pointer.
>
> One more query in I am getting this error from CommonFSUtils.
> invokeSetStoragePolicy()
> <
> https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533&gt
> ;.
> I am using hadoop 3.0.0 and in FilterFileSystem (which LocalFileSystem
> extends from) I do see the method&nbsp; setStoragePolicy
> <
> https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637&gt
> ;
> .
>
> Can someone explain how is this NoSuchMethodException is being thrown or I
> am looking at the wrong code path for LocalFileSystem?
>
> On Tue, Jul 21, 2020 at 7:04 PM Sean Busbey <busbey@apache.org&gt; wrote:
>
> &gt; that is the detail message of why it determined that the FileSystem
> &gt; doesn't support setStoragePolicy. As opposed to e.g. a security
> &gt; manager denying access to introspect the methods available.
> &gt;
> &gt; On Tue, Jul 21, 2020 at 8:24 AM Debraj Manna <
> subharaj.manna@gmail.com&gt;
> &gt; wrote:
> &gt; &gt;
> &gt; &gt; I understood the "util.CommonFSUtils: FileSystem doesn't support
> &gt; &gt; setStoragePolicy;" part.
> &gt; &gt;
> &gt; &gt; Can you let me know why it is saying
> "java.lang.NoSuchMethodException:
> &gt; &gt; org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy2020-07-20
> &gt; &gt; 06:02:24,859 WARN&nbsp; [StoreOpener-1588230740-1]" ?
> &gt; &gt;
> &gt; &gt;
> &gt; &gt; On Tue, Jul 21, 2020 at 6:43 PM zheng wang <18031031@qq.com&gt;
> wrote:
> &gt; &gt;
> &gt; &gt; &gt; This log info just as a warning that cant make it disappear
> for now,
> &gt; but
> &gt; &gt; &gt; will not impact anything, so you can just ignore it in
> local mode.
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt; ------------------&amp;nbsp;原始邮件&amp;nbsp;------------------
> &gt; &gt; &gt; 发件人:
> &gt; &gt;
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> "user"
> &gt; &gt;
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> <
> &gt; &gt; &gt; subharaj.manna@gmail.com&amp;gt;;
> &gt; &gt; &gt; 发送时间:&amp;nbsp;2020年7月21日(星期二) 晚上9:19
> &gt; &gt; &gt; 收件人:&amp;nbsp;"Hbase-User"<user@hbase.apache.org&amp;gt;;
> &gt; &gt; &gt;
> &gt; &gt; &gt; 主题:&amp;nbsp;Re: HBase 2.1.0 - NoSuchMethodException
> &gt; &gt; &gt; org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt; Thanks for replying.
> &gt; &gt; &gt;
> &gt; &gt; &gt; Yes it is a single node hbase cluster. I am not specifying
> any storage
> &gt; &gt; &gt; policy. Looking at the HStore
> &gt; &gt; &gt; <
> &gt; &gt; &gt;
> &gt;
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt
> &gt
> <https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt&gt>;
> &gt; &gt; ;
> &gt; &gt; &gt; code it appears even if no storage policy is specified it
> will take
> &gt; HOT.
> &gt; &gt; &gt;
> &gt; &gt; &gt; Can you explain this a bit more how can I get around this
> error or in a
> &gt; &gt; &gt; single node hbase cluster I should be ignoring this?
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt; On Tue, Jul 21, 2020 at 3:03 PM zheng wang <18031031@qq.com&amp;gt;
> wrote:
> &gt; &gt; &gt;
> &gt; &gt; &gt; &amp;gt; LocalFileSystem? The&amp;amp;nbsp;setStoragePolicy
> could only be used
> &gt; in
> &gt; &gt; &gt; &amp;gt; distributed hdfs.
> &gt; &gt; &gt; &amp;gt; &amp;amp;nbsp;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> ------------------&amp;amp;nbsp;原始邮件&amp;amp;nbsp;------------------
> &gt; &gt; &gt; &amp;gt; 发件人:
> &gt; &gt; &gt;
> &gt;
> &amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> &gt; &gt; &gt; "user"
> &gt; &gt; &gt;
> &gt;
> &amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> &gt; &gt; &gt; <
> &gt; &gt; &gt; &amp;gt; subharaj.manna@gmail.com&amp;amp;gt;;
> &gt; &gt; &gt; &amp;gt; 发送时间:&amp;amp;nbsp;2020年7月21日(星期二) 下午5:58
> &gt; &gt; &gt; &amp;gt; 收件人:&amp;amp;nbsp;"Hbase-User"<
> user@hbase.apache.org&amp;amp;gt;;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; 主题:&amp;amp;nbsp;HBase 2.1.0 -
> NoSuchMethodException
> &gt; &gt; &gt; &amp;gt;
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; Hi
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; I am using HBase 2.1.0 with Hadoop 3.0.0. In hbase
> master logs I
> &gt; am
> &gt; &gt; &gt; &amp;gt; seeing a warning like below
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; 2020-07-20 06:02:24,859 WARN&amp;amp;nbsp;
> [StoreOpener-1588230740-1]
> &gt; &gt; &gt; &amp;gt; util.CommonFSUtils: FileSystem doesn't support
> setStoragePolicy;
> &gt; &gt; &gt; &amp;gt; HDFS-6584, HDFS-9345 not available. This is normal
> and expected on
> &gt; &gt; &gt; &amp;gt; earlier Hadoop versions.
> &gt; &gt; &gt; &amp;gt; java.lang.NoSuchMethodException:
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path,
> &gt; &gt; &gt; &amp;gt; java.lang.String)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt; java.lang.Class.getDeclaredMethod(Class.java:2130)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:577)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:558)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:526)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:194)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.regionserver.HStore.<init&amp;amp;gt;(HStore.java:264)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt; java.lang.Thread.run(Thread.java:748)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; As per my understanding, this error should not be
> coming with
> &gt; Hadoop
> &gt; &gt; &gt; &amp;gt; 3.0.0. Can someone let me know if my understanding
> is correct or
> &gt; what
> &gt; &gt; &gt; &amp;gt; could be going wrong here?
> &gt;
> &gt;
> &gt;
> &gt; --
> &gt; Sean
> &gt;
Re: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
Posted by Debraj Manna <su...@gmail.com>.
Thanks all my doubts are resolved now.
On Wed, Jul 22, 2020 at 4:46 PM zheng wang <18...@qq.com> wrote:
> And getMethod does.
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人:
> "user"
> <
> subharaj.manna@gmail.com>;
> 发送时间: 2020年7月22日(星期三) 晚上6:44
> 收件人: "Hbase-User"<user@hbase.apache.org>;
>
> 主题: Re: HBase 2.1.0 - NoSuchMethodException
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
>
>
>
> Yes I am using hadoop 3.0.0.
>
> On Wed, Jul 22, 2020 at 12:31 PM zheng wang <18031031@qq.com> wrote:
>
> > Are you sure you are using hadoop3.0.0?
> >
> >
> >
> >
> > ------------------&nbsp;原始邮件&nbsp;------------------
> > 发件人:
> >
> "user"
> >
> <
> > subharaj.manna@gmail.com&gt;;
> > 发送时间:&nbsp;2020年7月22日(星期三) 凌晨3:17
> > 收件人:&nbsp;"Hbase-User"<user@hbase.apache.org&gt;;
> >
> > 主题:&nbsp;Re: HBase 2.1.0 - NoSuchMethodException
> > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> >
> >
> >
> > Thanks for the pointer.
> >
> > One more query in I am getting this error from CommonFSUtils.
> > invokeSetStoragePolicy()
> > <
> >
> https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533&gt
> >
> <https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533&gt>>;
> ;.
> > I am using hadoop 3.0.0 and in FilterFileSystem (which LocalFileSystem
> > extends from) I do see the method&nbsp; setStoragePolicy
> > <
> >
> https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637&gt
> >
> <https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637&gt>>;
> ;
> > .
> >
> > Can someone explain how is this NoSuchMethodException is being thrown
> or I
> > am looking at the wrong code path for LocalFileSystem?
> >
> > On Tue, Jul 21, 2020 at 7:04 PM Sean Busbey <busbey@apache.org&gt;
> wrote:
> >
> > &gt; that is the detail message of why it determined that the
> FileSystem
> > &gt; doesn't support setStoragePolicy. As opposed to e.g. a
> security
> > &gt; manager denying access to introspect the methods available.
> > &gt;
> > &gt; On Tue, Jul 21, 2020 at 8:24 AM Debraj Manna <
> > subharaj.manna@gmail.com&gt;
> > &gt; wrote:
> > &gt; &gt;
> > &gt; &gt; I understood the "util.CommonFSUtils: FileSystem
> doesn't support
> > &gt; &gt; setStoragePolicy;" part.
> > &gt; &gt;
> > &gt; &gt; Can you let me know why it is saying
> > "java.lang.NoSuchMethodException:
> > &gt; &gt;
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy2020-07-20
> > &gt; &gt; 06:02:24,859 WARN&nbsp;
> [StoreOpener-1588230740-1]" ?
> > &gt; &gt;
> > &gt; &gt;
> > &gt; &gt; On Tue, Jul 21, 2020 at 6:43 PM zheng wang <
> 18031031@qq.com&gt;
> > wrote:
> > &gt; &gt;
> > &gt; &gt; &gt; This log info just as a warning that cant
> make it disappear
> > for now,
> > &gt; but
> > &gt; &gt; &gt; will not impact anything, so you can just
> ignore it in
> > local mode.
> > &gt; &gt; &gt;
> > &gt; &gt; &gt;
> > &gt; &gt; &gt;
> ------------------&amp;nbsp;原始邮件&amp;nbsp;------------------
> > &gt; &gt; &gt; 发件人:
> > &gt; &gt;
> >
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > "user"
> > &gt; &gt;
> >
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > <
> > &gt; &gt; &gt; subharaj.manna@gmail.com&amp;gt;;
> > &gt; &gt; &gt; 发送时间:&amp;nbsp;2020年7月21日(星期二) 晚上9:19
> > &gt; &gt; &gt; 收件人:&amp;nbsp;"Hbase-User"<
> user@hbase.apache.org&amp;gt;;
> > &gt; &gt; &gt;
> > &gt; &gt; &gt; 主题:&amp;nbsp;Re: HBase 2.1.0 -
> NoSuchMethodException
> > &gt; &gt; &gt;
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> > &gt; &gt; &gt;
> > &gt; &gt; &gt;
> > &gt; &gt; &gt;
> > &gt; &gt; &gt; Thanks for replying.
> > &gt; &gt; &gt;
> > &gt; &gt; &gt; Yes it is a single node hbase cluster. I
> am not specifying
> > any storage
> > &gt; &gt; &gt; policy. Looking at the HStore
> > &gt; &gt; &gt; <
> > &gt; &gt; &gt;
> > &gt;
> >
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt
> >
> <https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt>>;
> &gt
> > <
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt&gt>
> ;;
> > &gt; &gt; ;
> > &gt; &gt; &gt; code it appears even if no storage policy
> is specified it
> > will take
> > &gt; HOT.
> > &gt; &gt; &gt;
> > &gt; &gt; &gt; Can you explain this a bit more how can I
> get around this
> > error or in a
> > &gt; &gt; &gt; single node hbase cluster I should be
> ignoring this?
> > &gt; &gt; &gt;
> > &gt; &gt; &gt;
> > &gt; &gt; &gt; On Tue, Jul 21, 2020 at 3:03 PM zheng wang
> <18031031@qq.com&amp;gt;
> > wrote:
> > &gt; &gt; &gt;
> > &gt; &gt; &gt; &amp;gt; LocalFileSystem?
> The&amp;amp;nbsp;setStoragePolicy
> > could only be used
> > &gt; in
> > &gt; &gt; &gt; &amp;gt; distributed hdfs.
> > &gt; &gt; &gt; &amp;gt; &amp;amp;nbsp;
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt; &amp;gt;
> >
> ------------------&amp;amp;nbsp;原始邮件&amp;amp;nbsp;------------------
> > &gt; &gt; &gt; &amp;gt; 发件人:
> > &gt; &gt; &gt;
> > &gt;
> >
> &amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > &gt; &gt; &gt; "user"
> > &gt; &gt; &gt;
> > &gt;
> >
> &amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > &gt; &gt; &gt; <
> > &gt; &gt; &gt; &amp;gt; subharaj.manna@gmail.com
> &amp;amp;gt;;
> > &gt; &gt; &gt; &amp;gt;
> 发送时间:&amp;amp;nbsp;2020年7月21日(星期二) 下午5:58
> > &gt; &gt; &gt; &amp;gt;
> 收件人:&amp;amp;nbsp;"Hbase-User"<
> > user@hbase.apache.org&amp;amp;gt;;
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt; &amp;gt; 主题:&amp;amp;nbsp;HBase
> 2.1.0 -
> > NoSuchMethodException
> > &gt; &gt; &gt; &amp;gt;
> > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt; &amp;gt; Hi
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt; &amp;gt; I am using HBase 2.1.0 with
> Hadoop 3.0.0. In hbase
> > master logs I
> > &gt; am
> > &gt; &gt; &gt; &amp;gt; seeing a warning like below
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt; &amp;gt; 2020-07-20 06:02:24,859
> WARN&amp;amp;nbsp;
> > [StoreOpener-1588230740-1]
> > &gt; &gt; &gt; &amp;gt; util.CommonFSUtils:
> FileSystem doesn't support
> > setStoragePolicy;
> > &gt; &gt; &gt; &amp;gt; HDFS-6584, HDFS-9345 not
> available. This is normal
> > and expected on
> > &gt; &gt; &gt; &amp;gt; earlier Hadoop versions.
> > &gt; &gt; &gt; &amp;gt;
> java.lang.NoSuchMethodException:
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> > &gt;
> >
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path,
> > &gt; &gt; &gt; &amp;gt; java.lang.String)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> java.lang.Class.getDeclaredMethod(Class.java:2130)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> > &gt;
> >
> org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:577)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> > &gt;
> >
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:558)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> > &gt;
> >
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:526)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> > &gt;
> >
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:194)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> > &gt;
> >
> org.apache.hadoop.hbase.regionserver.HStore.<init&amp;amp;gt;(HStore.java:264)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> > &gt;
> >
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> > &gt;
> org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> > &gt;
> org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> > &gt;
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> > &gt;
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt;
> >
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> > &gt; at
> > &gt; &gt; &gt; &amp;gt;
> java.lang.Thread.run(Thread.java:748)
> > &gt; &gt; &gt; &amp;gt;
> > &gt; &gt; &gt; &amp;gt; As per my understanding, this
> error should not be
> > coming with
> > &gt; Hadoop
> > &gt; &gt; &gt; &amp;gt; 3.0.0. Can someone let me
> know if my understanding
> > is correct or
> > &gt; what
> > &gt; &gt; &gt; &amp;gt; could be going wrong here?
> > &gt;
> > &gt;
> > &gt;
> > &gt; --
> > &gt; Sean
> > &gt;
回复: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
Posted by zheng wang <18...@qq.com>.
And getMethod does.
------------------ 原始邮件 ------------------
发件人: "user" <subharaj.manna@gmail.com>;
发送时间: 2020年7月22日(星期三) 晚上6:44
收件人: "Hbase-User"<user@hbase.apache.org>;
主题: Re: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
Yes I am using hadoop 3.0.0.
On Wed, Jul 22, 2020 at 12:31 PM zheng wang <18031031@qq.com> wrote:
> Are you sure you are using hadoop3.0.0?
>
>
>
>
> ------------------&nbsp;原始邮件&nbsp;------------------
> 发件人:
> "user"
> <
> subharaj.manna@gmail.com&gt;;
> 发送时间:&nbsp;2020年7月22日(星期三) 凌晨3:17
> 收件人:&nbsp;"Hbase-User"<user@hbase.apache.org&gt;;
>
> 主题:&nbsp;Re: HBase 2.1.0 - NoSuchMethodException
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
>
>
>
> Thanks for the pointer.
>
> One more query in I am getting this error from CommonFSUtils.
> invokeSetStoragePolicy()
> <
> https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533&gt
> ;.
> I am using hadoop 3.0.0 and in FilterFileSystem (which LocalFileSystem
> extends from) I do see the method&nbsp; setStoragePolicy
> <
> https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637&gt
> ;
> .
>
> Can someone explain how is this NoSuchMethodException is being thrown or I
> am looking at the wrong code path for LocalFileSystem?
>
> On Tue, Jul 21, 2020 at 7:04 PM Sean Busbey <busbey@apache.org&gt; wrote:
>
> &gt; that is the detail message of why it determined that the FileSystem
> &gt; doesn't support setStoragePolicy. As opposed to e.g. a security
> &gt; manager denying access to introspect the methods available.
> &gt;
> &gt; On Tue, Jul 21, 2020 at 8:24 AM Debraj Manna <
> subharaj.manna@gmail.com&gt;
> &gt; wrote:
> &gt; &gt;
> &gt; &gt; I understood the "util.CommonFSUtils: FileSystem doesn't support
> &gt; &gt; setStoragePolicy;" part.
> &gt; &gt;
> &gt; &gt; Can you let me know why it is saying
> "java.lang.NoSuchMethodException:
> &gt; &gt; org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy2020-07-20
> &gt; &gt; 06:02:24,859 WARN&nbsp; [StoreOpener-1588230740-1]" ?
> &gt; &gt;
> &gt; &gt;
> &gt; &gt; On Tue, Jul 21, 2020 at 6:43 PM zheng wang <18031031@qq.com&gt;
> wrote:
> &gt; &gt;
> &gt; &gt; &gt; This log info just as a warning that cant make it disappear
> for now,
> &gt; but
> &gt; &gt; &gt; will not impact anything, so you can just ignore it in
> local mode.
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt; ------------------&amp;nbsp;原始邮件&amp;nbsp;------------------
> &gt; &gt; &gt; 发件人:
> &gt; &gt;
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> "user"
> &gt; &gt;
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> <
> &gt; &gt; &gt; subharaj.manna@gmail.com&amp;gt;;
> &gt; &gt; &gt; 发送时间:&amp;nbsp;2020年7月21日(星期二) 晚上9:19
> &gt; &gt; &gt; 收件人:&amp;nbsp;"Hbase-User"<user@hbase.apache.org&amp;gt;;
> &gt; &gt; &gt;
> &gt; &gt; &gt; 主题:&amp;nbsp;Re: HBase 2.1.0 - NoSuchMethodException
> &gt; &gt; &gt; org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt; Thanks for replying.
> &gt; &gt; &gt;
> &gt; &gt; &gt; Yes it is a single node hbase cluster. I am not specifying
> any storage
> &gt; &gt; &gt; policy. Looking at the HStore
> &gt; &gt; &gt; <
> &gt; &gt; &gt;
> &gt;
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt
> &gt
> <https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&amp;gt&gt>;
> &gt; &gt; ;
> &gt; &gt; &gt; code it appears even if no storage policy is specified it
> will take
> &gt; HOT.
> &gt; &gt; &gt;
> &gt; &gt; &gt; Can you explain this a bit more how can I get around this
> error or in a
> &gt; &gt; &gt; single node hbase cluster I should be ignoring this?
> &gt; &gt; &gt;
> &gt; &gt; &gt;
> &gt; &gt; &gt; On Tue, Jul 21, 2020 at 3:03 PM zheng wang <18031031@qq.com&amp;gt;
> wrote:
> &gt; &gt; &gt;
> &gt; &gt; &gt; &amp;gt; LocalFileSystem? The&amp;amp;nbsp;setStoragePolicy
> could only be used
> &gt; in
> &gt; &gt; &gt; &amp;gt; distributed hdfs.
> &gt; &gt; &gt; &amp;gt; &amp;amp;nbsp;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> ------------------&amp;amp;nbsp;原始邮件&amp;amp;nbsp;------------------
> &gt; &gt; &gt; &amp;gt; 发件人:
> &gt; &gt; &gt;
> &gt;
> &amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> &gt; &gt; &gt; "user"
> &gt; &gt; &gt;
> &gt;
> &amp;gt;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> &gt; &gt; &gt; <
> &gt; &gt; &gt; &amp;gt; subharaj.manna@gmail.com&amp;amp;gt;;
> &gt; &gt; &gt; &amp;gt; 发送时间:&amp;amp;nbsp;2020年7月21日(星期二) 下午5:58
> &gt; &gt; &gt; &amp;gt; 收件人:&amp;amp;nbsp;"Hbase-User"<
> user@hbase.apache.org&amp;amp;gt;;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; 主题:&amp;amp;nbsp;HBase 2.1.0 -
> NoSuchMethodException
> &gt; &gt; &gt; &amp;gt;
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; Hi
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; I am using HBase 2.1.0 with Hadoop 3.0.0. In hbase
> master logs I
> &gt; am
> &gt; &gt; &gt; &amp;gt; seeing a warning like below
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; 2020-07-20 06:02:24,859 WARN&amp;amp;nbsp;
> [StoreOpener-1588230740-1]
> &gt; &gt; &gt; &amp;gt; util.CommonFSUtils: FileSystem doesn't support
> setStoragePolicy;
> &gt; &gt; &gt; &amp;gt; HDFS-6584, HDFS-9345 not available. This is normal
> and expected on
> &gt; &gt; &gt; &amp;gt; earlier Hadoop versions.
> &gt; &gt; &gt; &amp;gt; java.lang.NoSuchMethodException:
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path,
> &gt; &gt; &gt; &amp;gt; java.lang.String)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt; java.lang.Class.getDeclaredMethod(Class.java:2130)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:577)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:558)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:526)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:194)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.regionserver.HStore.<init&amp;amp;gt;(HStore.java:264)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &gt;
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt;
> &amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;&amp;amp;nbsp;
> &gt; at
> &gt; &gt; &gt; &amp;gt; java.lang.Thread.run(Thread.java:748)
> &gt; &gt; &gt; &amp;gt;
> &gt; &gt; &gt; &amp;gt; As per my understanding, this error should not be
> coming with
> &gt; Hadoop
> &gt; &gt; &gt; &amp;gt; 3.0.0. Can someone let me know if my understanding
> is correct or
> &gt; what
> &gt; &gt; &gt; &amp;gt; could be going wrong here?
> &gt;
> &gt;
> &gt;
> &gt; --
> &gt; Sean
> &gt;
Re: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
Posted by Debraj Manna <su...@gmail.com>.
Yes I am using hadoop 3.0.0.
On Wed, Jul 22, 2020 at 12:31 PM zheng wang <18...@qq.com> wrote:
> Are you sure you are using hadoop3.0.0?
>
>
>
>
> ------------------ 原始邮件 ------------------
> 发件人:
> "user"
> <
> subharaj.manna@gmail.com>;
> 发送时间: 2020年7月22日(星期三) 凌晨3:17
> 收件人: "Hbase-User"<user@hbase.apache.org>;
>
> 主题: Re: HBase 2.1.0 - NoSuchMethodException
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
>
>
>
> Thanks for the pointer.
>
> One more query in I am getting this error from CommonFSUtils.
> invokeSetStoragePolicy()
> <
> https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533>
> ;.
> I am using hadoop 3.0.0 and in FilterFileSystem (which LocalFileSystem
> extends from) I do see the method setStoragePolicy
> <
> https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637>
> ;
> .
>
> Can someone explain how is this NoSuchMethodException is being thrown or I
> am looking at the wrong code path for LocalFileSystem?
>
> On Tue, Jul 21, 2020 at 7:04 PM Sean Busbey <busbey@apache.org> wrote:
>
> > that is the detail message of why it determined that the FileSystem
> > doesn't support setStoragePolicy. As opposed to e.g. a security
> > manager denying access to introspect the methods available.
> >
> > On Tue, Jul 21, 2020 at 8:24 AM Debraj Manna <
> subharaj.manna@gmail.com>
> > wrote:
> > >
> > > I understood the "util.CommonFSUtils: FileSystem doesn't support
> > > setStoragePolicy;" part.
> > >
> > > Can you let me know why it is saying
> "java.lang.NoSuchMethodException:
> > > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy2020-07-20
> > > 06:02:24,859 WARN [StoreOpener-1588230740-1]" ?
> > >
> > >
> > > On Tue, Jul 21, 2020 at 6:43 PM zheng wang <18031031@qq.com>
> wrote:
> > >
> > > > This log info just as a warning that cant make it disappear
> for now,
> > but
> > > > will not impact anything, so you can just ignore it in
> local mode.
> > > >
> > > >
> > > > ------------------&nbsp;原始邮件&nbsp;------------------
> > > > 发件人:
> > >
> >
> "user"
> > >
> >
> <
> > > > subharaj.manna@gmail.com&gt;;
> > > > 发送时间:&nbsp;2020年7月21日(星期二) 晚上9:19
> > > > 收件人:&nbsp;"Hbase-User"<user@hbase.apache.org&gt;;
> > > >
> > > > 主题:&nbsp;Re: HBase 2.1.0 - NoSuchMethodException
> > > > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> > > >
> > > >
> > > >
> > > > Thanks for replying.
> > > >
> > > > Yes it is a single node hbase cluster. I am not specifying
> any storage
> > > > policy. Looking at the HStore
> > > > <
> > > >
> >
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&gt
> >
> <https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&gt>>;
> > > ;
> > > > code it appears even if no storage policy is specified it
> will take
> > HOT.
> > > >
> > > > Can you explain this a bit more how can I get around this
> error or in a
> > > > single node hbase cluster I should be ignoring this?
> > > >
> > > >
> > > > On Tue, Jul 21, 2020 at 3:03 PM zheng wang <18031031@qq.com&gt;
> wrote:
> > > >
> > > > &gt; LocalFileSystem? The&amp;nbsp;setStoragePolicy
> could only be used
> > in
> > > > &gt; distributed hdfs.
> > > > &gt; &amp;nbsp;
> > > > &gt;
> > > > &gt;
> > > > &gt;
> ------------------&amp;nbsp;原始邮件&amp;nbsp;------------------
> > > > &gt; 发件人:
> > > >
> >
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > > > "user"
> > > >
> >
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > > > <
> > > > &gt; subharaj.manna@gmail.com&amp;gt;;
> > > > &gt; 发送时间:&amp;nbsp;2020年7月21日(星期二) 下午5:58
> > > > &gt; 收件人:&amp;nbsp;"Hbase-User"<
> user@hbase.apache.org&amp;gt;;
> > > > &gt;
> > > > &gt; 主题:&amp;nbsp;HBase 2.1.0 -
> NoSuchMethodException
> > > > &gt;
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> > > > &gt;
> > > > &gt;
> > > > &gt;
> > > > &gt; Hi
> > > > &gt;
> > > > &gt; I am using HBase 2.1.0 with Hadoop 3.0.0. In hbase
> master logs I
> > am
> > > > &gt; seeing a warning like below
> > > > &gt;
> > > > &gt; 2020-07-20 06:02:24,859 WARN&amp;nbsp;
> [StoreOpener-1588230740-1]
> > > > &gt; util.CommonFSUtils: FileSystem doesn't support
> setStoragePolicy;
> > > > &gt; HDFS-6584, HDFS-9345 not available. This is normal
> and expected on
> > > > &gt; earlier Hadoop versions.
> > > > &gt; java.lang.NoSuchMethodException:
> > > > &gt;
> > > > &gt;
> > > >
> >
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path,
> > > > &gt; java.lang.String)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt; java.lang.Class.getDeclaredMethod(Class.java:2130)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt;
> > > >
> >
> org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:577)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt;
> > > >
> >
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:558)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt;
> > > >
> >
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:526)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt;
> > > >
> >
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:194)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt;
> > > >
> >
> org.apache.hadoop.hbase.regionserver.HStore.<init&amp;gt;(HStore.java:264)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt;
> > > >
> >
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt;
> > org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt;
> > org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt;
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt;
> > > >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt;
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt;
> > > >
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt;
> > > >
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > &gt;
> > > >
> &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> > at
> > > > &gt; java.lang.Thread.run(Thread.java:748)
> > > > &gt;
> > > > &gt; As per my understanding, this error should not be
> coming with
> > Hadoop
> > > > &gt; 3.0.0. Can someone let me know if my understanding
> is correct or
> > what
> > > > &gt; could be going wrong here?
> >
> >
> >
> > --
> > Sean
> >
回复: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
Posted by zheng wang <18...@qq.com>.
Are you sure you are using hadoop3.0.0?
------------------ 原始邮件 ------------------
发件人: "user" <subharaj.manna@gmail.com>;
发送时间: 2020年7月22日(星期三) 凌晨3:17
收件人: "Hbase-User"<user@hbase.apache.org>;
主题: Re: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
Thanks for the pointer.
One more query in I am getting this error from CommonFSUtils.
invokeSetStoragePolicy()
<https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533>.
I am using hadoop 3.0.0 and in FilterFileSystem (which LocalFileSystem
extends from) I do see the method setStoragePolicy
<https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637>
.
Can someone explain how is this NoSuchMethodException is being thrown or I
am looking at the wrong code path for LocalFileSystem?
On Tue, Jul 21, 2020 at 7:04 PM Sean Busbey <busbey@apache.org> wrote:
> that is the detail message of why it determined that the FileSystem
> doesn't support setStoragePolicy. As opposed to e.g. a security
> manager denying access to introspect the methods available.
>
> On Tue, Jul 21, 2020 at 8:24 AM Debraj Manna <subharaj.manna@gmail.com>
> wrote:
> >
> > I understood the "util.CommonFSUtils: FileSystem doesn't support
> > setStoragePolicy;" part.
> >
> > Can you let me know why it is saying "java.lang.NoSuchMethodException:
> > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy2020-07-20
> > 06:02:24,859 WARN [StoreOpener-1588230740-1]" ?
> >
> >
> > On Tue, Jul 21, 2020 at 6:43 PM zheng wang <18031031@qq.com> wrote:
> >
> > > This log info just as a warning that cant make it disappear for now,
> but
> > > will not impact anything, so you can just ignore it in local mode.
> > >
> > >
> > > ------------------&nbsp;原始邮件&nbsp;------------------
> > > 发件人:
> > > "user"
> > > <
> > > subharaj.manna@gmail.com&gt;;
> > > 发送时间:&nbsp;2020年7月21日(星期二) 晚上9:19
> > > 收件人:&nbsp;"Hbase-User"<user@hbase.apache.org&gt;;
> > >
> > > 主题:&nbsp;Re: HBase 2.1.0 - NoSuchMethodException
> > > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> > >
> > >
> > >
> > > Thanks for replying.
> > >
> > > Yes it is a single node hbase cluster. I am not specifying any storage
> > > policy. Looking at the HStore
> > > <
> > >
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274&gt
> > > ;
> > > code it appears even if no storage policy is specified it will take
> HOT.
> > >
> > > Can you explain this a bit more how can I get around this error or in a
> > > single node hbase cluster I should be ignoring this?
> > >
> > >
> > > On Tue, Jul 21, 2020 at 3:03 PM zheng wang <18031031@qq.com&gt; wrote:
> > >
> > > &gt; LocalFileSystem? The&amp;nbsp;setStoragePolicy could only be used
> in
> > > &gt; distributed hdfs.
> > > &gt; &amp;nbsp;
> > > &gt;
> > > &gt;
> > > &gt; ------------------&amp;nbsp;原始邮件&amp;nbsp;------------------
> > > &gt; 发件人:
> > >
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > > "user"
> > >
> &gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> > > <
> > > &gt; subharaj.manna@gmail.com&amp;gt;;
> > > &gt; 发送时间:&amp;nbsp;2020年7月21日(星期二) 下午5:58
> > > &gt; 收件人:&amp;nbsp;"Hbase-User"<user@hbase.apache.org&amp;gt;;
> > > &gt;
> > > &gt; 主题:&amp;nbsp;HBase 2.1.0 - NoSuchMethodException
> > > &gt; org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> > > &gt;
> > > &gt;
> > > &gt;
> > > &gt; Hi
> > > &gt;
> > > &gt; I am using HBase 2.1.0 with Hadoop 3.0.0. In hbase master logs I
> am
> > > &gt; seeing a warning like below
> > > &gt;
> > > &gt; 2020-07-20 06:02:24,859 WARN&amp;nbsp; [StoreOpener-1588230740-1]
> > > &gt; util.CommonFSUtils: FileSystem doesn't support setStoragePolicy;
> > > &gt; HDFS-6584, HDFS-9345 not available. This is normal and expected on
> > > &gt; earlier Hadoop versions.
> > > &gt; java.lang.NoSuchMethodException:
> > > &gt;
> > > &gt;
> > >
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path,
> > > &gt; java.lang.String)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt; java.lang.Class.getDeclaredMethod(Class.java:2130)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt;
> > >
> org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:577)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt;
> > >
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:558)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt;
> > >
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:526)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt;
> > >
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:194)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt;
> > >
> org.apache.hadoop.hbase.regionserver.HStore.<init&amp;gt;(HStore.java:264)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt;
> > >
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt;
> org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt;
> org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt; java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt;
> > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt; java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt;
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt;
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > &gt;
> > > &amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;&amp;nbsp;
> at
> > > &gt; java.lang.Thread.run(Thread.java:748)
> > > &gt;
> > > &gt; As per my understanding, this error should not be coming with
> Hadoop
> > > &gt; 3.0.0. Can someone let me know if my understanding is correct or
> what
> > > &gt; could be going wrong here?
>
>
>
> --
> Sean
>
Re: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
Posted by Debraj Manna <su...@gmail.com>.
Thanks for the pointer.
One more query in I am getting this error from CommonFSUtils.
invokeSetStoragePolicy()
<https://github.com/apache/hbase/blob/branch-2.1/hbase-common/src/main/java/org/apache/hadoop/hbase/util/CommonFSUtils.java#L533>.
I am using hadoop 3.0.0 and in FilterFileSystem (which LocalFileSystem
extends from) I do see the method setStoragePolicy
<https://github.com/apache/hadoop/blob/release-3.0.0-RC1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FilterFileSystem.java#L637>
.
Can someone explain how is this NoSuchMethodException is being thrown or I
am looking at the wrong code path for LocalFileSystem?
On Tue, Jul 21, 2020 at 7:04 PM Sean Busbey <bu...@apache.org> wrote:
> that is the detail message of why it determined that the FileSystem
> doesn't support setStoragePolicy. As opposed to e.g. a security
> manager denying access to introspect the methods available.
>
> On Tue, Jul 21, 2020 at 8:24 AM Debraj Manna <su...@gmail.com>
> wrote:
> >
> > I understood the "util.CommonFSUtils: FileSystem doesn't support
> > setStoragePolicy;" part.
> >
> > Can you let me know why it is saying "java.lang.NoSuchMethodException:
> > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy2020-07-20
> > 06:02:24,859 WARN [StoreOpener-1588230740-1]" ?
> >
> >
> > On Tue, Jul 21, 2020 at 6:43 PM zheng wang <18...@qq.com> wrote:
> >
> > > This log info just as a warning that cant make it disappear for now,
> but
> > > will not impact anything, so you can just ignore it in local mode.
> > >
> > >
> > > ------------------ 原始邮件 ------------------
> > > 发件人:
> > > "user"
> > > <
> > > subharaj.manna@gmail.com>;
> > > 发送时间: 2020年7月21日(星期二) 晚上9:19
> > > 收件人: "Hbase-User"<user@hbase.apache.org>;
> > >
> > > 主题: Re: HBase 2.1.0 - NoSuchMethodException
> > > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> > >
> > >
> > >
> > > Thanks for replying.
> > >
> > > Yes it is a single node hbase cluster. I am not specifying any storage
> > > policy. Looking at the HStore
> > > <
> > >
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274>
> > > ;
> > > code it appears even if no storage policy is specified it will take
> HOT.
> > >
> > > Can you explain this a bit more how can I get around this error or in a
> > > single node hbase cluster I should be ignoring this?
> > >
> > >
> > > On Tue, Jul 21, 2020 at 3:03 PM zheng wang <18031031@qq.com> wrote:
> > >
> > > > LocalFileSystem? The&nbsp;setStoragePolicy could only be used
> in
> > > > distributed hdfs.
> > > > &nbsp;
> > > >
> > > >
> > > > ------------------&nbsp;原始邮件&nbsp;------------------
> > > > 发件人:
> > >
> >
> > > "user"
> > >
> >
> > > <
> > > > subharaj.manna@gmail.com&gt;;
> > > > 发送时间:&nbsp;2020年7月21日(星期二) 下午5:58
> > > > 收件人:&nbsp;"Hbase-User"<user@hbase.apache.org&gt;;
> > > >
> > > > 主题:&nbsp;HBase 2.1.0 - NoSuchMethodException
> > > > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> > > >
> > > >
> > > >
> > > > Hi
> > > >
> > > > I am using HBase 2.1.0 with Hadoop 3.0.0. In hbase master logs I
> am
> > > > seeing a warning like below
> > > >
> > > > 2020-07-20 06:02:24,859 WARN&nbsp; [StoreOpener-1588230740-1]
> > > > util.CommonFSUtils: FileSystem doesn't support setStoragePolicy;
> > > > HDFS-6584, HDFS-9345 not available. This is normal and expected on
> > > > earlier Hadoop versions.
> > > > java.lang.NoSuchMethodException:
> > > >
> > > >
> > >
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path,
> > > > java.lang.String)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > > java.lang.Class.getDeclaredMethod(Class.java:2130)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > >
> > >
> org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:577)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > >
> > >
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:558)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > >
> > >
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:526)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > >
> > >
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:194)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > >
> > >
> org.apache.hadoop.hbase.regionserver.HStore.<init&gt;(HStore.java:264)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > >
> > >
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > >
> org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > >
> org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > >
> > > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > >
> > > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
> at
> > > > java.lang.Thread.run(Thread.java:748)
> > > >
> > > > As per my understanding, this error should not be coming with
> Hadoop
> > > > 3.0.0. Can someone let me know if my understanding is correct or
> what
> > > > could be going wrong here?
>
>
>
> --
> Sean
>
Re: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
Posted by Sean Busbey <bu...@apache.org>.
that is the detail message of why it determined that the FileSystem
doesn't support setStoragePolicy. As opposed to e.g. a security
manager denying access to introspect the methods available.
On Tue, Jul 21, 2020 at 8:24 AM Debraj Manna <su...@gmail.com> wrote:
>
> I understood the "util.CommonFSUtils: FileSystem doesn't support
> setStoragePolicy;" part.
>
> Can you let me know why it is saying "java.lang.NoSuchMethodException:
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy2020-07-20
> 06:02:24,859 WARN [StoreOpener-1588230740-1]" ?
>
>
> On Tue, Jul 21, 2020 at 6:43 PM zheng wang <18...@qq.com> wrote:
>
> > This log info just as a warning that cant make it disappear for now, but
> > will not impact anything, so you can just ignore it in local mode.
> >
> >
> > ------------------ 原始邮件 ------------------
> > 发件人:
> > "user"
> > <
> > subharaj.manna@gmail.com>;
> > 发送时间: 2020年7月21日(星期二) 晚上9:19
> > 收件人: "Hbase-User"<user@hbase.apache.org>;
> >
> > 主题: Re: HBase 2.1.0 - NoSuchMethodException
> > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> >
> >
> >
> > Thanks for replying.
> >
> > Yes it is a single node hbase cluster. I am not specifying any storage
> > policy. Looking at the HStore
> > <
> > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274>
> > ;
> > code it appears even if no storage policy is specified it will take HOT.
> >
> > Can you explain this a bit more how can I get around this error or in a
> > single node hbase cluster I should be ignoring this?
> >
> >
> > On Tue, Jul 21, 2020 at 3:03 PM zheng wang <18031031@qq.com> wrote:
> >
> > > LocalFileSystem? The&nbsp;setStoragePolicy could only be used in
> > > distributed hdfs.
> > > &nbsp;
> > >
> > >
> > > ------------------&nbsp;原始邮件&nbsp;------------------
> > > 发件人:
> > >
> > "user"
> > >
> > <
> > > subharaj.manna@gmail.com&gt;;
> > > 发送时间:&nbsp;2020年7月21日(星期二) 下午5:58
> > > 收件人:&nbsp;"Hbase-User"<user@hbase.apache.org&gt;;
> > >
> > > 主题:&nbsp;HBase 2.1.0 - NoSuchMethodException
> > > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> > >
> > >
> > >
> > > Hi
> > >
> > > I am using HBase 2.1.0 with Hadoop 3.0.0. In hbase master logs I am
> > > seeing a warning like below
> > >
> > > 2020-07-20 06:02:24,859 WARN&nbsp; [StoreOpener-1588230740-1]
> > > util.CommonFSUtils: FileSystem doesn't support setStoragePolicy;
> > > HDFS-6584, HDFS-9345 not available. This is normal and expected on
> > > earlier Hadoop versions.
> > > java.lang.NoSuchMethodException:
> > >
> > >
> > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path,
> > > java.lang.String)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > > java.lang.Class.getDeclaredMethod(Class.java:2130)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > >
> > org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:577)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > >
> > org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:558)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > >
> > org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:526)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > >
> > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:194)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > >
> > org.apache.hadoop.hbase.regionserver.HStore.<init&gt;(HStore.java:264)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > >
> > org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > > org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > > org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > >
> > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > >
> > &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > > java.lang.Thread.run(Thread.java:748)
> > >
> > > As per my understanding, this error should not be coming with Hadoop
> > > 3.0.0. Can someone let me know if my understanding is correct or what
> > > could be going wrong here?
--
Sean
Re: HBase 2.1.0 - NoSuchMethodException org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
Posted by Debraj Manna <su...@gmail.com>.
I understood the "util.CommonFSUtils: FileSystem doesn't support
setStoragePolicy;" part.
Can you let me know why it is saying "java.lang.NoSuchMethodException:
org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy2020-07-20
06:02:24,859 WARN [StoreOpener-1588230740-1]" ?
On Tue, Jul 21, 2020 at 6:43 PM zheng wang <18...@qq.com> wrote:
> This log info just as a warning that cant make it disappear for now, but
> will not impact anything, so you can just ignore it in local mode.
>
>
> ------------------ 原始邮件 ------------------
> 发件人:
> "user"
> <
> subharaj.manna@gmail.com>;
> 发送时间: 2020年7月21日(星期二) 晚上9:19
> 收件人: "Hbase-User"<user@hbase.apache.org>;
>
> 主题: Re: HBase 2.1.0 - NoSuchMethodException
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
>
>
>
> Thanks for replying.
>
> Yes it is a single node hbase cluster. I am not specifying any storage
> policy. Looking at the HStore
> <
> https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java#L274>
> ;
> code it appears even if no storage policy is specified it will take HOT.
>
> Can you explain this a bit more how can I get around this error or in a
> single node hbase cluster I should be ignoring this?
>
>
> On Tue, Jul 21, 2020 at 3:03 PM zheng wang <18031031@qq.com> wrote:
>
> > LocalFileSystem? The&nbsp;setStoragePolicy could only be used in
> > distributed hdfs.
> > &nbsp;
> >
> >
> > ------------------&nbsp;原始邮件&nbsp;------------------
> > 发件人:
> >
> "user"
> >
> <
> > subharaj.manna@gmail.com&gt;;
> > 发送时间:&nbsp;2020年7月21日(星期二) 下午5:58
> > 收件人:&nbsp;"Hbase-User"<user@hbase.apache.org&gt;;
> >
> > 主题:&nbsp;HBase 2.1.0 - NoSuchMethodException
> > org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy
> >
> >
> >
> > Hi
> >
> > I am using HBase 2.1.0 with Hadoop 3.0.0. In hbase master logs I am
> > seeing a warning like below
> >
> > 2020-07-20 06:02:24,859 WARN&nbsp; [StoreOpener-1588230740-1]
> > util.CommonFSUtils: FileSystem doesn't support setStoragePolicy;
> > HDFS-6584, HDFS-9345 not available. This is normal and expected on
> > earlier Hadoop versions.
> > java.lang.NoSuchMethodException:
> >
> >
> org.apache.hadoop.fs.LocalFileSystem.setStoragePolicy(org.apache.hadoop.fs.Path,
> > java.lang.String)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > java.lang.Class.getDeclaredMethod(Class.java:2130)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> >
> org.apache.hadoop.hbase.util.CommonFSUtils.invokeSetStoragePolicy(CommonFSUtils.java:577)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> >
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:558)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> >
> org.apache.hadoop.hbase.util.CommonFSUtils.setStoragePolicy(CommonFSUtils.java:526)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> >
> org.apache.hadoop.hbase.regionserver.HRegionFileSystem.setStoragePolicy(HRegionFileSystem.java:194)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> >
> org.apache.hadoop.hbase.regionserver.HStore.<init&gt;(HStore.java:264)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> >
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:5731)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1059)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:1056)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> >
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> >
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at
> > java.lang.Thread.run(Thread.java:748)
> >
> > As per my understanding, this error should not be coming with Hadoop
> > 3.0.0. Can someone let me know if my understanding is correct or what
> > could be going wrong here?