You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Ming Ma (JIRA)" <ji...@apache.org> on 2011/08/10 03:47:27 UTC
[jira] [Created] (HBASE-4184) CatalogJanitor doesn't work properly
when "fs.default.name" isn't set in config file.
CatalogJanitor doesn't work properly when "fs.default.name" isn't set in config file.
-------------------------------------------------------------------------------------
Key: HBASE-4184
URL: https://issues.apache.org/jira/browse/HBASE-4184
Project: HBase
Issue Type: Bug
Components: master
Reporter: Ming Ma
Assignee: Ming Ma
In our system, hbase.rootdir is set to a hdfs path and hbase can figure out the FileSystem and set "fs.default.name" accordingly on the Configuration object and pass around including to RS. That is handled in HMaster.java and MasterFileSystem.java.
CatalogJanitor uses deprecated HRegionInfo.getTableDesc. The method creates a default configuration and get FileSystem from there. That will be RawLocalFileSystem. It returns the following exception.
java.lang.IllegalArgumentException: Wrong FS: hdfs://sea-esxi-0:54310/tmp/hbase/
testtb/.tableinfo, expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:454)
at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:67)
at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:307)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1085)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1110)
at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:494)
at org.apache.hadoop.hbase.util.FSUtils.getTableInfoModtime(FSUtils.java:833)
at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:127)
at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:99)
at org.apache.hadoop.hbase.HRegionInfo.getTableDesc(HRegionInfo.java:560)
at org.apache.hadoop.hbase.master.CatalogJanitor$1.compare(CatalogJanitor.java:118)
at org.apache.hadoop.hbase.master.CatalogJanitor$1.compare(CatalogJanitor.java:110)
at java.util.TreeMap.put(TreeMap.java:530) at org.apache.hadoop.hbase.master.CatalogJanitor$2.visit(CatalogJanitor.java:138)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4184) CatalogJanitor doesn't work properly
when "fs.default.name" isn't set in config file.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-4184.
--------------------------
Resolution: Fixed
Fix Version/s: 0.92.0
Hadoop Flags: [Reviewed]
Committed to TRUNK. Thanks for the patch Ming.
> CatalogJanitor doesn't work properly when "fs.default.name" isn't set in config file.
> -------------------------------------------------------------------------------------
>
> Key: HBASE-4184
> URL: https://issues.apache.org/jira/browse/HBASE-4184
> Project: HBase
> Issue Type: Bug
> Components: master
> Reporter: Ming Ma
> Assignee: Ming Ma
> Fix For: 0.92.0
>
> Attachments: HBASE-4184-trunk.patch
>
>
> In our system, hbase.rootdir is set to a hdfs path and hbase can figure out the FileSystem and set "fs.default.name" accordingly on the Configuration object and pass around including to RS. That is handled in HMaster.java and MasterFileSystem.java.
> CatalogJanitor uses deprecated HRegionInfo.getTableDesc. The method creates a default configuration and get FileSystem from there. That will be RawLocalFileSystem. It returns the following exception.
> java.lang.IllegalArgumentException: Wrong FS: hdfs://sea-esxi-0:54310/tmp/hbase/
> testtb/.tableinfo, expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:454)
> at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:67)
> at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:307)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1085)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1110)
> at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:494)
> at org.apache.hadoop.hbase.util.FSUtils.getTableInfoModtime(FSUtils.java:833)
> at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:127)
> at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:99)
> at org.apache.hadoop.hbase.HRegionInfo.getTableDesc(HRegionInfo.java:560)
> at org.apache.hadoop.hbase.master.CatalogJanitor$1.compare(CatalogJanitor.java:118)
> at org.apache.hadoop.hbase.master.CatalogJanitor$1.compare(CatalogJanitor.java:110)
> at java.util.TreeMap.put(TreeMap.java:530) at org.apache.hadoop.hbase.master.CatalogJanitor$2.visit(CatalogJanitor.java:138)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4184) CatalogJanitor doesn't work properly
when "fs.default.name" isn't set in config file.
Posted by "Ming Ma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ming Ma updated HBASE-4184:
---------------------------
Attachment: HBASE-4184-trunk.patch
Change CatalogJanitor.java to use HRegionInfo.getTableName() instead.
Should we actually remove HRegionInfo.getTableDesc()? I know it isn't backward compatible. But the function doesn't seem provide much value.
> CatalogJanitor doesn't work properly when "fs.default.name" isn't set in config file.
> -------------------------------------------------------------------------------------
>
> Key: HBASE-4184
> URL: https://issues.apache.org/jira/browse/HBASE-4184
> Project: HBase
> Issue Type: Bug
> Components: master
> Reporter: Ming Ma
> Assignee: Ming Ma
> Attachments: HBASE-4184-trunk.patch
>
>
> In our system, hbase.rootdir is set to a hdfs path and hbase can figure out the FileSystem and set "fs.default.name" accordingly on the Configuration object and pass around including to RS. That is handled in HMaster.java and MasterFileSystem.java.
> CatalogJanitor uses deprecated HRegionInfo.getTableDesc. The method creates a default configuration and get FileSystem from there. That will be RawLocalFileSystem. It returns the following exception.
> java.lang.IllegalArgumentException: Wrong FS: hdfs://sea-esxi-0:54310/tmp/hbase/
> testtb/.tableinfo, expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:454)
> at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:67)
> at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:307)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1085)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1110)
> at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:494)
> at org.apache.hadoop.hbase.util.FSUtils.getTableInfoModtime(FSUtils.java:833)
> at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:127)
> at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:99)
> at org.apache.hadoop.hbase.HRegionInfo.getTableDesc(HRegionInfo.java:560)
> at org.apache.hadoop.hbase.master.CatalogJanitor$1.compare(CatalogJanitor.java:118)
> at org.apache.hadoop.hbase.master.CatalogJanitor$1.compare(CatalogJanitor.java:110)
> at java.util.TreeMap.put(TreeMap.java:530) at org.apache.hadoop.hbase.master.CatalogJanitor$2.visit(CatalogJanitor.java:138)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4184) CatalogJanitor doesn't work
properly when "fs.default.name" isn't set in config file.
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082092#comment-13082092 ]
Ted Yu commented on HBASE-4184:
-------------------------------
+1 on patch.
Andrew argued that HRegionInfo.getTableDesc() should be kept for one major release before being removed.
> CatalogJanitor doesn't work properly when "fs.default.name" isn't set in config file.
> -------------------------------------------------------------------------------------
>
> Key: HBASE-4184
> URL: https://issues.apache.org/jira/browse/HBASE-4184
> Project: HBase
> Issue Type: Bug
> Components: master
> Reporter: Ming Ma
> Assignee: Ming Ma
> Attachments: HBASE-4184-trunk.patch
>
>
> In our system, hbase.rootdir is set to a hdfs path and hbase can figure out the FileSystem and set "fs.default.name" accordingly on the Configuration object and pass around including to RS. That is handled in HMaster.java and MasterFileSystem.java.
> CatalogJanitor uses deprecated HRegionInfo.getTableDesc. The method creates a default configuration and get FileSystem from there. That will be RawLocalFileSystem. It returns the following exception.
> java.lang.IllegalArgumentException: Wrong FS: hdfs://sea-esxi-0:54310/tmp/hbase/
> testtb/.tableinfo, expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:454)
> at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:67)
> at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:307)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1085)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1110)
> at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:494)
> at org.apache.hadoop.hbase.util.FSUtils.getTableInfoModtime(FSUtils.java:833)
> at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:127)
> at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:99)
> at org.apache.hadoop.hbase.HRegionInfo.getTableDesc(HRegionInfo.java:560)
> at org.apache.hadoop.hbase.master.CatalogJanitor$1.compare(CatalogJanitor.java:118)
> at org.apache.hadoop.hbase.master.CatalogJanitor$1.compare(CatalogJanitor.java:110)
> at java.util.TreeMap.put(TreeMap.java:530) at org.apache.hadoop.hbase.master.CatalogJanitor$2.visit(CatalogJanitor.java:138)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4184) CatalogJanitor doesn't work
properly when "fs.default.name" isn't set in config file.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082849#comment-13082849 ]
Hudson commented on HBASE-4184:
-------------------------------
Integrated in HBase-TRUNK #2108 (See [https://builds.apache.org/job/HBase-TRUNK/2108/])
HBASE-4184 CatalogJanitor doesn't work properly when fs.default.name isn't set in config file.
stack :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
* /hbase/trunk/CHANGES.txt
> CatalogJanitor doesn't work properly when "fs.default.name" isn't set in config file.
> -------------------------------------------------------------------------------------
>
> Key: HBASE-4184
> URL: https://issues.apache.org/jira/browse/HBASE-4184
> Project: HBase
> Issue Type: Bug
> Components: master
> Reporter: Ming Ma
> Assignee: Ming Ma
> Fix For: 0.92.0
>
> Attachments: HBASE-4184-trunk.patch
>
>
> In our system, hbase.rootdir is set to a hdfs path and hbase can figure out the FileSystem and set "fs.default.name" accordingly on the Configuration object and pass around including to RS. That is handled in HMaster.java and MasterFileSystem.java.
> CatalogJanitor uses deprecated HRegionInfo.getTableDesc. The method creates a default configuration and get FileSystem from there. That will be RawLocalFileSystem. It returns the following exception.
> java.lang.IllegalArgumentException: Wrong FS: hdfs://sea-esxi-0:54310/tmp/hbase/
> testtb/.tableinfo, expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:454)
> at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:67)
> at org.apache.hadoop.fs.RawLocalFileSystem.listStatus(RawLocalFileSystem.java:307)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1085)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1110)
> at org.apache.hadoop.fs.ChecksumFileSystem.listStatus(ChecksumFileSystem.java:494)
> at org.apache.hadoop.hbase.util.FSUtils.getTableInfoModtime(FSUtils.java:833)
> at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:127)
> at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:99)
> at org.apache.hadoop.hbase.HRegionInfo.getTableDesc(HRegionInfo.java:560)
> at org.apache.hadoop.hbase.master.CatalogJanitor$1.compare(CatalogJanitor.java:118)
> at org.apache.hadoop.hbase.master.CatalogJanitor$1.compare(CatalogJanitor.java:110)
> at java.util.TreeMap.put(TreeMap.java:530) at org.apache.hadoop.hbase.master.CatalogJanitor$2.visit(CatalogJanitor.java:138)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira