You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "stack (JIRA)" <ji...@apache.org> on 2009/06/26 00:52:07 UTC
[jira] Resolved: (HBASE-1092) FileNotFoundException trying to get
index file size for metrics
[ https://issues.apache.org/jira/browse/HBASE-1092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack resolved HBASE-1092.
--------------------------
Resolution: Cannot Reproduce
Haven't seen since. Resolving.
> FileNotFoundException trying to get index file size for metrics
> ---------------------------------------------------------------
>
> Key: HBASE-1092
> URL: https://issues.apache.org/jira/browse/HBASE-1092
> Project: Hadoop HBase
> Issue Type: Bug
> Environment: apurtell cluster; trunk on hadoop 0.18.1.
> Reporter: stack
> Assignee: stack
>
> This is an odd one. We open a region and load up its store files. Part of loading store files is confirming presence of MapFile index files (reconstituting them even if missing). The below log is of region open and then seconds later, failing to find the index files when we go to look at them for sake of metrics:
> {code}
> 2008-12-28 00:06:19,330 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Worker: MSG_REGION_OPEN: content,10a1c144cf729885001e71a5ff5108dc,1230416158498
> 2008-12-28 00:06:19,330 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Opening region content,10a1c144cf729885001e71a5ff5108dc,1230416158498/2030495720
> 2008-12-28 00:06:19,337 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/url/info/4139998553412763261, isReference=false, sequence id=13310628, length=54275, majorCompaction=false
> 2008-12-28 00:06:19,368 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/url/info/4467064002967944271, isReference=false, sequence id=9760762, length=432827, majorCompaction=false
> 2008-12-28 00:06:19,373 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/url/info/5563124412728188459, isReference=false, sequence id=12406630, length=22596, majorCompaction=false
> 2008-12-28 00:06:19,379 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/url/info/7040855870376599550, isReference=false, sequence id=12795530, length=4163, majorCompaction=false
> 2008-12-28 00:06:19,379 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 4 file(s) in hstore 2030495720/url, max sequence id 13310628
> 2008-12-28 00:06:19,496 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Applied 0, skipped 2078 because sequence id <= 13310628
> 2008-12-28 00:06:19,687 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/info/info/4076087643455354411, isReference=false, sequence id=9760762, length=1491212, majorCompaction=false
> 2008-12-28 00:06:19,691 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/info/info/4178444212265859440, isReference=false, sequence id=13310628, length=156148, majorCompaction=false
> 2008-12-28 00:06:19,697 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/info/info/7223110203696352566, isReference=false, sequence id=12406630, length=84614, majorCompaction=false
> 2008-12-28 00:06:19,703 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/info/info/8629305049543986640, isReference=false, sequence id=12795530, length=9293, majorCompaction=false
> 2008-12-28 00:06:19,704 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 4 file(s) in hstore 2030495720/info, max sequence id 13310628
> 2008-12-28 00:06:19,773 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /data/hbase/content/compaction.dir/888098363/content/mapfiles/3159110292991346789 to /data/hbase/content/888098363/content/mapfiles/6086595812879433437
> 2008-12-28 00:06:19,787 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 888098363/content store size is 172.4m
> 2008-12-28 00:06:19,791 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Compaction size of 888098363/info: 1.4m; Skipped 2 file(s), size: 1221094
> 2008-12-28 00:06:19,801 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Started compaction of 2 file(s) into /data/hbase/content/compaction.dir/888098363/info/mapfiles/2317624608256855622
> 2008-12-28 00:06:19,821 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Applied 0, skipped 2078 because sequence id <= 13310628
> 2008-12-28 00:06:19,879 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /data/hbase/content/compaction.dir/888098363/info/mapfiles/2317624608256855622 to /data/hbase/content/888098363/info/mapfiles/2738033219360665217
> 2008-12-28 00:06:19,883 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/content/info/2253403598042963153, isReference=false, sequence id=13310628, length=12491312, majorCompaction=false
> 2008-12-28 00:06:19,887 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/content/info/4596284995168293361, isReference=false, sequence id=9760762, length=158138153, majorCompaction=false
> 2008-12-28 00:06:19,896 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 888098363/info store size is 1.4m
> 2008-12-28 00:06:19,898 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Compaction size of 888098363/url: 405.9k; Skipped 2 file(s), size: 339984
> 2008-12-28 00:06:19,904 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Started compaction of 2 file(s) into /data/hbase/content/compaction.dir/888098363/url/mapfiles/1108952767537472093
> 2008-12-28 00:06:19,924 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/content/info/7464468628166271570, isReference=false, sequence id=12406630, length=11058492, majorCompaction=false
> 2008-12-28 00:06:19,930 DEBUG org.apache.hadoop.hbase.regionserver.HStore: loaded /data/hbase/content/2030495720/content/info/7703689073557380324, isReference=false, sequence id=12795530, length=183695, majorCompaction=false
> 2008-12-28 00:06:19,931 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Loaded 4 file(s) in hstore 2030495720/content, max sequence id 13310628
> 2008-12-28 00:06:19,958 DEBUG org.apache.hadoop.hbase.regionserver.HStore: moving /data/hbase/content/compaction.dir/888098363/url/mapfiles/1108952767537472093 to /data/hbase/content/888098363/url/mapfiles/4113677425818108069
> 2008-12-28 00:06:19,976 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Completed compaction of 888098363/url store size is 403.6k
> 2008-12-28 00:06:19,978 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region content,846510f382e9a0ae5655f03cb772830d,1230384248550 in 0sec
> 2008-12-28 00:06:19,979 INFO org.apache.hadoop.hbase.regionserver.HRegion: starting compaction on region content,e2f10daf46269ad3cc25766aa3bf48c4,1230416165744
> 2008-12-28 00:06:19,981 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Compaction size of 2100409777/content: 170.2m; Skipped 1 file(s), size: 144757669
> 2008-12-28 00:06:19,996 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Started compaction of 2 file(s) into /data/hbase/content/compaction.dir/2100409777/content/mapfiles/7166480659109830957
> 2008-12-28 00:06:20,033 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Applied 0, skipped 2078 because sequence id <= 13310628
> 2008-12-28 00:06:20,127 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Deleting old log file: hdfs://sjdc-atr-dc-1.atr.trendmicro.com:50000/data/hbase/content/2030495720/oldlogfile.log
> 2008-12-28 00:06:20,137 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Next sequence id for region content,10a1c144cf729885001e71a5ff5108dc,1230416158498 is 13310629
> 2008-12-28 00:06:20,138 INFO org.apache.hadoop.hbase.regionserver.HRegion: region content,10a1c144cf729885001e71a5ff5108dc,1230416158498/2030495720 available
> ....
> 2008-12-28 00:06:21,008 DEBUG org.apache.hadoop.hbase.regionserver.HStore: Started compaction of 2 file(s) into /data/hbase/content/compaction.dir/2100409777/url/mapfiles/744229794307693860
> 2008-12-28 00:06:21,030 WARN org.apache.hadoop.hbase.regionserver.HRegionServer: error getting store file index size for 2030495720/content: java.io.FileNotFoundException: File does not exist: hdfs://sjdc-atr-dc-1.atr.trendmicro.com:50000/data/hbase/content/2030495720/content/mapfiles/7703689073557380324/index
> at org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:394)
> at org.apache.hadoop.hbase.regionserver.HStoreFile.indexLength(HStoreFile.java:488)
> at org.apache.hadoop.hbase.regionserver.HStore.getStorefilesIndexSize(HStore.java:2174)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.doMetrics(HRegionServer.java:936)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:333)
> at java.lang.Thread.run(Thread.java:619)
> 2008-12-28 00:06:21,032 WARN org.apache.hadoop.hbase.regionserver.HRegionServer: error getting store file index size for 2030495720/info: java.io.FileNotFoundException: File does not exist: hdfs://sjdc-atr-dc-1.atr.trendmicro.com:50000/data/hbase/content/2030495720/info/mapfiles/8629305049543986640/index
> at org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:394)
> at org.apache.hadoop.hbase.regionserver.HStoreFile.indexLength(HStoreFile.java:488)
> at org.apache.hadoop.hbase.regionserver.HStore.getStorefilesIndexSize(HStore.java:2174)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.doMetrics(HRegionServer.java:936)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:333)
> at java.lang.Thread.run(Thread.java:619)
> 2008-12-28 00:06:21,053 WARN org.apache.hadoop.hbase.regionserver.HRegionServer: error getting store file index size for 2030495720/url: java.io.FileNotFoundException: File does not exist: hdfs://sjdc-atr-dc-1.atr.trendmicro.com:50000/data/hbase/content/2030495720/url/mapfiles/7040855870376599550/index
> at org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:394)
> at org.apache.hadoop.hbase.regionserver.HStoreFile.indexLength(HStoreFile.java:488)
> at org.apache.hadoop.hbase.regionserver.HStore.getStorefilesIndexSize(HStore.java:2174)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.doMetrics(HRegionServer.java:936)
> at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:333)
> at java.lang.Thread.run(Thread.java:619)
> ...
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.