You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Anoop Sam John (JIRA)" <ji...@apache.org> on 2012/06/05 20:34:23 UTC
[jira] [Created] (HBASE-6164) Correct the bug in block encoding
usage in bulkload
Anoop Sam John created HBASE-6164:
-------------------------------------
Summary: Correct the bug in block encoding usage in bulkload
Key: HBASE-6164
URL: https://issues.apache.org/jira/browse/HBASE-6164
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.96.0, 0.94.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.1
Address the issue raised under HBASE-6040
https://issues.apache.org/jira/browse/HBASE-6040?focusedCommentId=13289334&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13289334
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6164) Correct the bug in block encoding
usage in bulkload
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ramkrishna.s.vasudevan updated HBASE-6164:
------------------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk and 0.94
Thanks for the patch Anoop.
Thanks for the review Ted.
> Correct the bug in block encoding usage in bulkload
> ---------------------------------------------------
>
> Key: HBASE-6164
> URL: https://issues.apache.org/jira/browse/HBASE-6164
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.96.0, 0.94.1
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6164_94.patch, 6164_Trunk.patch
>
>
> Address the issue raised under HBASE-6040
> https://issues.apache.org/jira/browse/HBASE-6040?focusedCommentId=13289334&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13289334
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6164) Correct the bug in block encoding
usage in bulkload
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291630#comment-13291630 ]
Anoop Sam John commented on HBASE-6164:
---------------------------------------
@Stack
Can you have a look at this patch pls.
> Correct the bug in block encoding usage in bulkload
> ---------------------------------------------------
>
> Key: HBASE-6164
> URL: https://issues.apache.org/jira/browse/HBASE-6164
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.96.0, 0.94.1
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6164_94.patch, 6164_Trunk.patch
>
>
> Address the issue raised under HBASE-6040
> https://issues.apache.org/jira/browse/HBASE-6040?focusedCommentId=13289334&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13289334
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6164) Correct the bug in block encoding
usage in bulkload
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290145#comment-13290145 ]
Hadoop QA commented on HBASE-6164:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12531097/6164_Trunk.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 5 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestAtomicOperation
org.apache.hadoop.hbase.replication.TestReplication
org.apache.hadoop.hbase.TestZooKeeper
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2115//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2115//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2115//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2115//console
This message is automatically generated.
> Correct the bug in block encoding usage in bulkload
> ---------------------------------------------------
>
> Key: HBASE-6164
> URL: https://issues.apache.org/jira/browse/HBASE-6164
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.96.0, 0.94.1
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6164_94.patch, 6164_Trunk.patch
>
>
> Address the issue raised under HBASE-6040
> https://issues.apache.org/jira/browse/HBASE-6040?focusedCommentId=13289334&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13289334
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6164) Correct the bug in block encoding
usage in bulkload
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13289632#comment-13289632 ]
Anoop Sam John commented on HBASE-6164:
---------------------------------------
To solve this
1. Move the writing of the DataBlockEncoding meta data into HFile.Writer
2. This needs change in the signature of HFileDataBlockEncoder#saveMetadata(StoreFile.Writer storeFileWriter) tosaveMetadata(HFile.Writer writer)
Will give the patch by tomorrow.
> Correct the bug in block encoding usage in bulkload
> ---------------------------------------------------
>
> Key: HBASE-6164
> URL: https://issues.apache.org/jira/browse/HBASE-6164
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.96.0, 0.94.1
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 0.96.0, 0.94.1
>
>
> Address the issue raised under HBASE-6040
> https://issues.apache.org/jira/browse/HBASE-6040?focusedCommentId=13289334&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13289334
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6164) Correct the bug in block encoding
usage in bulkload
Posted by "ramkrishna.s.vasudevan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291631#comment-13291631 ]
ramkrishna.s.vasudevan commented on HBASE-6164:
-----------------------------------------------
@Stack
Could you take a look and commit this if you feel it is fine?. Internally the patch has been tested also. so I am +1 on the change, if you are ok with the change in api params.
> Correct the bug in block encoding usage in bulkload
> ---------------------------------------------------
>
> Key: HBASE-6164
> URL: https://issues.apache.org/jira/browse/HBASE-6164
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.96.0, 0.94.1
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6164_94.patch, 6164_Trunk.patch
>
>
> Address the issue raised under HBASE-6040
> https://issues.apache.org/jira/browse/HBASE-6040?focusedCommentId=13289334&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13289334
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6164) Correct the bug in block encoding
usage in bulkload
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anoop Sam John updated HBASE-6164:
----------------------------------
Status: Patch Available (was: Open)
> Correct the bug in block encoding usage in bulkload
> ---------------------------------------------------
>
> Key: HBASE-6164
> URL: https://issues.apache.org/jira/browse/HBASE-6164
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.96.0, 0.94.1
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6164_94.patch, 6164_Trunk.patch
>
>
> Address the issue raised under HBASE-6040
> https://issues.apache.org/jira/browse/HBASE-6040?focusedCommentId=13289334&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13289334
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6164) Correct the bug in block encoding
usage in bulkload
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290190#comment-13290190 ]
Anoop Sam John commented on HBASE-6164:
---------------------------------------
Ted
The MR code deals with HFile Writer. That is why moved to close() in HFile.Writer
Also HFile deals with dataBlockEncoder other than this saveMetaData part.. That is why I thought it can be moved..
> Correct the bug in block encoding usage in bulkload
> ---------------------------------------------------
>
> Key: HBASE-6164
> URL: https://issues.apache.org/jira/browse/HBASE-6164
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.96.0, 0.94.1
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6164_94.patch, 6164_Trunk.patch
>
>
> Address the issue raised under HBASE-6040
> https://issues.apache.org/jira/browse/HBASE-6040?focusedCommentId=13289334&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13289334
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6164) Correct the bug in block encoding
usage in bulkload
Posted by "Anoop Sam John (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Anoop Sam John updated HBASE-6164:
----------------------------------
Attachment: 6164_Trunk.patch
6164_94.patch
With this it is working fine.
> Correct the bug in block encoding usage in bulkload
> ---------------------------------------------------
>
> Key: HBASE-6164
> URL: https://issues.apache.org/jira/browse/HBASE-6164
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.96.0, 0.94.1
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6164_94.patch, 6164_Trunk.patch
>
>
> Address the issue raised under HBASE-6040
> https://issues.apache.org/jira/browse/HBASE-6040?focusedCommentId=13289334&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13289334
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6164) Correct the bug in block encoding
usage in bulkload
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400011#comment-13400011 ]
Hudson commented on HBASE-6164:
-------------------------------
Integrated in HBase-0.94-security #37 (See [https://builds.apache.org/job/HBase-0.94-security/37/])
HBASE-6164 Correct the bug in block encoding usage in bulkload
Submitted by:Anoop
Reviewed by:Ted (Revision 1352221)
Result = SUCCESS
ramkrishna :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoder.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoderImpl.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/NoOpDataBlockEncoder.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java
> Correct the bug in block encoding usage in bulkload
> ---------------------------------------------------
>
> Key: HBASE-6164
> URL: https://issues.apache.org/jira/browse/HBASE-6164
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.96.0, 0.94.1
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6164_94.patch, 6164_Trunk.patch
>
>
> Address the issue raised under HBASE-6040
> https://issues.apache.org/jira/browse/HBASE-6040?focusedCommentId=13289334&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13289334
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6164) Correct the bug in block encoding
usage in bulkload
Posted by "Zhihong Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290189#comment-13290189 ]
Zhihong Ted Yu commented on HBASE-6164:
---------------------------------------
Patch looks good.
{code}
- // Save data block encoder metadata in the file info.
- dataBlockEncoder.saveMetadata(this);
{code}
Why is the above method lifted out of StoreFile ?
> Correct the bug in block encoding usage in bulkload
> ---------------------------------------------------
>
> Key: HBASE-6164
> URL: https://issues.apache.org/jira/browse/HBASE-6164
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.96.0, 0.94.1
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6164_94.patch, 6164_Trunk.patch
>
>
> Address the issue raised under HBASE-6040
> https://issues.apache.org/jira/browse/HBASE-6040?focusedCommentId=13289334&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13289334
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6164) Correct the bug in block encoding
usage in bulkload
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397778#comment-13397778 ]
Hudson commented on HBASE-6164:
-------------------------------
Integrated in HBase-0.94 #263 (See [https://builds.apache.org/job/HBase-0.94/263/])
HBASE-6164 Correct the bug in block encoding usage in bulkload
Submitted by:Anoop
Reviewed by:Ted (Revision 1352221)
Result = SUCCESS
ramkrishna :
Files :
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoder.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileDataBlockEncoderImpl.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/io/hfile/NoOpDataBlockEncoder.java
* /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
* /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java
> Correct the bug in block encoding usage in bulkload
> ---------------------------------------------------
>
> Key: HBASE-6164
> URL: https://issues.apache.org/jira/browse/HBASE-6164
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.96.0, 0.94.1
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 0.96.0, 0.94.1
>
> Attachments: 6164_94.patch, 6164_Trunk.patch
>
>
> Address the issue raised under HBASE-6040
> https://issues.apache.org/jira/browse/HBASE-6040?focusedCommentId=13289334&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13289334
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HBASE-6164) Correct the bug in block encoding
usage in bulkload
Posted by "Lars Hofhansl (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-6164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lars Hofhansl closed HBASE-6164.
--------------------------------
> Correct the bug in block encoding usage in bulkload
> ---------------------------------------------------
>
> Key: HBASE-6164
> URL: https://issues.apache.org/jira/browse/HBASE-6164
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.94.1, 0.96.0
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Fix For: 0.94.1, 0.96.0
>
> Attachments: 6164_94.patch, 6164_Trunk.patch
>
>
> Address the issue raised under HBASE-6040
> https://issues.apache.org/jira/browse/HBASE-6040?focusedCommentId=13289334&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13289334
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira