You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Gregory Chanan (Created) (JIRA)" <ji...@apache.org> on 2012/02/02 01:11:53 UTC
[jira] [Created] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Fix TestHFileOutputFormat to work against hadoop 0.23
-----------------------------------------------------
Key: HBASE-5317
URL: https://issues.apache.org/jira/browse/HBASE-5317
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 0.92.0, 0.94.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Running
mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
yields this on 0.92:
Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
Tests in error:
test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
It looks like on trunk, this also results in an error:
testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215069#comment-13215069 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
I will try again when I get home.
I noticed weird connection issues at work - inability to use Colloquy, etc
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200580#comment-13200580 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
Here was the command I used:
{code}
mvn clean -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
{code}
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201491#comment-13201491 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
Not every directory under hbase.rootdir is deemed corresponding to a table.
See the following javadoc in FSUtils.getTableDirs():
{code}
* @return All the table directories under <code>rootdir</code>. Ignore non table hbase folders such as
* .logs, .oldlogs, .corrupt, .META., and -ROOT- folders.
{code}
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gregory Chanan updated HBASE-5317:
----------------------------------
Attachment: HBASE-5317to0.92.patch
patch for 0.92
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, HBASE-5317to0.92.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207280#comment-13207280 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
I agree with the proposal.
Another option is to put the tests which depend on hbase jar(s) into a dedicated category. That category would run after hbase jar(s) are built.
Thanks for digging.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214025#comment-13214025 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
@Ted:
Checked it on a mac, seeing the same failure as you. Can you confirm something for me? If you cd to target/org.apache.hadoop.mapred.MiniMRCluster and run the following commands, do you see something similar?
$ find . -name stderr
./org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_2/application_1329939539132_0001/container_1329939539132_0001_01_000001/stderr
find . -name syslog
$
$ cat ./org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_2/application_1329939539132_0001/container_1329939539132_0001_01_000001/stderr
/bin/bash: /bin/java: No such file or directory
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200632#comment-13200632 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
@Ted:
your output is correct. There is another patch required to get past the TestTable error and to the "Error starting MRAppMaster." I didn't include it because it doesn't solve the issue, only lets you further along. I wanted to leave the patch in a "fresh" state that actually solves the issue addressed by the JIRA.
Perhaps the best thing to do is narrow the scope of this JIRA (to just get test_TIMERANGE and testColumnFamilyCompression to pass) and then open a new JIRA to solve the remaining issues. Do you think that makes the most sense?
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gregory Chanan updated HBASE-5317:
----------------------------------
Attachment: HBASE-5317-v3.patch
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222119#comment-13222119 ]
Hudson commented on HBASE-5317:
-------------------------------
Integrated in HBase-0.92-security #96 (See [https://builds.apache.org/job/HBase-0.92-security/96/])
HBASE-5317 Fix TestHFileOutputFormat to work against hadoop 0.23
(Gregory Taylor) (Revision 1293306)
Result = FAILURE
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/pom.xml
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestMetaMigrationRemovingHTD.java
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Fix For: 0.92.1, 0.94.0
>
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, HBASE-5317to0.92.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201841#comment-13201841 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
@Ted:
FSUtils.getTableDirs() excludes a specific list of directories. Specifically:
{code}
Arrays.asList(new String[]{ HREGION_LOGDIR_NAME, HREGION_OLDLOGDIR_NAME,
CORRUPT_DIR_NAME, Bytes.toString(META_TABLE_NAME),
Bytes.toString(ROOT_TABLE_NAME), SPLIT_LOGDIR_NAME }));
{code}
So if the MiniMRCluster creates a "target" directory, it will be returned.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213147#comment-13213147 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
Based on latest patch, I got:
{code}
Failed tests: testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
{code}
Here is one of the test failures:
{code}
testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat) Time elapsed: 29.375 sec <<< FAILURE!
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:478)
at org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testExcludeMinorCompaction(TestHFileOutputFormat.java:723)
{code}
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gregory Chanan updated HBASE-5317:
----------------------------------
Attachment: HBASE-5317-v6.patch
This is just to match up the version on Review Board. Only minor spacing differences.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209929#comment-13209929 ]
jiraposter@reviews.apache.org commented on HBASE-5317:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3939/#review5179
-----------------------------------------------------------
Good progress.
Please submit to Hadoop QA for test run.
src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
<https://reviews.apache.org/r/3939/#comment11338>
Please put a comma or period after avilable which is misspelled.
- Ted
On 2012-02-16 21:57:38, Gregory Chanan wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3939/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-02-16 21:57:38)
bq.
bq.
bq. Review request for hbase, Ted Yu and Michael Stack.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. NOTE: To get TestHFileOutputFormat passing also requires HBASE-5346, which has been reviewed but not committed yet.
bq.
bq. Fixes a few issues interacting with the new mapreduce in 0.23:
bq. - Maven dependency issue
bq. - Change the hbaseRootDir so that mrv2 output is not confused with an hbase table
bq. - Create Jars necessary to run mrv2 jobs at unit test time (before hbase jars have been built)
bq.
bq.
bq. This addresses bug HBASE-5317.
bq. https://issues.apache.org/jira/browse/HBASE-5317
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. pom.xml f24b389
bq. src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java c5222c2
bq. src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java ff38731
bq. src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java ec9d581
bq.
bq. Diff: https://reviews.apache.org/r/3939/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Gregory
bq.
bq.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205983#comment-13205983 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
@Gregory:
I think we should try approach #1 first.
Thanks for your persistence.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gregory Chanan updated HBASE-5317:
----------------------------------
Resolution: Fixed
Fix Version/s: 0.94.0
0.92.1
Status: Resolved (was: Patch Available)
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Fix For: 0.92.1, 0.94.0
>
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, HBASE-5317to0.92.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200650#comment-13200650 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
The subject of this JIRA is for all of TestHFileOutputFormat to pass for 0.23
I think we should try to achieve that goal.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200644#comment-13200644 ]
Hadoop QA commented on HBASE-5317:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12513229/HBASE-5317-v0.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -136 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 156 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/904//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/904//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/904//console
This message is automatically generated.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201431#comment-13201431 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
I don't see ClassNotFoundException in test output but the following may provide a clue:
{code}
2012-02-06 09:44:48,377 WARN [main] mapreduce.JobSubmitter(139): Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
2012-02-06 09:44:48,380 WARN [main] mapreduce.JobSubmitter(241): No job jar file set. User classes may not be found. See Job or Job#setJar(String).
2012-02-06 09:44:51,163 WARN [ContainersLauncher #0] nodemanager.DefaultContainerExecutor(192): Exit code from task is : 127
2012-02-06 09:44:51,165 WARN [ContainersLauncher #0] launcher.ContainerLaunch(273): Container exited with a non-zero exit code 127
{code}
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215066#comment-13215066 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
I am unable to reproduce either on my machine or on a coworker's Mac.
Is it possible for you to try it on another machine? Perhaps some (security?) setting on your machine makes mapreduce unable to open the required port or something?
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209938#comment-13209938 ]
jiraposter@reviews.apache.org commented on HBASE-5317:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3939/#review5180
-----------------------------------------------------------
Minor nits. +1 otherwise.
pom.xml
<https://reviews.apache.org/r/3939/#comment11342>
Whats DS? (Sorry if I'm being extra thick here)
src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
<https://reviews.apache.org/r/3939/#comment11343>
rad!
src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
<https://reviews.apache.org/r/3939/#comment11345>
Excellent
src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
<https://reviews.apache.org/r/3939/#comment11346>
Is this because we're running against target/classes dir?
src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
<https://reviews.apache.org/r/3939/#comment11347>
Something is wrong w/ the indent here.
- Michael
On 2012-02-16 21:57:38, Gregory Chanan wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3939/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-02-16 21:57:38)
bq.
bq.
bq. Review request for hbase, Ted Yu and Michael Stack.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. NOTE: To get TestHFileOutputFormat passing also requires HBASE-5346, which has been reviewed but not committed yet.
bq.
bq. Fixes a few issues interacting with the new mapreduce in 0.23:
bq. - Maven dependency issue
bq. - Change the hbaseRootDir so that mrv2 output is not confused with an hbase table
bq. - Create Jars necessary to run mrv2 jobs at unit test time (before hbase jars have been built)
bq.
bq.
bq. This addresses bug HBASE-5317.
bq. https://issues.apache.org/jira/browse/HBASE-5317
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. pom.xml f24b389
bq. src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java c5222c2
bq. src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java ff38731
bq. src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java ec9d581
bq.
bq. Diff: https://reviews.apache.org/r/3939/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Gregory
bq.
bq.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215222#comment-13215222 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
I was able to get the test to pass on Linux for patch v6.
I will integrate patch v6 later if I don't hear objection.
Thanks for the persistent digging, Gregory.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213155#comment-13213155 ]
Hadoop QA commented on HBASE-5317:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515510/HBASE-5317-v6.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -136 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 159 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1000//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1000//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1000//console
This message is automatically generated.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215017#comment-13215017 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
{code}
echo ${JAVA_HOME}
/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
mvn -Dhadoop.profile=23 clean test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
{code}
Same test failures, e.g.:
{code}
testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat) Time elapsed: 60.857 sec <<< FAILURE!
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.runIncrementalPELoad(TestHFileOutputFormat.java:478)
at org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.doIncrementalLoadTest(TestHFileOutputFormat.java:390)
at org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.testMRIncrementalLoadWithSplit(TestHFileOutputFormat.java:369)
{code}
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213212#comment-13213212 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
@Ted:
Hmm...just to check I fetched the latest applied the patch and both:
{code}
mvn -Dhadoop.profile=23 clean test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
{code}
and
{code}
mvn clean test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
{code}
passed. Can you clarify exactly what you ran?
I don't know what is going on. I've noticed those tests have been failing Hadoop QA as well for a few days, and I can't reproduce those either. Do you have any idea what is going on?
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214037#comment-13214037 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
I do see similar output:
{code}
LM-SJN-00713032:org.apache.hadoop.mapred.MiniMRCluster zhihyu$ find . -name stderr
./org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_3/application_1329947593635_0001/container_1329947593635_0001_01_000001/stderr
LM-SJN-00713032:org.apache.hadoop.mapred.MiniMRCluster zhihyu$ find . -name syslog
LM-SJN-00713032:org.apache.hadoop.mapred.MiniMRCluster zhihyu$ cat org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_3/application_1329947593635_0001/container_1329947593635_0001_01_000001/stderr
/bin/bash: /bin/java: No such file or directory
{code}
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215784#comment-13215784 ]
Hudson commented on HBASE-5317:
-------------------------------
Integrated in HBase-0.92 #302 (See [https://builds.apache.org/job/HBase-0.92/302/])
HBASE-5317 Fix TestHFileOutputFormat to work against hadoop 0.23
(Gregory Taylor) (Revision 1293306)
Result = SUCCESS
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/pom.xml
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
* /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestMetaMigrationRemovingHTD.java
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, HBASE-5317to0.92.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Jimmy Xiang (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200575#comment-13200575 ]
Jimmy Xiang commented on HBASE-5317:
------------------------------------
@Ted, did you run it with Hadoop 0.23?
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201460#comment-13201460 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
Configuration.handleDeprecation() is private.
We may need to borrow deprecatedKeyMap and come up with good strategy of providing up-to-date config parameters to MRv2.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210639#comment-13210639 ]
Hudson commented on HBASE-5317:
-------------------------------
Integrated in HBase-0.92 #287 (See [https://builds.apache.org/job/HBase-0.92/287/])
HBASE-5317 Fix testColumnFamilyCompression and test_TIMERANGE in TestHFileOutputFormat (Revision 1245724)
Result = SUCCESS
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215442#comment-13215442 ]
stack commented on HBASE-5317:
------------------------------
It can go into 0.92 if you make a version (I see a bunch of failures trying to apply trunk patch). Thanks Gregory.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201444#comment-13201444 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
For HBASE-5317-v1.patch, I think we shouldn't simply catch TableExistsException.
We should add missing Configuration parameters in MRv2 so that there is no TableExistsException during test.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gregory Chanan updated HBASE-5317:
----------------------------------
Attachment: HBASE-5317-v4.patch
@stack:
I'm new to git and didn't add one of the files I modified :).
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201848#comment-13201848 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
Right.
Can we find out why this "target" directory was created ?
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215437#comment-13215437 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
Thanks, Ted.
I'll grant the license in the future.
Any objection to also getting this into 0.92?
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13205982#comment-13205982 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
Ok, did some more digging.
In 0.23 the MRAppMaster tries to load up the OutputFileFormat, so we have to load that, which we didn't have to do prior to 0.23.
It looks like: TableMapReduceUtil.addDependencyJars(job)
is supposed to do this, but when I run it, no jars can be found for the classes associated with the job.
By default, it looks like the classpath in the test is:
{basedir}/target/test-classes:{basedir}/target/classes:...
but the relevant jars are in:
target/
target/hbase-{VERSION}/hbase-{VERSION}/
By the way, when I add hbase jars manually to "tmpjars" [which is what addDependencyJars does], the test passes.
So it seems like the solution is either:
1) Modify the classpath to include those jars
2) Build a jar myself of the relevant classes
I'm not sure what the correct answer is; please advise.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200303#comment-13200303 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
testMRIncrementalLoad (and probably the other 2) fail with TestTable; this is pretty easy to get past.
But after that MR Job fails with:
2012-02-03 17:02:37,982 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
org.apache.hadoop.yarn.YarnException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.mapreduce.HFileOutputFormat
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRAppMaster.java:320)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.init(MRAppMaster.java:213)
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210741#comment-13210741 ]
Hadoop QA commented on HBASE-5317:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515057/HBASE-5317-v3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -136 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 158 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/985//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/985//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/985//console
This message is automatically generated.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207090#comment-13207090 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
@Evert:
Yes, that is the idea. I'm hoping to have a patch that does that and solves the jar issue above by the end of today.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215024#comment-13215024 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
{code}
LM-SJN-00713032:org.apache.hadoop.mapred.MiniMRCluster zhihyu$ find . -name stderr
./org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_0/application_1330028372796_0001/container_1330028372796_0001_01_000007/stderr
./org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_2/application_1330028372796_0001/container_1330028372796_0001_01_000001/stderr
./org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_2/application_1330028372796_0001/container_1330028372796_0001_01_000006/stderr
./org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_3/application_1330028372796_0001/container_1330028372796_0001_01_000004/stderr
./org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_3/application_1330028372796_0001/container_1330028372796_0001_01_000005/stderr
./org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-1_2/application_1330028372796_0001/container_1330028372796_0001_01_000002/stderr
./org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-1_3/application_1330028372796_0001/container_1330028372796_0001_01_000003/stderr
LM-SJN-00713032:org.apache.hadoop.mapred.MiniMRCluster zhihyu$ cat org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_0/application_1330028372796_0001/container_1330028372796_0001_01_000007/stderr
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
LM-SJN-00713032:org.apache.hadoop.mapred.MiniMRCluster zhihyu$ cat org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-1_3/application_1330028372796_0001/container_1330028372796_0001_01_000003/stderr
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
{code}
>From org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-0_0/application_1330028372796_0001/container_1330028372796_0001_01_000007/syslog:
{code}
2012-02-23 12:22:34,208 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Assiging lm-sjn-00713032.corp.ebay.com:61263 with 1 to fetcher#2
2012-02-23 12:22:34,208 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: assigned 1 of 1 to lm-sjn-00713032.corp.ebay.com:61263 to fetcher#2
2012-02-23 12:22:34,469 WARN [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.Fetcher: Failed to connect to lm-sjn-00713032.corp.ebay.com:61263 with 1 map outputs
java.io.IOException: Server returned HTTP response code: 503 for URL: http://lm-sjn-00713032.corp.ebay.com:61263/mapOutput?job=job_1330028372796_0001&reduce=0&map=attempt_1330028372796_0001_m_000000_1
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:220)
at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:152)
2012-02-23 12:22:34,471 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Reporting fetch failure for attempt_1330028372796_0001_m_000000_1 to jobtracker.
2012-02-23 12:22:34,471 FATAL [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: Shuffle failed with too many fetch failures and insufficient progress!
2012-02-23 12:22:34,472 INFO [fetcher#2] org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler: lm-sjn-00713032.corp.ebay.com:61263 freed by fetcher#2 in 264s
2012-02-23 12:22:34,472 ERROR [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:zhihyu (auth:SIMPLE) cause:org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in shuffle in fetcher#2
2012-02-23 12:22:34,472 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in shuffle in fetcher#2
at org.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:123)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:371)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
Caused by: java.io.IOException: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out.
at org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler.checkReducerHealth(ShuffleScheduler.java:253)
at org.apache.hadoop.mapreduce.task.reduce.ShuffleScheduler.copyFailed(ShuffleScheduler.java:187)
at org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:247)
at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:152)
2012-02-23 12:22:34,476 INFO [main] org.apache.hadoop.mapred.Task: Runnning cleanup for the task
{code}
Looks like 503 caused fetch failure.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201473#comment-13201473 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
@Ted:
I agree we shouldn't just catch the TableExistsException -- that's why I didn't initially post that part of the patch. My recollection of that issue was that the MiniMRCluster is creating a "target" directory in hbase.rootdir [I could be wrong about the exact location]. When we call "table.getTableDescriptor()" it can't get a table descriptor for "target", so throws a TableExistsException. Can the handleDeprecation call prevent the target directory from being created? Or are you thinking of something else?
It also seems a little strange to me that calling "table.getTableDescriptor();" tries to get table descriptors for *everything* in hbase.rootdir. Why should I get an exception thrown if hbase can't find a TableDescriptor for "target" when I am only asking about "table"?
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200661#comment-13200661 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
@Ted:
Yes, that is the problem that causes the "TestTable" error. I'll post a patch for that tomorrow.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200651#comment-13200651 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
I found the following in test output (when using hadoop 0.23):
{code}
2012-02-04 20:18:30,563 WARN [main] conf.Configuration(326): mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address2012-02-04 20:18:30,578 WARN [main] conf.Configuration(343): mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
2012-02-04 20:18:30,579 INFO [main] mapreduce.HFileOutputFormat(320): Writing partition information to /Users/zhihyu/trunk-hbase/target/test-data/67a9efc5-8b7c-4afd-b516-39dd454334b8/mapred-working-dir/partitions_864c11de-4d4f-45a3-ba94-26aac1c751e6
2012-02-04 20:18:30,604 WARN [IPC Server handler 4 on 63830] master.HMaster(1725): Failed getting all descriptors
org.apache.hadoop.hbase.TableExistsException: No descriptor for target, hdfs://localhost:63821/user/zhihyu
at org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:164)
at org.apache.hadoop.hbase.util.FSTableDescriptors.getAll(FSTableDescriptors.java:182)
at org.apache.hadoop.hbase.master.HMaster.getHTableDescriptors(HMaster.java:1723)
{code}
When using hadoop 1.0, there is no such error.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202813#comment-13202813 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
The new MiniMRCluster in 0.23 copies some new MrApp jars into the working directory of the filesystem (which is the home directory of that file system):
{code}
FileSystem fs = FileSystem.get(conf);
Path testRootDir = new Path("target", caller.getName() + "-tmpDir")
.makeQualified(fs);
Path appJar = new Path(testRootDir, "MRAppJar.jar");
// Copy MRAppJar and make it private.
Path appMasterJar = new Path(MiniMRYarnCluster.APPJAR);
fs.copyFromLocalFile(appMasterJar, appJar);
{code}
This also happens to be the HBASE_DIR:
{code}
Path hbaseRootdir = fs.makeQualified(fs.getHomeDirectory());
this.conf.set(HConstants.HBASE_DIR, hbaseRootdir.toString());
{code}
I created a subdirectory under fs.getHomeDirectory called "hbase" and made this the HBASE_DIR. This gets past the "TestTable" issue and back to the ClassNotFoundException.
Since these issues appear to be issues with the test utilities, rather than the test themselves, I am going to create a subtask for fixing the test themselves (which will just be the v0 patch).
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215410#comment-13215410 ]
Hudson commented on HBASE-5317:
-------------------------------
Integrated in HBase-TRUNK-security #121 (See [https://builds.apache.org/job/HBase-TRUNK-security/121/])
HBASE-5317 Fix TestHFileOutputFormat to work against hadoop 0.23 (Gregory Chanan) (Revision 1293049)
Result = SUCCESS
tedyu :
Files :
* /hbase/trunk/pom.xml
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMetaMigrationRemovingHTD.java
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214998#comment-13214998 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
Okay, this looks like a mvn issue.
On linux, if JAVA_HOME is not set, mvn sets JAVA_HOME to be whatever shows up in "mvn --version"
On mac, mvn appears to not set JAVA_HOME if it is not already set.
So if you set JAVA_HOME to "/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home" the test should pass.
The new version of mapreduce requires JAVA_HOME to be set and tries to execute $JAVA_HOME + "bin/java" -- they should probably report a bug if JAVA_HOME is not set.
Let me know if you want me to do anything else wrt this issue.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Evert Arckens (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206877#comment-13206877 ]
Evert Arckens commented on HBASE-5317:
--------------------------------------
@Gregory:
I'm also encountering the "No descriptor for target" issue when using HBaseTestingUtility. And after reading this issue and HBASE-4025, it seems to me having HBaseTestingUtility create a subdirectory "hbase" is the right thing to do here. Are you effectively planning to do so?
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210678#comment-13210678 ]
jiraposter@reviews.apache.org commented on HBASE-5317:
------------------------------------------------------
bq. On 2012-02-17 00:49:13, Michael Stack wrote:
bq. > Minor nits. +1 otherwise.
Thanks for the reviews, Ted and Stack.
bq. On 2012-02-17 00:49:13, Michael Stack wrote:
bq. > pom.xml, line 2001
bq. > <https://reviews.apache.org/r/3939/diff/1/?file=75517#file75517line2001>
bq. >
bq. > Whats DS? (Sorry if I'm being extra thick here)
Distributed Shell (hadoop-yarn-applications-distributedshell)
bq. On 2012-02-17 00:49:13, Michael Stack wrote:
bq. > src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java, line 515
bq. > <https://reviews.apache.org/r/3939/diff/1/?file=75519#file75519line515>
bq. >
bq. > Is this because we're running against target/classes dir?
Right. HBase is a single module project and the maven test phase is before the maven package phase (when the jars are built).
This is discussed briefly on the JIRA, but I think the correct long-term solution is to make HBase a multi-module project (see HBASE-4336) and make a test module that depends on the core module.
- Gregory
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3939/#review5180
-----------------------------------------------------------
On 2012-02-16 21:57:38, Gregory Chanan wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3939/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-02-16 21:57:38)
bq.
bq.
bq. Review request for hbase, Ted Yu and Michael Stack.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. NOTE: To get TestHFileOutputFormat passing also requires HBASE-5346, which has been reviewed but not committed yet.
bq.
bq. Fixes a few issues interacting with the new mapreduce in 0.23:
bq. - Maven dependency issue
bq. - Change the hbaseRootDir so that mrv2 output is not confused with an hbase table
bq. - Create Jars necessary to run mrv2 jobs at unit test time (before hbase jars have been built)
bq.
bq.
bq. This addresses bug HBASE-5317.
bq. https://issues.apache.org/jira/browse/HBASE-5317
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. pom.xml f24b389
bq. src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java c5222c2
bq. src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java ff38731
bq. src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java ec9d581
bq.
bq. Diff: https://reviews.apache.org/r/3939/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Gregory
bq.
bq.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5317:
------------------------------
Status: Patch Available (was: Open)
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201488#comment-13201488 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
handleDeprecation() call may not prevent the target directory from being created.
We should try to find out why 'target' directory was only created for hadoop 0.23.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214353#comment-13214353 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
Thanks very much, Ted.
Could also you post the output of "mvn --version" where you ran the test? I'm particularly interested in the value of "Java home:" -- is there something valid at the location it is giving you?
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gregory Chanan updated HBASE-5317:
----------------------------------
Attachment: HBASE-5317-v5.patch
One more try at this...
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213130#comment-13213130 ]
jiraposter@reviews.apache.org commented on HBASE-5317:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3939/
-----------------------------------------------------------
(Updated 2012-02-22 00:01:33.392779)
Review request for hbase, Ted Yu and Michael Stack.
Changes
-------
Small changes as suggested by Ted and stack.
Also updates a test case that was assuming the root dir was the homedir.
Summary
-------
NOTE: To get TestHFileOutputFormat passing also requires HBASE-5346, which has been reviewed but not committed yet.
Fixes a few issues interacting with the new mapreduce in 0.23:
- Maven dependency issue
- Change the hbaseRootDir so that mrv2 output is not confused with an hbase table
- Create Jars necessary to run mrv2 jobs at unit test time (before hbase jars have been built)
This addresses bug HBASE-5317.
https://issues.apache.org/jira/browse/HBASE-5317
Diffs (updated)
-----
pom.xml 066c027
src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java c5222c2
src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java ff38731
src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java ec9d581
src/test/java/org/apache/hadoop/hbase/client/TestMetaMigrationRemovingHTD.java 7c5e108
Diff: https://reviews.apache.org/r/3939/diff
Testing
-------
Thanks,
Gregory
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212154#comment-13212154 ]
Hudson commented on HBASE-5317:
-------------------------------
Integrated in HBase-0.92-security #91 (See [https://builds.apache.org/job/HBase-0.92-security/91/])
HBASE-5317 Fix testColumnFamilyCompression and test_TIMERANGE in TestHFileOutputFormat (Revision 1245724)
Result = FAILURE
stack :
Files :
* /hbase/branches/0.92/CHANGES.txt
* /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gregory Chanan updated HBASE-5317:
----------------------------------
Attachment: HBASE-5317-v0.patch
Patch v0 fixes test_TIMERANGE and testColumnFamilyCompression
Other tests are a little more involved.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212413#comment-13212413 ]
Hadoop QA commented on HBASE-5317:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515326/HBASE-5317-v5.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -136 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 159 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/996//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/996//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/996//console
This message is automatically generated.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213221#comment-13213221 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
Here is what I did on MacBook:
{code}
527 patch -p0 -i HBASE-5317-v6.patch
528 mvn clean -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
{code}
I will attach test output.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13213229#comment-13213229 ]
Hadoop QA commented on HBASE-5317:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515531/TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 48 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1003//console
This message is automatically generated.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13214356#comment-13214356 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
{code}
LM-SJN-00713032:205-hadoop zhihyu$ mvn --version
Apache Maven 3.0.3 (r1075438; 2011-02-28 09:31:09-0800)
Maven home: /usr/share/maven
Java version: 1.6.0_29, vendor: Apple Inc.
Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Default locale: en_US, platform encoding: MacRoman
OS name: "mac os x", version: "10.6.8", arch: "x86_64", family: "mac"
LM-SJN-00713032:205-hadoop zhihyu$ ls /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
bin bundle include lib man
{code}
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207272#comment-13207272 ]
Gregory Chanan commented on HBASE-5317:
---------------------------------------
@Ted:
I tried approach #1, but it seems like it is not kosher to have the test depend on the maven packaging step. For example:
{code}
mvn -Dhadoop.profile=23 -DskipTests package
mvn -Dhadoop.profile=23 test ...
{code}
works, but
{code}
mvn -Dhadoop.profile=23 clean test ...
{code}
fails because the jars aren't even built without the package step.
#2 will probably work in both cases. It don't think it's great either, because ideally the tests should exercise the jars, because that is what downstream users would be running against. I think testing against the jars requires making hbase a multi-module project and is therefore probably:
0) outside the scope of this JIRA
1) a lot of work
2) should be discussed in the community
Therefore, I propose getting attempt #2 to work and filing a JIRA to figure out a better packaging story so we can later get rid of #2.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215290#comment-13215290 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
Integrated patch v6 to TRUNK.
Thanks for the patch Gregory.
In the future, please grant license to Apache when you attach patches.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215477#comment-13215477 ]
Hadoop QA commented on HBASE-5317:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515901/HBASE-5317to0.92.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1040//console
This message is automatically generated.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, HBASE-5317to0.92.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215703#comment-13215703 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
Integrated to 0.92 branch.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, HBASE-5317to0.92.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212386#comment-13212386 ]
Hadoop QA commented on HBASE-5317:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12515322/HBASE-5317-v4.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -136 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 159 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/995//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/995//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/995//console
This message is automatically generated.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-5317) Fix
TestHFileOutputFormat to work against hadoop 0.23
Posted by "Zhihong Yu (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201460#comment-13201460 ]
Zhihong Yu edited comment on HBASE-5317 at 2/6/12 6:47 PM:
-----------------------------------------------------------
Configuration.handleDeprecation() is private.
We may need to borrow deprecatedKeyMap and come up with good strategy of providing up-to-date config parameters to MRv2 (when hadoop.profile property carries value of 23).
was (Author: zhihyu@ebaysf.com):
Configuration.handleDeprecation() is private.
We may need to borrow deprecatedKeyMap and come up with good strategy of providing up-to-date config parameters to MRv2.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "jiraposter@reviews.apache.org (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209784#comment-13209784 ]
jiraposter@reviews.apache.org commented on HBASE-5317:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3939/
-----------------------------------------------------------
Review request for hbase, Ted Yu and Michael Stack.
Summary
-------
NOTE: To get TestHFileOutputFormat passing also requires HBASE-5346, which has been reviewed but not committed yet.
Fixes a few issues interacting with the new mapreduce in 0.23:
- Maven dependency issue
- Change the hbaseRootDir so that mrv2 output is not confused with an hbase table
- Create Jars necessary to run mrv2 jobs at unit test time (before hbase jars have been built)
This addresses bug HBASE-5317.
https://issues.apache.org/jira/browse/HBASE-5317
Diffs
-----
pom.xml f24b389
src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java c5222c2
src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java ff38731
src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java ec9d581
Diff: https://reviews.apache.org/r/3939/diff
Testing
-------
Thanks,
Gregory
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Gregory Chanan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gregory Chanan updated HBASE-5317:
----------------------------------
Attachment: HBASE-5317-v1.patch
This patch gets past the "TestTable" errors. Now, the problem is the job never completes (you get an assertion of job.waitForCompletion). From digging through the mapreduce logs, I found the ClassNotFoundException listed above.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13200348#comment-13200348 ]
Zhihong Yu commented on HBASE-5317:
-----------------------------------
Running TestHFileOutputFormat on MacBook, I don't see the 'Error starting MRAppMaster' message above.
Three tests failed (after applying patch):
{code}
testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
{code}
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216339#comment-13216339 ]
Hudson commented on HBASE-5317:
-------------------------------
Integrated in HBase-TRUNK #2669 (See [https://builds.apache.org/job/HBase-TRUNK/2669/])
HBASE-5317 Fix TestHFileOutputFormat to work against hadoop 0.23 (Gregory Chanan) (Revision 1293049)
Result = SUCCESS
tedyu :
Files :
* /hbase/trunk/pom.xml
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMetaMigrationRemovingHTD.java
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.92.0, 0.94.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, HBASE-5317to0.92.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211118#comment-13211118 ]
stack commented on HBASE-5317:
------------------------------
Gregory it failed with this:
{code}
[INFO] --- maven-compiler-plugin:2.0.2:testCompile (default-testCompile) @ hbase ---
[INFO] Compiling 331 source files to /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/target/test-classes
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 23.287s
[INFO] Finished at: Sat Feb 18 01:46:56 UTC 2012
[INFO] Final Memory: 41M/424M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.0.2:testCompile (default-testCompile) on project hbase: Compilation failure
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/src/test/java/org/apache/hadoop/hbase/client/TestMetaMigrationRemovingHTD.java:[79,33] cannot find symbol
[ERROR] symbol : method getDefaultRootDirPath()
[ERROR] location: class org.apache.hadoop.hbase.HBaseTestingUtility
[ERROR] -> [Help 1]
{code}
What you reckon thats about? Is it the 0.23 profile? leaking?
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Zhihong Yu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhihong Yu updated HBASE-5317:
------------------------------
Attachment: TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch, HBASE-5317-v3.patch, HBASE-5317-v4.patch, HBASE-5317-v5.patch, HBASE-5317-v6.patch, TEST-org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat.xml
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5317) Fix TestHFileOutputFormat to work
against hadoop 0.23
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201085#comment-13201085 ]
Hadoop QA commented on HBASE-5317:
----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12513394/HBASE-5317-v1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -136 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 156 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.replication.TestReplicationPeer
org.apache.hadoop.hbase.mapreduce.TestImportTsv
org.apache.hadoop.hbase.mapred.TestTableMapReduce
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/906//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/906//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/906//console
This message is automatically generated.
> Fix TestHFileOutputFormat to work against hadoop 0.23
> -----------------------------------------------------
>
> Key: HBASE-5317
> URL: https://issues.apache.org/jira/browse/HBASE-5317
> Project: HBase
> Issue Type: Bug
> Components: test
> Affects Versions: 0.94.0, 0.92.0
> Reporter: Gregory Chanan
> Assignee: Gregory Chanan
> Attachments: HBASE-5317-v0.patch, HBASE-5317-v1.patch
>
>
> Running
> mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> yields this on 0.92:
> Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found
> Tests in error:
> test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)
> testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> It looks like on trunk, this also results in an error:
> testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): TestTable
> I have a patch that fixes testColumnFamilyCompression and test_TIMERANGE, but haven't fixed the other 3 yet.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira