You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jonathan Hsieh (JIRA)" <ji...@apache.org> on 2011/08/29 06:02:39 UTC
[jira] [Created] (HBASE-4269) Add tests and restore semantics to
TableImportFormat/TableRecordReader
Add tests and restore semantics to TableImportFormat/TableRecordReader
----------------------------------------------------------------------
Key: HBASE-4269
URL: https://issues.apache.org/jira/browse/HBASE-4269
Project: HBase
Issue Type: Improvement
Components: mapred, mapreduce, test
Reporter: Jonathan Hsieh
Fix For: 0.90.5
HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4269) Add tests and restore semantics to
TableImportFormat/TableRecordReader
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hsieh updated HBASE-4269:
----------------------------------
Status: Patch Available (was: Open)
> Add tests and restore semantics to TableImportFormat/TableRecordReader
> ----------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "Jonathan Hsieh (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13231356#comment-13231356 ]
Jonathan Hsieh commented on HBASE-4269:
---------------------------------------
Jan,
When I did this patch, the two versions had different error recovery path and I made them similar. UnknownScannerException is a subclass DoNotRetryIOException so I chose that instead.
I'm assuming this is causing some pain now -- how is this affecting the job you are running? (is it catching and rethrowing other exceptions as well?)
If there is something we need to change I'm fine with that. Let's file a new issue -- this patch has been in included in a few releases now.
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.90.5, 0.92.0
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093156#comment-13093156 ]
Ted Yu commented on HBASE-4269:
-------------------------------
Would src/test/java/org/apache/hadoop/hbase/ be a better place to host TestTableInputFormat ?
The reason being that it tests both mapred and mapreduce input formats.
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableImportFormat/TableRecordReader
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13092626#comment-13092626 ]
jiraposter@reviews.apache.org commented on HBASE-4269:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1675/
-----------------------------------------------------------
Review request for hbase, Ted Yu and Ming Ma.
Summary
-------
commit d2bbee8696d54245b046158bacb5ab1a210e5df6
Author: Jonathan Hsieh <jo...@cloudera.com>
Date: Sun Aug 28 19:10:57 2011 -0700
HBASE-4269: Add tests and restore semantics to TableImportFormat/TableRecordReader
Adding tests cases and restoring sematics from HBASE-4196 patch.
This addresses bug HBASE-4269.
https://issues.apache.org/jira/browse/HBASE-4269
Diffs
-----
src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java 54df2cf
src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java c5fc496
src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java PRE-CREATION
Diff: https://reviews.apache.org/r/1675/diff
Testing
-------
New unit tests pass, running other tests.
Thanks,
jmhsieh
> Add tests and restore semantics to TableImportFormat/TableRecordReader
> ----------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093082#comment-13093082 ]
jiraposter@reviews.apache.org commented on HBASE-4269:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1675/#review1682
-----------------------------------------------------------
Ship it!
+1
nit: test is excellent but looks like could have gone into TestTableInputFormatScan?
Lets commit.
- Michael
On 2011-08-29 12:10:51, jmhsieh wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1675/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-29 12:10:51)
bq.
bq.
bq. Review request for hbase, Ted Yu and Ming Ma.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. commit d2bbee8696d54245b046158bacb5ab1a210e5df6
bq. Author: Jonathan Hsieh <jo...@cloudera.com>
bq. Date: Sun Aug 28 19:10:57 2011 -0700
bq.
bq. HBASE-4269: Add tests and restore semantics to TableInputFormat/TableRecordReader
bq.
bq. Adding tests cases and restoring sematics from HBASE-4196 patch.
bq.
bq.
bq. This addresses bug HBASE-4269.
bq. https://issues.apache.org/jira/browse/HBASE-4269
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java c5fc496
bq. src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java 54df2cf
bq.
bq. Diff: https://reviews.apache.org/r/1675/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. New unit tests pass, running other tests.
bq.
bq.
bq. Thanks,
bq.
bq. jmhsieh
bq.
bq.
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093510#comment-13093510 ]
Hudson commented on HBASE-4269:
-------------------------------
Integrated in HBase-TRUNK #2159 (See [https://builds.apache.org/job/HBase-TRUNK/2159/])
HBASE-4269 Add tests and restore semantics to TableInputFormat/TableRecordReader
(Jonathan Hsieh)
tedyu :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java
* /hbase/trunk/src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13092667#comment-13092667 ]
jiraposter@reviews.apache.org commented on HBASE-4269:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1675/#review1675
-----------------------------------------------------------
Nice work.
src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java
<https://reviews.apache.org/r/1675/#comment3823>
Please add javadoc stating that this test covers org.apache.hadoop.hbase.mapred namespace.
src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java
<https://reviews.apache.org/r/1675/#comment3826>
I think calling implementation class' ctor is better, following example of runTestMapreduce().
src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java
<https://reviews.apache.org/r/1675/#comment3824>
Please add javadoc stating that this test covers org.apache.hadoop.hbase.mapreduce namespace.
src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java
<https://reviews.apache.org/r/1675/#comment3825>
Please wrap long line.
- Ted
On 2011-08-29 04:20:16, jmhsieh wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1675/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-29 04:20:16)
bq.
bq.
bq. Review request for hbase, Ted Yu and Ming Ma.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. commit d2bbee8696d54245b046158bacb5ab1a210e5df6
bq. Author: Jonathan Hsieh <jo...@cloudera.com>
bq. Date: Sun Aug 28 19:10:57 2011 -0700
bq.
bq. HBASE-4269: Add tests and restore semantics to TableImportFormat/TableRecordReader
bq.
bq. Adding tests cases and restoring sematics from HBASE-4196 patch.
bq.
bq.
bq. This addresses bug HBASE-4269.
bq. https://issues.apache.org/jira/browse/HBASE-4269
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java 54df2cf
bq. src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java c5fc496
bq. src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/1675/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. New unit tests pass, running other tests.
bq.
bq.
bq. Thanks,
bq.
bq. jmhsieh
bq.
bq.
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4269) Add tests and restore semantics to
TableImportFormat/TableRecordReader
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hsieh reassigned HBASE-4269:
-------------------------------------
Assignee: Jonathan Hsieh
> Add tests and restore semantics to TableImportFormat/TableRecordReader
> ----------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "Jan Lukavsky (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250541#comment-13250541 ]
Jan Lukavsky commented on HBASE-4269:
-------------------------------------
Hi Jonathan,
I will file a new issue with description of issues it causes us.
Thanks.
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.90.5, 0.92.0
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093422#comment-13093422 ]
Ted Yu commented on HBASE-4269:
-------------------------------
Integrated to branch and TRUNK.
Thanks for the patch Jonathan.
Thanks for the review Michael.
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093240#comment-13093240 ]
Ted Yu commented on HBASE-4269:
-------------------------------
We can keep the new test in o.a.h.hbase.mapred
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093229#comment-13093229 ]
stack commented on HBASE-4269:
------------------------------
@Jon Don't worry about it; lets commit (I wanted to amorticize cluster startup -- thats what takes the time in my experience)
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hsieh updated HBASE-4269:
----------------------------------
Attachment: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
Updated patch to address review comments.
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "Jan Lukavsky (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230972#comment-13230972 ]
Jan Lukavsky commented on HBASE-4269:
-------------------------------------
Hi,
I think patch to this issue changed semantics for mapreduce API. In HBASE-4196 there was no change in semantics in org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl, the only change was in org.apache.hadoop.hbase.mapred.TableRecordReaderImpl (where the catch of UnknownScannerException was changed to IOException). Now the semantics of mapreduce API is different of the one before HBASE-4196, and I think this should be reverted. Is there any reason why to have different semantics for the two APIs? Wouldn't it be better to accept the change of semantics in HBASE-4196? Are there any negative side-effects of this change? I don't see any discussion of the type "do we need to change the semantics back"?
Thanks for reply :)
Jan
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.90.5, 0.92.0
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13092795#comment-13092795 ]
jiraposter@reviews.apache.org commented on HBASE-4269:
------------------------------------------------------
bq. On 2011-08-29 06:54:46, Ted Yu wrote:
bq. > src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java, line 107
bq. > <https://reviews.apache.org/r/1675/diff/1/?file=36176#file36176line107>
bq. >
bq. > Please add javadoc stating that this test covers org.apache.hadoop.hbase.mapred namespace.
done
bq. On 2011-08-29 06:54:46, Ted Yu wrote:
bq. > src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java, line 137
bq. > <https://reviews.apache.org/r/1675/diff/1/?file=36176#file36176line137>
bq. >
bq. > Please add javadoc stating that this test covers org.apache.hadoop.hbase.mapreduce namespace.
done
bq. On 2011-08-29 06:54:46, Ted Yu wrote:
bq. > src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java, line 145
bq. > <https://reviews.apache.org/r/1675/diff/1/?file=36176#file36176line145>
bq. >
bq. > Please wrap long line.
done
bq. On 2011-08-29 06:54:46, Ted Yu wrote:
bq. > src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java, line 113
bq. > <https://reviews.apache.org/r/1675/diff/1/?file=36176#file36176line113>
bq. >
bq. > I think calling implementation class' ctor is better, following example of runTestMapreduce().
i think you mean spelling out the class completely on the constructor like this right?
org.apache.hadoop.hbase.mapred.TableRecordReader trr =
new org.apache.hadoop.hbase.mapred.TableRecordReader();
did you mean more than that?
- jmhsieh
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1675/#review1675
-----------------------------------------------------------
On 2011-08-29 12:10:51, jmhsieh wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1675/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-29 12:10:51)
bq.
bq.
bq. Review request for hbase, Ted Yu and Ming Ma.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. commit d2bbee8696d54245b046158bacb5ab1a210e5df6
bq. Author: Jonathan Hsieh <jo...@cloudera.com>
bq. Date: Sun Aug 28 19:10:57 2011 -0700
bq.
bq. HBASE-4269: Add tests and restore semantics to TableInputFormat/TableRecordReader
bq.
bq. Adding tests cases and restoring sematics from HBASE-4196 patch.
bq.
bq.
bq. This addresses bug HBASE-4269.
bq. https://issues.apache.org/jira/browse/HBASE-4269
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java c5fc496
bq. src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java 54df2cf
bq.
bq. Diff: https://reviews.apache.org/r/1675/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. New unit tests pass, running other tests.
bq.
bq.
bq. Thanks,
bq.
bq. jmhsieh
bq.
bq.
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4269:
--------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13092794#comment-13092794 ]
jiraposter@reviews.apache.org commented on HBASE-4269:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1675/
-----------------------------------------------------------
(Updated 2011-08-29 12:10:51.509870)
Review request for hbase, Ted Yu and Ming Ma.
Changes
-------
Addressed comments
Summary (updated)
-------
commit d2bbee8696d54245b046158bacb5ab1a210e5df6
Author: Jonathan Hsieh <jo...@cloudera.com>
Date: Sun Aug 28 19:10:57 2011 -0700
HBASE-4269: Add tests and restore semantics to TableInputFormat/TableRecordReader
Adding tests cases and restoring sematics from HBASE-4196 patch.
This addresses bug HBASE-4269.
https://issues.apache.org/jira/browse/HBASE-4269
Diffs (updated)
-----
src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java c5fc496
src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java PRE-CREATION
src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java 54df2cf
Diff: https://reviews.apache.org/r/1675/diff
Testing
-------
New unit tests pass, running other tests.
Thanks,
jmhsieh
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4269) Add tests and restore semantics to
TableImportFormat/TableRecordReader
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hsieh updated HBASE-4269:
----------------------------------
Attachment: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
> Add tests and restore semantics to TableImportFormat/TableRecordReader
> ----------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093220#comment-13093220 ]
Jonathan Hsieh commented on HBASE-4269:
---------------------------------------
@Stack: The test you mention sets up a lot more stuff (and takes a long time to run) so I opted not to. Maybe rename the test class to TestTableInputFormatScanFailures?
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13093216#comment-13093216 ]
Jonathan Hsieh commented on HBASE-4269:
---------------------------------------
@Ted Most of the MR tests seem to live in o.a.h.hbase.mapreduce instead of mapred. Would it make more sense to move it there? IMO it just doesn't feel like it should be in o.a.h.hbase.
(yeahm, I started working on the mapred package and then added it the mapreduce tests afterwards.)
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13092832#comment-13092832 ]
jiraposter@reviews.apache.org commented on HBASE-4269:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1675/#review1678
-----------------------------------------------------------
Ship it!
- Ted
On 2011-08-29 12:10:51, jmhsieh wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1675/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-08-29 12:10:51)
bq.
bq.
bq. Review request for hbase, Ted Yu and Ming Ma.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. commit d2bbee8696d54245b046158bacb5ab1a210e5df6
bq. Author: Jonathan Hsieh <jo...@cloudera.com>
bq. Date: Sun Aug 28 19:10:57 2011 -0700
bq.
bq. HBASE-4269: Add tests and restore semantics to TableInputFormat/TableRecordReader
bq.
bq. Adding tests cases and restoring sematics from HBASE-4196 patch.
bq.
bq.
bq. This addresses bug HBASE-4269.
bq. https://issues.apache.org/jira/browse/HBASE-4269
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/main/java/org/apache/hadoop/hbase/mapreduce/TableRecordReaderImpl.java c5fc496
bq. src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java PRE-CREATION
bq. src/main/java/org/apache/hadoop/hbase/mapred/TableRecordReaderImpl.java 54df2cf
bq.
bq. Diff: https://reviews.apache.org/r/1675/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. New unit tests pass, running other tests.
bq.
bq.
bq. Thanks,
bq.
bq. jmhsieh
bq.
bq.
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch, 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4269) Add tests and restore semantics to
TableInputFormat/TableRecordReader
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu updated HBASE-4269:
--------------------------
Summary: Add tests and restore semantics to TableInputFormat/TableRecordReader (was: Add tests and restore semantics to TableImportFormat/TableRecordReader)
> Add tests and restore semantics to TableInputFormat/TableRecordReader
> ---------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
> Attachments: 0001-HBASE-4269-Add-tests-and-restore-semantics-to-TableI.patch
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4269) Add tests and restore semantics to
TableImportFormat/TableRecordReader
Posted by "Jonathan Hsieh (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hsieh updated HBASE-4269:
----------------------------------
Affects Version/s: 0.90.5
0.92.0
> Add tests and restore semantics to TableImportFormat/TableRecordReader
> ----------------------------------------------------------------------
>
> Key: HBASE-4269
> URL: https://issues.apache.org/jira/browse/HBASE-4269
> Project: HBase
> Issue Type: Improvement
> Components: mapred, mapreduce, test
> Affects Versions: 0.92.0, 0.90.5
> Reporter: Jonathan Hsieh
> Assignee: Jonathan Hsieh
> Fix For: 0.90.5
>
>
> HBASE-4196 Modified the semantics of failures in TableImportFormat/TableRecordReader, and had no tests cases. This patch restores semantics to rethrow when a DoNotRetryIOException is triggered and adds test cases.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira