You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2011/07/06 06:56:16 UTC
[jira] [Created] (HBASE-4065) TableOutputFormat ignores failure to
create table instance
TableOutputFormat ignores failure to create table instance
----------------------------------------------------------
Key: HBASE-4065
URL: https://issues.apache.org/jira/browse/HBASE-4065
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3
Reporter: Todd Lipcon
Fix For: 0.90.4
If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4065) TableOutputFormat ignores failure to
create table instance
Posted by "Brock Noland (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland reassigned HBASE-4065:
-----------------------------------
Assignee: Brock Noland
> TableOutputFormat ignores failure to create table instance
> ----------------------------------------------------------
>
> Key: HBASE-4065
> URL: https://issues.apache.org/jira/browse/HBASE-4065
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Reporter: Todd Lipcon
> Assignee: Brock Noland
> Fix For: 0.94.0
>
>
> If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
> Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4065) TableOutputFormat ignores failure to
create table instance
Posted by "Brock Noland (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated HBASE-4065:
--------------------------------
Attachment: HBASE-4065.2.patch
Good call.
Attached checks for tableName.length() <= 0.
> TableOutputFormat ignores failure to create table instance
> ----------------------------------------------------------
>
> Key: HBASE-4065
> URL: https://issues.apache.org/jira/browse/HBASE-4065
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Reporter: Todd Lipcon
> Assignee: Brock Noland
> Fix For: 0.94.0
>
> Attachments: HBASE-4065.1.patch, HBASE-4065.2.patch
>
>
> If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
> Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4065) TableOutputFormat ignores failure
to create table instance
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088387#comment-13088387 ]
Ted Yu commented on HBASE-4065:
-------------------------------
+1 on patch version 2.
> TableOutputFormat ignores failure to create table instance
> ----------------------------------------------------------
>
> Key: HBASE-4065
> URL: https://issues.apache.org/jira/browse/HBASE-4065
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Reporter: Todd Lipcon
> Assignee: Brock Noland
> Fix For: 0.94.0
>
> Attachments: HBASE-4065.1.patch, HBASE-4065.2.patch
>
>
> If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
> Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4065) TableOutputFormat ignores failure to
create table instance
Posted by "Brock Noland (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated HBASE-4065:
--------------------------------
Attachment: HBASE-4065.1.patch
Attached throws a RumtimeException since ZKUtil.applyClusterKeyToConf also throws IOException and probably belongs in setConf. Also this throws the error earlier in the process.
> TableOutputFormat ignores failure to create table instance
> ----------------------------------------------------------
>
> Key: HBASE-4065
> URL: https://issues.apache.org/jira/browse/HBASE-4065
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Reporter: Todd Lipcon
> Assignee: Brock Noland
> Fix For: 0.94.0
>
> Attachments: HBASE-4065.1.patch
>
>
> If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
> Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HBASE-4065) TableOutputFormat ignores
failure to create table instance
Posted by "Brock Noland (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on HBASE-4065 started by Brock Noland.
> TableOutputFormat ignores failure to create table instance
> ----------------------------------------------------------
>
> Key: HBASE-4065
> URL: https://issues.apache.org/jira/browse/HBASE-4065
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Reporter: Todd Lipcon
> Assignee: Brock Noland
> Fix For: 0.94.0
>
> Attachments: HBASE-4065.1.patch, HBASE-4065.2.patch
>
>
> If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
> Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4065) TableOutputFormat ignores failure to
create table instance
Posted by "Brock Noland (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated HBASE-4065:
--------------------------------
Hadoop Flags: [Reviewed]
Status: Patch Available (was: In Progress)
Submitting patch, I am not entirely sure how to get jenkins to run but I think this might be it.
> TableOutputFormat ignores failure to create table instance
> ----------------------------------------------------------
>
> Key: HBASE-4065
> URL: https://issues.apache.org/jira/browse/HBASE-4065
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Reporter: Todd Lipcon
> Assignee: Brock Noland
> Fix For: 0.94.0
>
> Attachments: HBASE-4065.1.patch, HBASE-4065.2.patch
>
>
> If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
> Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4065) TableOutputFormat ignores failure to
create table instance
Posted by "Brock Noland (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated HBASE-4065:
--------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Marking issue resolve.
> TableOutputFormat ignores failure to create table instance
> ----------------------------------------------------------
>
> Key: HBASE-4065
> URL: https://issues.apache.org/jira/browse/HBASE-4065
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Reporter: Todd Lipcon
> Assignee: Brock Noland
> Fix For: 0.94.0
>
> Attachments: HBASE-4065.1.patch, HBASE-4065.2.patch
>
>
> If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
> Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4065) TableOutputFormat ignores failure
to create table instance
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088975#comment-13088975 ]
Hudson commented on HBASE-4065:
-------------------------------
Integrated in HBase-TRUNK #2128 (See [https://builds.apache.org/job/HBase-TRUNK/2128/])
HBASE-4065 TableOutputFormat ignores failure to create table instance
(Brock Noland)
tedyu :
Files :
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java
* /hbase/trunk/CHANGES.txt
> TableOutputFormat ignores failure to create table instance
> ----------------------------------------------------------
>
> Key: HBASE-4065
> URL: https://issues.apache.org/jira/browse/HBASE-4065
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Reporter: Todd Lipcon
> Assignee: Brock Noland
> Fix For: 0.94.0
>
> Attachments: HBASE-4065.1.patch, HBASE-4065.2.patch
>
>
> If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
> Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4065) TableOutputFormat ignores failure
to create table instance
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088309#comment-13088309 ]
Ted Yu commented on HBASE-4065:
-------------------------------
In mapred/TableOutputFormat.java:
{code}
} catch(IOException e) {
LOG.error(e);
throw e;
}
{code}
Should we make their behavior consistent ?
> TableOutputFormat ignores failure to create table instance
> ----------------------------------------------------------
>
> Key: HBASE-4065
> URL: https://issues.apache.org/jira/browse/HBASE-4065
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Reporter: Todd Lipcon
> Assignee: Brock Noland
> Fix For: 0.94.0
>
> Attachments: HBASE-4065.1.patch
>
>
> If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
> Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4065) TableOutputFormat ignores failure
to create table instance
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088363#comment-13088363 ]
Ted Yu commented on HBASE-4065:
-------------------------------
We should handle IOE out of ZKUtil.applyClusterKeyToConf() as well
So the patch is fine.
Should the following check tableName's length as well ?
{code}
+ if(tableName == null) {
+ throw new IllegalArgumentException("Must specify table name");
{code}
Thanks for the work, Brock.
> TableOutputFormat ignores failure to create table instance
> ----------------------------------------------------------
>
> Key: HBASE-4065
> URL: https://issues.apache.org/jira/browse/HBASE-4065
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Reporter: Todd Lipcon
> Assignee: Brock Noland
> Fix For: 0.94.0
>
> Attachments: HBASE-4065.1.patch
>
>
> If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
> Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4065) TableOutputFormat ignores failure to
create table instance
Posted by "Jonathan Hsieh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hsieh updated HBASE-4065:
----------------------------------
Fix Version/s: 0.92.0
> TableOutputFormat ignores failure to create table instance
> ----------------------------------------------------------
>
> Key: HBASE-4065
> URL: https://issues.apache.org/jira/browse/HBASE-4065
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Reporter: Todd Lipcon
> Assignee: Brock Noland
> Fix For: 0.92.0, 0.94.0
>
> Attachments: HBASE-4065.1.patch, HBASE-4065.2.patch
>
>
> If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
> Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4065) TableOutputFormat ignores failure
to create table instance
Posted by "Ted Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088864#comment-13088864 ]
Ted Yu commented on HBASE-4065:
-------------------------------
Integrated to branch and TRUNK.
Thanks for the patch Brock.
> TableOutputFormat ignores failure to create table instance
> ----------------------------------------------------------
>
> Key: HBASE-4065
> URL: https://issues.apache.org/jira/browse/HBASE-4065
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Reporter: Todd Lipcon
> Assignee: Brock Noland
> Fix For: 0.94.0
>
> Attachments: HBASE-4065.1.patch, HBASE-4065.2.patch
>
>
> If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
> Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4065) TableOutputFormat ignores failure
to create table instance
Posted by "Brock Noland (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088320#comment-13088320 ]
Brock Noland commented on HBASE-4065:
-------------------------------------
In the mapred package, HTable is created inside getRecordWriter where it can throw an IOE. In mapreduce HTable is created in setConf where an IOE cannot be thrown. I am fine with moving the HTable creation to getRecordWriter like mapred, but that still leaves us with the fact that ZKUtil.applyClusterKeyToConf throws IOE.
My assumption was that an IOE from ZKUtil.applyClusterKeyToConf should be considered fatal as well? That may certainly not be true and we could simply move HTable creation to getRecordWriter.
> TableOutputFormat ignores failure to create table instance
> ----------------------------------------------------------
>
> Key: HBASE-4065
> URL: https://issues.apache.org/jira/browse/HBASE-4065
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Reporter: Todd Lipcon
> Assignee: Brock Noland
> Fix For: 0.94.0
>
> Attachments: HBASE-4065.1.patch
>
>
> If TableOutputFormat in the new API fails to create a table, it simply logs this at ERROR level and then continues on its way. Then, the first write() to the table will throw a NPE since table hasn't been set.
> Instead, it should probably rethrow the exception as a RuntimeException in setConf, or do what the old-API TOF does and not create the HTable instance until getRecordWriter, where it can throw an IOE.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira