You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Alexey Zotov (Created) (JIRA)" <ji...@apache.org> on 2011/10/21 15:04:32 UTC
[jira] [Created] (HBASE-4644) LoadIncrementalHFiles ignores
additional configurations
LoadIncrementalHFiles ignores additional configurations
-------------------------------------------------------
Key: HBASE-4644
URL: https://issues.apache.org/jira/browse/HBASE-4644
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3
Environment: Centos 5.5, Cloudera cdh3u1 distribution.
Reporter: Alexey Zotov
Method run ignores configuration if LoadIncrementalHFiles was runned from user's application:
{code}
LoadIncrementalHFiles hFilesMergeTask = new LoadIncrementalHFiles(conf);
ToolRunner.run(hFilesMergeTask, args);
{code}
Quick fix:
{code}
--- LoadIncrementalHFiles.java 2011-07-18 08:20:38.000000000 +0400
+++ LoadIncrementalHFiles.java 2011-10-19 18:08:31.228972054 +0400
@@ -447,14 +446,20 @@
if (!tableExists) this.createTable(tableName,dirPath);
Path hfofDir = new Path(dirPath);
- HTable table = new HTable(tableName);
+ HTable table;
+ Configuration configuration = getConf();
+ if (configuration != null) {
+ table = new HTable(configuration, tableName);
+ } else {
+ table = new HTable(tableName);
+ }
doBulkLoad(hfofDir, table);
return 0;
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4644) LoadIncrementalHFiles ignores
additional configurations
Posted by "Ted Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132663#comment-13132663 ]
Ted Yu commented on HBASE-4644:
-------------------------------
@Alexey:
Can you put the suggested fix into a patch file ?
If the patch for latest 0.90 branch doesn't apply cleanly to 0.92/TRUNK, separate patches should be attached.
> LoadIncrementalHFiles ignores additional configurations
> -------------------------------------------------------
>
> Key: HBASE-4644
> URL: https://issues.apache.org/jira/browse/HBASE-4644
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Environment: Centos 5.5, Cloudera cdh3u1 distribution.
> Reporter: Alexey Zotov
> Assignee: Alexey Zotov
> Priority: Minor
> Labels: Configuration, LoadIncrementalHFiles
>
> Method run ignores configuration, which was passed in as constructor argument:
> {code}
> LoadIncrementalHFiles hFilesMergeTask = new LoadIncrementalHFiles(conf);
> ToolRunner.run(hFilesMergeTask, args);
> {code}
> This happens because HTable creation (_new HTable(tableName);_ in LoadIncrementalHFiles.run() method) skips existing configuration and tries to create a new one for HTable. If there is no hbase-site.xml in classpath, previously loaded properties (via -conf <configuration file>) will be missed.
> Quick fix:
> {code}
> --- LoadIncrementalHFiles.java 2011-07-18 08:20:38.000000000 +0400
> +++ LoadIncrementalHFiles.java 2011-10-19 18:08:31.228972054 +0400
> @@ -447,14 +446,20 @@
> if (!tableExists) this.createTable(tableName,dirPath);
>
> Path hfofDir = new Path(dirPath);
> - HTable table = new HTable(tableName);
> + HTable table;
> + Configuration configuration = getConf();
> + if (configuration != null) {
> + table = new HTable(configuration, tableName);
> + } else {
> + table = new HTable(tableName);
> + }
>
> doBulkLoad(hfofDir, table);
> return 0;
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4644) LoadIncrementalHFiles ignores
additional configurations
Posted by "Alexey Zotov (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexey Zotov updated HBASE-4644:
--------------------------------
Description:
Method run ignores configuration, which was passed in as constructor argument:
{code}
LoadIncrementalHFiles hFilesMergeTask = new LoadIncrementalHFiles(conf);
ToolRunner.run(hFilesMergeTask, args);
{code}
This happens because HTable creation (_new HTable(tableName);_ in LoadIncrementalHFiles.run() method) skips existing configuration and tries to create a new one for HTable. If there is no hbase-site.xml in classpath, previously loaded properties (via -conf <configuration file>) will be missed.
Quick fix:
{code}
--- LoadIncrementalHFiles.java 2011-07-18 08:20:38.000000000 +0400
+++ LoadIncrementalHFiles.java 2011-10-19 18:08:31.228972054 +0400
@@ -447,14 +446,20 @@
if (!tableExists) this.createTable(tableName,dirPath);
Path hfofDir = new Path(dirPath);
- HTable table = new HTable(tableName);
+ HTable table;
+ Configuration configuration = getConf();
+ if (configuration != null) {
+ table = new HTable(configuration, tableName);
+ } else {
+ table = new HTable(tableName);
+ }
doBulkLoad(hfofDir, table);
return 0;
}
{code}
was:
Method run ignores configuration if LoadIncrementalHFiles was runned from user's application:
{code}
LoadIncrementalHFiles hFilesMergeTask = new LoadIncrementalHFiles(conf);
ToolRunner.run(hFilesMergeTask, args);
{code}
Quick fix:
{code}
--- LoadIncrementalHFiles.java 2011-07-18 08:20:38.000000000 +0400
+++ LoadIncrementalHFiles.java 2011-10-19 18:08:31.228972054 +0400
@@ -447,14 +446,20 @@
if (!tableExists) this.createTable(tableName,dirPath);
Path hfofDir = new Path(dirPath);
- HTable table = new HTable(tableName);
+ HTable table;
+ Configuration configuration = getConf();
+ if (configuration != null) {
+ table = new HTable(configuration, tableName);
+ } else {
+ table = new HTable(tableName);
+ }
doBulkLoad(hfofDir, table);
return 0;
}
{code}
Priority: Minor (was: Major)
> LoadIncrementalHFiles ignores additional configurations
> -------------------------------------------------------
>
> Key: HBASE-4644
> URL: https://issues.apache.org/jira/browse/HBASE-4644
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Environment: Centos 5.5, Cloudera cdh3u1 distribution.
> Reporter: Alexey Zotov
> Priority: Minor
> Labels: Configuration, LoadIncrementalHFiles
>
> Method run ignores configuration, which was passed in as constructor argument:
> {code}
> LoadIncrementalHFiles hFilesMergeTask = new LoadIncrementalHFiles(conf);
> ToolRunner.run(hFilesMergeTask, args);
> {code}
> This happens because HTable creation (_new HTable(tableName);_ in LoadIncrementalHFiles.run() method) skips existing configuration and tries to create a new one for HTable. If there is no hbase-site.xml in classpath, previously loaded properties (via -conf <configuration file>) will be missed.
> Quick fix:
> {code}
> --- LoadIncrementalHFiles.java 2011-07-18 08:20:38.000000000 +0400
> +++ LoadIncrementalHFiles.java 2011-10-19 18:08:31.228972054 +0400
> @@ -447,14 +446,20 @@
> if (!tableExists) this.createTable(tableName,dirPath);
>
> Path hfofDir = new Path(dirPath);
> - HTable table = new HTable(tableName);
> + HTable table;
> + Configuration configuration = getConf();
> + if (configuration != null) {
> + table = new HTable(configuration, tableName);
> + } else {
> + table = new HTable(tableName);
> + }
>
> doBulkLoad(hfofDir, table);
> return 0;
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4644) LoadIncrementalHFiles ignores
additional configurations
Posted by "Todd Lipcon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132746#comment-13132746 ]
Todd Lipcon commented on HBASE-4644:
------------------------------------
Dup of HBASE-3714?
> LoadIncrementalHFiles ignores additional configurations
> -------------------------------------------------------
>
> Key: HBASE-4644
> URL: https://issues.apache.org/jira/browse/HBASE-4644
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Environment: Centos 5.5, Cloudera cdh3u1 distribution.
> Reporter: Alexey Zotov
> Assignee: Alexey Zotov
> Priority: Minor
> Labels: Configuration, LoadIncrementalHFiles
>
> Method run ignores configuration, which was passed in as constructor argument:
> {code}
> LoadIncrementalHFiles hFilesMergeTask = new LoadIncrementalHFiles(conf);
> ToolRunner.run(hFilesMergeTask, args);
> {code}
> This happens because HTable creation (_new HTable(tableName);_ in LoadIncrementalHFiles.run() method) skips existing configuration and tries to create a new one for HTable. If there is no hbase-site.xml in classpath, previously loaded properties (via -conf <configuration file>) will be missed.
> Quick fix:
> {code}
> --- LoadIncrementalHFiles.java 2011-07-18 08:20:38.000000000 +0400
> +++ LoadIncrementalHFiles.java 2011-10-19 18:08:31.228972054 +0400
> @@ -447,14 +446,20 @@
> if (!tableExists) this.createTable(tableName,dirPath);
>
> Path hfofDir = new Path(dirPath);
> - HTable table = new HTable(tableName);
> + HTable table;
> + Configuration configuration = getConf();
> + if (configuration != null) {
> + table = new HTable(configuration, tableName);
> + } else {
> + table = new HTable(tableName);
> + }
>
> doBulkLoad(hfofDir, table);
> return 0;
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4644) LoadIncrementalHFiles ignores
additional configurations
Posted by "Ted Yu (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu resolved HBASE-4644.
---------------------------
Resolution: Duplicate
Duplicate with HBASE-3714
> LoadIncrementalHFiles ignores additional configurations
> -------------------------------------------------------
>
> Key: HBASE-4644
> URL: https://issues.apache.org/jira/browse/HBASE-4644
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Environment: Centos 5.5, Cloudera cdh3u1 distribution.
> Reporter: Alexey Zotov
> Assignee: Alexey Zotov
> Priority: Minor
> Labels: Configuration, LoadIncrementalHFiles
>
> Method run ignores configuration, which was passed in as constructor argument:
> {code}
> LoadIncrementalHFiles hFilesMergeTask = new LoadIncrementalHFiles(conf);
> ToolRunner.run(hFilesMergeTask, args);
> {code}
> This happens because HTable creation (_new HTable(tableName);_ in LoadIncrementalHFiles.run() method) skips existing configuration and tries to create a new one for HTable. If there is no hbase-site.xml in classpath, previously loaded properties (via -conf <configuration file>) will be missed.
> Quick fix:
> {code}
> --- LoadIncrementalHFiles.java 2011-07-18 08:20:38.000000000 +0400
> +++ LoadIncrementalHFiles.java 2011-10-19 18:08:31.228972054 +0400
> @@ -447,14 +446,20 @@
> if (!tableExists) this.createTable(tableName,dirPath);
>
> Path hfofDir = new Path(dirPath);
> - HTable table = new HTable(tableName);
> + HTable table;
> + Configuration configuration = getConf();
> + if (configuration != null) {
> + table = new HTable(configuration, tableName);
> + } else {
> + table = new HTable(tableName);
> + }
>
> doBulkLoad(hfofDir, table);
> return 0;
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4644) LoadIncrementalHFiles ignores
additional configurations
Posted by "Ted Yu (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ted Yu reassigned HBASE-4644:
-----------------------------
Assignee: Alexey Zotov
> LoadIncrementalHFiles ignores additional configurations
> -------------------------------------------------------
>
> Key: HBASE-4644
> URL: https://issues.apache.org/jira/browse/HBASE-4644
> Project: HBase
> Issue Type: Bug
> Affects Versions: 0.90.3
> Environment: Centos 5.5, Cloudera cdh3u1 distribution.
> Reporter: Alexey Zotov
> Assignee: Alexey Zotov
> Priority: Minor
> Labels: Configuration, LoadIncrementalHFiles
>
> Method run ignores configuration, which was passed in as constructor argument:
> {code}
> LoadIncrementalHFiles hFilesMergeTask = new LoadIncrementalHFiles(conf);
> ToolRunner.run(hFilesMergeTask, args);
> {code}
> This happens because HTable creation (_new HTable(tableName);_ in LoadIncrementalHFiles.run() method) skips existing configuration and tries to create a new one for HTable. If there is no hbase-site.xml in classpath, previously loaded properties (via -conf <configuration file>) will be missed.
> Quick fix:
> {code}
> --- LoadIncrementalHFiles.java 2011-07-18 08:20:38.000000000 +0400
> +++ LoadIncrementalHFiles.java 2011-10-19 18:08:31.228972054 +0400
> @@ -447,14 +446,20 @@
> if (!tableExists) this.createTable(tableName,dirPath);
>
> Path hfofDir = new Path(dirPath);
> - HTable table = new HTable(tableName);
> + HTable table;
> + Configuration configuration = getConf();
> + if (configuration != null) {
> + table = new HTable(configuration, tableName);
> + } else {
> + table = new HTable(tableName);
> + }
>
> doBulkLoad(hfofDir, table);
> return 0;
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira