You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jan Lukavsky (JIRA)" <ji...@apache.org> on 2011/08/30 09:44:38 UTC
[jira] [Created] (HBASE-4297) TableMapReduceUtil overwrites user
supplied options
TableMapReduceUtil overwrites user supplied options
---------------------------------------------------
Key: HBASE-4297
URL: https://issues.apache.org/jira/browse/HBASE-4297
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.90.4
Reporter: Jan Lukavsky
Attachments: HBASE-4297.patch
Job configuration is overwritten by hbase-default and hbase-site in TableMapReduceUtil.initTable(Mapper|Reducer)Job, causing unexpected behavior in the following code:
{noformat}
Configuration conf = HBaseConfiguration.create();
// change keyvalue size
conf.setInt("hbase.client.keyvalue.maxsize", 20971520);
Job job = new Job(conf, ...);
TableMapReduceUtil.initTableMapperJob(...);
// the job doesn't have the option changed, uses it from hbase-site or hbase-default
job.submit();
{noformat}
Although in this case it could be fixed by moving the set() after initTableMapperJob(), in case where user wants to change some option using GenericOptionsParser and -D this is impossible, making this cool feature useless.
In the 0.20.x era this code behaved as expected. The solution of this problem should be that we don't overwrite the options, but just read them if they are missing.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4297) TableMapReduceUtil overwrites user
supplied options
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100031#comment-13100031 ]
stack commented on HBASE-4297:
------------------------------
+1 on the patch. Does it work for you Jan? Any side effects that you've seen?
> TableMapReduceUtil overwrites user supplied options
> ---------------------------------------------------
>
> Key: HBASE-4297
> URL: https://issues.apache.org/jira/browse/HBASE-4297
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Jan Lukavsky
> Attachments: HBASE-4297.patch
>
>
> Job configuration is overwritten by hbase-default and hbase-site in TableMapReduceUtil.initTable(Mapper|Reducer)Job, causing unexpected behavior in the following code:
> {noformat}
> Configuration conf = HBaseConfiguration.create();
> // change keyvalue size
> conf.setInt("hbase.client.keyvalue.maxsize", 20971520);
> Job job = new Job(conf, ...);
> TableMapReduceUtil.initTableMapperJob(...);
> // the job doesn't have the option changed, uses it from hbase-site or hbase-default
> job.submit();
> {noformat}
> Although in this case it could be fixed by moving the set() after initTableMapperJob(), in case where user wants to change some option using GenericOptionsParser and -D this is impossible, making this cool feature useless.
> In the 0.20.x era this code behaved as expected. The solution of this problem should be that we don't overwrite the options, but just read them if they are missing.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4297) TableMapReduceUtil overwrites user
supplied options
Posted by "Jonathan Hsieh (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hsieh updated HBASE-4297:
----------------------------------
Fix Version/s: 0.90.5
> TableMapReduceUtil overwrites user supplied options
> ---------------------------------------------------
>
> Key: HBASE-4297
> URL: https://issues.apache.org/jira/browse/HBASE-4297
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Jan Lukavsky
> Fix For: 0.90.5
>
> Attachments: HBASE-4297.patch
>
>
> Job configuration is overwritten by hbase-default and hbase-site in TableMapReduceUtil.initTable(Mapper|Reducer)Job, causing unexpected behavior in the following code:
> {noformat}
> Configuration conf = HBaseConfiguration.create();
> // change keyvalue size
> conf.setInt("hbase.client.keyvalue.maxsize", 20971520);
> Job job = new Job(conf, ...);
> TableMapReduceUtil.initTableMapperJob(...);
> // the job doesn't have the option changed, uses it from hbase-site or hbase-default
> job.submit();
> {noformat}
> Although in this case it could be fixed by moving the set() after initTableMapperJob(), in case where user wants to change some option using GenericOptionsParser and -D this is impossible, making this cool feature useless.
> In the 0.20.x era this code behaved as expected. The solution of this problem should be that we don't overwrite the options, but just read them if they are missing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4297) TableMapReduceUtil overwrites user
supplied options
Posted by "Jan Lukavsky (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jan Lukavsky updated HBASE-4297:
--------------------------------
Attachment: HBASE-4297.patch
Attaching patch for this issue.
> TableMapReduceUtil overwrites user supplied options
> ---------------------------------------------------
>
> Key: HBASE-4297
> URL: https://issues.apache.org/jira/browse/HBASE-4297
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Jan Lukavsky
> Attachments: HBASE-4297.patch
>
>
> Job configuration is overwritten by hbase-default and hbase-site in TableMapReduceUtil.initTable(Mapper|Reducer)Job, causing unexpected behavior in the following code:
> {noformat}
> Configuration conf = HBaseConfiguration.create();
> // change keyvalue size
> conf.setInt("hbase.client.keyvalue.maxsize", 20971520);
> Job job = new Job(conf, ...);
> TableMapReduceUtil.initTableMapperJob(...);
> // the job doesn't have the option changed, uses it from hbase-site or hbase-default
> job.submit();
> {noformat}
> Although in this case it could be fixed by moving the set() after initTableMapperJob(), in case where user wants to change some option using GenericOptionsParser and -D this is impossible, making this cool feature useless.
> In the 0.20.x era this code behaved as expected. The solution of this problem should be that we don't overwrite the options, but just read them if they are missing.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4297) TableMapReduceUtil overwrites user
supplied options
Posted by "Jan Lukavsky (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100347#comment-13100347 ]
Jan Lukavsky commented on HBASE-4297:
-------------------------------------
Hi Stack,
I've tested the patch against cdh3u1 and it works fine for us. I haven't seen any negative side affects so far.
> TableMapReduceUtil overwrites user supplied options
> ---------------------------------------------------
>
> Key: HBASE-4297
> URL: https://issues.apache.org/jira/browse/HBASE-4297
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Jan Lukavsky
> Attachments: HBASE-4297.patch
>
>
> Job configuration is overwritten by hbase-default and hbase-site in TableMapReduceUtil.initTable(Mapper|Reducer)Job, causing unexpected behavior in the following code:
> {noformat}
> Configuration conf = HBaseConfiguration.create();
> // change keyvalue size
> conf.setInt("hbase.client.keyvalue.maxsize", 20971520);
> Job job = new Job(conf, ...);
> TableMapReduceUtil.initTableMapperJob(...);
> // the job doesn't have the option changed, uses it from hbase-site or hbase-default
> job.submit();
> {noformat}
> Although in this case it could be fixed by moving the set() after initTableMapperJob(), in case where user wants to change some option using GenericOptionsParser and -D this is impossible, making this cool feature useless.
> In the 0.20.x era this code behaved as expected. The solution of this problem should be that we don't overwrite the options, but just read them if they are missing.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4297) TableMapReduceUtil overwrites user
supplied options
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100519#comment-13100519 ]
Hudson commented on HBASE-4297:
-------------------------------
Integrated in HBase-TRUNK #2188 (See [https://builds.apache.org/job/HBase-TRUNK/2188/])
HBASE-4297 TableMapReduceUtil overwrites user supplied options
stack :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/HBaseConfiguration.java
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java
> TableMapReduceUtil overwrites user supplied options
> ---------------------------------------------------
>
> Key: HBASE-4297
> URL: https://issues.apache.org/jira/browse/HBASE-4297
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Jan Lukavsky
> Attachments: HBASE-4297.patch
>
>
> Job configuration is overwritten by hbase-default and hbase-site in TableMapReduceUtil.initTable(Mapper|Reducer)Job, causing unexpected behavior in the following code:
> {noformat}
> Configuration conf = HBaseConfiguration.create();
> // change keyvalue size
> conf.setInt("hbase.client.keyvalue.maxsize", 20971520);
> Job job = new Job(conf, ...);
> TableMapReduceUtil.initTableMapperJob(...);
> // the job doesn't have the option changed, uses it from hbase-site or hbase-default
> job.submit();
> {noformat}
> Although in this case it could be fixed by moving the set() after initTableMapperJob(), in case where user wants to change some option using GenericOptionsParser and -D this is impossible, making this cool feature useless.
> In the 0.20.x era this code behaved as expected. The solution of this problem should be that we don't overwrite the options, but just read them if they are missing.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4297) TableMapReduceUtil overwrites user
supplied options
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-4297:
-------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed to TRUNK. Thank you for the patch Jan.
> TableMapReduceUtil overwrites user supplied options
> ---------------------------------------------------
>
> Key: HBASE-4297
> URL: https://issues.apache.org/jira/browse/HBASE-4297
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Jan Lukavsky
> Attachments: HBASE-4297.patch
>
>
> Job configuration is overwritten by hbase-default and hbase-site in TableMapReduceUtil.initTable(Mapper|Reducer)Job, causing unexpected behavior in the following code:
> {noformat}
> Configuration conf = HBaseConfiguration.create();
> // change keyvalue size
> conf.setInt("hbase.client.keyvalue.maxsize", 20971520);
> Job job = new Job(conf, ...);
> TableMapReduceUtil.initTableMapperJob(...);
> // the job doesn't have the option changed, uses it from hbase-site or hbase-default
> job.submit();
> {noformat}
> Although in this case it could be fixed by moving the set() after initTableMapperJob(), in case where user wants to change some option using GenericOptionsParser and -D this is impossible, making this cool feature useless.
> In the 0.20.x era this code behaved as expected. The solution of this problem should be that we don't overwrite the options, but just read them if they are missing.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4297) TableMapReduceUtil overwrites user
supplied options
Posted by "Jan Lukavsky (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-4297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jan Lukavsky updated HBASE-4297:
--------------------------------
Status: Patch Available (was: Open)
> TableMapReduceUtil overwrites user supplied options
> ---------------------------------------------------
>
> Key: HBASE-4297
> URL: https://issues.apache.org/jira/browse/HBASE-4297
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.90.4
> Reporter: Jan Lukavsky
> Attachments: HBASE-4297.patch
>
>
> Job configuration is overwritten by hbase-default and hbase-site in TableMapReduceUtil.initTable(Mapper|Reducer)Job, causing unexpected behavior in the following code:
> {noformat}
> Configuration conf = HBaseConfiguration.create();
> // change keyvalue size
> conf.setInt("hbase.client.keyvalue.maxsize", 20971520);
> Job job = new Job(conf, ...);
> TableMapReduceUtil.initTableMapperJob(...);
> // the job doesn't have the option changed, uses it from hbase-site or hbase-default
> job.submit();
> {noformat}
> Although in this case it could be fixed by moving the set() after initTableMapperJob(), in case where user wants to change some option using GenericOptionsParser and -D this is impossible, making this cool feature useless.
> In the 0.20.x era this code behaved as expected. The solution of this problem should be that we don't overwrite the options, but just read them if they are missing.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira