You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "huan (JIRA)" <ji...@apache.org> on 2019/01/09 03:57:00 UTC
[jira] [Updated] (HBASE-21699) Create table failed when using
SPLITS_FILE => 'splits.txt'
[ https://issues.apache.org/jira/browse/HBASE-21699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
huan updated HBASE-21699:
-------------------------
Description:
Hi all:
When I ran
{code:java}
create 't1', 'f1', SPLITS_FILE => 'splits.txt'
{code}
on HBase2.0.0, it failed, and no detailed error info, just like below:
{code:java}
ERROR:
Creates a table. Pass a table name, and a set of column family
specifications (at least one), and, optionally, table configuration.
Column specification can be a simple string (name), or a dictionary
(dictionaries are described below in main help output), necessarily
including NAME attribute.
Examples:
{code}
So I opened the debug:
{code:java}
hbase shell -d
{code}
and
{code:java}
ERROR:
Backtrace: org.apache.hadoop.hbase.util.Bytes.toBytes(org/apache/hadoop/hbase/util/Bytes.java:732)
org.apache.hadoop.hbase.HTableDescriptor.setValue(org/apache/hadoop/hbase/HTableDescriptor.java:190){code}
But it works on branch 1.2.0.
so I view the source code, I find the issue is because the below code:
{code:java}
// admin.rb
if arg.key?(SPLITS_FILE)
splits_file = arg.delete(SPLITS_FILE)
unless File.exist?(splits_file)
raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
end
arg[SPLITS] = []
File.foreach(splits_file) do |line|
arg[SPLITS].push(line.chomp)
end
htd.setValue(SPLITS_FILE, arg[SPLITS_FILE])
end
{code}
{code:java}
// HTableDescriptor part
public HTableDescriptor setValue(String key, String value) {
getDelegateeForModification().setValue(Bytes.toBytes(key), Bytes.toBytes(value));
return this;
}
{code}
{code:java}
// Bytes part
public static byte[] toBytes(String s) {
try {
return s.getBytes(UTF8_CSN);
} catch (UnsupportedEncodingException e) {
// should never happen!
throw new IllegalArgumentException("UTF8 decoding is not supported", e);
}
}
{code}
Call flow is:
{code:java}
admin.rb ---> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE]) ---> Bytes.toBytes(key) && Bytes.toBytes(value) {code}
from Bytes.toBytes, if s is null, the function will throw NullPointerException, but HTableDescriptor.setValue(String key, String value) does not check key and value.
in admin.rb, it use arg.delete(SPLITS_FILE) to get the value, but this means, after using arg.delete(SPLITS_FILE), arg[SPLITS_FILE] will return nil. so HTableDescriptor.setValue(String key, String value) does not check key and value is root cause.
why branch below 2.0.0 works fine, because old code is :
{code:java}
public HTableDescriptor setValue(String key, String value) {
if (value == null) {
remove(key);
} else {
setValue(Bytes.toBytes(key), Bytes.toBytes(value));
}
return this;
}
{code}
it check the value.
since branch 2.0.0, HBase add new function called 'TableDescriptorBuilder'
it included:
{code:java}
public ModifyableTableDescriptor setValue(String key, String value) {
return setValue(toBytesOrNull(key, Bytes::toBytes),
toBytesOrNull(value, Bytes::toBytes));
}
{code}
it checked key and value, but HTableDescriptor.setValue(String key, String value) does not call it, so
just change HTableDescriptor.setValue(String key, String value) to call it, it will works
> Create table failed when using SPLITS_FILE => 'splits.txt'
> -----------------------------------------------------------
>
> Key: HBASE-21699
> URL: https://issues.apache.org/jira/browse/HBASE-21699
> Project: HBase
> Issue Type: Bug
> Components: Client
> Affects Versions: 3.0.0, 2.0.0, 2.0.1, 2.1.1, 2.0.2, 2.0.3, 2.1.2, 2.0.4
> Reporter: huan
> Priority: Minor
>
> Hi all:
> When I ran
> {code:java}
> create 't1', 'f1', SPLITS_FILE => 'splits.txt'
> {code}
>
> on HBase2.0.0, it failed, and no detailed error info, just like below:
>
> {code:java}
> ERROR:
> Creates a table. Pass a table name, and a set of column family
> specifications (at least one), and, optionally, table configuration.
> Column specification can be a simple string (name), or a dictionary
> (dictionaries are described below in main help output), necessarily
> including NAME attribute.
> Examples:
> {code}
> So I opened the debug:
>
>
> {code:java}
> hbase shell -d
> {code}
> and
>
>
> {code:java}
> ERROR:
> Backtrace: org.apache.hadoop.hbase.util.Bytes.toBytes(org/apache/hadoop/hbase/util/Bytes.java:732)
> org.apache.hadoop.hbase.HTableDescriptor.setValue(org/apache/hadoop/hbase/HTableDescriptor.java:190){code}
>
>
> But it works on branch 1.2.0.
> so I view the source code, I find the issue is because the below code:
> {code:java}
> // admin.rb
> if arg.key?(SPLITS_FILE)
> splits_file = arg.delete(SPLITS_FILE)
> unless File.exist?(splits_file)
> raise(ArgumentError, "Splits file #{splits_file} doesn't exist")
> end
> arg[SPLITS] = []
> File.foreach(splits_file) do |line|
> arg[SPLITS].push(line.chomp)
> end
> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE])
> end
> {code}
>
>
> {code:java}
> // HTableDescriptor part
> public HTableDescriptor setValue(String key, String value) {
> getDelegateeForModification().setValue(Bytes.toBytes(key), Bytes.toBytes(value));
> return this;
> }
> {code}
>
>
>
> {code:java}
> // Bytes part
> public static byte[] toBytes(String s) {
> try {
> return s.getBytes(UTF8_CSN);
> } catch (UnsupportedEncodingException e) {
> // should never happen!
> throw new IllegalArgumentException("UTF8 decoding is not supported", e);
> }
> }
> {code}
>
> Call flow is:
> {code:java}
> admin.rb ---> htd.setValue(SPLITS_FILE, arg[SPLITS_FILE]) ---> Bytes.toBytes(key) && Bytes.toBytes(value) {code}
>
> from Bytes.toBytes, if s is null, the function will throw NullPointerException, but HTableDescriptor.setValue(String key, String value) does not check key and value.
> in admin.rb, it use arg.delete(SPLITS_FILE) to get the value, but this means, after using arg.delete(SPLITS_FILE), arg[SPLITS_FILE] will return nil. so HTableDescriptor.setValue(String key, String value) does not check key and value is root cause.
> why branch below 2.0.0 works fine, because old code is :
>
> {code:java}
> public HTableDescriptor setValue(String key, String value) {
> if (value == null) {
> remove(key);
> } else {
> setValue(Bytes.toBytes(key), Bytes.toBytes(value));
> }
> return this;
> }
> {code}
> it check the value.
>
> since branch 2.0.0, HBase add new function called 'TableDescriptorBuilder'
> it included:
>
> {code:java}
> public ModifyableTableDescriptor setValue(String key, String value) {
> return setValue(toBytesOrNull(key, Bytes::toBytes),
> toBytesOrNull(value, Bytes::toBytes));
> }
> {code}
> it checked key and value, but HTableDescriptor.setValue(String key, String value) does not call it, so
> just change HTableDescriptor.setValue(String key, String value) to call it, it will works
>
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)