You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Dan Di Spaltro <da...@gmail.com> on 2014/03/26 23:42:00 UTC
Question on Binary indexes
I have a table with two indexes
create table auth.account_users
(
ordered_id BINARY(16) not null PRIMARY KEY,
account_id BINARY(16) not null,
user_id BINARY(16) not null,
default BOOLEAN,
created_at TIMESTAMP
) VERSIONS=1,IMMUTABLE_ROWS=true
CREATE INDEX auth_account_users_user_idx ON auth.account_users(user_id)
CREATE INDEX auth_account_users_account_idx ON
auth.account_users(account_id)
The creation works fine on this but during insert I get the following
message, I am wondering is this intended? Judging by the code it should
work... I am using the 4.0 RC btw.
java.lang.IllegalArgumentException: Unsupported non nullable index type
BINARY
at
org.apache.phoenix.util.IndexUtil.getIndexColumnDataType(IndexUtil.java:81)
at
org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMaintainer.java:345)
at
org.apache.phoenix.index.IndexMaintainer.buildUpdateMutation(IndexMaintainer.java:391)
at org.apache.phoenix.util.IndexUtil.generateIndexData(IndexUtil.java:185)
at org.apache.phoenix.execute.MutationState$1.next(MutationState.java:203)
at org.apache.phoenix.execute.MutationState$1.next(MutationState.java:186)
at org.apache.phoenix.execute.MutationState.commit(MutationState.java:347)
at
org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:351)
at
org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:229)
at
org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:185)
at
org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:146)
at
org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:151)
--
Dan Di Spaltro
Re: Question on Binary indexes
Posted by James Taylor <ja...@apache.org>.
Yes, a BIGINT (fixed 8 byte Long) may be used in an index. Pretty much
every fixed sized type *except* BINARY may be used. Because a column value
that maps to a KeyValue may be null (i.e. absent), we need a way of
representing that in the index row key. With numeric types we have that by
converting them to a DECIMAL.
File a JIRA, though. There are ways to encode BINARY data that would leave
us a way to represent the absence of a value.
On Sun, Mar 30, 2014 at 9:21 PM, Dan Di Spaltro <da...@gmail.com>wrote:
> Yeah I guess I was just assuming like a Long (fixed 8 bytes) it could be
> treated the same. Actually that reminds me, can you use a long to index?
> Does it internally maintain the 8 byte binary array?
>
>
> On Fri, Mar 28, 2014 at 10:23 AM, James Taylor <ja...@apache.org>wrote:
>
>> One option would be to base64 encode your fixed length binary as a
>> String. Then store it that way in Phoenix and you'd be able to index it.
>>
>> It's possible that we could encode binary data in a way that would let us
>> index it - how about filing a JIRA for that?
>>
>> Thanks,
>> James
>>
>>
>> On Fri, Mar 28, 2014 at 3:09 AM, Dan Di Spaltro <da...@gmail.com>wrote:
>>
>>> So James' ticket seems to indicate this is intended behavior, have you
>>> resolved that? It seems like an index on arbitrarily fixed-with bytes
>>> aren't a bad thing.
>>>
>>>
>>> On Wed, Mar 26, 2014 at 3:49 PM, Samarth Jain <sa...@gmail.com>wrote:
>>>
>>>> Hi Dan,
>>>>
>>>> Thanks for reporting.
>>>>
>>>> I am working on a fix for this. Please see:
>>>> https://issues.apache.org/jira/browse/PHOENIX-899
>>>>
>>>> Thanks,
>>>> Samarth
>>>>
>>>>
>>>>
>>>> On Wed, Mar 26, 2014 at 3:42 PM, Dan Di Spaltro <
>>>> dan.dispaltro@gmail.com> wrote:
>>>>
>>>>> I have a table with two indexes
>>>>>
>>>>> create table auth.account_users
>>>>> (
>>>>> ordered_id BINARY(16) not null PRIMARY KEY,
>>>>> account_id BINARY(16) not null,
>>>>> user_id BINARY(16) not null,
>>>>> default BOOLEAN,
>>>>> created_at TIMESTAMP
>>>>> ) VERSIONS=1,IMMUTABLE_ROWS=true
>>>>>
>>>>>
>>>>> CREATE INDEX auth_account_users_user_idx ON
>>>>> auth.account_users(user_id)
>>>>>
>>>>> CREATE INDEX auth_account_users_account_idx ON
>>>>> auth.account_users(account_id)
>>>>>
>>>>> The creation works fine on this but during insert I get the following
>>>>> message, I am wondering is this intended? Judging by the code it should
>>>>> work... I am using the 4.0 RC btw.
>>>>>
>>>>> java.lang.IllegalArgumentException: Unsupported non nullable index
>>>>> type BINARY
>>>>> at
>>>>> org.apache.phoenix.util.IndexUtil.getIndexColumnDataType(IndexUtil.java:81)
>>>>> at
>>>>> org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMaintainer.java:345)
>>>>> at
>>>>> org.apache.phoenix.index.IndexMaintainer.buildUpdateMutation(IndexMaintainer.java:391)
>>>>> at
>>>>> org.apache.phoenix.util.IndexUtil.generateIndexData(IndexUtil.java:185)
>>>>> at
>>>>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:203)
>>>>> at
>>>>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:186)
>>>>> at
>>>>> org.apache.phoenix.execute.MutationState.commit(MutationState.java:347)
>>>>> at
>>>>> org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:351)
>>>>> at
>>>>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:229)
>>>>> at
>>>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:185)
>>>>> at
>>>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:146)
>>>>> at
>>>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:151)
>>>>>
>>>>> --
>>>>> Dan Di Spaltro
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Dan Di Spaltro
>>>
>>
>>
>
>
> --
> Dan Di Spaltro
>
Re: Question on Binary indexes
Posted by Dan Di Spaltro <da...@gmail.com>.
Yeah I guess I was just assuming like a Long (fixed 8 bytes) it could be
treated the same. Actually that reminds me, can you use a long to index?
Does it internally maintain the 8 byte binary array?
On Fri, Mar 28, 2014 at 10:23 AM, James Taylor <ja...@apache.org>wrote:
> One option would be to base64 encode your fixed length binary as a String.
> Then store it that way in Phoenix and you'd be able to index it.
>
> It's possible that we could encode binary data in a way that would let us
> index it - how about filing a JIRA for that?
>
> Thanks,
> James
>
>
> On Fri, Mar 28, 2014 at 3:09 AM, Dan Di Spaltro <da...@gmail.com>wrote:
>
>> So James' ticket seems to indicate this is intended behavior, have you
>> resolved that? It seems like an index on arbitrarily fixed-with bytes
>> aren't a bad thing.
>>
>>
>> On Wed, Mar 26, 2014 at 3:49 PM, Samarth Jain <sa...@gmail.com>wrote:
>>
>>> Hi Dan,
>>>
>>> Thanks for reporting.
>>>
>>> I am working on a fix for this. Please see:
>>> https://issues.apache.org/jira/browse/PHOENIX-899
>>>
>>> Thanks,
>>> Samarth
>>>
>>>
>>>
>>> On Wed, Mar 26, 2014 at 3:42 PM, Dan Di Spaltro <dan.dispaltro@gmail.com
>>> > wrote:
>>>
>>>> I have a table with two indexes
>>>>
>>>> create table auth.account_users
>>>> (
>>>> ordered_id BINARY(16) not null PRIMARY KEY,
>>>> account_id BINARY(16) not null,
>>>> user_id BINARY(16) not null,
>>>> default BOOLEAN,
>>>> created_at TIMESTAMP
>>>> ) VERSIONS=1,IMMUTABLE_ROWS=true
>>>>
>>>>
>>>> CREATE INDEX auth_account_users_user_idx ON auth.account_users(user_id)
>>>>
>>>> CREATE INDEX auth_account_users_account_idx ON
>>>> auth.account_users(account_id)
>>>>
>>>> The creation works fine on this but during insert I get the following
>>>> message, I am wondering is this intended? Judging by the code it should
>>>> work... I am using the 4.0 RC btw.
>>>>
>>>> java.lang.IllegalArgumentException: Unsupported non nullable index type
>>>> BINARY
>>>> at
>>>> org.apache.phoenix.util.IndexUtil.getIndexColumnDataType(IndexUtil.java:81)
>>>> at
>>>> org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMaintainer.java:345)
>>>> at
>>>> org.apache.phoenix.index.IndexMaintainer.buildUpdateMutation(IndexMaintainer.java:391)
>>>> at
>>>> org.apache.phoenix.util.IndexUtil.generateIndexData(IndexUtil.java:185)
>>>> at
>>>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:203)
>>>> at
>>>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:186)
>>>> at
>>>> org.apache.phoenix.execute.MutationState.commit(MutationState.java:347)
>>>> at
>>>> org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:351)
>>>> at
>>>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:229)
>>>> at
>>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:185)
>>>> at
>>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:146)
>>>> at
>>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:151)
>>>>
>>>> --
>>>> Dan Di Spaltro
>>>>
>>>
>>>
>>
>>
>> --
>> Dan Di Spaltro
>>
>
>
--
Dan Di Spaltro
Re: Question on Binary indexes
Posted by James Taylor <ja...@apache.org>.
One option would be to base64 encode your fixed length binary as a String.
Then store it that way in Phoenix and you'd be able to index it.
It's possible that we could encode binary data in a way that would let us
index it - how about filing a JIRA for that?
Thanks,
James
On Fri, Mar 28, 2014 at 3:09 AM, Dan Di Spaltro <da...@gmail.com>wrote:
> So James' ticket seems to indicate this is intended behavior, have you
> resolved that? It seems like an index on arbitrarily fixed-with bytes
> aren't a bad thing.
>
>
> On Wed, Mar 26, 2014 at 3:49 PM, Samarth Jain <sa...@gmail.com>wrote:
>
>> Hi Dan,
>>
>> Thanks for reporting.
>>
>> I am working on a fix for this. Please see:
>> https://issues.apache.org/jira/browse/PHOENIX-899
>>
>> Thanks,
>> Samarth
>>
>>
>>
>> On Wed, Mar 26, 2014 at 3:42 PM, Dan Di Spaltro <da...@gmail.com>wrote:
>>
>>> I have a table with two indexes
>>>
>>> create table auth.account_users
>>> (
>>> ordered_id BINARY(16) not null PRIMARY KEY,
>>> account_id BINARY(16) not null,
>>> user_id BINARY(16) not null,
>>> default BOOLEAN,
>>> created_at TIMESTAMP
>>> ) VERSIONS=1,IMMUTABLE_ROWS=true
>>>
>>>
>>> CREATE INDEX auth_account_users_user_idx ON auth.account_users(user_id)
>>>
>>> CREATE INDEX auth_account_users_account_idx ON
>>> auth.account_users(account_id)
>>>
>>> The creation works fine on this but during insert I get the following
>>> message, I am wondering is this intended? Judging by the code it should
>>> work... I am using the 4.0 RC btw.
>>>
>>> java.lang.IllegalArgumentException: Unsupported non nullable index type
>>> BINARY
>>> at
>>> org.apache.phoenix.util.IndexUtil.getIndexColumnDataType(IndexUtil.java:81)
>>> at
>>> org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMaintainer.java:345)
>>> at
>>> org.apache.phoenix.index.IndexMaintainer.buildUpdateMutation(IndexMaintainer.java:391)
>>> at
>>> org.apache.phoenix.util.IndexUtil.generateIndexData(IndexUtil.java:185)
>>> at
>>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:203)
>>> at
>>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:186)
>>> at
>>> org.apache.phoenix.execute.MutationState.commit(MutationState.java:347)
>>> at
>>> org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:351)
>>> at
>>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:229)
>>> at
>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:185)
>>> at
>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:146)
>>> at
>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:151)
>>>
>>> --
>>> Dan Di Spaltro
>>>
>>
>>
>
>
> --
> Dan Di Spaltro
>
Re: Question on Binary indexes
Posted by Dan Di Spaltro <da...@gmail.com>.
So James' ticket seems to indicate this is intended behavior, have you
resolved that? It seems like an index on arbitrarily fixed-with bytes
aren't a bad thing.
On Wed, Mar 26, 2014 at 3:49 PM, Samarth Jain <sa...@gmail.com>wrote:
> Hi Dan,
>
> Thanks for reporting.
>
> I am working on a fix for this. Please see:
> https://issues.apache.org/jira/browse/PHOENIX-899
>
> Thanks,
> Samarth
>
>
>
> On Wed, Mar 26, 2014 at 3:42 PM, Dan Di Spaltro <da...@gmail.com>wrote:
>
>> I have a table with two indexes
>>
>> create table auth.account_users
>> (
>> ordered_id BINARY(16) not null PRIMARY KEY,
>> account_id BINARY(16) not null,
>> user_id BINARY(16) not null,
>> default BOOLEAN,
>> created_at TIMESTAMP
>> ) VERSIONS=1,IMMUTABLE_ROWS=true
>>
>>
>> CREATE INDEX auth_account_users_user_idx ON auth.account_users(user_id)
>>
>> CREATE INDEX auth_account_users_account_idx ON
>> auth.account_users(account_id)
>>
>> The creation works fine on this but during insert I get the following
>> message, I am wondering is this intended? Judging by the code it should
>> work... I am using the 4.0 RC btw.
>>
>> java.lang.IllegalArgumentException: Unsupported non nullable index type
>> BINARY
>> at
>> org.apache.phoenix.util.IndexUtil.getIndexColumnDataType(IndexUtil.java:81)
>> at
>> org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMaintainer.java:345)
>> at
>> org.apache.phoenix.index.IndexMaintainer.buildUpdateMutation(IndexMaintainer.java:391)
>> at
>> org.apache.phoenix.util.IndexUtil.generateIndexData(IndexUtil.java:185)
>> at org.apache.phoenix.execute.MutationState$1.next(MutationState.java:203)
>> at
>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:186)
>> at org.apache.phoenix.execute.MutationState.commit(MutationState.java:347)
>> at
>> org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:351)
>> at
>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:229)
>> at
>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:185)
>> at
>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:146)
>> at
>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:151)
>>
>> --
>> Dan Di Spaltro
>>
>
>
--
Dan Di Spaltro
Re: Question on Binary indexes
Posted by Samarth Jain <sa...@gmail.com>.
Hi Dan,
Thanks for reporting.
I am working on a fix for this. Please see:
https://issues.apache.org/jira/browse/PHOENIX-899
Thanks,
Samarth
On Wed, Mar 26, 2014 at 3:42 PM, Dan Di Spaltro <da...@gmail.com>wrote:
> I have a table with two indexes
>
> create table auth.account_users
> (
> ordered_id BINARY(16) not null PRIMARY KEY,
> account_id BINARY(16) not null,
> user_id BINARY(16) not null,
> default BOOLEAN,
> created_at TIMESTAMP
> ) VERSIONS=1,IMMUTABLE_ROWS=true
>
>
> CREATE INDEX auth_account_users_user_idx ON auth.account_users(user_id)
>
> CREATE INDEX auth_account_users_account_idx ON
> auth.account_users(account_id)
>
> The creation works fine on this but during insert I get the following
> message, I am wondering is this intended? Judging by the code it should
> work... I am using the 4.0 RC btw.
>
> java.lang.IllegalArgumentException: Unsupported non nullable index type
> BINARY
> at
> org.apache.phoenix.util.IndexUtil.getIndexColumnDataType(IndexUtil.java:81)
> at
> org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMaintainer.java:345)
> at
> org.apache.phoenix.index.IndexMaintainer.buildUpdateMutation(IndexMaintainer.java:391)
> at
> org.apache.phoenix.util.IndexUtil.generateIndexData(IndexUtil.java:185)
> at org.apache.phoenix.execute.MutationState$1.next(MutationState.java:203)
> at
> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:186)
> at org.apache.phoenix.execute.MutationState.commit(MutationState.java:347)
> at
> org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:351)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:229)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:185)
> at
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:146)
> at
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:151)
>
> --
> Dan Di Spaltro
>