You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Dan Di Spaltro <da...@gmail.com> on 2014/03/26 23:42:00 UTC

Question on Binary indexes

I have a table with two indexes

create table auth.account_users
    (
    ordered_id BINARY(16) not null PRIMARY KEY,
    account_id BINARY(16) not null,
    user_id BINARY(16) not null,
    default BOOLEAN,
    created_at TIMESTAMP
    ) VERSIONS=1,IMMUTABLE_ROWS=true


CREATE INDEX auth_account_users_user_idx ON auth.account_users(user_id)

CREATE INDEX auth_account_users_account_idx ON
auth.account_users(account_id)

The creation works fine on this but during insert I get the following
message, I am wondering is this intended?  Judging by the code it should
work... I am using the 4.0 RC btw.

java.lang.IllegalArgumentException: Unsupported non nullable index type
BINARY
at
org.apache.phoenix.util.IndexUtil.getIndexColumnDataType(IndexUtil.java:81)
 at
org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMaintainer.java:345)
at
org.apache.phoenix.index.IndexMaintainer.buildUpdateMutation(IndexMaintainer.java:391)
 at org.apache.phoenix.util.IndexUtil.generateIndexData(IndexUtil.java:185)
at org.apache.phoenix.execute.MutationState$1.next(MutationState.java:203)
 at org.apache.phoenix.execute.MutationState$1.next(MutationState.java:186)
at org.apache.phoenix.execute.MutationState.commit(MutationState.java:347)
 at
org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:351)
at
org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:229)
 at
org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:185)
at
org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:146)
 at
org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:151)

-- 
Dan Di Spaltro

Re: Question on Binary indexes

Posted by James Taylor <ja...@apache.org>.
Yes, a BIGINT (fixed 8 byte Long) may be used in an index. Pretty much
every fixed sized type *except* BINARY may be used. Because a column value
that maps to a KeyValue may be null (i.e. absent), we need a way of
representing that in the index row key. With numeric types we have that by
converting them to a DECIMAL.

File a JIRA, though. There are ways to encode BINARY data that would leave
us a way to represent the absence of a value.


On Sun, Mar 30, 2014 at 9:21 PM, Dan Di Spaltro <da...@gmail.com>wrote:

> Yeah I guess I was just assuming like a Long (fixed 8 bytes) it could be
> treated the same.  Actually that reminds me, can you use a long to index?
>  Does it internally maintain the 8 byte binary array?
>
>
> On Fri, Mar 28, 2014 at 10:23 AM, James Taylor <ja...@apache.org>wrote:
>
>> One option would be to base64 encode your fixed length binary as a
>> String. Then store it that way in Phoenix and you'd be able to index it.
>>
>> It's possible that we could encode binary data in a way that would let us
>> index it - how about filing a JIRA for that?
>>
>> Thanks,
>> James
>>
>>
>> On Fri, Mar 28, 2014 at 3:09 AM, Dan Di Spaltro <da...@gmail.com>wrote:
>>
>>> So James' ticket seems to indicate this is intended behavior, have you
>>> resolved that?  It seems like an index on arbitrarily fixed-with bytes
>>> aren't a bad thing.
>>>
>>>
>>> On Wed, Mar 26, 2014 at 3:49 PM, Samarth Jain <sa...@gmail.com>wrote:
>>>
>>>> Hi Dan,
>>>>
>>>> Thanks for reporting.
>>>>
>>>> I am working on a fix for this. Please see:
>>>> https://issues.apache.org/jira/browse/PHOENIX-899
>>>>
>>>> Thanks,
>>>> Samarth
>>>>
>>>>
>>>>
>>>> On Wed, Mar 26, 2014 at 3:42 PM, Dan Di Spaltro <
>>>> dan.dispaltro@gmail.com> wrote:
>>>>
>>>>> I have a table with two indexes
>>>>>
>>>>> create table auth.account_users
>>>>>     (
>>>>>     ordered_id BINARY(16) not null PRIMARY KEY,
>>>>>     account_id BINARY(16) not null,
>>>>>     user_id BINARY(16) not null,
>>>>>     default BOOLEAN,
>>>>>     created_at TIMESTAMP
>>>>>     ) VERSIONS=1,IMMUTABLE_ROWS=true
>>>>>
>>>>>
>>>>> CREATE INDEX auth_account_users_user_idx ON
>>>>> auth.account_users(user_id)
>>>>>
>>>>> CREATE INDEX auth_account_users_account_idx ON
>>>>> auth.account_users(account_id)
>>>>>
>>>>> The creation works fine on this but during insert I get the following
>>>>> message, I am wondering is this intended?  Judging by the code it should
>>>>> work... I am using the 4.0 RC btw.
>>>>>
>>>>> java.lang.IllegalArgumentException: Unsupported non nullable index
>>>>> type BINARY
>>>>> at
>>>>> org.apache.phoenix.util.IndexUtil.getIndexColumnDataType(IndexUtil.java:81)
>>>>>  at
>>>>> org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMaintainer.java:345)
>>>>> at
>>>>> org.apache.phoenix.index.IndexMaintainer.buildUpdateMutation(IndexMaintainer.java:391)
>>>>>  at
>>>>> org.apache.phoenix.util.IndexUtil.generateIndexData(IndexUtil.java:185)
>>>>> at
>>>>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:203)
>>>>>  at
>>>>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:186)
>>>>> at
>>>>> org.apache.phoenix.execute.MutationState.commit(MutationState.java:347)
>>>>>  at
>>>>> org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:351)
>>>>> at
>>>>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:229)
>>>>>  at
>>>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:185)
>>>>> at
>>>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:146)
>>>>>  at
>>>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:151)
>>>>>
>>>>> --
>>>>> Dan Di Spaltro
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Dan Di Spaltro
>>>
>>
>>
>
>
> --
> Dan Di Spaltro
>

Re: Question on Binary indexes

Posted by Dan Di Spaltro <da...@gmail.com>.
Yeah I guess I was just assuming like a Long (fixed 8 bytes) it could be
treated the same.  Actually that reminds me, can you use a long to index?
 Does it internally maintain the 8 byte binary array?


On Fri, Mar 28, 2014 at 10:23 AM, James Taylor <ja...@apache.org>wrote:

> One option would be to base64 encode your fixed length binary as a String.
> Then store it that way in Phoenix and you'd be able to index it.
>
> It's possible that we could encode binary data in a way that would let us
> index it - how about filing a JIRA for that?
>
> Thanks,
> James
>
>
> On Fri, Mar 28, 2014 at 3:09 AM, Dan Di Spaltro <da...@gmail.com>wrote:
>
>> So James' ticket seems to indicate this is intended behavior, have you
>> resolved that?  It seems like an index on arbitrarily fixed-with bytes
>> aren't a bad thing.
>>
>>
>> On Wed, Mar 26, 2014 at 3:49 PM, Samarth Jain <sa...@gmail.com>wrote:
>>
>>> Hi Dan,
>>>
>>> Thanks for reporting.
>>>
>>> I am working on a fix for this. Please see:
>>> https://issues.apache.org/jira/browse/PHOENIX-899
>>>
>>> Thanks,
>>> Samarth
>>>
>>>
>>>
>>> On Wed, Mar 26, 2014 at 3:42 PM, Dan Di Spaltro <dan.dispaltro@gmail.com
>>> > wrote:
>>>
>>>> I have a table with two indexes
>>>>
>>>> create table auth.account_users
>>>>     (
>>>>     ordered_id BINARY(16) not null PRIMARY KEY,
>>>>     account_id BINARY(16) not null,
>>>>     user_id BINARY(16) not null,
>>>>     default BOOLEAN,
>>>>     created_at TIMESTAMP
>>>>     ) VERSIONS=1,IMMUTABLE_ROWS=true
>>>>
>>>>
>>>> CREATE INDEX auth_account_users_user_idx ON auth.account_users(user_id)
>>>>
>>>> CREATE INDEX auth_account_users_account_idx ON
>>>> auth.account_users(account_id)
>>>>
>>>> The creation works fine on this but during insert I get the following
>>>> message, I am wondering is this intended?  Judging by the code it should
>>>> work... I am using the 4.0 RC btw.
>>>>
>>>> java.lang.IllegalArgumentException: Unsupported non nullable index type
>>>> BINARY
>>>> at
>>>> org.apache.phoenix.util.IndexUtil.getIndexColumnDataType(IndexUtil.java:81)
>>>>  at
>>>> org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMaintainer.java:345)
>>>> at
>>>> org.apache.phoenix.index.IndexMaintainer.buildUpdateMutation(IndexMaintainer.java:391)
>>>>  at
>>>> org.apache.phoenix.util.IndexUtil.generateIndexData(IndexUtil.java:185)
>>>> at
>>>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:203)
>>>>  at
>>>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:186)
>>>> at
>>>> org.apache.phoenix.execute.MutationState.commit(MutationState.java:347)
>>>>  at
>>>> org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:351)
>>>> at
>>>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:229)
>>>>  at
>>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:185)
>>>> at
>>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:146)
>>>>  at
>>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:151)
>>>>
>>>> --
>>>> Dan Di Spaltro
>>>>
>>>
>>>
>>
>>
>> --
>> Dan Di Spaltro
>>
>
>


-- 
Dan Di Spaltro

Re: Question on Binary indexes

Posted by James Taylor <ja...@apache.org>.
One option would be to base64 encode your fixed length binary as a String.
Then store it that way in Phoenix and you'd be able to index it.

It's possible that we could encode binary data in a way that would let us
index it - how about filing a JIRA for that?

Thanks,
James


On Fri, Mar 28, 2014 at 3:09 AM, Dan Di Spaltro <da...@gmail.com>wrote:

> So James' ticket seems to indicate this is intended behavior, have you
> resolved that?  It seems like an index on arbitrarily fixed-with bytes
> aren't a bad thing.
>
>
> On Wed, Mar 26, 2014 at 3:49 PM, Samarth Jain <sa...@gmail.com>wrote:
>
>> Hi Dan,
>>
>> Thanks for reporting.
>>
>> I am working on a fix for this. Please see:
>> https://issues.apache.org/jira/browse/PHOENIX-899
>>
>> Thanks,
>> Samarth
>>
>>
>>
>> On Wed, Mar 26, 2014 at 3:42 PM, Dan Di Spaltro <da...@gmail.com>wrote:
>>
>>> I have a table with two indexes
>>>
>>> create table auth.account_users
>>>     (
>>>     ordered_id BINARY(16) not null PRIMARY KEY,
>>>     account_id BINARY(16) not null,
>>>     user_id BINARY(16) not null,
>>>     default BOOLEAN,
>>>     created_at TIMESTAMP
>>>     ) VERSIONS=1,IMMUTABLE_ROWS=true
>>>
>>>
>>> CREATE INDEX auth_account_users_user_idx ON auth.account_users(user_id)
>>>
>>> CREATE INDEX auth_account_users_account_idx ON
>>> auth.account_users(account_id)
>>>
>>> The creation works fine on this but during insert I get the following
>>> message, I am wondering is this intended?  Judging by the code it should
>>> work... I am using the 4.0 RC btw.
>>>
>>> java.lang.IllegalArgumentException: Unsupported non nullable index type
>>> BINARY
>>> at
>>> org.apache.phoenix.util.IndexUtil.getIndexColumnDataType(IndexUtil.java:81)
>>>  at
>>> org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMaintainer.java:345)
>>> at
>>> org.apache.phoenix.index.IndexMaintainer.buildUpdateMutation(IndexMaintainer.java:391)
>>>  at
>>> org.apache.phoenix.util.IndexUtil.generateIndexData(IndexUtil.java:185)
>>> at
>>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:203)
>>>  at
>>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:186)
>>> at
>>> org.apache.phoenix.execute.MutationState.commit(MutationState.java:347)
>>>  at
>>> org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:351)
>>> at
>>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:229)
>>>  at
>>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:185)
>>> at
>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:146)
>>>  at
>>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:151)
>>>
>>> --
>>> Dan Di Spaltro
>>>
>>
>>
>
>
> --
> Dan Di Spaltro
>

Re: Question on Binary indexes

Posted by Dan Di Spaltro <da...@gmail.com>.
So James' ticket seems to indicate this is intended behavior, have you
resolved that?  It seems like an index on arbitrarily fixed-with bytes
aren't a bad thing.


On Wed, Mar 26, 2014 at 3:49 PM, Samarth Jain <sa...@gmail.com>wrote:

> Hi Dan,
>
> Thanks for reporting.
>
> I am working on a fix for this. Please see:
> https://issues.apache.org/jira/browse/PHOENIX-899
>
> Thanks,
> Samarth
>
>
>
> On Wed, Mar 26, 2014 at 3:42 PM, Dan Di Spaltro <da...@gmail.com>wrote:
>
>> I have a table with two indexes
>>
>> create table auth.account_users
>>     (
>>     ordered_id BINARY(16) not null PRIMARY KEY,
>>     account_id BINARY(16) not null,
>>     user_id BINARY(16) not null,
>>     default BOOLEAN,
>>     created_at TIMESTAMP
>>     ) VERSIONS=1,IMMUTABLE_ROWS=true
>>
>>
>> CREATE INDEX auth_account_users_user_idx ON auth.account_users(user_id)
>>
>> CREATE INDEX auth_account_users_account_idx ON
>> auth.account_users(account_id)
>>
>> The creation works fine on this but during insert I get the following
>> message, I am wondering is this intended?  Judging by the code it should
>> work... I am using the 4.0 RC btw.
>>
>> java.lang.IllegalArgumentException: Unsupported non nullable index type
>> BINARY
>> at
>> org.apache.phoenix.util.IndexUtil.getIndexColumnDataType(IndexUtil.java:81)
>>  at
>> org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMaintainer.java:345)
>> at
>> org.apache.phoenix.index.IndexMaintainer.buildUpdateMutation(IndexMaintainer.java:391)
>>  at
>> org.apache.phoenix.util.IndexUtil.generateIndexData(IndexUtil.java:185)
>> at org.apache.phoenix.execute.MutationState$1.next(MutationState.java:203)
>>  at
>> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:186)
>> at org.apache.phoenix.execute.MutationState.commit(MutationState.java:347)
>>  at
>> org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:351)
>> at
>> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:229)
>>  at
>> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:185)
>> at
>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:146)
>>  at
>> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:151)
>>
>> --
>> Dan Di Spaltro
>>
>
>


-- 
Dan Di Spaltro

Re: Question on Binary indexes

Posted by Samarth Jain <sa...@gmail.com>.
Hi Dan,

Thanks for reporting.

I am working on a fix for this. Please see:
https://issues.apache.org/jira/browse/PHOENIX-899

Thanks,
Samarth



On Wed, Mar 26, 2014 at 3:42 PM, Dan Di Spaltro <da...@gmail.com>wrote:

> I have a table with two indexes
>
> create table auth.account_users
>     (
>     ordered_id BINARY(16) not null PRIMARY KEY,
>     account_id BINARY(16) not null,
>     user_id BINARY(16) not null,
>     default BOOLEAN,
>     created_at TIMESTAMP
>     ) VERSIONS=1,IMMUTABLE_ROWS=true
>
>
> CREATE INDEX auth_account_users_user_idx ON auth.account_users(user_id)
>
> CREATE INDEX auth_account_users_account_idx ON
> auth.account_users(account_id)
>
> The creation works fine on this but during insert I get the following
> message, I am wondering is this intended?  Judging by the code it should
> work... I am using the 4.0 RC btw.
>
> java.lang.IllegalArgumentException: Unsupported non nullable index type
> BINARY
> at
> org.apache.phoenix.util.IndexUtil.getIndexColumnDataType(IndexUtil.java:81)
>  at
> org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMaintainer.java:345)
> at
> org.apache.phoenix.index.IndexMaintainer.buildUpdateMutation(IndexMaintainer.java:391)
>  at
> org.apache.phoenix.util.IndexUtil.generateIndexData(IndexUtil.java:185)
> at org.apache.phoenix.execute.MutationState$1.next(MutationState.java:203)
>  at
> org.apache.phoenix.execute.MutationState$1.next(MutationState.java:186)
> at org.apache.phoenix.execute.MutationState.commit(MutationState.java:347)
>  at
> org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:351)
> at
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:229)
>  at
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:185)
> at
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:146)
>  at
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:151)
>
> --
> Dan Di Spaltro
>