Phoenix Secondary index question

[Udit Mehta <um...@groupon.com>]

Hi All,

I am facing an issue while working with secondary indexes on a Phoenix
view/table that is created on top of an existing Hbase table.

I basically have an existing Hbase table with a few rows and I then build a
Phoenix view on this table. This works fine and any updates directly to the
Hbase table are reflected in the Phoenix view. Now the problem arises the
moment I create a secondary index on the phoenix table and then update the
underlying Hbase table. I dont see the updates being reflected in either of
the Phoenix tables ie the original one and the indexed one.

Is this behavior or am I missing something?

Also is it a good idea to have such a design where we update the Hbase
table directly with a Phoenix table/view created on top of it? We had to
choose this design due to the backup/snapshot features Hbase provides. I
could not find any such mechanism in Phoenix so not sure what the best
approach is here.

Looking forward to your ideas.

Thanks in advance,
Udit

Re: Phoenix Secondary index question

[Mich Talebzadeh <mi...@gmail.com>]

Phoenix indexes are covering indexes that are physically created in Hbase
database.

To maintain these Phoenix indexes you need to ingest data through Phoenix
as opposed to Hbase. You can of course manually rebuild Phoenix indexes but
that is neither practical or cheap time wise.

You also have the option of org.apache.phoenix.mapreduce.CsvBulkLoadTool to
load data at command level into Hbase via Phoenix skin.

HTH

Dr Mich Talebzadeh



LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*



http://talebzadehmich.wordpress.com


*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.



On 4 January 2017 at 19:47, Udit Mehta <um...@groupon.com> wrote:

> Thanks James for the reply.
>
> Can you also tell me how to go about taking backups/snapshots in Phoenix
> similar to what Hbase has. I assume i cant use the snapshot feature Hbase
> offers for the same reason you cited above.
>
> Thanks again,
> Udit
>
> On Tue, Jan 3, 2017 at 5:26 PM, James Taylor <ja...@apache.org>
> wrote:
>
>> Hi Udit,
>> You'll need to use Phoenix APIs when you update your data if you want
>> Phoenix to keep you index in sync with your table.
>> Thanks,
>> James
>>
>> On Tue, Jan 3, 2017 at 1:32 PM, Udit Mehta <um...@groupon.com> wrote:
>>
>>> Hi All,
>>>
>>> I am facing an issue while working with secondary indexes on a Phoenix
>>> view/table that is created on top of an existing Hbase table.
>>>
>>> I basically have an existing Hbase table with a few rows and I then
>>> build a Phoenix view on this table. This works fine and any updates
>>> directly to the Hbase table are reflected in the Phoenix view. Now the
>>> problem arises the moment I create a secondary index on the phoenix table
>>> and then update the underlying Hbase table. I dont see the updates being
>>> reflected in either of the Phoenix tables ie the original one and the
>>> indexed one.
>>>
>>> Is this behavior or am I missing something?
>>>
>>> Also is it a good idea to have such a design where we update the Hbase
>>> table directly with a Phoenix table/view created on top of it? We had to
>>> choose this design due to the backup/snapshot features Hbase provides. I
>>> could not find any such mechanism in Phoenix so not sure what the best
>>> approach is here.
>>>
>>> Looking forward to your ideas.
>>>
>>> Thanks in advance,
>>> Udit
>>>
>>
>>
>

Re: Phoenix Secondary index question

[Jeremy Huffman <je...@jeremyhuffman.com>]

Udit once they have been persisted Phoenix tables and indexes are just
normal rows in HBase tables. HBase backups and replication would work just
fine. It may be helpful to make a small table with index, upsert a few rows
and look at the raw entries with the HBase shell to understand what Phoenix
is doing for you behind the scenes.


As for Snapshot functionality, Phoenix has explicit support through
the CurrentSCN
property; usage is demonstrated in the FAQ.

http://phoenix.apache.org/faq.html#Can_phoenix_work_on_tables_with_arbitrary_timestamp_as_flexible_as_HBase_API


On Wed, Jan 4, 2017 at 2:47 PM Udit Mehta <um...@groupon.com> wrote:

> Thanks James for the reply.
>
> Can you also tell me how to go about taking backups/snapshots in Phoenix
> similar to what Hbase has. I assume i cant use the snapshot feature Hbase
> offers for the same reason you cited above.
>
> Thanks again,
> Udit
>
> On Tue, Jan 3, 2017 at 5:26 PM, James Taylor <ja...@apache.org>
> wrote:
>
> Hi Udit,
> You'll need to use Phoenix APIs when you update your data if you want
> Phoenix to keep you index in sync with your table.
> Thanks,
> James
>
> On Tue, Jan 3, 2017 at 1:32 PM, Udit Mehta <um...@groupon.com> wrote:
>
> Hi All,
>
> I am facing an issue while working with secondary indexes on a Phoenix
> view/table that is created on top of an existing Hbase table.
>
> I basically have an existing Hbase table with a few rows and I then build
> a Phoenix view on this table. This works fine and any updates directly to
> the Hbase table are reflected in the Phoenix view. Now the problem arises
> the moment I create a secondary index on the phoenix table and then update
> the underlying Hbase table. I dont see the updates being reflected in
> either of the Phoenix tables ie the original one and the indexed one.
>
> Is this behavior or am I missing something?
>
> Also is it a good idea to have such a design where we update the Hbase
> table directly with a Phoenix table/view created on top of it? We had to
> choose this design due to the backup/snapshot features Hbase provides. I
> could not find any such mechanism in Phoenix so not sure what the best
> approach is here.
>
> Looking forward to your ideas.
>
> Thanks in advance,
> Udit
>
>
>
>
>
>
>
>
>

Re: Phoenix Secondary index question

[Udit Mehta <um...@groupon.com>]

Thanks James for the reply.

Can you also tell me how to go about taking backups/snapshots in Phoenix
similar to what Hbase has. I assume i cant use the snapshot feature Hbase
offers for the same reason you cited above.

Thanks again,
Udit

On Tue, Jan 3, 2017 at 5:26 PM, James Taylor <ja...@apache.org> wrote:

> Hi Udit,
> You'll need to use Phoenix APIs when you update your data if you want
> Phoenix to keep you index in sync with your table.
> Thanks,
> James
>
> On Tue, Jan 3, 2017 at 1:32 PM, Udit Mehta <um...@groupon.com> wrote:
>
>> Hi All,
>>
>> I am facing an issue while working with secondary indexes on a Phoenix
>> view/table that is created on top of an existing Hbase table.
>>
>> I basically have an existing Hbase table with a few rows and I then build
>> a Phoenix view on this table. This works fine and any updates directly to
>> the Hbase table are reflected in the Phoenix view. Now the problem arises
>> the moment I create a secondary index on the phoenix table and then update
>> the underlying Hbase table. I dont see the updates being reflected in
>> either of the Phoenix tables ie the original one and the indexed one.
>>
>> Is this behavior or am I missing something?
>>
>> Also is it a good idea to have such a design where we update the Hbase
>> table directly with a Phoenix table/view created on top of it? We had to
>> choose this design due to the backup/snapshot features Hbase provides. I
>> could not find any such mechanism in Phoenix so not sure what the best
>> approach is here.
>>
>> Looking forward to your ideas.
>>
>> Thanks in advance,
>> Udit
>>
>
>

Re: Phoenix Secondary index question

[James Taylor <ja...@apache.org>]

Hi Udit,
You'll need to use Phoenix APIs when you update your data if you want
Phoenix to keep you index in sync with your table.
Thanks,
James

On Tue, Jan 3, 2017 at 1:32 PM, Udit Mehta <um...@groupon.com> wrote:

> Hi All,
>
> I am facing an issue while working with secondary indexes on a Phoenix
> view/table that is created on top of an existing Hbase table.
>
> I basically have an existing Hbase table with a few rows and I then build
> a Phoenix view on this table. This works fine and any updates directly to
> the Hbase table are reflected in the Phoenix view. Now the problem arises
> the moment I create a secondary index on the phoenix table and then update
> the underlying Hbase table. I dont see the updates being reflected in
> either of the Phoenix tables ie the original one and the indexed one.
>
> Is this behavior or am I missing something?
>
> Also is it a good idea to have such a design where we update the Hbase
> table directly with a Phoenix table/view created on top of it? We had to
> choose this design due to the backup/snapshot features Hbase provides. I
> could not find any such mechanism in Phoenix so not sure what the best
> approach is here.
>
> Looking forward to your ideas.
>
> Thanks in advance,
> Udit
>