You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Alex Kamil <al...@gmail.com> on 2014/09/17 06:28:23 UTC
speed up ALTER
Is there a faster alternative to ALTER statement?
I'm trying to add ~800 columns to a table (3 regionservers, salt_bucket=3)
and it takes >2 min. As I generate about 300 of these statements during
bootstrap it takes hours to finish
May be via direct update to meta tables via HBase API or even HDFS files,
anything goes
Re: speed up ALTER
Posted by Alex Kamil <al...@gmail.com>.
for reference, after replacing ALTER with CREATE it brought down total
bootstrap time from 5hr to 5min, about 1-1.5 sec per CREATE stmt (with 2000
columns per table)
it looks like "ALTER TABLE mytable ADD IF NOT EXISTS col1 varchar, col2
.." generates individual calls for each column (based on zookeeper trace),
while CREATE does it in one shot. probably expected behavior.
On Wed, Sep 17, 2014 at 1:03 AM, Alex Kamil <al...@gmail.com> wrote:
> oh, nevermind, there is CSV bulk loader
> <http://phoenix.apache.org/bulk_dataload.html>, somehow I missed it
>
> On Wed, Sep 17, 2014 at 12:53 AM, Alex Kamil <al...@gmail.com> wrote:
>
>> ..looks like there is a way to create/update Hbase table via MapReduce
>> job and TableOutputFormat, can it be applied to phoenix metadata imports
>>
>> On Wed, Sep 17, 2014 at 12:28 AM, Alex Kamil <al...@gmail.com>
>> wrote:
>>
>>> Is there a faster alternative to ALTER statement?
>>>
>>> I'm trying to add ~800 columns to a table (3 regionservers,
>>> salt_bucket=3) and it takes >2 min. As I generate about 300 of these
>>> statements during bootstrap it takes hours to finish
>>>
>>> May be via direct update to meta tables via HBase API or even HDFS
>>> files, anything goes
>>>
>>
>>
>
Re: speed up ALTER
Posted by Alex Kamil <al...@gmail.com>.
oh, nevermind, there is CSV bulk loader
<http://phoenix.apache.org/bulk_dataload.html>, somehow I missed it
On Wed, Sep 17, 2014 at 12:53 AM, Alex Kamil <al...@gmail.com> wrote:
> ..looks like there is a way to create/update Hbase table via MapReduce job
> and TableOutputFormat, can it be applied to phoenix metadata imports
>
> On Wed, Sep 17, 2014 at 12:28 AM, Alex Kamil <al...@gmail.com> wrote:
>
>> Is there a faster alternative to ALTER statement?
>>
>> I'm trying to add ~800 columns to a table (3 regionservers,
>> salt_bucket=3) and it takes >2 min. As I generate about 300 of these
>> statements during bootstrap it takes hours to finish
>>
>> May be via direct update to meta tables via HBase API or even HDFS files,
>> anything goes
>>
>
>
Re: speed up ALTER
Posted by Alex Kamil <al...@gmail.com>.
..looks like there is a way to create/update Hbase table via MapReduce job
and TableOutputFormat, can it be applied to phoenix metadata imports
On Wed, Sep 17, 2014 at 12:28 AM, Alex Kamil <al...@gmail.com> wrote:
> Is there a faster alternative to ALTER statement?
>
> I'm trying to add ~800 columns to a table (3 regionservers, salt_bucket=3)
> and it takes >2 min. As I generate about 300 of these statements during
> bootstrap it takes hours to finish
>
> May be via direct update to meta tables via HBase API or even HDFS files,
> anything goes
>