You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Satish Iyengar <sa...@gmail.com> on 2015/09/01 16:02:41 UTC
PhoenixHBaseStorage issues with Salted table
Good Morning,
I am using Phoenix 4.2.2 with Hbase 0.98 on Horton Works HDP 2.2.
My Phoenix table looks like this
CREATE TABLE checks(hashed_key varchar(32), enterprise_id bigint not
null, business_date
date not null, location_id bigintnot null, cost_center_code varchar,
cost_center_name
varchar, day_part varchar, check_number varchar, total decimal(15,2),
item_sum decimal(15,2), item_count decimal(15,2), discount_sum decimal(15,2
), service_charge_sum decimal(15,2), tax_sumdecimal(15,2), payment_sum
decimal(15,2), service_charge_percent decimal(15,2), void_sum
decimal(15,2) CONSTRAINT
pk PRIMARY KEY (hashed_key, enterprise_id, business_date, location_id,
cost_center_code, cost_center_name, day_part))SALT_BUCKETS=32, VERSIONS=1;
My pig script to store in the above table is
c = foreach b generate md5hasher(event_group_key) as hashed_key,
enterprise_id, ToDate(business_date, 'yyyy-MM-dd') asbusiness_date,
location_id, (chararray) cost_center_code, normalize(cost_center_name) as
cost_center_name, day_part_nameas day_part, (chararray) check_number,
aggregate_total as total, aggregate_item_sum as item_sum,
aggregate_item_count asitem_count, aggregate_discount_sum as discount_sum,
aggregate_servicecharge_sum as service_charge_sum, aggregate_tax_sumas
tax_sum, aggregate_payment_sum as payment_sum,
aggregate_servicecharge_percent as service_charge_percent,
aggregate_void_sum as void_sum;
STORE c into 'hbase://checks' using
org.apache.phoenix.pig.PhoenixHBaseStorage('zkquorum','-batchSize 5000'
);
I am getting exception
Error: java.lang.RuntimeException: Unable to process column _SALT:BINARY,
innerMessage=org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (
22005): Type mismatch. BINARY cannot be coerced to VARCHAR at org.apache.
phoenix.pig.hadoop.PhoenixRecord.write(PhoenixRecord.java:80) at org.apache.
phoenix.pig.hadoop.PhoenixRecordWriter.write(PhoenixRecordWriter.java:71) at
org.apache.phoenix.pig.hadoop.PhoenixRecordWriter.write(PhoenixRecordWriter.
java:41) at org.apache.phoenix.pig.PhoenixHBaseStorage.putNext(
PhoenixHBaseStorage.java:172) at org.apache.pig.backend.hadoop.
executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(
PigOutputFormat.java:136)
How can I get around this? Am I missing something?
Furthermore, Is it possible to provide a KeyPrefix Split policy (HBase
provided) with Phoenix table?
Thanks,
Satish
--
Satish Iyengar
"Anyone who has never made a mistake has never tried anything new."
Albert Einstein
Re: PhoenixHBaseStorage issues with Salted table
Posted by Ravi Kiran <ma...@gmail.com>.
Hi Satish,
This was reported and fixed as part of
https://issues.apache.org/jira/browse/PHOENIX-2181. For a quick turnaround,
you can do this.
STORE c into 'hbase://checks/enterprise_id,business_date....' using
org.apache.phoenix.pig.PhoenixHBaseStorage('zkquorum','-batchSize 5000'
);
Primarily, you would need to explicitly pass all the columns .
Thanks
Ravi
On Tue, Sep 1, 2015 at 7:02 AM, Satish Iyengar <sa...@gmail.com> wrote:
>
>
> Good Morning,
>
> I am using Phoenix 4.2.2 with Hbase 0.98 on Horton Works HDP 2.2.
>
> My Phoenix table looks like this
>
> CREATE TABLE checks(hashed_key varchar(32), enterprise_id bigint not null, business_date
> date not null, location_id bigintnot null, cost_center_code varchar, cost_center_name
> varchar, day_part varchar, check_number varchar, total decimal(15,2),
> item_sum decimal(15,2), item_count decimal(15,2), discount_sum decimal(15,
> 2), service_charge_sum decimal(15,2), tax_sumdecimal(15,2), payment_sum
> decimal(15,2), service_charge_percent decimal(15,2), void_sum decimal(15,2
> ) CONSTRAINT pk PRIMARY KEY (hashed_key, enterprise_id, business_date,
> location_id, cost_center_code, cost_center_name, day_part))SALT_BUCKETS=
> 32, VERSIONS=1;
>
> My pig script to store in the above table is
>
> c = foreach b generate md5hasher(event_group_key) as hashed_key,
> enterprise_id, ToDate(business_date, 'yyyy-MM-dd') asbusiness_date,
> location_id, (chararray) cost_center_code, normalize(cost_center_name) as
> cost_center_name, day_part_nameas day_part, (chararray) check_number,
> aggregate_total as total, aggregate_item_sum as item_sum,
> aggregate_item_count asitem_count, aggregate_discount_sum as discount_sum
> , aggregate_servicecharge_sum as service_charge_sum, aggregate_tax_sumas
> tax_sum, aggregate_payment_sum as payment_sum,
> aggregate_servicecharge_percent as service_charge_percent,
> aggregate_void_sum as void_sum;
>
>
> STORE c into 'hbase://checks' using
> org.apache.phoenix.pig.PhoenixHBaseStorage('zkquorum','-batchSize
> 5000');
>
>
> I am getting exception
>
> Error: java.lang.RuntimeException: Unable to process column _SALT:BINARY,
> innerMessage=org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (
> 22005): Type mismatch. BINARY cannot be coerced to VARCHAR at org.apache.
> phoenix.pig.hadoop.PhoenixRecord.write(PhoenixRecord.java:80) at org.
> apache.phoenix.pig.hadoop.PhoenixRecordWriter.write(PhoenixRecordWriter.
> java:71) at org.apache.phoenix.pig.hadoop.PhoenixRecordWriter.write(
> PhoenixRecordWriter.java:41) at org.apache.phoenix.pig.PhoenixHBaseStorage
> .putNext(PhoenixHBaseStorage.java:172) at org.apache.pig.backend.hadoop.
> executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(
> PigOutputFormat.java:136)
>
> How can I get around this? Am I missing something?
>
> Furthermore, Is it possible to provide a KeyPrefix Split policy (HBase
> provided) with Phoenix table?
>
> Thanks,
> Satish
>
>
>
> --
> Satish Iyengar
>
> "Anyone who has never made a mistake has never tried anything new."
> Albert Einstein
>