[jira] [Created] (TRAFODION-2748) Upsert performance when source is a Hive scan

["Suresh Subbiah (JIRA)" <ji...@apache.org>]

Suresh Subbiah created TRAFODION-2748:
-----------------------------------------

             Summary: Upsert performance when source is a Hive scan
                 Key: TRAFODION-2748
                 URL: https://issues.apache.org/jira/browse/TRAFODION-2748
             Project: Apache Trafodion
          Issue Type: Bug
          Components: sql-cmp
    Affects Versions: 1.3-incubating
            Reporter: Suresh Subbiah
            Assignee: Suresh Subbiah
            Priority: Minor
             Fix For: 1.3-incubating


From Eric Owhadi.

In the following set of steps, the cqd GEN_DP2I_SIZE_DOWN should not have been necessary. The quesize should be determined by rowset size. However we were not flagging Hive scans as returning multiple rows in OltOptInfo class in the generator. This cause the initial queue size to be set too low. The fix could help improve performance for other query types with Hive scan too.

cqd GEN_DP2I_SIZE_DOWN '1024';
cqd UPD_ORDERED 'OFF';
cqd HBASE_ROWSET_VSBB_SIZE '20480';

  cqd allow_nullable_unique_key_constraint 'ON';
  CREATE TABLE STORE_SALES5(
    ss_sold_date_sk int,
  ss_sold_time_sk int,
  ss_item_sk int,
  ss_customer_sk int,
  ss_cdemo_sk int,
  ss_hdemo_sk int,
  ss_addr_sk int,
  ss_store_sk int,
  ss_promo_sk int,
  ss_ticket_number int,
  ss_quantity int,
  ss_wholesale_cost real,
  ss_list_price real,
  ss_sales_price real,
  ss_ext_discount_amt real,
  ss_ext_sales_price real,
  ss_ext_wholesale_cost real,
  ss_ext_list_price real,
  ss_ext_tax real,
  ss_coupon_amt real,
  ss_net_paid real,
  ss_net_paid_inc_tax real,
  ss_net_profit real)
  store by (ss_sold_date_sk, ss_sold_time_sk)
  Salt using 96 partitions on (ss_sold_date_sk, ss_sold_time_sk)
    HBASE_OPTIONS
  (
    DATA_BLOCK_ENCODING = 'FAST_DIFF',
    COMPRESSION = 'SNAPPY',
    MEMSTORE_FLUSH_SIZE = '1400000000',
    MAX_FILESIZE = '107374182400',
    BLOOMFILTER = 'NONE',
    BLOCKSIZE = '524280'

  )
;

 alter 'TRAF_1500000:TRAFODION.ERIC.STORE_SALES5', CONFIGURATION => {'hbase.hstore.compaction.max.size' => '4000000000'}
prepare s from 
upsert using load into store_sales5 select * from hive.tpcds_sf10000.store_sales_ext;




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)