[jira] [Created] (PHOENIX-3778) Local Index - Writes get exponentially slower as batch size increases

["Mujtaba Chohan (JIRA)" <ji...@apache.org>]

Mujtaba Chohan created PHOENIX-3778:
---------------------------------------

             Summary: Local Index - Writes get exponentially slower as batch size increases
                 Key: PHOENIX-3778
                 URL: https://issues.apache.org/jira/browse/PHOENIX-3778
             Project: Phoenix
          Issue Type: Bug
    Affects Versions: 4.10.0
            Reporter: Mujtaba Chohan


{noformat}
CREATE TABLE IF NOT EXISTS T (
    PK_PART_A CHAR(15) NOT NULL,
    PK_PART_B CHAR(3) NOT NULL,
    PK_PART_C CHAR(15) NOT NULL,
    PK_PART_DATE DATE NOT NULL,
    PK_PART_LAST CHAR(15) NOT NULL,
    VAL_ID CHAR(15), 
    CONSTRAINT PK PRIMARY KEY 
    (
        PK_PART_A, 
        PK_PART_B,
        PK_PART_C,
        PK_PART_DATE,
        PK_PART_LAST
    )
) IMMUTABLE_ROWS=true

CREATE LOCAL INDEX IF NOT EXISTS T_IDX ON T (PK_PART_DATE)
{noformat}

With HBase 0.98.23 and head of 4.x. writes to local index gets exponentially slower as batch size increases compared to global index when data is upserted to base table with index. 

Following is the average time per batch comparing local vs global index. Tested on cluster with 12GB heap per region server and also on single node machine with 4GB heap. Memory usage was within limits with minimal GC activity on client and server. [~samarthjain] and I looked for anything obvious for CPU in jvisualvm but nothing popped up.

\\
| *Batch Size* | *Local Index* time in ms per batch | *Global Index* time in ms per batch |
| 10         | 15                                 | 11                                  |
| 100        | 175                                | 15                                  |
| 500        | 3500                               | 50                                  |
| 1000       | 150000                             | 95                                  |
\\
[~rajeshbabu]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)