[jira] [Updated] (PHOENIX-2995) Write performance severely degrades with large number of views

["Mujtaba Chohan (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PHOENIX-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mujtaba Chohan updated PHOENIX-2995:
------------------------------------
    Attachment: upsert_rate.png

> Write performance severely degrades with large number of views 
> ---------------------------------------------------------------
>
>                 Key: PHOENIX-2995
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-2995
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Mujtaba Chohan
>              Labels: Argus
>         Attachments: upsert_rate.png
>
>
> Write performance for each 1K batch degrades significantly when there are *10K* views being written in random with default {{phoenix.client.maxMetaDataCacheSize}}. With all views created, upsert rate remains around 25 seconds per 1K batch i.e. ~2K rows/min upsert rate. 
> When {{phoenix.client.maxMetaDataCacheSize}} is increased to 100MB+ then view does not need to get re-resolved and upsert rate gets back to normal ~60K rows/min.
> With *100K* views and {{phoenix.client.maxMetaDataCacheSize}} set to 1GB, I wasn't able create all 100K views as upsert time for each 1K batch keeps on steadily increasing. 
> Following graph shows 1K batch upsert rate over time with variation of number of views. Rows are upserted to random views {{CREATE VIEW IF NOT EXISTS ... APPEND_ONLY_SCHEMA = true, UPDATE_CACHE_FREQUENCY=900000}} is executed before upsert statement.
> Base table is also created with {{APPEND_ONLY_SCHEMA = true, UPDATE_CACHE_FREQUENCY = 900000, AUTO_PARTITION_SEQ}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)