You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "Hudson (JIRA)" <ji...@apache.org> on 2014/08/06 19:24:12 UTC
[jira] [Commented] (PHOENIX-1149) Index mutations should be
committed as batches while building local index
[ https://issues.apache.org/jira/browse/PHOENIX-1149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14087917#comment-14087917 ]
Hudson commented on PHOENIX-1149:
---------------------------------
FAILURE: Integrated in Phoenix | Master | Hadoop1 #318 (See [https://builds.apache.org/job/Phoenix-master-hadoop1/318/])
PHOENIX-1149 Index mutations should be committed as batches while building local index (Rajeshbabu) (jtaylor: rev 0d8c04cf8061bf658afab0241453fea662c1d22b)
* phoenix-core/src/main/java/org/apache/phoenix/coprocessor/UngroupedAggregateRegionObserver.java
* phoenix-core/src/main/java/org/apache/phoenix/compile/WhereOptimizer.java
> Index mutations should be committed as batches while building local index
> -------------------------------------------------------------------------
>
> Key: PHOENIX-1149
> URL: https://issues.apache.org/jira/browse/PHOENIX-1149
> Project: Phoenix
> Issue Type: Bug
> Reporter: rajeshbabu
> Assignee: rajeshbabu
> Fix For: 5.0.0, 4.1
>
> Attachments: PHOENIX-1149.patch
>
>
> Currently while building local indexes batchSize is 0 so we are not committing the index mutations until full region scan is done and which may cause high memory usage or OOM with huge data.
> {code}
> // Commit in batches based on UPSERT_BATCH_SIZE_ATTRIB in config
> if (!indexMutations.isEmpty() && batchSize > 0 && indexMutations.size() % batchSize == 0) {
> HRegion indexRegion = getIndexRegion(c.getEnvironment());
> // Get indexRegion corresponding to data region
> commitBatch(indexRegion, indexMutations, null);
> indexMutations.clear();
> }
> {code}
> Thanks for finding this [~mujtabachohan].
--
This message was sent by Atlassian JIRA
(v6.2#6252)