[jira] [Commented] (PHOENIX-3788) GLOBAL_MUTATION_BATCH_SIZE should reflect size of chunked batches

["Geoffrey Jacoby (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037547#comment-16037547 ] 

Geoffrey Jacoby commented on PHOENIX-3788:
------------------------------------------

[~jamestaylor], I tried implementing the approach you explain above, and ran into an issue on the last step: recording on client-side the chunked bytes the server side used in Delete scans and UPSERT SELECTs. In both cases the server-side MutationPlan execute returns a MutationState, but it's empty except for the max batch size and the total mutation count of the operation. In addition to the total mutation count, I also need the total byte size of all mutations. 

However, in MutationPlan.execute I only have access to an iterator of Tuples, and the ColumnProjector, and neither seems to have a way to estimate the byte size of the underlying keyvalues (since Tuples don't seem to have an iterator over their KeyValues). 

Is there a simple solution I'm missing, or does this need further enhancement to the Tuple interface/some other dependency?

> GLOBAL_MUTATION_BATCH_SIZE should reflect size of chunked batches
> -----------------------------------------------------------------
>
>                 Key: PHOENIX-3788
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3788
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.10.0
>            Reporter: Geoffrey Jacoby
>            Assignee: Geoffrey Jacoby
>             Fix For: 4.11.0
>
>         Attachments: PHOENIX-3788.patch
>
>
> As part of PHOENIX-541, we started chunking large MutationStates into multiple smaller batches transparently. However, the relevant metric, GLOBAL_MUTATION_BATCH_SIZE, still is updated with the total batch size, not the size of each chunk. This means you can't see the actual batch sizes which are being submitted to HBase. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)