You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "James Taylor (JIRA)" <ji...@apache.org> on 2017/01/19 23:33:26 UTC
[jira] [Comment Edited] (PHOENIX-541) Make mutable batch size
bytes-based instead of row-based
[ https://issues.apache.org/jira/browse/PHOENIX-541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15830818#comment-15830818 ]
James Taylor edited comment on PHOENIX-541 at 1/19/17 11:33 PM:
----------------------------------------------------------------
Sorry to leave this hanging, [~gjacoby]. I think this looks good to go - definitely an improvement. I do think we should do these couple of other items (and you're right, it's only calculated/used in that one place), but we can do that in a separate JIRA (filed PHOENIX-3612 for this):
- Make similar changes QueryServices.MAX_MUTATION_SIZE_ATTRIB - making it byte-based instead of row-count-based. Usage of this config parameter would be isolated to MutationState, I believe. We should be able to come up with an accurate size based on the underlying Mutation and/or Delete info we store in PRowImpl.
- Have a reasonable (smaller) default for the new QueryServices.MAX_MUTATION_SIZE_BYTES_ATTRIB
[~samarthjain] - would you mind giving this a quick look and committing if it looks fine?
was (Author: jamestaylor):
Sorry to leave this hanging, [~gjacoby]. I think this looks good to go - definitely an improvement. I do think we should do these couple of other items (and you're right, it's only calculated/used in that one place), but we can do that in a separate JIRA:
- Make similar changes QueryServices.MAX_MUTATION_SIZE_ATTRIB - making it byte-based instead of row-count-based. Usage of this config parameter would be isolated to MutationState, I believe. We should be able to come up with an accurate size based on the underlying Mutation and/or Delete info we store in PRowImpl.
- Have a reasonable (smaller) default for the new QueryServices.MAX_MUTATION_SIZE_BYTES_ATTRIB
[~samarthjain] - would you mind giving this a quick look and committing if it looks fine?
> Make mutable batch size bytes-based instead of row-based
> --------------------------------------------------------
>
> Key: PHOENIX-541
> URL: https://issues.apache.org/jira/browse/PHOENIX-541
> Project: Phoenix
> Issue Type: Improvement
> Affects Versions: 3.0-Release
> Reporter: mujtaba
> Assignee: Geoffrey Jacoby
> Labels: newbie
> Fix For: 4.10.0
>
> Attachments: PHOENIX-541.patch, PHOENIX-541-v2.patch
>
>
> With current configuration of row-count based mutable batch size, ideal value for batch size is around 800 rather then current 15k when creating indexes based on memory consumption, CPU and GC (data size: key: ~60 bytes, 14 integer column in separate CFs)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)