You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Abhishek Chennaka (Code Review)" <ge...@cloudera.org> on 2023/09/18 21:37:11 UTC

[kudu-CR] [server]KUDU-3513 Decrease default value of —log container preallocate bytes

Abhishek Chennaka has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20489


Change subject: [server]KUDU-3513 Decrease default value of —log_container_preallocate_bytes
......................................................................

[server]KUDU-3513 Decrease default value of —log_container_preallocate_bytes

This changes the default value of —log_container_preallocate_bytes to 1MB.
In situations where there is a table with large number of columns
(for example 1000) and one row of data we end up with large amount of disk
space being pre-allocated currently (in our example 32*1000MB). This might
not be significant if we have large number of rows and fewer columns but
becomes more pronounced in the other case. As we continue to explore the
limitations of Kudu with large number of columns, this needs to be addressed.

Change-Id: I3dc424f0a6086bb80d904fac94f9327a3bf9423a
---
M src/kudu/fs/log_block_manager.cc
1 file changed, 1 insertion(+), 1 deletion(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/89/20489/1
-- 
To view, visit http://gerrit.cloudera.org:8080/20489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I3dc424f0a6086bb80d904fac94f9327a3bf9423a
Gerrit-Change-Number: 20489
Gerrit-PatchSet: 1
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>

[kudu-CR] [server]KUDU-3513 Decrease default value of —log container preallocate bytes

Posted by "Yingchun Lai (Code Review)" <ge...@cloudera.org>.
Yingchun Lai has posted comments on this change. ( http://gerrit.cloudera.org:8080/20489 )

Change subject: [server]KUDU-3513 Decrease default value of —log_container_preallocate_bytes
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/20489/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/20489/1//COMMIT_MSG@12
PS1, Line 12: This might
            : not be significant if we have large number of rows and fewer columns but
            : becomes more pronounced in the other case
> The pre-allocation allows for container's data to be more "contiguous" on d
I encountered the same issue about the pre-allocate space when tables have large number of columns.
But it's not a good idea to change the default config value IMO, it changes the default behavior.



-- 
To view, visit http://gerrit.cloudera.org:8080/20489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3dc424f0a6086bb80d904fac94f9327a3bf9423a
Gerrit-Change-Number: 20489
Gerrit-PatchSet: 1
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Yingchun Lai <la...@apache.org>
Gerrit-Comment-Date: Wed, 11 Oct 2023 03:16:00 +0000
Gerrit-HasComments: Yes

[kudu-CR] [server]KUDU-3513 Decrease default value of —log container preallocate bytes

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/20489 )

Change subject: [server]KUDU-3513 Decrease default value of —log_container_preallocate_bytes
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/20489/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/20489/1//COMMIT_MSG@12
PS1, Line 12: This might
            : not be significant if we have large number of rows and fewer columns but
            : becomes more pronounced in the other case
The pre-allocation allows for container's data to be more "contiguous" on disk to increase the performance of sequential reading.

Did you do any performance-related tests to assess performance impact of this patch?



-- 
To view, visit http://gerrit.cloudera.org:8080/20489
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I3dc424f0a6086bb80d904fac94f9327a3bf9423a
Gerrit-Change-Number: 20489
Gerrit-PatchSet: 1
Gerrit-Owner: Abhishek Chennaka <ac...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <al...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Tue, 19 Sep 2023 00:28:19 +0000
Gerrit-HasComments: Yes