You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kudu.apache.org by "Todd Lipcon (JIRA)" <ji...@apache.org> on 2016/11/28 20:13:58 UTC
[jira] [Commented] (KUDU-1764) Log block manager leaks disk space
at the end of full containers
[ https://issues.apache.org/jira/browse/KUDU-1764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15702995#comment-15702995 ]
Todd Lipcon commented on KUDU-1764:
-----------------------------------
For example:
{noformat}
[todd@vd0340 data]$ du -sm fdba6629bd1a48608c3e3f5551ee2bcf.data
30 fdba6629bd1a48608c3e3f5551ee2bcf.data
[todd@vd0340 data]$ filefrag -v -b fdba6629bd1a48608c3e3f5551ee2bcf.data | tail -15
283 10304954 1941265850 1941265090 2
284 10339630 1941886254 1941265852 2
285 10381588 1942636820 1941886256 2
286 10417560 1943102872 1942636822 2
287 10420946 1943139026 1943102874 4
288 10423842 1943141922 1943139030 2
289 10489036 1944046796 1943141924 820 unwritten
290 10489856 1944125440 1944047616 4096 unwritten
291 10493952 1944182784 1944129536 4096 unwritten
292 10498048 1944227840 1944186880 4096 unwritten
293 10502144 1944281088 1944231936 4096 unwritten
294 10506240 1944334336 1944285184 4096 unwritten
295 10510336 1944387584 1944338432 4096 unwritten
296 10514432 1944432640 1944391680 4024 unwritten,eof
fdba6629bd1a48608c3e3f5551ee2bcf.data: 297 extents found
[todd@vd0340 data]$ echo $[4096*6 + 4024 + 820]
29420
[todd@vd0340 data]$ ls -l fdba6629bd1a48608c3e3f5551ee2bcf.data
-rw-r--r-- 1 kudu kudu 10770898944 Oct 17 2015 fdba6629bd1a48608c3e3f5551ee2bcf.data
[todd@vd0340 data]$ ls -lh fdba6629bd1a48608c3e3f5551ee2bcf.data
-rw-r--r-- 1 kudu kudu 11G Oct 17 2015 fdba6629bd1a48608c3e3f5551ee2bcf.data
{noformat}
> Log block manager leaks disk space at the end of full containers
> ----------------------------------------------------------------
>
> Key: KUDU-1764
> URL: https://issues.apache.org/jira/browse/KUDU-1764
> Project: Kudu
> Issue Type: Bug
> Components: fs
> Affects Versions: 1.0.0, 1.1.0
> Reporter: Todd Lipcon
> Assignee: Adar Dembo
> Priority: Critical
>
> 阿香 reported on the mailing list that his total disk space usage is significantly more than the metric reported by the LBM usage metric.
> Adar and I investigated and found the likely cause: when writing to LBM containers, the LBM preallocates disk space at the end of the file where the next blocks will be written. However, the preallocated space can extend past the "full" container size, and when the container is declared full, we don't truncate or punch away that preallocated space.
> Thus, even when a container has been mostly punched out, we still end up with ~15-30MB of wasted disk space at the end.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)