You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Roshan Naik (JIRA)" <ji...@apache.org> on 2013/04/04 20:35:15 UTC
[jira] [Commented] (FLUME-1966) FileChannel should intelligently
decide on default minimumRequiredSpace
[ https://issues.apache.org/jira/browse/FLUME-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622633#comment-13622633 ]
Roshan Naik commented on FLUME-1966:
------------------------------------
Brock, What is the significance of minimumRequiredSpace ? Not much mentioned in the User guide. Is it possible to eliminate this setting instead ?
> FileChannel should intelligently decide on default minimumRequiredSpace
> -----------------------------------------------------------------------
>
> Key: FLUME-1966
> URL: https://issues.apache.org/jira/browse/FLUME-1966
> Project: Flume
> Issue Type: Bug
> Components: File Channel
> Reporter: Brock Noland
>
> Right now the default minimumRequiredSpace is set to 500MB. As opposed to having a fixed number, I think we should have some kind of algorithm that finds a proper default based on the disk in question. This assumes the user has not set minimumRequiredSpace, in which case the default calcuated below will be ignored.
> The issue is we would like a large minimumRequiredSpace so we can be sure to stop writing before a disk fills up in production, where disks are generally going to be quite large. However, we don't want it so large that the file channel doesn't work out of the box when testing it on VMs or laptops.
> Something like this might work:
> default = max(min(10% disk capacity, 5GB), 500MB)
> Basically we always set the default to a minimum of 500MB. In production, typically we have 500GB or larger disks we'll set it to 5GB. On VM/laptop machines with smaller disks, we'll set it to 10% the disk or 500MB whichever is larger.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira