You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Amareshwari Sri Ramadasu (JIRA)" <ji...@apache.org> on 2007/10/15 14:49:50 UTC
[jira] Issue Comment Edited: (HADOOP-1788) Increase the buffer size
of pipes from 1k to 128k
[ https://issues.apache.org/jira/browse/HADOOP-1788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534843 ]
amareshwari edited comment on HADOOP-1788 at 10/15/07 5:48 AM:
----------------------------------------------------------------------------
Thanks Christian.
I ran my 500KB write example with this patch and now, there is 128KB write followed by the rest.
mmap2(NULL, 507904, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7ec4000
write(3, "oooooooooooooooooooooooooooooooo"..., 131072) = 131072
write(3, "oooooooooooooooooooooooooooooooo"..., 393216) = 393216
munmap(0xb7ec4000, 507904) = 0
Patch is fine, except some clarifications
1. 128*1028 has to be replaced by 128*1024.
2. do we need the change in procedure progress() ?
3. do we require setsockopt change now?
4. should we do setvbuf for the case of 'hadoop.pipes.command.file' environment also?
was (Author: amareshwari):
Thanks Christian.
I ran my 500KB write example with this patch and now, there is 128KB write followed by the rest.
mmap2(NULL, 507904, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7ec4000
write(3, "oooooooooooooooooooooooooooooooo"..., 131072) = 131072
write(3, "oooooooooooooooooooooooooooooooo"..., 393216) = 393216
munmap(0xb7ec4000, 507904) = 0
Patch is fine, except some clarifications
1. 128*1028 has to be replaced by 128*1024.
2. do we need the change in procedure progress() ?
4. do we require setsockopt change now?
3. should we do setvbuf for the case of 'hadoop.pipes.command.file' environment also?
> Increase the buffer size of pipes from 1k to 128k
> -------------------------------------------------
>
> Key: HADOOP-1788
> URL: https://issues.apache.org/jira/browse/HADOOP-1788
> Project: Hadoop
> Issue Type: Bug
> Components: pipes
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sri Ramadasu
> Attachments: patch-1788-1.txt, patch-1788.txt
>
>
> Currently pipes applications use 1k writes to the socket and it should be larger to increase throughput.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.