[jira] [Updated] (OOZIE-3382) [SSH action] Optimize process streams draining

["Andras Piros (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OOZIE-3382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andras Piros updated OOZIE-3382:
--------------------------------
    Summary: [SSH action] Optimize process streams draining  (was: Optimize SshActionExecutor's drainBuffers method)

> [SSH action] Optimize process streams draining
> ----------------------------------------------
>
>                 Key: OOZIE-3382
>                 URL: https://issues.apache.org/jira/browse/OOZIE-3382
>             Project: Oozie
>          Issue Type: Improvement
>            Reporter: Andras Salamon
>            Assignee: Andras Salamon
>            Priority: Major
>         Attachments: OOZIE-3382-01.patch, OOZIE-3382-02.patch, OOZIE-3382-03.patch, OOZIE-3382-04.patch
>
>
> OOZIE-3354 improved {{SshActionExecutor}} to avoid {{Process#waitFor()}} blocks and modified the {{drainBuffers}} method to keep draining the standard output (and standard error) continuously.
> Right now the speed of the drain is hardwired. As long as the process is running the method only reads 1024 bytes in each cycle (half a second) which can take very long time if we want to drain several megabytes (for instance {{oozie.servlet.CallbackServlet.max.data.len}} is increased).
> Let's optimize the draining.
> We can either read 1024 bytes multiple times in each cycle (as long as there are data in the buffer), or we can increase the value of the buffer size (1024). 
> In the latter case the default of the buffer size could be half of the {{oozie.servlet.CallbackServlet.max.data.len}} value, but we also need an additional property to specify the buffer size (to avoid memory problems because of using a very big buffer). We can keep 1024 as a minimum buffer size. 
> It would be also useful to refactor the code and put the buffer draining into a separate class and create unit tests for the class. Using this class in {{ShellMain}} to avoid code duplication would also be very useful, but we have to fix OOZIE-3359 first.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)